Mapping array type events for data lake insertion by using the REST API

The Platform Service logical interface array JSON type is not supported for incoming events for use by Maximo Asset Monitor. To use array data with Platform Service you must map either the entire array or the individual array items to logical interface properties by converting them into strings

Overview

The Platform Service user interface is used to create physical and logical interfaces for data ingestion to the data lake. You can use either Last Event Cache or file uploads to define the events to be included in the physical interface, and then use simple base type mapping between the physical and logical interfaces.

However, the Platform Service Web UI mapping does not currently support the array JSON type. To map array type properties you must use the Platform Service REST API.

In this example we are using the device type SimDevType with one registered device: simdev01
For the SimDevType physical interface schema, go here.

The SimDevType events include array JSON type properties.
The following shows the content of the IoT event named "example" which includes three array JSON type properties: arr01, arr02, and arr03. These cannot be directly consumed by Maximo Asset Monitor.


{
"EventName" : "anExampleEvent",
"EventDescription" : "defines the format of an example IoT event with arrays",
"EventID": "example",
"Payload" : {"d": {"ver" : "1.3",
"powerCons" : 12.64,
"currTemp" : 51,
"arr01" : [ 1.1, 2.22, 3.333, 4.4444, 5.55, 6.666],
"arr02" : [ "Abc", "Def", "Ghi" ],
"arr03":[ 123, 456, 789, 1010 ]
 } } }

The goal of this task is to map the array properties into three corresponding string type properties: compobj1, compobj2 and compobj3.
Each of these properties is a string that concatenates the first, second, and third array items correspondingly.
Example: "compobj1": "$string({'x': $event.d.arr01[0], 'y': $event.d.arr02[0], 'z': $event.d.arr03[0]})",
From the sample event: compobj1 = 'x':1.1, 'y':"Abc", 'z':123

Update the logical interface mapping by using the REST API

Important: The following array mapping steps must be done after you have created the interfaces with the UI, but before you activate them.

To complete the array event type mapping, complete the following steps:

Step 1: Get the schema ID and logical interface ID.
Step 2: Get the draft logical interface schema content.
Step 3: Update the draft logical interface schema.
Add, delete or modify the schema properties, then use the modified schema contents file to update the logical interface schema.
Step 4: Get the draft mappings that are associated with a specified device type.
Step 5: Update the draft mappings.
Add, delete, or modify the physical interface to logical interface mappings. Use the modified mappings file to update the device type mappings.
Step 6: Validate the draft logical interface configuration.
Step 7: Activate the configuration.

Note:: All API calls are provided as cURL examples.
For more information about the Platform Service state management REST API calls that are used in this example, see the IBM Watson IoT Platform HTTP REST API Docs.

Step 1: Get the schema ID and logical interface ID

Use the GET /draft/device/types/{deviceType}/logicalinterfaces call to get the draft logical interfaces associated with the SimDevType device type.
Example:
curl -X GET "https://<orgId>.internetofthings.ibmcloud.com/api/v0002/draft/device/types/SimDevType/logicalinterfaces" -H "accept: application/json" -H "authorization: Basic <Base64-encoded-credentials>"
Where <orgId> is the unique six character identifier for your account and <Base64-encoded-credentials> is the base64-encoded API credentials for that account. For information about the API credentials, see the REST APIs topic.

Identify the correct logical interface for the device type.
The logicalinterfaces end point returns an array that includes information for all logical interfaces for a single device type.
Note: The id and schemaId in the following sample are used throughout this task.
Sample API call return:


[
{
 "name": "SimDevType_LI",
 "description": "",
 "id": "{logicalInterfaceId}",
 "schemaId": "{schemaId}",
 "refs": {
   "schema": "/api/v0002/draft/schemas/{schemaId}"
 },
 "version": "draft",
 "created": "2018-06-08T12:29:50Z",
 "createdBy": "user@address.com",
 "updated": "2018-06-08T12:29:50Z",
 "updatedBy": "user@address.com"
}
]

Step 2: Get the draft logical interface schema contents

Use the GET /draft/schemas/{schemaId}/content API call to retrieve the content of the draft schema definition file with the specified ID.
Example:
curl -X GET "https://<orgId>.internetofthings.ibmcloud.com/api/v0002/draft/schemas/{schemaId}/content" -H "accept: application/json" -H "authorization: Basic <Base64-encoded-credentials>"

Save the draft logical interface schema contents to a file.
[The array type properties are by default included as ...Item_01 and ...ItemLast properties. ]
Sample physical interface schema file:


{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "schema-title-52395315",
"description": "",
"properties": {
 "version": {
   "type": "string"
 },
 "kWh": {
   "type": "number",
   "default": 1
 },
 "temperature": {
   "type": "number",
   "default": 0
 },
 "arr01_Item01": {
   "type": "number"
 },
 "arr01_ItemLast": {
   "type": "number"
 },
 "arr02_Item01": {
   "type": "string"
 },
 "arr02_ItemLast": {
   "type": "string"
 },
 "arr03_Item01": {
   "type": "number"
 },
 "arr03_ItemLast": {
   "type": "number"
 }
}
}

Step 3: Update the logical schema contents

Add the combobj logical interface schema properties that will be the targets of your array mapping, then use the modified schema contents file to update the logical interface schema.

Update the array entries to match array properties.
Add the compobj propereties that you will map the individual array components to:
```
"compobj1": {
   "type": "string"
 },
 "compobj2": {
   "type": "string"
 },
 "compobj3": {
   "type": "string"
 }
```
Use the PUT /draft/schemas/{schemaId}/content call to update the content of a draft schema definition file.
Example:
curl -X PUT "https://<orgId>.internetofthings.ibmcloud.com/api/v0002/draft/schemas/{schemaId}/content" -H "accept: application/json" -H "authorization: Basic <Base64-encoded-credentials>" -H "content-type: multipart/form-data" -d {"schemaFile":{}}

Verify that the return code is 204 for a successful update.
Sample schema content after content update:
Note: Modified items are highlighted.


{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"title": "schema-title-52395315",
"description": "",
"properties": {
 "version": {
   "type": "string"
 },
 "kWh": {
   "type": "number",
   "default": 1
 },
 "temperature": {
   "type": "number",
   "default": 0
 },
 "arr01_Item01": {
   "type": "number",
   "default": 1, // <- Modified
   "minimum": 0.5 // <- Modified
 },
 "arr01_ItemLast": {
   "type": "number"
 },
 "arr02_Item01": {
   "type": "string"
 },
 "arr02_ItemLast": {
   "type": "string"
 },
 "arr03_Item01": {
   "type": "number",
   "default": 0, // <- Modified
   "minimum": -50 // <- Modified
 },
 "arr03_ItemLast": {
   "type": "number"
 },
 "compobj1": { // <- Modified
   "type": "string" // <- Modified
 },
 "compobj2": { // <- Modified
   "type": "string" // <- Modified
 },
 "compobj3": { // <- Modified
   "type": "string" // <- Modified
 }
}
}

Step 4: Get the draft mappings that are associated with a specified device type.

Use the GET /draft/device/types/{deviceType}/mappingsGet call to the draft mappings associated with a device type.
Example:
curl -X GET "https://<orgId>.internetofthings.ibmcloud.com/api/v0002/draft/device/types/SimDevType/mappings" -H "accept: application/json" -H "authorization: Basic <Base64-encoded-credentials>"

Identify the correct logical interface for the device type.
The mappingsGet end point returns an array that includes information for all mappings for a single device type.
Sample API call return:


[
{
 "logicalInterfaceId": "{logicalInterfaceId}",
 "notificationStrategy": "on-state-change",
 "propertyMappings": {
   "example": { "version": "$event.d.ver",
 "kWh": "$event.d.powerCons",
 "temperature": "$event.d.currTemp",
 "arr02_ItemLast": "$event.d.arr02[-1]",
 "arr03_ItemLast": "$event.d.arr03[-1]",
 "arr01_Item01": "$event.d.arr01[0]",
 "arr01_ItemLast": "$event.d.arr01[-1]",
 "arr02_Item01": "$event.d.arr02[0]",
 "arr03_Item01": "$event.d.arr03[0]"
 } }, "version": "draft", "created": "2018-06-08T12:29:50Z", "createdBy": "user@address.com", "updated": "2018-06-08T15:39:20Z", "updatedBy": "a-ju1wwa-ploazvcz5l"
      } ]

Save the draft mappings to a file.

Step 5: Modify the mappings.

Add the compobj mappings to the logical interface mappings, then use the modified mappings file to update the device type mappings.

Add the array entries to match array properties.
For example, add:


"compobj1": "$string({'x': $event.d.arr01[0], 'y': $event.d.arr02[0], 'z': $event.d.arr03[0]})",
"compobj2": "$string({'x': $event.d.arr01[1], 'y': $event.d.arr02[1], 'z': $event.d.arr03[1]})",
"compobj3": "$string({'x': $event.d.arr01[2], 'y': $event.d.arr02[2], 'z': $event.d.arr03[2]})"

Each of these mappings concatenates the first, second, and third array items correspondingly.
Tip: You can also convert an entire JSON array into a string. An example of this is: "entire_arr01" : "$string($event.d.arr01)"

Use the PUT /draft/device/types/SimDevType/mappings/{logicalInterfaceId} REST API call to update the mapping.
Example:
curl -X PUT "https://<orgId>.internetofthings.ibmcloud.com/api/v0002/draft/device/types/SimDevType/mappings/{logicalInterfaceId}" -H "accept: application/json" -H "authorization: Basic <Base64-encoded-credentials>" -H "content-type: application/json" -d "{ \"logicalInterfaceId\": \"{logicalInterfaceId}\", \"notificationStrategy\": \"on-state-change\", \"propertyMappings\": { \"example\": { \"version\": \"$event.d.ver\", \"kWh\": \"$event.d.powerCons\", \"arr02_ItemLast\": \"$event.d.arr02[-1]\", \"temperature\": \"$event.d.currTemp\", \"arr03_ItemLast\": \"$event.d.arr03[-1]\", \"arr01_Item01\": \"$event.d.arr01[0]\", \"arr01_ItemLast\": \"$event.d.arr01[-1]\", \"arr02_Item01\": \"$event.d.arr02[0]\", \"arr03_Item01\": \"$event.d.arr03[0]\", \"compobj1\": \"$string({'x': $event.d.arr01[0], 'y': $event.d.arr02[0], 'z': $event.d.arr03[0]})\", \"compobj2\": \"$string({'x': $event.d.arr01[1], 'y': $event.d.arr02[1], 'z': $event.d.arr03[1]})\", \"compobj3\": \"$string({'x': $event.d.arr01[2], 'y': $event.d.arr02[2], 'z': $event.d.arr03[2]})\" } }, \"version\": \"draft\", \"created\": \"2018-06-08T12:29:50Z\", \"createdBy\": \"user@address.com\", \"updated\": \"2018-06-08T15:39:20Z\", \"updatedBy\": \"a-ju1wwa-ploazvcz5l\" }"
Sample API call return:
Note: Modified items are highlighted.
```
{
"logicalInterfaceId": "{logicalInterfaceId}",
"notificationStrategy": "on-state-change",
"propertyMappings": {
 "example": {
   "version": "$event.d.ver",
   "kWh":      "$event.d.powerCons",
   "arr02_ItemLast":  "$event.d.arr02[-1]",
   "temperature":      "$event.d.currTemp",
   "arr03_ItemLast": "$event.d.arr03[-1]",
   "arr01_Item01":   "$event.d.arr01[0]",
   "arr01_ItemLast": "$event.d.arr01[-1]",
   "arr02_Item01":   "$event.d.arr02[0]",
   "arr03_Item01":   "$event.d.arr03[0]",
   "compobj1": "$string({'x': $event.d.arr01[0], 'y':  $event.d.arr02[0], 'z': $event.d.arr03[0]})", // <- Modified
   "compobj2": "$string({'x': $event.d.arr01[1], 'y': $event.d.arr02[1], 'z': $event.d.arr03[1]})", // <- Modified
   "compobj3": "$string({'x': $event.d.arr01[2], 'y': $event.d.arr02[2], 'z': $event.d.arr03[2]})" // <- Modified
 }
},
"version": "draft",
"created": "2018-06-08T12:29:50Z",
"createdBy": "user@address.com",
"updated": "2018-06-08T15:39:20Z",
"updatedBy": "a-ju1wwa-ploazvcz5l"
}
```

Step 6: Validate the draft logical interface configuration.

Use the PATCH /draft/logicalinterfaces/{logicalInterfaceId} API call with the validate-configuration operation to validate the draft logical interface.
Example:
curl -X PATCH "https://{orgId}.internetofthings.ibmcloud.com/api/v0002/draft/logicalinterfaces/{logicalInterfaceId}" -H "accept: application/json" -H "content-type: application/json" -d "{ \"operation\": \"validate-configuration\"}"

Sample API call return:


{
  "message": "CUDIM0302I: State update configuration for Logical Interface '{logicalInterfaceName}' is valid.",
  "details": {
    "id": "CUDIM0302I",
    "properties": [
      "Logical Interface",
      "{logicalInterfaceName}"
    ]
  },
  "failures": []
}

Step 7: Activate the configuration.

Use the PATCH /draft/logicalinterfaces/{logicalInterfaceId} API call with the activate-configuration operation to activate the draft logical interface.
Example:
curl -X PATCH "https://{orgId}.internetofthings.ibmcloud.com/api/v0002/draft/logicalinterfaces/{logicalInterfaceId}" -H "accept: application/json" -H "content-type: application/json" -d "{ \"operation\": \"activate-configuration\"}"

Sample API call return:


{
    "message": "CUDIM0300I: State update configuration for Logical Interface '{logicalInterfaceName}' has been successfully submitted for activation.",
    "details": {
        "id": "CUDIM0300I",
        "properties": [
            "Logical Interface",
            "{logicalInterfaceName}"
        ]
    },
    "failures": []
}

Tip: You can also activate the logical interface from the Platform Service dashboard.

In the Platform Service user interface, go to Devices and then select the Device Type tab.
Select your device type and then select the Interfaces tab.
Click the Activate button to activate the draft logical interface.

After the logical interface schema is activated, any new array items from the received IoT event are combined into logical interface properties properties compobj1, compobj2 and compobj3 and converted to strings.

Sample draft physical interface schema

This is the example IoT event schema for SimDevType_PI:

curl -X GET "https://<orgId>.internetofthings.ibmcloud.com/api/v0002/schemas/5b1a63065908010033644197/content" -H "accept: application/json" -H "authorization: Basic <Base64-encoded-credentials>"

Returns:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "title": "example",            
  "description": "",
  "properties": {
    "d": {
      "type": "object",
      "properties": {
        "ver": {
          "type": "string",
          "default": ""
        },
        "powerCons": {
          "type": "number",
          "format": "float",
          "default": 0
        },
        "currTemp": {
          "type": "number",
          "format": "float",
          "default": 0
        },
        "arr01": {
          "default": [],
          "type": "array",
          "items": {
            "type": "number"
          }
        },
        "arr02": {
          "default": [],
          "type": "array",
          "items": {
            "type": "string"
          }
        },
        "arr03": {
          "default": [],
          "type": "array",
          "items": {
            "type": "number"
          }
        }
      }
    }
  }
}