Mapping expression language
You can use the mapping expression language to manipulate and combine data, and to format the results of any queries that you might run on your processed data.
The mapping expression language is a subset of JSONata and can be used when defining mappings or creating rules.
JSONata is a lightweight query and transformation language for JSON data. A JSONata Exerciser tool is also available, providing a quick and convenient way to try out JSONata.
The following information shows the key operators and functions that are currently supported, along with some examples of how you might use them.
Expression language and notification rules
The mapping expression language has been extended for use with the data management Using notification rules and actions through the
introduction of $state and $instance variables which are defined
for use in expressions.
- The
$statevariable is used to reference properties on the device state in an expression, for example,$state.temperature. - The
$instancevariable is used to reference attributes and metadata on the device instance in an expression, for example,$instance.metadata.tempThresholdMax.
You might want to use these variables so that you do not need to hard-code values in your notification rule condition expressions, and can set a different threshold value for each device instance.
JSONata operators
All JSONata Operators are supported, except as listed:
- Path
operators
Not supported:^(…)(Order-by)
- Numeric operators
- Comparison operators
- Boolean operators
- Other
operators
Not supported:~>(Chain):=(Variable binding)... ~> | ... | ... |(Transform)
- Use Parenthesis
( )for expression grouping and to alter operator precedence. - Use single or double quotes to surround strings and property names, for example
$event.object.'ab'. - Use back-ticks to surround property names that contain special characters, including spaces, for
example:
{"x y":22}.`x y` - Use back-ticks to surround property names that begin with a number, for example
`7emperature`
JSONata functions
- All String functions
- All Numeric functions
- All Numeric aggregation functions
- All Boolean functions
- The following Array functions:
- $count
- $append
Extending the mapping expression language
The mapping expression language has been extended for use with the data management feature through the introduction of $event, $state and $instance variables. JSON is bound to these variables before the expression is evaluated. The following table provides an overview of these variables, which are defined for use in expressions:
| Variable | Example input JSON | Example as an expression | Use to... |
|---|---|---|---|
$event |
{"t": 34.5} |
$event.t |
Select a property of an inbound event to use in an expression. |
$state |
{"temperature": 34.5,"humidity": 78 } |
$state.temperature |
Select a property on the device state to use in an expression. |
$instance |
{"deviceId": "tSensor","typeId": "humiditySensor","metadata":
{"temp_adjustment": 50}} |
$instance.metadata.temp_adjustment |
Select device or device type attribute to use in an expression. |
The following example uses the following object as the input to an event:
{
"temperature": 35,
"humidity": 72,
"pressure": 1024
}
The object is converted to an array by using the following expression:
[$event.temperature, $event.humidity, $event.pressure]
The expression results in the generation of the following output:
[
35,
72,
1024
]
You can reverse this example, and convert an array from the input to an object. The following example uses the following array as the input to an event:
{
"readings": [
35,
72,
1024
]
}
The array is converted to an object by using the following expression:
{"temperature": $event.readings[0], "humidity": $event.readings[1], "pressure": $event.readings[2]}
The expression results in the generation of the following output:
{
"temperature": 35,
"humidity": 72,
"pressure": 1024
}
You can also define an expression that combines the use of these variables. In the following example, a temp_adjustment property is defined in the device metadata and is used to calibrate the event reading. The property is defined in one mapping but can be applied to many devices.
"propertyMappings" : {
"tevt" : {
"temperature" : "$event.t + $instance.metadata.temp_adjustment"
}
},
The dot operator "." is used for object access with a literal key, for example $event.object.hh.
The first expression, which in the example is $event.t, is constrained to a
specific property, either in the event ($event) or the state ($state) or the metadata ($instance).
The second expression, which in the example is $instance.metadata.temp_adjustment,
contains the information that you might want to access.
Language guide
- All Simple queries are supported.
- Predicate queries are supported, except for wildcards.
- Parenthesized expressions are supported, except for code blocks.
- The following Conditional expressions are supported:
- Conditional logic
- Variables
Variables are supported through the use of the$instance,$state, and$eventvariables as part of the extended mapping language which is specific this data management feature.Note: The$and$$variables that are used in JSONata are not currently supported. - Invocation of most predefined functions.
Note: You cannot define your own functions.
Constructing output
You can specify how processed data is presented in the output by using array constructors or object constructors.
Supported array constructors
Arrays can be constructed by enclosing a comma-separated list of literals or expressions in a
pair of brackets. Commas are used to separate multiple expressions within the array constructor
inlcuding sequence generation, for example, [1, 3-1] = [1,2], [1..3] ->
[1,2,3] and [1..3, 7..9] -> [1,2,3,7,8,9].
Supported object constructors
You can construct JSON objects in your output by using a pair of braces {} that
contain key values or pairs separated by commas, with each key and value separated by a colon, for
example, {key1 : value1, key2:value2} or {"hello" : "world"}. The
object key must be a string.
Example: Using arrays to process and report temperature data
The following sections build on the example in Getting started with data management by using REST APIs to show how you might use arrays to maintain a sliding window of temperature readings, and calculate the current sum or average of the reading contained within that window.
A sliding window stores data in the order of arrival. Rather than keeping all the data ever inserted, sliding windows can be configured to evict data in an incremental fashion. When a sliding window fills up, any future insertion results in the eviction of the oldest data item in that window.
The following example shows a sliding window that is configured with a count-based eviction policy of size 5:
() -> (1) -> (2, 1) -> (3, 2, 1) -> (4, 3, 2, 1) -> (5, 4, 3, 2, 1) -> (6, 5, 4, 3, 2) -> (7, 6, 5, 4, 3) -> ...
The following example shows how the logical interface schema file configuration that is shown in
step 7 of the Step-by-step guide
1, can be modified by adding an array called tempReadings. The array is used
to maintain a window of the last 5 values that are sent from devices for the last 5 events. If there
are no values stored in tempReadings, the array is set to 0 and
the next received reading is appended on to the array which grows until 5 readings are received.
After 5 readings are received, a new reading results in the removal of the oldest reading from the
window.
tempReadings as "required", and
as an array by default.{
"$schema": "http://json-schema.org/draft-04/schema#",
"definitions": {},
"properties": {
"temperature": {
"description" : "latest temperature reading in degrees Celsius",
"type" : "number",
"minimum" : -273.15,
"default" : 0.0
},
"tempAverage": {
"type": "number"
},
"tempReadings": {
"default": [],
"items": {
"type": "number"
},
"type": "array"
},
"tempSum": {
"type": "number"
}
},
"required": [
"tempReadings"
],
"type": "object"
}
The following section shows an example of how you can configure the mappings resource to calculate the average temperature reading, and the sum of all temperature readings based on the readings that are contained within the current sliding window:
[
{
"created": "2017-10-13T09:21:36Z",
"createdBy": "admin",
"logicalInterfaceId": "5846ec826522050001db0e12",
"notificationStrategy": "on-state-change",
"propertyMappings": {
"tevt": {
"tempAverage": "$average($count($state.tempReadings)<5?$append($state.tempReadings, $event.t):$append($state.tempReadings[[1..4]], $event.t))",
"tempReadings": "$count($state.tempReadings)<5?$append($state.tempReadings, $event.t):$append($state.tempReadings[[1..4]], $event.t)",
"tempSum": "$sum($count($state.tempReadings)<5?$append($state.tempReadings, $event.t):$append($state.tempReadings[[1..4]], $event.t))"
}
},
"updated": "2017-10-13T10:05:40Z",
"updatedBy": "a-8x7nmj-9iqt56kfil",
"version": "active"
}
]
$state.tempReadings array is recalculated before it is used in
the $average and $sum functions. The recalculation of the array is required to ensure that the array
contains the current values when the tempAverage or tempSum
expression is evaluated, because the order of the mapping expressions cannot be controlled.The following example shows the average temperature and the summed temperature based on a sliding window of 5 temperature readings:
{
"state": {
"tempAverage": 18557.6,
"tempReadings": [
17591,
10262,
25621,
16676,
22638
],
"tempSum": 92788
},
"timestamp": "2017-10-13T11:07:20Z",
"updated": "2017-10-13T11:05:40Z"
}
Handling mismatches between mapping expression and input data
A state update might fail when one of the mapping expressions contains a reference to input data that is not specified in the published event.
For example, you might configure the following expressions:
temperature = $event.t
humidity = $event.hum
where t is an optional property for the event.
If an event that contains only humidity data is received, for example,
{"humidity":22}, then the expression temperature = $event.t fails
to evaluate, because the optional t property is not specified in the published device
event.
The state update fails. The humidity state property is not updated, and an error message is published to the MQTT error topic for the device:
iot-2/type/${typeId}/id/${deviceId}/err/data
To prevent state update failures because of unspecified optional data, you can use the $exists function in combination with a ternary conditional to specify a default value for optional properties. The following example defines a default value of 0 for the t property:
"tempEvent:
{
"temperature": "$exists($event.t)?$event.t:0",
"humidity": "$event.hum"
}
By defining a default value for the optional property in this way, the expression evaluates
successfully, even when the t property is not specified in the published device
event.
Therefore, if the event {"humidity":22} is received, the state update completes
successfully and the device state is set to {"humidity":22, "temperature":0}.