Contents


Analyzing IoT device movement data

Capture and send sensor data using IBM Watson IoT Platform, and then analyze movement patterns using SPSS Modeler

Comments

This tutorial is based on the Harlem Shake game that Romeo Kienzler developed and presented in his tutorial titled Create a fun, simple IoT accelerometer game. Kienzler's game uses simple movement data from a smartphone, streams the data to the cloud, captures the data in a Cloudant database, and then analyzes and determines the winner using IBM Watson Studio (formerly IBM Data Science Experience).

In this tutorial, we'll start with Kienzler's basics by also using IBM Cloud (formerly IBM Bluemix) and the IBM Watson IoT Platform services, including Node-RED, MQTT (in Watson IoT Platform), and Cloudant. We'll differ from his Harlem Shake game in these ways:

  • We'll use additional sensor data besides simple accelerometer input.
  • We'll use SPSS Modeler (instead of IBM Watson Studio) to analyze the data. SPSS Modeler is a code-less analysis and model generation and deployment. It is perfect for a business user with general knowledge about the data, but no coding background. (IBM Watson Studio, which might be preferred by data scientists familiar with coding in Open Source environments, uses R, Python, or both.)

Playing with sensor input from a smartphone is fun, but are there business applications for sensor data? The short answer is, yes. Imagine you have a production facility and the sensor data tells you that anytime a truck is driving near it, the production quality lowers. That would be useful information to know. A "movement" is nothing more than a combination of three-dimensional sensor data about position and speed. Keep in mind there is more than moving sensors and you can consider other sensor input like temperature, pressure, or even optical recognition of the person who enters a shop. All this data can influence business key performance indicators (KPIs) like quality, throughput, sales, or the efficiency of people working. Now, if you know what the situation is, you could take action on and improve the situation.

We will build our game in several steps:

  • Create the base application, connectivity, and data storage using the Internet of Things Platform service, Cloudant, and Node-RED.
  • Check that the data is stored in Cloudant.
  • Modify the database to collect data for three different movement types.
  • Use SPSS Modeler to create an analysis stream using data from the Cloudant database.
  • Create a classification model in SPSS Modeler by transforming the timestamp, calculating a new measure of the energy used, telling SPSS Modeler what the target is to be predicted, and by adding a classification algorithm.
  • Deploy and test the classification model manually by collecting scoring data, modifying the existing stream in SPSS Modeler to analyze and score the movement data.
  • Add a new branch to the SPSS stream for live scoring.
  • Use IBM Machine Learning in IBM Cloud to deploy the SPSS Modeler stream.

What you’ll need to build your app

1

Create the base application and verify the data in the Cloudant database

This first step is a big one. As we mentioned before, this tutorial is based on the Harlem Shake game published by Romeo Kienzler. To get started, go to Create a fun, simple IoT accelerometer game, and complete the steps in Steps 1 through 5. When you're asked to name your application, you can call it anything you like, but for the examples in this tutorial I called my application anythinguniquewilldods123.

Before you come back to this tutorial to extend Kienzler's work, you'll have deployed a game application using one-click deployment, replaced the Internet of Things Platform service, ensured the MQTT message broker can receive data, set up a Cloudant NoSQL database to store the data, and streamed the data to Cloudant using Node-RED. (Don't worry – it's not as much work as it sounds, and it's fun!)

After completing those steps in Kienzler's article, we will next check if data arrives in the table. Ensure the game app on your smartphone is still sending data by looking at the debug tab in Node-RED.

Everything should look good. The smartphone shakes, the data streams up to the cloud, and the database is holding the data. But at this point, we still do not know if data actually arrives in the Cloudant instance. Kienzler's approach was to use IBM Watson Studio, but we're going to use SPSS Modeler. Before we get to that point, however, we need to make a few more changes.

Remember, Cloudant is a NoSQL ("Not-Only-SQL") database system that we use to store the data. We can perform a basic check to see if data is arriving by using the Cloudant Dashboard.

  1. You should already be logged in to your IBM Cloud account. If not, log in.
  2. From the hamburger menu, select Dashboard.
  3. The Dashboard shows your applications and services. Look in the Services section to see your instance of a Cloudant NoSQL database. This was created with the IoT Starter Kit in Step 2 of Kienzler's tutorial.
  4. Click the line with your Cloudant NoSQL DB instance.
  5. On the Cloudant Service Details page, click Launch. The Cloudant Dashboard opens in a new window.
  6. From the left menu, click the database icon to see a list of your Cloudant databases.
  7. Find the harlemshake database and note the # of Docs stored.
  8. Activate the IoT sensors on your smartphone.

    To activate your IoT sensors on your smartphone, do the following:

    1. On your smartphone use the link you wrote down from Step 1, of Kienzler's tutorial.
    2. Log in using your own unique alphanumeric ID and 8-digit character alphabetic password (a-z). Do not include blank spaces.
    3. Wait until the app status shows "Connected" and starts counting the published messages being submitted.
  9. Refresh the Cloudant Dashboard page and observe that the # of Docs increased. (Later, we see the data using SPSS Modeler.)
2

Replace the existing Cloudant database so you can collect extended data points

We need to modify the Harlem Shake application to not only record the x, y, z position data of the smartphone, but also the acceleration data and a few more data points.

2a

Replace the existing Cloudant database

  1. You should already have your Node-RED instance open. If not, open it.

    To open your Node-RED instance, do the following:

    1. Go to the IBM Cloud Dashboard.
    2. Under Cloud Foundry Apps, click the link for your app (for example, mine is called anythinguniquewilldods123).
    3. Next to the application's name, click Visit App.
    4. Click Go to your Node-RED flow editor.
    5. Log in to your game app's Node-RED instance using the user name and password you assigned yourself during Kienzler's tutorial.

    The following flow should be displayed.

  2. To store more data points, we need a new database. We'll replace the existing harlemshake database with one of our own called iotmovements.
    1. Double-click the Cloudant node named harlemshake.
    2. In the Database field, enter the name for the new database iotmovements.
    3. Click Done.

      Note: Do not deploy and test yet.

  3. Disconnect the Cloudant node. We want to avoid "unclean data" in our new database. We reconnect it later. Leave the debug node connected.
  4. Double-click the function node and extend the JavaScript code to include acceleration data and the timestamp of the message. Replace the existing code with this code:
    	msg.payload =
    	{
    	X : msg.payload.d.ax,
    	Y : msg.payload.d.ay,
    	Z : msg.payload.d.az,
    	alpha: msg.payload.d.oa,
    	beta: msg.payload.d.ob,
    	gamma: msg.payload.d.og,
    	stime: msg.payload.d.ts,
    	mtype: "roll",
    	SENSORID : msg.payload.d.id
    	};
    	return msg;

    The app running on your smartphone delivers the parameters X, Y, Z, alpha, beta, gamma, and stime. The mtype parameter is set to the first of our experimental movement types. We go deeper into this parameter in the coming steps.

    Optionally, you can give the function a useful name (I used flatten_for_training.). The final result should look like this.

  5. Click Done.
  6. Click Deploy. When you activate the smartphone app, the output on the debug tab should look similar to this.

Note: The SENSORID value should reflect the alphanumeric ID you used when you activated your smartphone. Do not worry about the format of the timestamp. We transform it later into a more readable format.

2b

Modify the database to collect data for three different movement types

We record some example data for three different movement types as illustrated in Figure 1:

  • Roll – hold the smartphone horizontally and turn the smartphone away from you continuously so you quickly alternate between seeing the front and back of the smartphone.
  • Turn – hold the smartphone vertically and turn it to the right continuously so you quickly alternate between seeing the front and the back of the smartphone.
  • Wiggle – hold the smartphone vertically and then rotate your hand from side to side (or left to right).
Figure 1. Three different movement types illustrated

You can experiment with your own ideas later.

To identify the movement types in the data records, we have to tag them inside the code. We already did this for the first movement type roll in the previous step. For testing, we disconnected the Cloudant node. We will finalize the pre-work now, and start the collection of sample data.

  1. Reconnect the Cloudant node to the function node. The flow should look like this.
  2. Click Deploy.
  3. Collect data for the rollmtype.
    1. Re-activate the IoT sensors on your smartphone.
    2. As soon as the app starts sending data, hold your smartphone horizontally, as if you want to take a landscape picture. Start rolling your smartphone at a steady pace for at least 45-60 seconds. (In my case, 45 seconds resulted in an increase of about 500 documents in the database.)

      Note: If you watch the debug tab in your Node-RED flow editor while doing this you might see red error messages. The free edition of Cloudant only handles 10 events per second and sometimes the smartphone is faster. We can ignore this slight loss of data.

    3. De-activate the IoT sensors on your smartphone by turning it off or deactivating the browser app.
  4. Collect data for the turnmtype.
    1. Go to the Node-RED flow editor and double-click the function node.
    2. In the code, find roll and replace with turn.
    3. Click Done.
    4. Click Deploy.
    5. Re-activate the IoT sensors on your smartphone.
    6. As soon as the app starts sending data, hold the smartphone vertically and start rotating it at a steady pace. (Try to use approximately the same timeframe as before.)
    7. Check the test results of the new movement in the debug tab.
    8. De-activate the IoT sensors on your smartphone.
  5. Collect data for the wigglemtype.
    1. Go to the Node-RED flow editor and double-click the function node.
    2. In the code, find turn and replace with wiggle.
    3. Click Done.
    4. Click Deploy.
    5. Re-activate the IoT sensors on your smartphone.
    6. As soon as the app starts sending data, hold your arm vertically, the smartphone slightly tilted, and start rotating the hand with the smartphone back and forth (or left to right) at a steady pace. (Try to use approximately the same timeframe as before.)
    7. Check the test results of the new movement in the debug tab.
  6. Disconnect the Cloudant node to avoid further data being recorded.
  7. Click Deploy.

Now that we recorded sample data for different movement types in the database, we need to connect our database to our analysis tool, SPSS Modeler. We build a statistical model and "teach it" the data structure of the movements.

3

Install and configure the Cloudant Extension to connect SPSS Modeler to the database

Before we can connect our database to our modeling tool, we first have to install the SPSS Modeler. The exact steps depend on your version of SPSS Modeler and your operating system. If you already have SPSS Modeler installed, you can use your installation.

SPSS Modeler is open for extensions using public assets. We use one of these public assets to connect to the Cloudant database. With a working installation, install these necessary extensions.

  1. Go to the Downloads section of the SPSS Predictive Analytics website, scroll down and click Get R/Python. Use the table to find the correct version of R, and follow the link provided to download and install R on your computer. (I am using SPSS Modeler 18.1 and R 3.2.2.)
  2. In GitHub, go to R essentials for Modeler. Find the correct version for your platform and SPSS Modeler and install.
  3. In GitHub, install Cloudant Extension for SPSS Modeler.
  4. Extract the Cloudant Extension file.

    The Cloudant Extension comes in a compressed file (.zip) with an example SPSS stream. We need to modify the example stream to connect to our own Cloudant database.

    Note: The compressed file also contains documentation in PDF format in the Example folder. See the document named Mining Cloud Data with SPSS.

  5. In SPSS Modeler, in the Example folder, open the cloudant_import_demo_complete_real_sensor.str stream. The upper left of the example stream contains a simple connection to the Cloudant database and an output to a table.
  6. Delete everything from the stream except the User Input node and the simple connection to the Cloudant node.
  7. Save the stream with a new name. Keep the SPSS Modeler window open.

    Note: The Cloudant node is based on a general R node and therefore needs an input node. For this reason, we find the User Input node here. Do not delete it, even though it does not do anything relevant.

    The stream in SPSS Modeler now should look like this.

  8. For connecting to the Cloudant node, we need to replace the connection credentials with the ones from our own Cloudant instance. You can look them up in your IBM Cloud dashboard:
    1. Log in to IBM Cloud.
    2. From the hamburger menu, select Dashboard.
    3. From the IBM Cloud Dashboard, under Cloud Foundry Apps, click your application name.
    4. Click Connections.
    5. On the Cloudant tile, click View credentials. We need some of the string values.
    6. Use the Copy button to copy the complete text. Paste it into a text editor of your choice to keep it for later reference.
  9. Return to the open SPSS Modeler stream, double-click the Cloudant node to open the configuration for the node.
  10. Copy the values for host, username, and password from your text note and paste the values into the matching fields in the Cloudant node.

    Note: Do not copy the quotation marks.

  11. In the database field, enter iotmovements. The result should look similar to this.
  12. Click OK.
  13. Right-click the Table node and click Run to look at the movement data from your movement type recordings.

We now have a working connection between our SPSS Modeler desktop-based analysis workbench tool and a cloud-based database. The data can be analyzed just like any local database or file.

We next create a predictive model that "knows" how to identify the movement type out of the raw data.

4

Create a classification model stream in SPSS Modeler

SPSS Modeler "learns" from existing data to create a model and uses the resulting model to apply what it has learned to new data.

In this step, we teach the model about the data we recorded. The resulting model "knows" the combination of parameters to identify the different movement types.

In statistical terms, this is a classification or decision tree model. The huge advantage of SPSS Modeler is that we do not have to know anything more about statistics – the tool finds the correct model automatically.

Our tutorial reflects a fact about real-life data science projects: eighty to ninety percent of the work is getting the data and transforming it in some way. In our case, we need to perform two simple steps:

  • Make the timestamp usable for further analysis.
  • Calculate a new measure out of the recorded raw data.
4a

Transform the timestamp

The timestamp is a good example of raw data provided by a sensor that has to be transformed to be usable. The timestamp from the smartphone app is just a serial number based on 1 January 1970 as starting point. We can set this reference point in the stream's properties.

  1. In SPSS Modeler, select File > Stream Properties.
  2. On the Options tab, select Date/Time.
  3. For Date baseline, enter 1970.
  4. Click OK.
  5. Save the stream.
  6. In the stream, select the Cloudant node.
  7. In the lower pallet, from the Field Ops tab, double-click the Derive node and add it after the Cloudant node.
  8. Double-click the Derive node.
  9. On the Annotations tab, enter a Custom name timestamp for the node.
  10. Select the Settings tab.
  11. For the Derive field, enter timestamp.
  12. In the Formula field, type or paste this code: datetime_timestamp(stime/1000)

    The result should look like this.

  13. Click OK.

The new field contains a time and date value that SPSS Modeler can use for graphs and analysis. We do not use it in this tutorial – it is just an example transformation – but you can pick it up later if you want to chart some data.

4b

Calculate a new measure for the energy used

The raw data for the movement in X, Y, and Z direction might not be sufficient for a good prediction. It is very typical for a data science project to use the raw data to calculate a new measure. In his original tutorial, Kienzler calculates the overall energy (something like the relative movement in all directions). We pick up this example here again by adding another Derive node.

  1. Select the timestamp node.
  2. From the lower pallet, double-click the Derive node. This inserts the new node with a connection right after the timestamp node.
  3. On the Annotations tab of the new node, enter the Custom name energy for the node.
  4. Select the Settings tab.
  5. For the Derive field, enter energy.
  6. In the Formula field, type or paste this code:

    sqrt((X*X)+(Y*Y)+(Z*Z))

    The result should look like this.

  7. Click OK.
4c

Tell SPSS Modeler what the target is to be predicted

In this step, we tell SPSS Modeler what exactly should be predicted – What is our target? This is done using a Type node.

After the second Derive node called energy, add a new Type node.

  1. Drag a Type node to the stream after the Energy node.
  2. To connect the node to the stream, select the Energy node, press F2, and drag the connection to the new Type node.
  3. Double-click the Type node and click Read Values. Wait for the process to finish.
  4. On the Types tab, find the rows for stime and timestamp.
  5. Change the value for both in the Role column to None.

    Note: We need the timestamp and date and time primarily for graphing results later. These fields are not considered in the automatic creation of the model.

  6. For mtype, change the value in the Role column to Target.
  7. Click OK.
  8. Click Save.
4d

Add a classification model

We use a classification model to "learn" the connection between the raw data, the calculated energy, and the movement types. The model learns which combination of input data is typically observed for the three different movement types.

SPSS Modeler knows different algorithms for a classification (or decision tree) and even can find the best working model automatically. Because we want to deploy the model later to IBM Machine Learning in IBM Cloud we use one specific algorithm here.

  1. From the Modeling palette, add a C&R Tree node after the Type node. It is automatically named mtype, following the target selected before.
  2. Right-click the new mtype model node and select Run. Wait for the model nugget to appear.

    Note: I selected the C&R Tree model as an example. In some cases – depending on your individual data – this model might not be usable and you will get an error message. You can try other models, but be aware that not all models run in the cloud deployment. You can, for example, use the Logistic model (regression). This might not be the best algorithm, but it should work in many cases.

  3. Double-click the model nugget named mtype to inspect the result. The expanded decision tree on the left might look similar to this. (The exact results depend heavily on the movement data you recorded yourself).
  4. Close the window.
5

Deploy and test the classification model manually

We now have all the knowledge about moving the smartphone programmed into the classification model nugget. We can now collect some new data (without the movement type) and let the model tell us which movement type this might be. We call this the "deployment of the model."

In this step, we collect some new data and find out how the smartphone moved.

5a

Create a Cloudant database to collect scoring data

We set up a new Cloudant database to collect the scoring data (raw data as before, but without the movement type identification - imagine someone moved the smartphone in a hidden place) and run it manually in SPSS Modeler.

First, create the new database instance.

  1. You should already be logged in to your IBM Cloud account. If not, log in.
  2. From the hamburger menu, select Dashboard.
  3. The Dashboard shows your applications and services. In the Services section, you see your instance of a Cloudant NoSQL database service.
  4. Click the line with the Cloudant NoSQL DB instance.
  5. On the Cloudant Service Details page, click Launch. The Cloudant Dashboard opens in a new window.
  6. From the left menu, click the database icon to see a list of your Cloudant databases.
  7. Click Create Database and enter iotmovements_scoring as the name.
  8. Click Create.
5b

Modify the existing Cloudant node

We created a new empty database instance and we redirect the sensor data collected from the smartphone into this new instance.

  1. Start the Node-RED flow editor for your Cloud Foundry application. Your flow should look similar to this.
  2. Double-click the Cloudant node.
  3. Change the Database name to iotmovements_scoring to match the name used in IBM Cloud.
  4. Click Done.
  5. Double-click the function node.
  6. Change the string value for mtype to something neutral.

    mtype: "-",

    Note: For technical reasons, do not delete the line!

  7. Connect the function node to the Cloudant node.
  8. Click Deploy.
5c

Test to be sure the movement is being received by the database

We've created the database and given it the code to collect the data. Now it's time to test it.

  1. Re-activate the IoT sensors on your smartphone by opening the browser app as before.
  2. After the application connects, perform some different test movements.
  3. Remember what you did and de-activate the smartphone. This produces some new data in the database that we use for scoring now.
  4. After you record some data, you might want to disconnect the Cloudant database in the Node-RED flow, to avoid accidentally recording data.

    Note: If you do, don't forget to deploy!

5d

Create a new stream in SPSS Modeler to analyze the movement data (scoring)

We need to create a working deployment for predicting the movement types.

  1. In SPSS Modeler, duplicate the first three nodes: User Input, Cloudant, and Table. (Select the nodes; copy and paste them.)
  2. Double-click the new Cloudant node.
  3. On the Connection settings tab, change the name of the database to iotmovements_scoring.
  4. Click OK.
  5. Test the connection by right-clicking the Table node and selecting Run.
  6. Copy and paste the Derive nodes date_time and Energy and connect both new nodes to the new Cloudant node.
  7. Connect the last node (the Energy node) from the new stream to the mtype model nugget from the old stream.

    Note: The new model was automatically connected to the original data. We are replacing this connection here with the new stream part with the scoring data.

    1. Right-click the Energy node, select Connect, and then click the mtype model nugget.

      Note: If needed, you might want to first move the nugget node on the canvas by dragging it.

    2. Select Replace.
    3. Select the mtype model nugget, and in the lower pallet area on the Output tab find the Table node and double-click it to add it right after the mtype Model nugget.
    4. Right-click the Table and select Run.
    5. In the resulting table, scroll to the right and you see the predicted movement types plus a score for this prediction.

      Note: The nearer this score is to 1.0, the better the prediction.

We now have a working deployment for finding a "best guess" on the movement types. Sometimes it might be more useful to score the sensor data "on the fly" instead of recording it in a database before scoring.

6

Add a new branch to the SPSS stream for live scoring

The IBM Cloud platform lets you quickly and easily deploy a predictive data stream to the cloud. It is immediately available for scoring without the need for your own software infrastructure.

The Machine Learning service in IBM Cloud can make use of a SPSS Modeler stream file (.str) for cloud-based scoring. You can use a Cloud Foundry App (like the one we are already using) to feed data into this stream. We again use Node-RED to set up the needed data flow. The Machine Learning service omits (or cuts off) the first and the last node from the stream and places the remains into the Node-RED data stream. We will try this again to completely understand the idea.

Review the existing SPSS Modeler stream. It currently looks like this.

Imagine how the stream would look if the first and the last node are cut off. This wouldn't work. The User Input node at the beginning is purely technical (remember, the Cloudant node needs an input) and without the last node there is no output at all. What we don't need is the Cloudant node for the deployment because the data flows in directly from the Cloud Foundry App. However, we still need the same structure of fields for the rest of the stream. There is an easy help for this.

  1. If you want to keep your stream as it is so far, make a backup copy or copy and paste the part shown above inside the same .str file.
  2. Delete the Table node below the Cloudant node because we do not need this.
  3. Right-click the Cloudant node, and select Generate User Input Node.
  4. Rename the new node from user input to scoringdata.
  5. Connect the new node to the date_time node. This provides the correct field structure for the stream. The new node will not be used, as it will be cut off when the stream is deployed in the cloud.
  6. After the mtype model node, add a Table as an output. This table will also be cut off later and replaced by the post-scoring data flow in Node-RED.
  7. Specify which node is the one to be used in scoring in this stream: Right-click the new Table node and select Use as scoring branch. The relevant part of the stream for the cloud-based scoring is highlighted in green color. The result should look like this.

    Note: We could delete the two User Input and the Cloudant nodes at the left, but I always keep these fragments because it documents what I did before.

  8. Save the file with a new name and remember the location in your file system.
7

Use IBM Machine Learning in IBM Cloud to deploy the SPSS Modeler stream

We prepared the SPSS stream file for deployment in the IBM Cloud Machine Learning service. Now we have everything prepared for the deployment itself.

  1. Log in to IBM Cloud and, if necessary, open the IBM Cloud Dashboard.
  2. Click the name of your Cloud Foundry app to open its console.

    Note: Do not click the link in the Route column.

  3. From the left menu, click Connections.
  4. At the top of the window, in the upper right, click Connect New.
  5. From the list of services, select Machine Learning service.
  6. In the Connect to field, check that your app is selected and click Create.
  7. On the console for the newly created service, from the menu on the left, click Service credentials. No credentials are listed so far.
  8. On the top of the empty list, click New Credentials.
  9. In the line of the new entry, click View Credentials and copy the contents of the text field into a text file. We need the access_key and the url later.
  10. From the menu on the left, click Manage to go back to the console of the Machine Learning service.
  11. Below SPSS Streams Service, click Launch Dashboard.
  12. Click the Dashboard tab. Here, you can upload the .str file that you edited and saved in the previous step.
  13. Either drag and drop the .str file or use the Select File button to find the file.
  14. Specify a Context id. Enter a unique name like iotmovements into the field and click Deploy.

The list now should look similar to this.

8

Set up the data flow for live scoring

We can now connect all the pieces:

  • The IoT app contains the sensor simulation that runs on a smartphone.
  • Each data set from the sensor is delivered to the Machine Learning service that contains the SPSS scoring model, and not stored in a database.
  • This data stream needs to be created now with Node-RED.
8a

Log in to IBM Cloud and launch the Cloud Foundry App in Node-RED

First, we have to work in Node-RED again.

  1. Go back to the IBM Cloud dashboard and again launch the console of the Cloud Foundry App by clicking the app name.
  2. Click Visit App URL and then click Go to your Node-REDflow editor and log in, if necessary. Your flow diagram should look similar to this – with or without the connection to the Cloudant node on the right. (You might have disconnected it earlier.)
8b

Replace the existing database with the online scoring database

We replace the current database storage with an online scoring. If you are familiar with the environment, you might also establish both database storage and online scoring.

  1. Disconnect the existing function node from the IBM IoT node, but do not delete the node.
  2. Add a new function node and connect the IBM IoT node to the new one.
  3. Insert an http request node and connect it after the new functionnode.
  4. The URL to call the API of the Machine Learning service has the following structure: https://<URL>/pm/v1/score/<CONTEXTID>?accesskey=<ACCESS_KEY>
    Use a text editor to fill in the variables.
    1. Use the url and access key from the credentials for the Machine Learning service that you created in the previous step (step 9).
    2. Use the Context id that you specified when uploading the SPSS stream in the previous step (step 14). For example, my complete string looks like this. https://ibm-watson-ml.mybluemix.net/pm/v1/score/iotmovements?accesskey=7Axxxxxxxx1WXWeZv
  5. Copy and paste the completed string into the URL field of the http request node.
  6. In the Return field, select a parsed JSON object.
  7. Click Done.
  8. The input for the http request is a complex object that contains a tuple for the headers of the fields and a matching tuple with the necessary values. We use the function node before the http request node for the necessary transformation and do not go into too much detail here. This function connects to the scoringdata node that we previously created and renamed.
    1. Double-click the function node and copy the following code into the Function field:
      		msg.headers = { "Content-type" : "application/json"  };
      		msg.payload =
      		{
      		    "tablename":"scoringdata",
      		    "header":["X_id", "X_rev", "X", "Y", "Z", "alpha", "beta","gamma","stime","mtype","SENSORID"],
      		    "data":[["X_id","x_rev", msg.payload.d.ax, msg.payload.d.ay, msg.payload.d.az, msg.payload.d.oa, msg.payload.d.ob, msg.payload.d.og, msg.payload.d.ts, "-", msg.payload.d.id]]
      		};
      		return msg;
    2. Name the function node with a useful name.
    3. Click Done.

      The final result should look like this.

8c

Test the scoring

After replacing the database with the cloud-based scoring, we can test to see if the scoring is working.

  1. Connect a debug node after the http request node.
  2. Click Deploy.
  3. If it is not already active, re-activate the IoT sensors on your smartphone.
  4. In Node-RED, go to the debug tab and compare it to the following example. You can expand the shown object appropriately.
8d

Make the JSON object more usable

Transform the resulting JSON object into a more usable format that contains only the necessary information.

  1. Re-use the older and disconnected function node (or add a new one, if you deleted it) and copy the following code into the Function field. This picks the needed entries out of the stacked arrays from within the JSON object.
    		msg.payload =
    		{
    		 X : msg.payload[0].data[0][2],
    		 Y : msg.payload[0].data[0][3],
    		 Z : msg.payload[0].data[0][4],
    		 alpha : msg.payload[0].data[0][5],
    		 beta : msg.payload[0].data[0][6],
    		 gamma : msg.payload[0].data[0][7], 
    		 timestamp : msg.payload[0].data[0][8],
    		 device : msg.payload[0].data[0][10],
    		 energy : msg.payload[0].data[0][12],
    		 pred_mtype : msg.payload[0].data[0][13],
    		 pred_score : msg.payload[0].data[0][14]
    		};
    		return msg;
  2. Click Done.
  3. Connect the debug node at the end of the flow.
  4. Click Deploy. Another test using the smartphone should result in some debug output similar to this. Take a special note of the predicted movement type and the prediction score.

Finally, for later reporting you might want to display the results on a web page, collect it for another database, or both. For the database, you should know now what to do. For the web page, this might be an interesting tutorial for you to write yourself.

Conclusion

This tutorial was born from my personal confusion about IoT, cloud, sensors, and especially the question "where is the magic?" Technically, we connected a sensor (our smartphones) to the cloud, stored the produced data in the cloud, and analyzed the data using SPSS Modeler. We created a model that is able to decide about "what happened with the device" just by looking at the sensor data. And, we learned different ways to bring the created model to life.

For me, the best learning was about all the interconnection on the IBM Cloud platform. Before IBM Cloud I did many similar test setups and much of the configuration and simple setup steps were confusing. There are a lot of moving parts to consider and it helps to know that IBM Cloud is one place to go and to get started. Furthermore, it was great to use the statistical and predictive workbench SPSS Modeler in connection with a cloud setup.

What should you do next or how can you learn more? Feel free to extend in all directions. The first logical step would be to make the model more accurate. Consider working on your own real-world example. If you find your own realistic case with real-world sensor data, start with something small to get a basic understanding of the data and especially the business case behind it. For me, there is absolutely no need to work with or on data without a proper goal or business case.

From a business standpoint, the question would be very down to earth, for example, "how to get sensor data from older machines without replacing them?" On the data side, ask yourself "do we really need all the detailed sensor data that all the world will produce within the next years?" If so, how do we decide what data is important? One answer to this question could be to extend the architecture by cognitive methods that you can also find in IBM Cloud.


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Internet of Things, Cognitive computing, Information Management
ArticleID=1052252
ArticleTitle=Analyzing IoT device movement data
publish-date=11212017