How I built a TJBot photo booth using Node-RED

Share this post:

As an IBM Developer Advocate, I get to come up with cool demos for events and conferences that showcases the technology IBM Cloud has to offer developers. For a community event, I built a TJBot photo booth, using the famous TJBot to capture photos using a Raspberry Pi camera and send them via Twilio.

If you’re feeling brave, you can go right to the code on GitHub. Otherwise, read on to discover how I turned this powerful little robot into a photo booth!

Now, photo booths aren’t new. But have you ever had a cardboard box take your photo and send you a text message? And yes, for those curious readers, we routinely “test” the demo “for quality assurance purposes.” It’s a tough job!

This post details the various parts of how it was built and the decisions I made. There are two parts to this project, running Node-RED on the Raspberry Pi and running Node-RED in IBM Bluemix, IBM’s cloud platform.

Raspberry Pi

Let’s start with Node-RED running on the Raspberry Pi.

The TJBot subscribes to a takephoto command from the Watson IoT Platform. When a command is received, the LED is changed to the color red to signal that something is about to happen, in this case, taking a picture.

A camerastatus event is sent back to the Watson IoT Platform to inform any subscribers that TJBot is taking a photo. We’ll come back to how these simple events are used in Node-RED running on IBM Bluemix.

The Take Photo node uses the Raspberry Pi camera to capture an image. The node saves the photo as a file locally.

When the photo has been taken, another shine node is used to turn the LED off. Another camerastatus message is sent up to Watson IoT Platform alerting subscribers that the TJBot is uploading the photo.

The file on the Raspberry Pi where the photo was saved to is read into memory by the file node and is then passed to the Object Storage node. This node uploads the image to the Object Storage service in IBM Bluemix. The node returns a URL to the file.

A third camerastatus event is sent up to the Watson IoT Platform alerting subscribers that the TJBot is finished and ready to take another photo.

Lastly, an IoT event, phototaken, is published with the details of the uploaded file.

Still with me? Take a breath, I know that was a lot to digest. Here are a couple of things that were considered when building this project using the TJBot.

TJBot was originally conceived to be self-contained and powered by cognitive services from IBM Watson. But it can be more than that. In this project, TJBot is an IoT device that subscribes to commands and publishes events.

Sending multiple IoT events might end up being a very quick process with a fast Internet connection. However, when this was first shown at a community event in the middle of a city park, it was powered off a mobile hotspot. This can REALLY slow down the experience, which means keeping the user aware of what is happening. Each status event gives a clear picture (pardon the pun) of what the TJBot is doing.

Using the Watson IoT Platform in this manner was really easy. More often, lots of data points are published from IoT devices.

Node-RED in IBM Bluemix

Let’s transition to the cloud and see the components of controlling the TJBot Photo Booth. Before we dive into Node-RED in the Cloud, here are some of the design decisions that were made.

I used Node-RED to control the interface to showcase how Node-RED could be used both on the device and in the cloud, encouraging anyone, from beginner to advanced developer, that their ideas can be built by thinking in a logical process based flow. Yes, you can use any front-end, even a native mobile application.

Since Node-RED offers a set of nodes to build a dashboard, it was a good excuse to test it out. Here’s the application running in IBM Bluemix.

Photo booth interface that takes the photo, selects an photo, and sends a text message with the photo

Node-RED application in IBM Bluemix

I’ll walk through the user interface and show how Node-RED powers the functionality. In the top left corner is the Take Photo button.

This button sends a command via the Watson IoT Platform to the TJBot device to take a photo.

Configuring the ibmiot out node

The Bluemix application sits idle until an IoT event comes back from the TJBot. The status is displayed to alert the user if TJBot is ready to take a photo, taking a photo, or uploading a photo.

Three states of the status: ready to take a photo, taking a photo, uploading a photo, and back to ready state

Node-RED flow that responds to IoT events published by the TJBot

Notice I have a couple of debug nodes to see the IoT event data. Here’s what the debug window looks like when a photo is taken.

Debug messages as the photo is being taken, uploaded, and added to the gallery.

The first three types of events, taking, uploading, and ready are for display purposes. If the event is of type completed, the flow uses the IoT data to add the image to the gallery. The URL is publicly accessible, which will be useful later on when Twilio is called.

Debug of the IoT event when a photo is taken

Function node that adds the photo to the gallery.

The gallery template displays the photos that have been taken. Each time a new photo is added to gallery, the dashboard is updated.

Now that the picture has been added to the gallery, we wait for a user who wants to receive their photo. This next part describes the logic to controlling the UI and sending a picture to a phone number.

When the user clicks on one of the photos in the gallery, the template node labeled Gallery generates a message that the change node stores into a flow variable. The box on the right is also enabled to accept a phone number.

Some magic just happened in the background. When the preview image was shown, there’s an greeting message overlayed onto the image. The source of the image is actually a Cloud Function action that takes the photo and a watermark image and combines them together. Here’s an example of the code:

https://openwhisk.ng.bluemix.net/api/v1/web/openwhisk_space/default/watermark.http?watermark=watermark-image-url&image=photo-from-object-storage

We’ll use this Cloud Function action again later on.

When a phone number is entered, it is saved into a flow variable.

When the user clicks on the Send Photo button, another Cloud Function action is called with the phone number, image, and a message. I used this Cloud Function action I created previously to send messages to the Twilio API.

Notice the image URL? It’s the first Cloud Function action that combines the image with a watermark overlay! The value for the to: phone number is using the phone number stored in the flow variable from the phone number input node.

When the Cloud Function action returns, a notification is displayed to alert the user the photo was sent. Note, this really means it has been received by Twilio, but it might be queued in the system and delayed.

Back to the Node-RED dashboard, we clean up a few things. Disable the Send box, clear out the phone number from the input box, and reset the photo to a placeholder.

The last part is a delete button. The red X button removes the image from the gallery and from the Object Storage. The link node re-renders the gallery.

Lessons Learned

This project was really fun to build as I discovered a number of things:

• IoT events can be simple messages that don’t necessary have lots of data.
• The Node-RED dashboard can handle some complicated flows and offers the ability to enable/disable elements. Unfortunately you can’t hide elements, or I would have hidden the Send Photo box when not active.
• Node-RED dashboard templates don’t always render. Perhaps there’s a setting I missed, but the gallery doesn’t always display properly. Refreshing the page seems to fix it.
• The Phone # input has a delay. Early prototypes with a value of 300ms made the UI laggy and caused a bug where numbers were lost. Some users enter their number really fast while others enter it slow.
• Processing photos with the watermark using Cloud Function avoids having an application runtime waiting for those infrequent calls into action. The first prototype used an application runtime and ran all the time. Cloud Function runs on demand and you pay only for the time you use.
• People love to take photos. Make sure you include props to make the photo booth fun and viral. You might notice the light sabres and cardboard boxes.
• The gallery is stored in memory for simplicity. If you need to persist this data, connect a Cloudant node to save and load the data in/from a database.