July 8, 2016 | Written by: Josh Zheng
Share this post:
Welcome to another episode of “Teaching Robots How To Love.”
If you only want the tutorial on how to build the candy machine, scroll down to the Step-by-Step Tutorial section.
In my previous post, I wrote a Python script combining IBM Watson’s Speech to Text with Watson AlchemyLanguage Sentiment Analysis to determine the sentiment of speech. At the end of the post, I mentioned how I’d like to further explore what it’d be like to interact with a a candy machine with a speech-sentiment interface. Specifically, I wanted to build a candy machine that could understand speech and dispense different types of candy based on the sentiment of the words. Well …
I present you the first prototype of the Watson Naughty-or-Nice Candy Machine!
Though we had exposed wires and handwritten notes all over the place, I’m actually very happy with how it turned out. This picture was shot right before the opening of IBM Interconnect 2016 in February. The candy machine was on display for four days from February 22nd to February 25th. I expected a lot to go wrong during the conference because the machine went straight from my living room to a busy expo floor. But it performed way above my expectations.
This is a wider shot of the entire setup.
If you were at the conference, here’s how you would have operated the candy machine.
- You’d walk up to the microphone and press the blue button (Record) to start recording.
- You’d then speak into the microphone.
- And press the red button (Stop) when you were done talking.
- This is what you’d see on the screen if you were nice:
or this if you were mean:
Note: The transcription of what you said is the small words below the buttons.
The machine would dispense sweet candy (sweet M&Ms) from the dispenser on the right if you said something positive (sentiment score > 0). The machine would dispense sour candy (sour Skittles) if you said something negative (sentiment score < 0). Nothing happened if you had said something neutral.
Okay, now that you have an idea of what it’s suppose to do, shall we start building? Yes, totally.
Here’s an overview of what we’re doing.
- We’re connecting a laptop running a Flask web application (served on localhost) to an Arduino board.
- The laptop uses the Watson Speech to Text service for transcription and the Watson AlchemyLanguage sentiment analysis to score the sentiment of the words.
- The laptop then uses serial communication to pass the sentiment info to the Arduino board via the USB port.
- The Arduino board has a motor shield plugged on top of it. The shield is directly connected to the DC motors inside the candy dispensers.
- Based on the sentiment info from the laptop, the Arduino code determines which candy dispenser to turn on by controlling the voltage on its IO pins.
Got it? Below are the steps in more detail.
Step 0: What You Need:
- Code: All the code you need can be found in this repository. Go ahead and clone the repo and I’ll walk you through how to set everything up in the following steps. Go ahead and do ‘git clone https://github.com/boxcarton/say-something-nice.git’
- Credentials for the Watson Speech to Text service and the Watson AlchemyLanguageservice. If you’ve never used them before, see my previous post on how to acquire the credentials. You need to get the credentials through Bluemix, but you’re not tied to Bluemix in any way afterwards. Note: Bluemix is IBM’s PaaS offering that let’s you deploy and manage your cloud applications.
- Two Sharper Image Candy Dispensers: I got the dispensers on sale from Bonton for $20 each. Note: This candy dispenser is unequivocally a one-star product — to directly quote a review from Amazon, “the mechanism dispenses between 0 and 35 pieces at random.” But don’t be discouraged, the inconsistency comes from the batteries’ inability to source enough current for the DC motors. We’ll be powering the motors (through the Arduino board) using an 12V-1.5A power adapter, which solves this problem. Another Note: It seems like this dispenser has been sold out from Bonton, you can still get one from Amazon for $36.99 each. The slightly cheaper version should also work for this project.
- Power Adapter: I found a 12V-1.5A power adapter under my bed. If you don’t have such superpowers, you can buy one like this on Amazon.
- Arduino Uno: I bought this Starter Kit from Vilros, but I’m pretty sure any Arduino board will do fine here. If you don’t get the kit, make sure to get the jumper wires (see #11), they’ll come in pretty handy.
- Motor Shield: I bought the Adafruit Motor/Stepper/Servo Shield. You can also buy it directly from Adafruit, but shipping is faster with Amazon. If you’re wondering why you need a motor shield, see this Quora post. TL;DR: More current for the motors and protects the Arduino from transient circuit behaviors.
- Extra Header Pins: You’ll be soldering header pins to the motor shield so you can plug it onto the Arduino. The Adafruit Motor Shield comes with just enough header pins, which is a problem if you screw up the soldering and need extra pins.
- Soldering Station: This is the soldering station I bought. It gets the job done. But if you’ll be soldering often, you’ll probably want something of higher quality. I recommend the Weller WESD51 Digital Soldering Station if that’s the case.
- Solder: This is the one I bought. But I found it to be too thick for the header pins we’re working with. Try getting something with a smaller diameter if you can, maybesomething like this.
- Solder Wick (Optional): Mandatory if you’re a soldering newb like me.
- Jumper Wires (Optional): Having these does make it easier to connect the motor shield to the candy machine. But if you already have copper wires, they should work fine.
- Noise-Cancelling Microphone (Optional): I got this one on Amazon after doing some research. It got the job done on the expo floor, but you can probably do better with more money. A noise-canceling microphone is only optional if you’re running this at home. It’s absolutely mandatory if you plan on having this in a noisy area.
- Extra Long Audio Stereo Cable (Optional): The Rode Video microphone above comes with an incredibly short cable because it’s meant to be attached to a video camera. You’ll need a longer cable like this one if you plan on having the microphone more than a few inches away from the laptop.
- Audio Port Splitter (Optional): Mandatory if you plan on using one of the newer Macbooks. In these newer laptops, Apple has combined the headphone and microphone ports into a single port. This is to go with the new Apple headphones that have combined headphone/microphone. You’ll notice that the the new Apple headphone plugs have three rings while most audio plugs still have two. This splitter will split the 3-ring port on your Macbook into normal headphone and microphone ports. Important Note: For the whole setup to work, you need to perform three steps in this order. 1. Plug a pair of normal headphones into the headphone port. 2. Plug the microphone into the microphone port. 3. Once both are plugged in, THEN plug the splitter into your laptop. This is the only way for your laptop to detect the external microphone.
Read more on Josh’s Medium blog.