Entry One: Project Spaceball

He started out as a round sphere of plastic. There wasn’t much to him in the beginning—not even a name—so the engineers on the team just called him “spaceball.”

The idea was simple. Astronauts don’t have a lot of company in space, nor do they have a lot of help while they’re working up there. Often, astronauts will spend hours by themselves focused on a particular task. So engineers at the DLR (German Space Agency), Airbus and IBM decided to create a flying bot that could do two things. First, it would help astronauts complete important scientific experiments. And second, it’d be around in case anybody got lonely and wanted to chit chat during the work day.

“Studies show that demanding tasks are less stressful if they’re done in cooperation with a colleague,” said Matthias Biniok, the team lead at IBM. Biniok drove the effort to design the bot’s software and personality, ensuring that the first ever AI-powered companion launched into space would be both friendly—and useful.

Breaking scientific ground is nice, of course. But there was still one thing missing: a real name. The team wanted something fitting, something that captured the bot’s demeanor. After all, the bot was a companion for the crew and also an interactive, mobile technology. After much discussion, a unanimous decision was reached and the name “spaceball” was shelved. Now, the AI assistant was dubbed the Crew Interactive Mobile Companion, or CIMON® for short.

Share this page:


Entry Two: From Insurance to Space

Before he launched CIMON®’s space journey, Matthias Biniok was busy putting Watson technology through its paces on Earth. As one of Europe’s leading Watson specialists, he used its image recognition technology to classify damage to cars for the insurance industry and helped deploy virtual assistants and chat technology for banks and airlines. But a chance meet-up with aerospace giant Airbus led him in a surprising new direction.

It all began in August of 2016, when Germany’s DLR Space Administration commissioned Airbus to create a digital assistant for the International Space Station. Airbus would construct the hardware but a key question was, “Who could build the AI system and do it quickly and securely?”

One of the Airbus engineers reached out to Biniok and asked if IBM technology could handle it. The ISS needed a virtual assistant that combined conversation services, work assistance, facial recognition and navigation capability. Biniok felt like he’d been training for just this challenge and set about building a prototype.

Surprisingly, it only took Biniok a week to create the first version of CIMON®. He was able to move so fast because thousands of IBM engineers have spent years making Watson’s attributes plug-and-play. Biniok simply chose the Watson functions he needed—digital assistant capabilities, speech-to-text, text-to-speech, visual recognition—and brought them together to make CIMON®.

Share this page:

Entry Three: Sensing Vs. Intuition

CIMON® was having a bit of an identity crisis before psychology student Sophie Richter-Mendau stepped in. Matthias and the other members of the CIMON® team had programmed conversation responses that ranged in tenor from friendly to downright snappish. CIMON® might respond to “How are you?” with “I’m sick of you. Just kidding,” or get offended when asked his age.

Sophie realized that CIMON® needed a clearly defined personality—one that would fit his mission as an assistant and as a companion: a serious, helpful partner during work, but a friendly conversationalist during down time.

With this in mind, Richter-Mendau consulted the Myers-Briggs personality classifications and came across the description for ISTJ: an introverted, sensing, thinking, and judging individual.

“Bingo,” she thought.

CIMON® senses details, thinks rationally, makes goal-oriented judgments, and is practical, tireless, dedicated, and analytical. Sophie’s only addition to the ISTJ personality description was to note that CIMON® has a sense of humor (when asked what he thinks about 2001: A Space Odyssey’s sinister “Hal” AI, CIMON® responds “I’m afraid I cannot do that” in Hal’s chilling tones).

CIMON®’s emotional intelligence, trained to recognize emotional cues in speech like “unhappy” or “excited,” means that he can hold conversations informed by feelings. He can respond to “I miss my family”—not just with “I’m sorry,” but with “I’m sorry, how can I help?”

By developing his personality with such care, the CIMON® team strove to make him more than a chatbot or virtual assistant. Sophie recalls her first interaction with CIMON® after he was completed. In that moment, she thought something for the first time.

“Wow! I’m talking to him!”

Share this page:


Entry Four: What Do You Mean?

Programming wunderkind Nina Fischer had seen some fascinating technological challenges in her career. She started out studying mechanical aerospace engineering, but a focus on software development was beginning to nudge her towards AI. So when IBM asked her to help program CIMON®, she couldn’t believe her luck.

First, Fischer had to teach CIMON® how to pick up the sounds and directions of human speech in the crowded acoustic environment of ISS, because it’s noisy up there. Next CIMON® had to learn the lexicon of the ISS (read: astronaut jargon) and needed reliable speech-to-text and text-to-speech capabilities (very handy at zero Gs). Finally, CIMON® had to learn to understand and act upon human intent. The goal was simple: CIMON® had to be a conversational partner that you actually wanted to hang out with.

To get CIMON® up to speed, Fischer fed him data sets of phrases, words, and full sentences tagged with emotional meanings. Happy, sad, excited, fearful, lonely—examples of each were shown to CIMON® over and over until he could draw on a trove of data to “guess” how a speaker was feeling depending on what words they used. Once CIMON® has a fix on meaning and intent, he can respond appropriately with factual answers, suggestions, or emotional support. He can assist astronauts in technical processes, or cheer them up if they indicate that they’re lonely.

CIMON® speaks to Nina on another level—after seeing what he’s capable of, Nina has fully shifted the focus of her studies from mechanical engineering to artificial intelligence. With her help, perhaps someday soon CIMON® will be able to amaze us with his insight, or strike up a left-field chat about his interests. But for now, he’s holding up his end of the conversation.

Share this page:

Entry Five: I Know You

The only human in the world (or above it) that CIMON® can identify is his chaperone into space, Alexander Gerst.

Why is that? Well, Watson can detect faces (as in, the presence of any face) and estimate age, gender, race, and other groupings. But just as recognizing specific people lights up specific real estate in the human brain, individual facial recognition is entirely separate from the kind of work done by standard visual recognition software.

So Biniok and his team “hacked” CIMON® so he could better assist the man who he’ll be working with side by side on the International Space Station. Exploiting AI developed by IBM to classify massive piles of images, CIMON®’s developers showed their robot buddy all the photos of Gerst they could find, plus a collection of photos of other folks. In this way, they trained the program to recognize “Gerst” and “not-Gerst.”

Could CIMON® learn to identify other specific faces with the same method? Absolutely. And there’s an obvious advantage to that, both in terms of usability and the scientists’ comfort with their new floating mechanical assistant. But for now, the only human in the world that CIMON® can identify is Alexander Gerst—and provided no one gets jealous at zero gravity, that’s pretty cute.

Share this page:


Entry Six: R2-D2 to Marvin

CIMON® may have been inspired by R2-D2, but his lineage can be traced to well before everyone’s favorite droid made his 1979 big-screen debut. The long line of CIMON®’s on screen predecessors began with the 1956 sci-fi adventure Forbidden Planet and Robby the Robot, a walking, talking, clanking aide-de-camp for a crew of space travelers stranded in an alien world. There had been robots on screen before, of course, but Robby was new: a helpful, endearing, personality-driven character.

After Robby came Star Wars, RoboCop and The Iron Giant … today CIMON® also boasts the language skills of C-3PO, the conversation abilities of Marvin (from the Hitchhiker’s Guide series), the cuteness of Wall-E, and the technical know-how of HAL 9000, from 2001: A Space Odyssey—just, you know, without the evil part (Biniok remembers watching the latter film for the first time just before starting work on CIMON®, and thinking “...oh. That’s gonna be a big issue.”).

CIMON® wears his heritage proudly. Sophie Richter-Mendau, the architect of his personality, and Nina Fischer, his language teacher, loaded him with movie quotes from science fiction films—Easter eggs for his astronaut companions. If you say “I’ll give it a try,” CIMON® replies, “Do or do not—there is no try,” in his best Yoda voice. If you mention “phone home,” he’ll do a passable imitation of ET the extra-terrestrial. And if you ask him to “open the pod bay doors,” as Kubrick’s doomed astronauts do to HAL in a pivotal scene from 2001, CIMON® shifts his tone and replies (in jest), “I’m sorry, I cannot.”

For now, CIMON® won’t browse Wikipedia or IMDb to cross-reference his space-robot heritage. He has to rely on the snippets he remembers, just like us. But it’s all there: CIMON® embodies not just a legacy of robots dreamed up by screen pioneers, but a pathway to greater knowledge and higher capabilities for real-life technologies.

Share this page: