The role of virtual assistants in driving – lessons from Ford hackathon and CES 2017

By | 4 minute read | January 18, 2017

leadspace image of las vegas skyline where the CES event is held

One thing was clear at CES 2017: virtual assistants from Amazon, Google, Microsoft, and IBM showed up in a broad range of robots, speakers, appliances and even vehicles. Amazon’s Alexa was the clear winner with an entourage of companies demonstrating new Skills that control their services and devices through Alexa commands. IBM demonstrated partnerships with JBL, Harman and Panasonic where Watson is exposed in intelligent hotel mirrors, speakers, and vehicles. Alexa wasn’t the only virtual assistant on display in vehicles: FCA and Ford had IBM Watson demos showing off similar capabilities.

The role of OEMs in vehicle entertainment and driver productivity

Now that connectivity is nearly ubiquitous with virtual assistants in cars and autonomous vehicles are on the horizon, automakers are envisioning an entirely different experience in the car. Drivers not only want to be entertained, but also productive during their long commutes. The question becomes, how do the OEMs provide this content and experience? Do they give it all up to Apple and Google via Car Play and Android Auto, or do they define their own branded experience that keeps them in control of the data coming from the vehicle and the quality and type of infotainment offered to their customers? Do they embed the content into the head unit, or do they beam it in via mobile applications? It’s too early to declare a winner, but momentum is growing in the direction taken by Ford and their Smart Device Link (SDL) technology.

Quickly driving unique content and services

SDL was developed by Ford and originally released as AppLink in 2013. Shortly thereafter, Ford released the technology as open source, making it available to the community free of charge to drive unique content and services quickly and do it through known channels like the Apple AppStore and the Google Play store. Developers embed the SDK in their application and get access to a rich library of services that allow them to control the application using buttons on the steering wheel, customize the touch screen interface, utilize the built-in microphone, speakers, and access a rich set of vehicle data.

To promote SDL in the industry, Ford has hosted hackathons annually for the past three years. The latest event was held at CTIA SuperMobility Week in September of 2016. We had a blast at the hack two years ago (read about our crazy Social Safe Driving concept here), so I decided to get our team together again this fall and we ended up winning! Our app is called “IBM Smart Drive”, a Virtual Assistant that makes your commute more enjoyable and productive.

Virtual assistants and improving the driver experience

When you consider the amount of time commuters spend in their car there is a lot of opportunity to improve the experience. We used the IBM Watson Conversation service in our SDL app to enable a natural language dialogue between the driver and their surroundings. The SDL software development kit has a feature that lets you stream audio from the microphone in the vehicle to the cloud. We used the IBM Watson iOS SDL to stream our conversation to Watson Speech to Text so that drivers can speak naturally and ask about the nearest points of interest, like “where’s the nearest restaurant?,” or “I’d like a coffee.” The Conversation service lets us understand the intent of the driver and then we call Yelp! to get the list of destinations and display them on the touch screen in the vehicle. The driver can choose to touch the button for the location they want, or simply say, “take me to the second one.” We then set the Sync navigation destination, or if not available, launch the navigation app on the phone.

One of the differentiators of SDL is getting access to vehicle data. We subscribe to the data values and publish them to the Watson IOT for Automotive cloud service for analysis later. When low on fuel, the driver is prompted by Watson he displays the nearest gas stations offering to set the navigation to the preferred destination.

Don’t miss any more conference calls from your car

One of the challenges I have while driving is that I don’t hear the prompt on my phone to join my next conference call. I’m often late, or I even forget because I’m listening to news or sports radio. IBM Smart Drive reads your phone’s calendar and tells you about your next meeting and offers to dial you into the teleconference one minute before if starts. Using the SDL features, the app dials the number to get you into the meeting on time.

Adding trip reports

We’ve just begun to explore the features of Watston IoT for Automotive and are currently adding trip reports to the app. As we collect the data from the vehicle and publish it the IBM cloud, we create analytical reports of your trips. Information like MPG, start time, end time, elapsed time, and detailed trajectory analysis allows us to create a snapshot for every drive you make. The Driver Behavior service provides all of this and more. Eventually we will predict your next route and let you know how weather will affect your commute.

This is just a quick snapshot of the features we are developing in IBM Smart Drive and plan to add much more. Stay tuned for a future post where we go into more detail about the app and the features enabled by IBM Watson IoT for Automotive.