November 8, 2016 | Written by: Scott Stockwell
Share this post:
The afternoon session of the Watson Developer Conference saw a spotlight featuring Chris Messina, Developer Experience Lead at Uber, Matt Makai, Developer Evangelist at Twilio, Amir Shevat, Director of Developer Relations, Slack. The group came together to talk all things ‘bot’. The panel was hosted by Beerud Sheth, Co-Founder and CEO of Gupshup.
The expert panel prepare to talk all things ‘bot’
How do you define a bot?
It’s how you have a conversation with software through a chat platform answered Amir. Chris felt that understanding the behaviour was the important element. Matt was more in agreement with Amir feeling that it was more about the interaction conversation.
Does text matter? Are other interfaces important?
Chris started off talking about our move from folders and files on computers into conversations with smart technology in our pockets. Music is a great example, you can talk to music services to find your music. As a user, you don’t really care how you ask, just that you have an answer. Amir brought up the topic of expense reports – not popular with anyone in the audience. Why? Because the user interaction to get your money is not enjoyable. If we had a personal assistant that made this easy for us, we’d probably enjoy it – and this is where bots come in. Does it matter if it’s text or voice? No – it’s about getting something done easily.
Do bots need to be intelligent?
Matt felt that until AI gets to the point of understanding intuitively, we would need to limit ourselves to ‘domain restricted bot’. This is where users are familiar with the goal and how to get there rather than a general conversation. Chris was clear that intelligence is critical. When you request a service like Uber, you want a car to turn up – not a pizza (although you might like the pizza!) For Chris, it’s about meeting or exceeding expectations by limiting themselves to what they know users want to achieve. Amir stepped back, considering that Artificial Intelligence is about specific use case. What’s in an image? What is the context of a discussion when a user needs help? Is the bot able to respond to sentiment such as frustration. Then there’s understanding what users say – and adding in the layer of conversational context. For example if you asked “how tall is Obama?” then asked “and his wife?” – the context of the previous question is key to understanding the second question.
Can a bot be perceived as intelligent?
Bots need to pass the beer test (would you take them for a beer because the service was good?) and not the Turing test (is this a human) – posed Amir. A bot acting consistently over time that responds in a similar way is likely to be perceived well. One that has different answers to similar questions is not going to be considered well thought Chris. You have to minimise the frustration with bots stated Matt. Rather than ‘I don’t understand that’, bots that can get across what they can do ‘I don’t understand that, but I can help with A, B, C’ is likely to have a much better user response. Matt thought it was important to plan for the pitfalls. “Intelligence is the absence of stupid” mentioned Beerud.
Amir was very clear – solve one thing well with your bot. If that works, then add more things, but always serve one purpose well at the outset. This can be bots that broaden services – or that go incredibly deep added Chris. ‘Restricted domain bot’ reminded Matt from an earlier question. Keep the scope tight – Twillio use a Slack bot that checks if phone numbers are Twilio numbers. Matt could do this manually, but using a bot to solve one specific problem has proven extremely useful.
Should bots have personalities?
Pixar’s character and script development for movies applies to bots too. Where did the bot start out, what’s its past, how did it get where it is today? It’s not just tone and scripting – humour is important too – and it’s one of the hardest areas to get right given the complexities of language and audience, said Chris. Amir mentioned that context is key too which Chris agreed with using the example of a weather forecasting cat which moved from humorous to being annoying very quickly. With anything you’re building – you have to know who you’re building it for, brought up Matt. This helps to design a bot that will be appropriate. Chris added that you have to add the place/time that the user is using the bot into the design considerations too. Amir mentioned that it’s value first – personality and humour are decorators, you have have to deliver the value first.
How do we track engagement?
Installs vs. uninstall is NOT the measure of engagement, said Amir. Measurement has to be done by cohort. What does that mean? Well, if you’re a travel bot, you have to measure against other travel bots. If you’re a sports bot, you have to measure against other sports bots. You can’t measure travel against sports as the cases are very different for the users. Chris mentioned that the outcome is a good measure. If the user is working with a bot on a task, measuring completion of that task using the bot is a good measure.
The panel then opened to the floor to questions
The hashtag at the time was a great way to pull similar conversations together – and invented by Chris on the panel! The panel was asked, what’s coming next. Amir mentioned that the @ sign was a great way to get attention and escalate content – and this will likely move forwards with bots. Chris wondered how bots using this approach with each other might change in the future. Matt brought up that data analysis on interaction with bots was likely to be a path to discover how we’ll want to work with bots or groups of bots in the future.
What tools should we use to measure bot performance? Amir mentioned that there are generally specific tools for specific bots and this was the best way to find the right tool for the analytics job.
How can we use AI to stitch different domain questions together and have either one bot deal with the need – or pull bots together to co-solve the answer? Amir recommended mapping individual paths to the solutions as a way to find out what could be combined and solved, and what had to be done individually.
Beerud brought up that reverse NLP can be used to train a bot. For example traveling on a journey can ‘flipped’ and help the bot to learn new paths. Bot to human, bot to big data, now bot to bot is becoming an area for investigation. If this is not in one bots domain, it could build it’s ability be knowing which other bots to ‘go to’ for help.
Privacy and bot ethics came up. Chris mentioned that like the ‘this call may be recorded’ message at the start of call centre conversations, bots may need to start to make the same considerations. Chris was keen to see this happen from the developer community.
Scale of 1-10 of bot evolution
Matt – we’ve come a long way ‘5’
Chris – ‘banana’
Amir – cave people talking about a spaceship ‘0.1’ but exponential growth.