Cognitive Computing

Improving Cognitive Computing with Human Interaction

Share this post:

With the advent of cognitive computing, two interesting questions arise:

  • What role do humans play in the new era?
  • How can a computer and humans work together to develop mutual trust to help solve the world’s most complicated problems?

The true potential of a cognitive system is its powerful data analytics and statistical reasoning. However, statistical trends or patterns may lead to false positive or negative answers to a specific question or for a specific situation.

The goal becomes continuously improving cognitive systems in real time for better reasoning. When Watson made a mistake in answering a question in the “US Cities” category in Jeopardy! five years ago, the confidence level for the answer was only 14 percent [1].Woman whispering into the ear of Pepper the robot.

Imagine if Watson worked with, rather than competed against, grand masters Ken Jennings and Brad Rutter? They could step in when they knew their “partner” was confused and could not make a confident choice. Or, what if Watson asked Ken or Brad to clear its confusion in order to make better reasoning? It could also accept human questioning on its answer and listen to human input and redo its reasoning based on the new input.

Cognitive reasoning can also become more personal and situational if the new input is real time in-situ data coming from mobile devices, wearables, and Internet of Things (IoT) sensors. Moreover, a cognitive system can be more intelligent if the system can take direct human input because of a human’s unique perspective, instinct, intuitiveness, or emotional context. Such human–cognitive interaction can greatly improve the accuracy of the system when solving real word complex and unpredictable problems.

Imagine how it could assist a person with low vision to recognize produce labels while shopping in a grocery store; assuming the person was wearing smart glasses to interact with the cognitive engine that is capable of image processing, and voice and text recognition.

The cognitive engine would detect a person who has walked into a local grocery store. The interaction begins with the person providing a description of the surroundings, such as “now in pain killer section of the pharmacy.” The device captures images of the item the person is viewing and would evaluate the images to determine if the quality of the text information on the image is sufficient for good text recognition.

If the engine determines that the quality is not good enough, it can speculate the cause, and ask the person for assistance. For example, it could ask, “turn your head to the left an inch,” “raise the box two inches,” “rotate the can a little,” or offer best guesses, such as, “I’m 57 percent confident the soup has 120 milligrams of sodium, rotate the can a little left to improve the confidence level.”

Once the engine determines that the quality of the images satisfies the recognition requirements, the engine does the text recognition and confirms, “updated, 92 percent confidence the soup has 120 milligrams of sodium.”

The engine may also ask the person for additional context. For example, when an image depicts “take a brea .” with a missing character, the engine is not able to make a determination from three possibilities: “take a bread,” “take a break,” and “take a breath,” each with a 30 percent confidence level.

The engine may ask the person, “are you in medication, food or another section?” With a reply of a “medication section,” the engine determines both “take a break” and “take a breath” are 90 percent likely to be the words. The person may respond to the engine that the words are “take a breath” for the engine to continuously learn and reason.

The human-cognitive interface allows for human and cognitive engine interaction through natural voice or text conversations, thus allowing the engine to adapt according to human behaviors and emotions.

Consider the example of a cognitive app for senior care. The initial interface might be based on an individual’s overall status of living – health, economic, social, psychological, and behavioral. The interface is also driven by changes of the status and living events such as going outdoors, traveling to see a doctor, or meeting a friend.

An individual’s emotion can be an important factor as an indicator of an overall quality of life to drive the changes of the interface as well. For example, a frustrated user may not want to continue the current task without additional help. If a cognitive engine can perceive if the human emotion changes, then it can provide contextual interactions through interfaces to better suit a senior’s needs.

There is no doubt that cognitive computing can help humans extend their expertise and improve their thinking. However, a cognitive engine’s data analytics and statistical reasoning can be augmented in real-time by human sensing and reasoning to improve the performance and accuracy of the cognitive engine in solving complex problems.

  1. John E. Kelly III, Computing, cognition and the future of knowing, October 2015.
More stories
By Si McAleer and Ruoyi Zhou on October 12, 2018

Reflection During National Disability Employment Awareness Month

National Disability Employment Awareness Month (NDEAM) is nearing its halfway point, and we would like to reflect on its theme of “America’s Workforce: Empowering All.” For more than 100 years, IBM has embodied that theme by practicing disability inclusion. While there is (always) still work to be done, we are excited and awed by what […]

Continue reading

By Yiannis Gkoufas on August 12, 2018

Puzzle Solving with Computer Vision and Watson Services

Who doesn’t love the challenge of solving a puzzle? Jigsaw puzzles are a popular hobby for all ages and have been an important tool in a child’s physical, cognitive and emotional development. Children can develop physical skills such as better hand/eye coordination; cognitive skills such as shape recognition, memory, and problem solving; and emotional skills […]

Continue reading

By Ram (P G) Ramachandran and Thomas Brunet on May 16, 2018

AI-Enabled Tools and Automation to Improve Accessibility

Try Automated Accessibility Tools wth Unlimited Scans through June The IBM Accessibility Research team is laser-focused on seamlessly integrating accessibility into the development process. We have developed a suite of IBM Automated Accessibility Tools to help development teams easily and effortlessly “make accessibility happen.” Teams that use our tools tell us: “The automated tests run […]

Continue reading