Innovations and research challenges in IoT
At IBM’s InterConnect event in March this year, Gabi Zodik, Director, IoT and Mobile Platforms gave a presentation entitled ‘IBM Research: IoT 2020 & Beyond.’ He was exploring what the future holds for the IoT, based on the research that IBM and our partners are undertaking together.
I’m here to take a look some of his interesting ideas and take you on a whistle-stop tour of what the future holds for three of the IoT’s supporting pillars: Cognitive, Edge Computing, and Security.
Setting the scene: IBM’s research and collaboration around the world
As you might imagine, IBM is dead keen on research. We collaborate extensively with academics, industry clients, partners and governments in locations all over the world to seek out the best and brightest minds and ideas.
We were also the first multinational corporation to establish a research institute in China, with the opening of a lab in Beijing in 1995. In 2008, another lab opened in Shanghai. And since 2010, we’ve established new labs in Brazil, Australia, Ireland, and East Africa – all growth regions filled with smart, talented people and no shortage of big, interesting challenges to work on.
Against this collaborative and exploratory backdrop, we’ve been making some exciting progress in the world of IoT. Let’s look at the field of Cognitive first.
Cognitive everywhere: what’s on the horizon?
Gabi’s vision of the cognitive future grounds itself in user interactions that are context-based, personalised and multi-modal. In other words, a user interface backed by technology that not only tailors its responses to the individual user and their current circumstances, but is able to use and understand multiple modalities, such as voice, gesture, text, video, and AR / VR.
Gone will be the apps we use today. Instead, they’ll be replaced by a new form of user interaction – by IoT systems that are flexible, responsive and personalised.
Cognitive in action
But what does a personalized, context-based IoT system look like exactly? The simplest way of understanding the potential of a such a system is to imagine it in action – so let’s put it to work diagnosing energy wastage in buildings.
Energy wastage diagnosis step-by-step:
- Collect the raw data by taking readings from sensor data influencing energy use
- Place the information into context: the sensor readings indicate abnormally high energy use
- Understand and reason: the system pinpoints Air Handling Unit 3 as the culprit for this unusual high energy use
- Use intelligence to offer actionable insights: the system checks the AHU maintenance schedule and sends for an engineer to fix the problem
- Learnings and recommendations for the future: the system provides the user with a maintenance schedule of components with abnormally high energy usage to pre-emptively address possible recurrence of the same issue
Deep cognitive: the next steps
In a use case like the one above, we start to see the advantage of cognitive analysis. But what about ‘deep cognitive’? Where are we today, and what can we expect from the future?
Today’s ‘deep cognitive’ systems generally offer analysis of unstructured data sets like conversations. However, their deep learning and reasoning capabilities are negligible as yet.
In the future, connected ‘things’ will be able to offer more than simple analysis of real-time data. The things themselves will encompass backend reasoning capabilities that enable them to respond proactively to any given scenario. They will continuously run in the background anticipating every move or action we plan to have in the future and provide us with a delightful service that is customized and personalized to our needs and preferences. They will no longer wait to be asked a question, for instance, but will infer the need to offer a solution (and what that solution should be) from their knowledge of their users’ daily habits, environment, schedule and personal preferences.
Of course, such a response requires continual monitoring by the connected things – of the environment around them, and, dare I say, of the people who use them as well. If, like me, you baulk a little at this Panopticon-esque vigilance, it’s worth mentioning that this visionary future comes with an element of choice. There’s still such a thing as the ‘off’ button, and presumably a system that offers such an extraordinary level of personalisation could be adapted to a user who prefers a greater degree of privacy, based on settings they control and decide upon themselves. Still, the questions of human control and security are important – vital, even – and these future IoT systems will need to come kitted out with superior security designed to protect the data our things will so assiduously collect. This is something we’ll examine later in the blog.
On a happier note, it is easy to see the fortuitous application of real-time data analysis. It can even be life-saving, as IBM has discovered thanks to our work with Medtronic – a glucose monitoring system to help diabetics manage their condition and maintain a high quality of life. The system monitors physical activity and nutrition in addition to glucose levels, feeding this information into a deep-learning model that can anticipate potential health problems and suggest preventative action on the user’s part – such as injecting insulin or having something to eat.
One thing we achieve with deep learning is to bring insights and recommendations for the people, of things they could not conclude on their own. This is because of the complex deep learning and reasoning models that are continuously running 24×7 in the background processing the huge amounts of sensor data or just interaction data.
Edge computing: all the benefits of cognitive computing, without sending data to the cloud
As we consider what the future holds for edge computing, it’s time to return to the question of security and of privacy. IBM is working to enable cognitive capabilities for Watson IoT solutions which do not require clients to send their proprietary data to the cloud. This means that clients keep hold of their data, but are still able to benefit from insights thrown up by its analysis.
But this is impossible, isn’t it? How can you analyse something that you can’t see? Well, apparently, you can – and to find out how, we’ll consider a simple use-case scenario.
Imagine that a manufacturing plant wants to augment its on-premises PMQ solutions with visual and audio analytics to check for defective parts onsite.
The client prefers not to send details of their plant operation to IBM, but still wants to benefit from the power of the AI learning available at the IBM Watson cloud services.
Here’s how we get around the problem:
- Images and sounds that demarcate faulty parts as opposed to functioning parts are uploaded to the Watson cloud
- Watson learns what a good part should look and sound like
- After learning on the cloud happens, an on-premises system inspects images and sounds during production to retrieve and cache those models at the premises
- The system can identify good and bad parts without sending production data to the cloud.
To sum up, by moving Watson IoT Cognition to your data you can:
- Harness the cognitive capabilities of Watson IoT cloud-based services
- Keep operational IoT data on the premises and maintain data privacy
- Reduce total solution cost
- Improve responsiveness and agility
So how close is IBM to achieving a model like this? Pretty close, apparently. We are actually building the enabling technologies right now – simplifying creation and deployment of cognitive IoT solutions using analytics for machine audio, images, video, anonymization and privacy.
One recent example is the FOAK we are running at Cenovus in Canada, here we deployed in this large facility several RPi devices that allow the employees to be connected and monitored to prevent various accidents without having to carry a mobile device with them, only the sensors. All the analytics is done on the edge on the RPi devices.
Edge computing in action
This solution has potential across many industries, including manufacturing, automotive, healthcare, mining, forestry, aerospace and defense and retail.
In a retail scenario, for example, intelligent asset and equipment management focuses on making shop floor equipment smarter and more reliable, to reduce downtime and increase performance.
IBM can make sense of the shop floor equipment by connecting it through our IoT platform, edge partners and ecosystem. The platform brings together data from sensors, manufacturing gateways and programmable logic controls so that retail clients can visualize the effectiveness of their equipment.
Beyond simple visualization, clients can also take advantage of advanced analytics, that provide predictions based on specific use cases such as maintenance or quality.
This is the job of IBM’s new Plant Performance Analytics. They come preloaded with valuable industry models for automotive – such as body and weld – to help automotive clients jump-start their analysis.
Other cognitive capabilities include Natural Language Processing (Watson’s uncanny ability to understand text by breaking it down grammatically, relationally and structurally), image analysis and text analysis. A piece of kit called ‘Equipment Advisor’ uses all three of these capabilities by ingesting text from manuals and logs to suggest ways to fix faulty equipment.
Security and Blockchain
Of course, all the solutions and technologies we’ve explored above rely on water-tight security, and we’ve identified three pivotal challenges:
- End-to-end device security
- Data protection
- IoT security analysis
End-to-end device security
Secure devices will enable device authentication to ensure all applications talk to the right IoT device. Device discovery is another key requirement that relies on authentication – to get an overview of what IoT devices are in use, and how trustworthy they are.
Secure IoT devices also require special hardware – an approach that already exists for traditional IT devices. The research challenge is to figure out how this can be achieved for IoT devices specifically.
IoT data provides a lot of information – some of which can be sensitive. Smart home data gives information about the people living in a house and whether they are at home, for instance. Handy material for a would-be burglar, unless it’s properly contained.
To combat this danger, enhanced privacy technologies for IoT data, such as data anonymization, are a vital requirement. The research challenge here is to keep a balance between privacy and usability – so that only necessary information is revealed to the IoT application.
IoT Security Analytics
Where IT devices are well protected in a data center, or are under the close control of the end user, IoT devices are often placed where there is little physical protection for them.
The job of IoT security analytics is to analyse the event streams produced by an IoT device, and detect anomalies that could indicate a breach. IBM is working to:
- Explore cognitive methods and technologies to identify anomalous events;
- Analyse event streams in real-time, with event analytics undertaken at the edge (i.e., close to the IoT device itself);
- Bridge the IT and IoT world around IT and IoT event analytics.
To understand the practical application of these technologies, let’s examine the way that IoT security and Blockchain can counteract a particular security issue. In this case, counterfeiting.
Point of Care Devices + IoT + Blockchain = no counterfeiting
In recent years, the spread of counterfeit goods has become a global problem. Counterfeiters generally do not comply with health and safety regulations, and consumers, buyers and importers have no way of finding out if the products meet safety regulations or not.
To combat this problem, IBM Research in Zurich developed a prototype that shows that cryptographic, temper-safe anchors can be embedded inside of Point of Care Devices and used to strongly link a physical product with its digital representation in a Blockchain system. This trusted linkage opens a variety of business use cases, from supply chain optimization and brand certification, to fraud detection of unauthorized replicas.
Voilà, the problem is solved, thanks to the coming together of Point of Care Devices, the IoT and Blockchain to create a secure solution.
The Haifa team is working to prevent counterfeirters in Kenya, where in many cases the drugs are replaced with fake ones during their shipment, using blockchain we can track all the way and make sure the original drugs were delivered to the doctor. In Africa Counterfeiters causes to millions of deaths every year.
So what’s in store for the IoT beyond 2020? Dr Curioni hopes that IBM can work to provide a ‘digital sixth sense’ – by combining computer science models and sensor technology. The goal is around augmentation and enhancement – allowing humans to connect with the physical world and cognitive objects in a way that simply isn’t possible today.
Within the next five years, you’ll be able to touch what you’re shopping for online thanks to sensory augmentation and substitution to create a beyond-the-pixel experience. Your mobile device will be able to distinguish fabrics, textures, and weaves so that you can feel a sweater, jacket, or upholstery – right through the screen.
The cradle of this technology is in haptic devices, which apply vibrations, forces or motions to the user to recreate the sense of touch. The gaming world has been using devices like this for years, and it’s already possible to recreate a sense of texture through vibration. But Dr Curioni believes that the next five years will see these vibrations translated into a dictionary of textures that match their physical counterparts. It will be possible, he posits, to match variable-frequency patterns of vibration to physical objects, to recreate the feel of a silk shirt, for instance.
Using digital image processing and digital image correlation, we can capture texture qualities in a Product Information Management (PIM) system to act as that dictionary. Retailers could then use it to match textures with their products and their products’ data – sizes, ingredients, dimensions, cost, and any other information the customer might expect. The dictionary of texture will also grow and evolve as we grow our appetite, usage and understanding of this kind of technology.
It sounds as though IBM Research has its work cut out. But the possibilities are exciting, and just around the corner.
To learn more about IBM’s research in security, cognitive and edge computing, you might find the following resources useful: