Asset Management

Why Pokémon Go will make you rethink IoT and AR

Share this post:

Pokémon Go already looks like a pretty major technology shift. It’s one of the most successful mobile games of all time, people are literally crashing into each other while playing, and the controversial headlines practically write themselves.

But the technology industry impact of Pokémon Go goes beyond just gameplay. At its core, Pokémon does a good job of integrating real-world paradigms like smartphone maps and cameras with virtual paradigms like animated cartoon characters — excuse me “fictional creatures” — and making it all work together in a brilliant way.

The game’s goal: capture as many characters around your town/city/country as possible, do battle with others, earn points and status, and so forth. If technology is part of your livelihood or you have no hobbies, I recommend trying it. Why? Because Pokémon Go showcases the parallel challenges for both augmented reality (AR) as well as the Internet of Things.

Augmented Reality 1.0

Pokémon Go utilizes AR and while today it is uncommon to see AR in IoT, it will play an important role the future as the simplicity of just pointing your smartphone at a “thing” and viewing its information shadow overlaid on your screen has many potential applications for IoT. You know, like this:

Terminator vision, via Jim Cameron

Terminator vision, via Jim Cameron

Information shadows include text and virtual object images that are superimposed over real-world images displayed on your smartphone screen, smart glass lenses, or other visually-based user interfaces. Here’s another example:

Information shadows in “Iron Man“

Information shadows in “Iron Man“

My experience with Pokémon Go suggests the game is a fair example of augmented reality, not a great example.

AR implemented in Pokemon Go. The bird’s name is Pidgey.

AR implemented in Pokemon Go. The bird’s name is Pidgey.

Throwing a ball at a Pokémon juxtaposed on the actual sidewalk in front of you is neat and adds to the fun, but to me there is no inherent need for AR in this game. Yet this will be the first AR experience for many and Pokémon Go also makes extensive use of maps and virtual objects that are placed at fixed geographic locations on a map. So this is all very IoT-like.

AR and IoT

AR matters for the IoT in particular because it provides a powerful paradigm for interacting with endpoints. Browser-based IoT interfaces are normal today, but the user experience for users that are close to the actual IoT endpoints can’t compare with a the intimacy and interaction opportunities for pointing a smartphone at a thing and being able to interact with its information shadow. This matters more and more as users in the field depend on the IoT: field service techs, truck drivers, nurses, soldiers, firefighters, oil rig workers, and may more. Putting aside their access to reliable and low-latency cloud services, the ability to point a phone and begin interacting with an endpoint via AR creates big opportunities for developers.

One more point about the coming collision of AR and the IoT: AR is about placing virtual objects and information shadows on top of real world objects, e.g. Pokémon Go. The IoT is focused today on adding non-AR digital information (e.g. sensor logs) to real-world objects. In both cases, the technologies seek to enhance our interaction with the real/physical world.

AR provides a new interaction paradigm that gels nicely with the needs of IoT users, albeit with local/on-premises users in mind. Similarly, IoT endpoints provide a new dimension of data that improves the performance of AR in many situations.

You can sort of tell when people are playing Pokemon Go …

You can sort of tell when people are playing Pokemon Go …

Why AR for IoT needs fixing

Obviously, AR is not right for all applications, but for many it will become an essential part of the IoT user experience. But before this happens for the IoT, the state of AR will have to overcome several hurdles:

  1. Poor location precision

If your smartphone thinks you are standing somewhere you are not, your smartphone (using its onboard compass) is probably also going to think you are pointing your phone at something you are not. This is a problem for many AR implementations today — which rely on your phone’s compass and geolocation capabilities, including GPS — seeking to serve accurate AR information shadows.

Unfortunately, location precision in smartphones remains surprisingly weak. Plus or minus an average of 30 meters, due to a variety of factors. My own experience using the map feature in Yelp, for example, is that the location precision in my iPhone is usually “good enough”.

Compass + GPS = Outdoor AR in Yelp

Compass + GPS = Outdoor AR in Yelp

However for applications where hyper-accurate and precise location can be mission-critical — like mobile advertising or augmented reality IoT applications— even a few meters of variance can lead to errors and failure. With Pokémon Go, I found outdoor location precision to be erratic at times, but indoor location precision was always a disaster. GPS doesn’t work indoors, cellular trilateration is an unsatisfactory substitute, and Wi-Fi location can actually result in even worse location precision. I fully expect the Bluetooth minions to be working to fix some of this, as everything looks like a nail when you think you have a hammer, but even this provides only a partial solution to both the location problem as well as broader AR bugs discussed below.

2. Reliance on the Cloud

If you point your phone at a thing and it only serves up an information shadow when a cloud server in, say, Ashland, Oregon is available, your AR experience will correlate with your ability to connect with that server. For me, this was the most frustrating aspect of playing Pokémon Go and it wasn’t limited to the AR features. Maybe the lag is just an overloaded data center phenomenon, but if the vision for AR is to rely on a cloud lookup every time we point our phones at something in order to view its “information shadow”, this AR business is going to take a while.

3. Poor mobile resolution

If you point your phone at something while you are moving at 20 mph in an attempt to view its AR information shadow, just know that Apple Maps will think you are somewhere different from where you were three seconds ago and your AR experience will likely vary. Same thing if you are standing still and you are pointing your phone at a moving object like a Ferrari or a dog running down the street.

AR works great when this thing is parked, not so much when it is moving …

AR works great when this thing is parked, not so much when it is moving …

The weak mobile capabilities of AR today is a subset of point #1 above and may be a less urgent bug/feature in Pokémon Go but nonetheless, it is of increasing importance for the IoT when required to connect under low power with a moving object (e.g. a car, bike, drone) or to connect while moving with a stationary object (e.g. highway infrastructure). Good augmented reality on smartphones — not necessarily the AR-lite on Pokémon Go — depends on precise, accurate location to determine where you are and in turn, at what are you pointing your phone. Today, this is hard with current smartphone and most IoT technologies.

The Ideal AR experience for Pokemon Go

I have not seen criteria for an “ideal” AR user experience vis-a-vis the IoT so here goes:

Pointing my phone at a thing and receiving a real-time information shadow about it, not a different thing located four feet away, while querying or controlling the thing in real-time, and without relying on a cloud connection, and with no regard to whether I am moving or standing still.

The AR in Pokémon Go does not meet the “ideal” criteria, but the game does provide a reference for future AR applications as well as a simulation of a wide area, open loop, multi-user AR IoT application that requires users to locate an object and interact with it, similar to the way public IoT networks will operate.

Simultaneously improving AR and IoT performance

At least part of the answer for better AR lies in work already underway with low power sensors and wireless communications.

RTLS: Some IoT endpoints outfitted with real-time, P2P wireless connectivity offer a bonus feature called real-time location services, or RTLS. A group of things scattered throughout a building can communicate amongst themselves in real-time and, using the strength of their radio signals and basic geometry, can help determine the location of a smartphone or any mobile endpoint to within one meter. This is generally very useful given the state of location precision generally, but particularly useful indoors, where location remains largely unresolved despite the best efforts of the Wi-Fi community and others. Specifically, consumer apps like mobile advertising and industrial apps like in-person measurements of environmental sensors on pieces of industrial equipment will benefit tremendously from this improvement.

P2P: Again, some types of low power IoT endpoints offer real-time, peer-to-peer networking. The ability to query a thing in real-time, without the need for a cloud lookup, brings obvious decreases in latency, but it also creates opportunities to directly query or control the thing being viewed. You can easily imagine AR-enabled IoT apps that are interactive and do more than just passively view endpoint data via a smartphone and provide entire user experiences, not dissimilar from the virtual experience of Pokémon Go, without the need for a cloud connection.

Instant-on Connections: A satisfactory AR experience enabled by IoT endpoints should not require lengthy handshaking or setup times every time a user comes into contact with an endpoint. Connections should be more or less instant. This is similarly true for connecting with moving objects or connecting with stationary objects while you are moving. Some low power IoT technologies offer such an instant-on feature.

The near-term steps to better AR

Unfortunately, most low power wireless IoT implementations are not real-time nor two-way in any meaningful sense: some continually broadcast a one-way unique ID (Bluetooth) that requires a cloud lookup, and other two-way technologies can take minutes just to get a two-way conversation going, forget about something as real-time and interactive as AR.

Getting to a real-time, P2P AR will require silicon support on the smartphone to support it. Today, getting new silicon onto a smartphone is like getting an expedition of 50 cats safely to the summit of Everest, therefore I see two paths to AR 2.0 while relying only existing silicon on a smartphone:

LTE: The cellular industry is doing its utmost to convince us all that they will finally deliver a wireless IoT standard that allows endpoints to communicate over several miles while preserving battery life of 3–5 years. And all with a chip that costs one dollar. Alas, these chips are not available today — yes I am shocked, shocked — and it could be years before they are, but when the time comes for NB-IoT (the most interesting of the various LTE-based IoT standards now in the works), it is possible — with help from a real-time networking stack, which NB-IoT lacks — to perform direct, P2P AR using your phone. LTE is everywhere on smartphones so this approach makes a lot of sense on the phone, but the idea of cellular carriers charging subscriptions for thousands of endpoints in an industrial plant, for example, is more fantasy than probability.

NFC: I’ve written about this before, but the NFC chip in most smartphones now can be re-purposed to support other frequencies and protocols including sub-1GHz frequencies where LPWAN technologies like LoRa or SigFox deploy. Retrofitting NFC to support one or more of these would be an IoT game changer and would cost next to nothing.

Some final thoughts on IoT application opportunities with next-gen AR

I am convinced, based really on my own experience with “AR-lite” in Pokémon Go, that AR is coming in a big way for the IoT, technology limitations notwithstanding. The cloud will probably serve as AR’s IoT training wheels at least initially, but real-time P2P connections are where the most exciting applications will be built.

Pokémon Go deals primarily with “virtual objects”, while the IoT deals with physical objects or things. Overlaying a virtual object on a physical IoT endpoint is something for developers to contemplate, as well as overlaying a physical IoT endpoint on a virtual object. Adding, for example, a low power wireless endpoint at the same location where a Pokestop is located presents interesting opportunities for game developers and IoT developers alike.

“Gamification” of the IoT is also intriguing from the perspective of mobile endpoints. In the Pokémon paradigm, there is no reason your dog could not be a mobile Pokestop. Or your car. Or your drone. Or you.

For the first time, I am considering the possibility that the entertainment industry might have a role to play in the IoT. By driving innovations in AR and through the sheer force of millions of AR users, the gaming industry has enough heft to demand real-world connectivity to their AR games. Nintendo, for example, is about to release a Pokémon Go Plus wearable device to help you play the game without using your phone. Extending this concept beyond the wearable will spawn a new category of hybrid AR/IoT gaming.

Learn more

Contextual understanding and analysis are all services that Watson can provide today to enable staff and technicians to catch issues early, maybe even before an issue turns into a real problem. Watson IoT can drastically change and improve how organizations keep up with thousands of machines, and hundreds of thousands of component parts.

To learn more about how IBM Watson IoT can help you obtain greater efficiency through smarter asset management, please read our white paper, Understanding the Value of Enterprise Asset Management or contact a representative today.  Here’s where you can access the IBM VR Speech Sandbox .

This article originally appeared on haystacktechnologies.com.

Add Comment
No Comments

Leave a Reply

Your email address will not be published.Required fields are marked *

More Asset Management Stories
By Rahul on April 28, 2017

IoT applications spanning across industries

Internet of Things examples extend from smart connected homes to wearables to healthcare. In fact, IoT is slowly becoming part of every aspect of our lives. Not only are Internet of Things applications enhancing our comfort, but they also give us more control to simplify routine work life and personal tasks. The recent hype about […]

Continue reading

By Devavrat Kulkarni on May 23, 2017

Parking in the smart city

Parking lots have largely been ignored when it comes to technological innovations and ideas. However, with the increasing number of cars on the street, along with the time taken to park them, the problem must now be effectively addressed. Devavrat Kulkarni, Senior Business Analyst at Maven Systems Pvt. Ltd., shares his insight on it as part of various […]

Continue reading

By Matt Alexander on June 23, 2017

Achieving EAM excellence in Oil and Gas

Oil and gas companies are facing major changes due to an era of lower oil and gas prices. Energy companies are increasing their investments in analytics, mobile, and Internet of Things capabilities. These investments will help them to reduce costs, improve operations and be more secure. The companies are connecting their processes, people and their […]

Continue reading