Your personal shopping assistant

Share this post:

It’s mobile, informative, and smart.

Have you ever found yourself in the supermarket staring at a shelf full of different cereal boxes, wishing someone could just point out the one with the best price, lowest sugar content, and the best reviews? New solutions for smarter retail will soon give customers the same type of information they get online when researching or comparing products – delivered inside the store as they shop.
A new augmented reality mobile shopping app being developed by researcher scientists at IBM’s lab in Haifa, Israel, is about to change the way we shop in stores. When shoppers use their smart phone or tablet video camera to pan over products on the shelf, the application will instantly display recommendations and offers based on their specific preferences.

Coupon via mobile shopping app.
“We’re going way beyond simple facial recognition for products to provide superimposed information that points out the products shopper prefer – whether based on previous purchases, price, consumer rating, sodium content, environmentally friendly packaging, or other considerations,” said Amnon Ribak, project leader for the augmented shopping advisor.
For example, a shopper looking for a high-quality facial moisturizer can specify important characteristics of the product, such as having a sun protection factor (SPF) of 15, is hypoallergenic, was not tested on animals, and that is on sale.
As the shopper points a smart device camera at the shelf of moisturizers in the pharmacy, the app recognizes the merchandise and displays information on the device’s screen, superimposed on the product images. It also highlights information based on the shopper’s stated preferences, and can offer coupons or special discounts that may apply.

“The idea of standing in an aisle in the supermarket and having your mobile device point out the gluten-free cookies you need can be a real time saver,” Ribak said. “This has the potential to completely change the shopping experience from one of hunting, reading, and searching to simply picking up those products you prefer.” 

Product recognition doesn’t require bar codes or RFID tags. It is done using a blend of different approaches, including image recognition for the packaging that matches the colors and shapes to a database collection, optical character recognition techniques, as well as on-shelf context and positioning. 

To develop this new technology, Ribak leads a team of research scientists whose image processing expertise has already contributed advances in such areas as license plate recognition at night or inclement weather for the Swedish toll road system; identification of malignant tumors in medical images; and optical recognition for ancient texts in libraries across Europe.

The researchers wrote algorithms that combine techniques used in facial recognition, color and shape matching, and associations with surrounding products. The app can take into account the mobile device’s camera angle, and distance from a shelf to help distinguish between products.
“Our first mission was to create our own mini-supermarket in the lab, so we could test the various approaches and challenges involved,” Ribak said. “We’ve already submitted a number of patent applications based on the new techniques we discovered to overcome the challenges in recognizing products with less than ideal lighting, shadows, and reflections.”
Aside from assisting shoppers, the new application could also help retail managers organize their stores, instantly point out what’s missing on the shelves, or summarize which products were on sale during the week. As more and more shoppers use the application, it will give retailers deeper insight into how their customers shop – which is an analytics goldmine for optimized shelf and store arrangement.   

Read more about IBM’s e-commerce technology on the Smarter Planet blog.

More stories

A new supercomputing-powered weather model may ready us for Exascale

In the U.S. alone, extreme weather caused some 297 deaths and $53.5 billion in economic damage in 2016. Globally, natural disasters caused $175 billion in damage. It’s essential for governments, business and people to receive advance warning of wild weather in order to minimize its impact, yet today the information we get is limited. Current […]

Continue reading

DREAM Challenge results: Can machine learning help improve accuracy in breast cancer screening?

        Breast Cancer is the most common cancer in women. It is estimated that one out of eight women will be diagnosed with breast cancer in their lifetime. The good news is that 99 percent of women whose breast cancer was detected early (stage 1 or 0) survive beyond five years after […]

Continue reading

Computational Neuroscience

New Issue of the IBM Journal of Research and Development   Understanding the brain’s dynamics is of central importance to neuroscience. Our ability to observe, model, and infer from neuroscientific data the principles and mechanisms of brain dynamics determines our ability to understand the brain’s unusual cognitive and behavioral capabilities. Our guest editors, James Kozloski, […]

Continue reading