The Brave New World of Computer Vision: 4 Things Retailers Need to Know


Computer vision in retail is a hot topic, but the technology, which can surpass human visual abilities, is being implemented across a breadth of industries. As a scientific discipline, computer vision is concerned with the theory and technology for building artificial systems that obtain information from images or multi-dimensional data, according to Science Daily.

Simply put, computer vision works by acquiring an image, processing the image, and then understanding or classifying the image.  Artificial intelligence (AI) can then take actions based on an understanding of the image.

While still in its infancy, computer vision is quickly becoming a part of everyday life. Over the next few years the technology will continue to scale at an accelerated pace and its impact will be felt by the public in a variety of industries.

Savvy retailers are beginning to experiment with applications of this next-gen technology, which will help them evolve with ever-changing shopping habits. Today 3% of retailers have computer vision technology in place, according to RIS’ “29th Annual Retail Technology Study: Retail Accelerates.” However, 40% plan to start or finish implementing the tech within the next two years.

Computer vision plays an important role in facial recognition, cashierless stores, inventory visibility, and visual search, four vital areas retailers are already putting the technology to work in. Here we take a deeper look at each of these applications and what they mean for both retailers already utilizing the tech and companies looking to adapt computer vision in the future.

Grab and Go

One of the most high profile applications of the technology in the retail sector is the cashierless or grab and go store. These next-gen stores allow shoppers to simply pick an item off the shelf, put it in their bag, and walk out. No stopping at the POS, no scanning of the product with a handheld device, nothing more than selecting a product and walking out with it.

Amazon is the highest profile use case of the grab and go store in the industry today. When the online behemoth went live with its first Amazon Go pilot in late 2016 it received wide-scale coverage as both shoppers and retailers alike were itching to get their first look at the cashierless technology in action.

The Go stores use a mix of next-gen technology, with computer vision at the forefront, to track customer movement in the store (thanks to hundreds of camera strategically placed throughout) and recognize when a product is selected and charge the customer’s account when they exit the store. 

"The majority of sensing is from above," the Chicago Tribune reports Dilip Kumar, vice president of technology for Amazon Go, said when the retailer opened its Chicago location. "Cameras figure out which interactions you have with the shelves. Computer vision figures out which items are taken. Machine-learning algorithms also determine which item it is."

To Amazon, scan and go stores represent more than just a science experiment; the retailer is on the record with its plan to open thousands of these stores over the next few years. And they are not alone.

Retailers across the industry are looking to get in on the act as they focus on building stores of the future focused on a frictionless, seamless customer experience. Sam’s Club for example went live with a new store late last year in Dallas, TX, dubbed Sam’s Club Now. The test lab store will be a mobile-first shopping experience powered by the new Sam’s Club Now app, allowing customers a cashierless, grab and go experience.

“We’re putting Sam’s Club on the cutting edge of innovation in the warehouse channel,” said Jamie IannoneCEO, and EVP of membership and technology. “Using all available technologies ― including computer vision, AR, machine learning, artificial intelligence, robotics, just to name a few ― we will redefine the retail experience today and into the future.”

While deep-pocketed industry disruptors like Amazon and Walmart are leading the way with computer vision powered grab and go stores, most of the industry is in wait and see mode. Just 7% of retailers have up-to-date grab and go technology in place today, according to RIS’ 29th Annual Retail Technology Study: Retail Accelerates.” But another 15% are either in the midst of a deployment or will start one in the next 12 months, indicating an eagerness among retail’s adventurous to bring the experience to market before the competition.

Inventory and Compliance

While exciting computer vision use cases like grab and go stores and visual search have the potential to make an immediate splash on the customer experience, other less sexy applications of the technology could potentially have the greatest long-term impact.

Retailers devote untold hours ensuring their inventory is accurate and up-to-date and stores are in compliance with safety and merchandising standards. Much of this work is done manually at the store level, with associates and managers tasked with constantly monitoring and scanning the store to ensure accurately and compliance. This outdated operational approach costs retailers thousands of man-hours every year, man-hours that could be reallocated to providing a differentiated customer experience.  

Savvy retailers are turning to robotics powered by computer vision technology to take over mundane store monitoring tasks. By deploying robots at the store level tasked with scanning shelves for both inventory and product compliance retailers can not only free up associates for customer-facing tasks, but they can greatly improve inventory accuracy.

Schnuck Markets invnetory robots traverse the store three times per day, scanning approximately 35,000 products per traversal.

Schnuck Markets had gone live with its autonomous shelf-scanning robot, Tally, to capturing deeper, real-time insights into on-shelf operations.

“Teammates are spending several hours a week completing tasks like inventory scanning and price tag auditing,” said Dave Steck, VP of IT, infrastructure and application development, Schnucks. “Since we implemented Tally, the robot now completes those mundane tasks, allowing teammates to focus on customer service.

The robots traverse the store three times per day, scanning approximately 35,000 products per traversal. On an average day, the Tally robots scans millions of products, giving Schnucks more accurate, frequent, and comprehensive insights into product flow and in-store operations.

“Tally has been instrumental in boosting the way our shelves are managed and how our stores function,” said Dave Steck, VP of IT, infrastructure and application development, Schnucks. “As a result, we’ve experienced a phenomenon we call ‘The Tally Effect,’ an immediate improvement in in-store operations and increased teammates productivity."

Facial Recognition

While a wide range of potential applications exist for face scanning biometric tech, which measures and matches unique characteristics to identify a subject, the implications in retail may be massive. The inclusion of facial-recognition software in personal devices, such as the iPhone X, has the potential to change how people shop. More than 1 billion smartphones will include facial-recognition features in the next two years, Forbes reports.

Inside retail stores, the tech can be used for both security and authentication, but not without concerns. Last Spring, The New York Times reported Madison Square Garden’s management has secretly been using facial recognition as part of its security measures, using cameras to capture images of people and then run the photos through an algorithm that compares them against a database of images. The news prompted public concern, including from New York City Council Member Ritchie Torres who in turn introduced a bill last October that requires businesses to notify customers of the use of biometric identifier technology.

Beyond the more obvious use case of security, retailers may be able to utilize the tech to benefit shoppers in a more personal way. Candy retailer Lolli and Pops uses facial recognition to identify loyalty members that opt into the program as they walk into the store. As shoppers enter a store, a camera will scan and a sophisticated algorithm will recognize their face and send that information to an app on the sales associate's tablet. The associate can access the individual's data, such as taste profile or allergies, and be able to offer personalized product recommendations via AI-enhanced analytics.

CaliBurger uses facial recognition software to identify registered customers and pull up their loyalty information.

Similarly Cali Group unveiled self-ordering kiosks at CaliBurger, which use facial recognition software to identify registered customers and pull up their loyalty accounts and order preferences. The kiosk pilot program began at CaliBurger’s Pasadena location where customers have the option of immediately activating their loyalty accounts as they approach the kiosks without needing to swipe a card or type in identifying information. The loyalty account shows a customer's favorite historical meal packages, enabling the customer to complete the ordering transaction in a matter of seconds.

Visual Search

Visual search technology is shaping online shopping, using relevant images to yield better search results than a traditional text search.

While visual search is beginning to enter the digital retail space, it remains a tiny fraction of total search volume, perhaps topping 1 billion monthly searches, compared with hundreds of billions of text searches, according to eMarketer. However, the tech has the potential to have a massive impact on online shopping, as younger shoppers enter the marketplace who are more comfortable with alternative searches to text. At 62%, visual search was the technology U.S. and U.K. millennial Internet users would feel most comfortable incorporating into their digital shopping experience, eMarketer found.

Online retailer eBay has been an early adopter of the tech. Its latest feature couples computer vision with relevant filters to allow shoppers to tap on a product listing to explore similar items. When shoppers open the eBay app, they can then tap on the three dots next to an item they would like to explore. This pulls up a screen for a number of options including the “looks like this” feature. Back in 2017 the company announced its Image Search function, which allows shoppers to share a photo into the eBay search bar to discover listings that match the item in the photo.

Omnichannel retailer Home Depot launched its visual search function in 2016, adding it in beta mode to its mobile app. Today consumers simply snap a picture of a product they are interested in and the solution instantly searches Home Depot’s extensive product catalog for that exact product and/or similar products. Typically, shoppers use the visual search function in one of two ways: to locate a product they are interested in, or to find complementary items to a product.

While some retailers are just getting their feet wet with the technology, some are already seeing the benefits. Forever 21’s AI-powered visual search and navigation feature debuted as “Discover Your Style” in the Forever 21 iOS app last May and was initially available for the dresses and tops categories. In the first month after launching the feature, the apparel retailer saw an increase in sales conversions and a 20% increase in average purchase value for the two test categories. This initial success led Forever 21 executives to fast-track the large-scale web integration of the technology, which uses AI, computer vision and natural language processing to provide shoppers with more relevant search results.


As retailers search for solutions to blur the lines between physical and digital retail, computer vision fits the bill. In addition, whether it’s in self-driving vehicles making deliveries, robots taking inventories, AI-driven security scans, or robots frying up tater tots to help humans avoid hot oil injuries, computer vision is an essential part of digitizing the retail workforce as well.

Privacy is bound to be a heated topic of discussion around implementing many of these futuristic solutions, but regardless, this is one next-gen tech retailers should be thinking about now as retail can reinvent itself in the blink of an eye.

This ad will auto-close in 10 seconds