Why Can't Retailers Move Faster? The Big Data and Analytics Roadblock
Retailers generally have a reputation for being slow to embrace the latest innovations in technology. According to data from the U.S. Census Bureau, the retail industry as a whole spends far less on technology than nearly all other service sectors.
Interestingly enough, one of the key reasons that retailers are slow to adopt new technology is that their existing technology infrastructures – the hardware and software systems that form the backbone of their IT operations – limit their ability to roll out promising new technologies. Retail Systems Research's 2013 Merchandising Benchmark report underscores this point: "45% of retail winners believe their existing technology infrastructure is preventing them from moving forward."
Big Data and analytics, currently the rage in many industries, provides a prime example. E-Tailers such as Amazon and Netflix pioneered Big Data and other analytics techniques during the last decade. But a recent research report from EKN, Big Data in Retail, points out that retailers in general spent less than half of 1% of their entire IT budget on Big Data projects in 2013. Moreover, they lack even the basic analytic maturity that is required to take advantage of Big Data insights: only 60% of retailers are able to perform even basic analysis and reporting tasks.
All of this puts retailers at a significant disadvantage. As the selling process becomes more and more personalized – promotions delivered directly to mobile devices, as an example – more data is generated and there are more opportunities for personalized promotions to perform either well or poorly. Big Data can play a big role in providing retailers with customer insights through such disparate data sets as sales receipts or transaction logs, website clickstream data, Facebook likes, tweets, product reviews, geolocation information, local events, weather patterns, flu trends, customer loyalty information, sales and promotion data, and much more.
Retailers' existing hardware infrastructures and legacy systems are barriers to implementing the very technologies that can help them become more successful. After all, most of these systems were designed and implemented well before the concept of Big Data existed. Retailers' outdated retail forecasting and analytic systems get around these limitations by taking shortcuts and making analytical compromises. In addition, retailers simply can't afford to invest in the in-house IT infrastructure that's needed to process staggeringly large amounts of disparate data.
The cloud changes all of this. The on-demand resources of the cloud provide retailers with unprecedented scale and ability to cost-effectively manage, mine and process data volumes that were previously unthinkable – and that's exactly what is required to gain insights from Big Data. The unlimited resources of the cloud are ideal for highly sophisticated, configurable and up-to-date forecasting science that utilizes all kinds of structured and unstructured data – science that simply can't be handled by traditional hardware-bound IT infrastructures.
With data growing inexorably, the cloud offers a way to keep computing power growing in a seamless fashion to meet the challenge of the ever-expanding data. Your own IT department doesn't need to constantly buy new machines; instead, by utilizing the elastic computing capacity of the cloud, a retailer can process extremely large amounts of data when it needs to do so, and then turn off those computing resources until the next time they are needed.
With these additional data sources and the unlimited elastic computing power of the cloud, retailers can begin to discriminate between different causal effects and to receive answers to questions which were difficult to answer in the era before Big Data: Was that promotion more effective in generating sales from existing customers or loyal ones? If we restricted the diaper promotion to particular days or week and times of day, what would be the financial impact? Did our turf builder sales increase because the promotion we ran was effective, or did they increase because of the shift in spring weather, this year versus last? How do pricing changes affect sales in stores that are very close to our primary competitor, versus stores that are geographically isolated from competition? What would be the impact of a paper towel and window cleaner bundling promotion?
The cloud holds the key for retailers that want to turn Big Data to their advantage – for better customer insights, more accurate forecasts, better merchandising decision, and every other kind of retail decision-making. As Retail Systems Research notes, retail winners are increasingly seeking more innovative delivery models such as cloud and SaaS, in which applications have been designed to take advantage of this newly available computing power. "With all the pressure consumers are placing on retailers today, it's hard to imagine replacing an application that has been 'working well' for five to ten years. But advances in technology have been coming quickly and furiously…with the advent of cloud and SaaS-based solutions, the time to value seems to be shrinking."
For retailers that are ready to embrace the power of Big Data and advanced analytics, the answers aren't in the traditional systems they've relied upon until now. Instead, the answer is in the cloud. And the good news is that as new retail applications exploit the power and promise of the cloud, retailers can move quickly to turn Big Data insights to their advantage – and to become more relevant to their customers.
Ron Menich, Ph.D. is EVP and chief data scientist at Predictix, a provider of assortment planning, allocation, replenishment and pricing software across retail segments.
Interestingly enough, one of the key reasons that retailers are slow to adopt new technology is that their existing technology infrastructures – the hardware and software systems that form the backbone of their IT operations – limit their ability to roll out promising new technologies. Retail Systems Research's 2013 Merchandising Benchmark report underscores this point: "45% of retail winners believe their existing technology infrastructure is preventing them from moving forward."
Big Data and analytics, currently the rage in many industries, provides a prime example. E-Tailers such as Amazon and Netflix pioneered Big Data and other analytics techniques during the last decade. But a recent research report from EKN, Big Data in Retail, points out that retailers in general spent less than half of 1% of their entire IT budget on Big Data projects in 2013. Moreover, they lack even the basic analytic maturity that is required to take advantage of Big Data insights: only 60% of retailers are able to perform even basic analysis and reporting tasks.
All of this puts retailers at a significant disadvantage. As the selling process becomes more and more personalized – promotions delivered directly to mobile devices, as an example – more data is generated and there are more opportunities for personalized promotions to perform either well or poorly. Big Data can play a big role in providing retailers with customer insights through such disparate data sets as sales receipts or transaction logs, website clickstream data, Facebook likes, tweets, product reviews, geolocation information, local events, weather patterns, flu trends, customer loyalty information, sales and promotion data, and much more.
Retailers' existing hardware infrastructures and legacy systems are barriers to implementing the very technologies that can help them become more successful. After all, most of these systems were designed and implemented well before the concept of Big Data existed. Retailers' outdated retail forecasting and analytic systems get around these limitations by taking shortcuts and making analytical compromises. In addition, retailers simply can't afford to invest in the in-house IT infrastructure that's needed to process staggeringly large amounts of disparate data.
The cloud changes all of this. The on-demand resources of the cloud provide retailers with unprecedented scale and ability to cost-effectively manage, mine and process data volumes that were previously unthinkable – and that's exactly what is required to gain insights from Big Data. The unlimited resources of the cloud are ideal for highly sophisticated, configurable and up-to-date forecasting science that utilizes all kinds of structured and unstructured data – science that simply can't be handled by traditional hardware-bound IT infrastructures.
With data growing inexorably, the cloud offers a way to keep computing power growing in a seamless fashion to meet the challenge of the ever-expanding data. Your own IT department doesn't need to constantly buy new machines; instead, by utilizing the elastic computing capacity of the cloud, a retailer can process extremely large amounts of data when it needs to do so, and then turn off those computing resources until the next time they are needed.
With these additional data sources and the unlimited elastic computing power of the cloud, retailers can begin to discriminate between different causal effects and to receive answers to questions which were difficult to answer in the era before Big Data: Was that promotion more effective in generating sales from existing customers or loyal ones? If we restricted the diaper promotion to particular days or week and times of day, what would be the financial impact? Did our turf builder sales increase because the promotion we ran was effective, or did they increase because of the shift in spring weather, this year versus last? How do pricing changes affect sales in stores that are very close to our primary competitor, versus stores that are geographically isolated from competition? What would be the impact of a paper towel and window cleaner bundling promotion?
The cloud holds the key for retailers that want to turn Big Data to their advantage – for better customer insights, more accurate forecasts, better merchandising decision, and every other kind of retail decision-making. As Retail Systems Research notes, retail winners are increasingly seeking more innovative delivery models such as cloud and SaaS, in which applications have been designed to take advantage of this newly available computing power. "With all the pressure consumers are placing on retailers today, it's hard to imagine replacing an application that has been 'working well' for five to ten years. But advances in technology have been coming quickly and furiously…with the advent of cloud and SaaS-based solutions, the time to value seems to be shrinking."
For retailers that are ready to embrace the power of Big Data and advanced analytics, the answers aren't in the traditional systems they've relied upon until now. Instead, the answer is in the cloud. And the good news is that as new retail applications exploit the power and promise of the cloud, retailers can move quickly to turn Big Data insights to their advantage – and to become more relevant to their customers.
Ron Menich, Ph.D. is EVP and chief data scientist at Predictix, a provider of assortment planning, allocation, replenishment and pricing software across retail segments.