How to Maximize the Potential of Machine Learning in Retail

For decades retail companies have been exploiting analytics within the different segments of their businesses, including marketing and operations. Such analytics are dusty, however, and have now come to an end. Traditional analytical methods are outdated; they require a lot of manual steps and the insights extracted cannot be easily generalized. Using analytics ultimately provides a low return if you include the amount of manpower needed allocating to run them. Then what’s next?

Machine learning (ML) can be viewed as an extension of analytics. Not only does ML leverage expertise and domain knowledge specific to a retail industry, it also explores new data sources that may be informative. Using ML overcomes the caveats of traditional analytics: state-of-the-art approaches are exploited, inferred insights become robust over different business scenarios, and automation oversteps human involvement and is less prone to errors. Eventually returns also increase. The retail industry has a lot of opportunities to leverage ML, from logistics optimization to customer care through the sale of data and insights to vendors, demand forecast, product recommendations, and promotion campaigns.

In this post I focus on promotion campaigning and see that some of these ML use cases are intertwined with it. “It can cost five times more to attract a new customer, than it does to retain an existing one.” Forbes, 2019

Targeting past or current customers through custom promotion campaigns is crucial for the retail industry. As a prerequisite, a lot of data needs collecting, usually through loyalty programs. Using ML to sketch custom promotion campaigns is usually a quick win for the sector: no additional data needs grabbing outside of loyalty programs and retail companies do not have to invest into external sources. The efforts required are not overwhelming and rely on the ML phase solely without a lot of new data ingestion and engineering, which can represent as much as a 40% cut in implementation costs compared to other ML use cases. Furthermore, validating ML-based campaigns is easy through A/B tests. Multiple marketing campaigns featuring selected products can be directly sketched from ML insights. Retail companies use them to target different customer audiences from their loyalty programs, leverage those most likely to respond.

Creating targeted promotion campaigns is a four-step funneling process:

1. Identify product trends to extract.
2. Select the right product(s) to feature in a dedicated campaign.
3. Segment customers.
4. Target the right customer segment with the most relevant campaign.

The process

1. Product trends

The very first quantitative task is to train and serve a thorough demand forecast for each product category you want to advertise, including trends and seasonalities. Historical sales data is usually enough. Demand (i.e., purchases) can be modeled for each product category within a time horizon. Advantages are twofold:

• The forecasting approach directly checks for global vs. seasonal trends.
• Useful business insights can be extracted for various types of products, including seasonal ones.

To solve such tasks, many approaches are valid, like additive models based either on Box-Jenkins ARIMA or Exponential Smoothing. Such models have various components modeling different factors: non-periodic demand changes but also periodic ones (weekly, annual, seasonal), holiday effects with irregularities, and an error term referring to unusual demand changes. The ML community uses such ETS (Error, Trend & Season) models extensively. There can be various implementations, for instance, in StatsModels or Prophet.

As for here at Aliz, we are used to developing and deploying them in Kubeflow.

2. Product selection for campaigns

Within the trending categories, we can select specific products to advertise, given many considerations including:

• Product trends within the time period of interest.
• Stock optimization (e.g. destock).
• KPI (e.g. profits, customer satisfaction via new product offer).
• Product availability (stock, renewal, etc.).

This task is a conventional optimization which does not have to be turned into an ML one.

3. Customer segmentation

The big picture is to discover your customers’ habits and tastes from their buying behavior. Customers are clustered given their purchase history which is collected through loyalty programs. Clustering can be interpreted as extracting similar patterns in the customers’ buying behaviors. Doing so usually helps to increase profits, while leveraging recommendation systems favors customer experience. Many ML approaches are suitable:

• Standard clustering models explore purchase history. Customers from the same clusters are assumed to be interested in the same products. This technique focuses on customer habits and can be exploited to push customers to consume more of what they have already tasted.
• Recommendation engine models (SVD, WALS, etc.) can be used to regroup customers potentially interested by the same product. This method focuses on new product recommendations and can be explored to bring new product experiences to the customers.

There are a lot of implementations in the ML community. Mainly coding in Python, we think that Scikit-learn gives the most exhaustive scope for clustering approaches. As for the recommendation models, we find some of them either in Scikit-learn or in Tensorflow. At Aliz, we usually run clustering models and recommendation engines in Kubeflow.

Targeting the right audience

The aim is to offer the right products to the most relevant customers while the means are the diverse marketing campaigns we sketch to feature such products. To do this, the findings extracted are combined from the most pertinent products for the company to advertise to the most relevant customer segments. This is about cross-referencing products with customer needs and tastes. Additional constraints can also be taken into consideration, like product availability in the customer’s favorite or nearest store. Despite being quantitative, this task covers more from optimization than ML.

How to measure the success of a marketing campaign

A promotion campaign impacts retail customers in three different aspects of their buying behavior: from customer visits to loyalty shift, through money spent. In a loyalty program, there are three types of customers:

• Neutrals have some go-to retailers for particular purchases but also shop around.
• Roamers always tend to shop around prior to making a purchase.
• Loyalists find a go-to retailer for their needs and stick with them.

Life stages influence customers’ loyalty behavior. Customers can be Loyalists for one product category and Roamers for another. Additionally, they will shift from Loyalist to Roamer as they are influenced by multiple external factors, e.g. their current financial and social stage, the presence of children in the household.

“Customers actually move back and forth on the shopping personalities spectrum several times during their lifetime. […] For instance, very young consumers and those aged 35 to 44 are most likely to be Roamers, while Millennials and roughly half of consumers 75+ tend to be Loyalists. […] When you become more budget-conscious, you become more of a Roamer. […] Even though parents are often time-strapped, they appear to be more status-driven and willing to shop around for incentives.” Gudat, Consumer Shopping Personalities and their Impact on Customer Brand Loyalty

Being revenue-focused is the key success of a promotion campaign. Roamers form the target audience most responsive to promotional offers. The greatest normalized revenue lifts are usually noticed from the Roamers segment. Some Roamers, may even become Loyalists for some product categories.

Loyalty shift

Roamers are also sensitive to new product recommendations. By definition, they are interested in product categories that they may not buy in their regular shops. It is essential to attract them with new products that they should potentially be interested in. In summary, defining success for a campaign is about thinking beyond revenue, namely from visits to loyalty shift. This can be illustrated into the life-cycle of a customer experience:

1. Revenue: Not only do shoppers buy the promotional products, they can also be tempted to to purchase other goods not recommended.
2. Satisfaction and loyalty: Analyze which product categories make your shoppers Loyalists and which categories they begin to buy.
3. Shopper visits: Customers can be convinced to visit your shops more often. Important measures to track are the frequency and velocity of visits.

Implementation and architecture (the techie part)

Many different models are involved when it comes to the implementation of an ML solution in the retail industry:

• Each product category has its dedicated demand forecast model.
• Clustering models explore customers’ purchase history data.
• Recommendation models are collaborative-filtering approaches, i.e., based on customer-product interactions.

On one hand, all of these models can be trained in parallel, which is an advantage both for computational time and performances (as model errors won’t accumulate on one another). On the other hand, the variety of models described requires the orchestration of a modeling pipeline so that the training phase works effectively and without friction, regardless of the frequency of retraining. Cloud Composer is a solution for orchestrating such a pipeline. It involves:

• Feature engineering steps tackled in BigQuery.
• Training phases of the diverse models run in Kubernetes Engine.

In most business scenarios, an asynchronous prediction-serving schema is enough. Basically, static and near-time features are engineered in BigQuery. The Kubernetes Engine serves the predictions of the various models, including demand forecasts, customer segmentations, and recommendations. On top of prediction serving, the Kubernetes Engine is also used to run some other tasks required, such as the optimization of business constraints for product selection and the cross-reference of products vs. customer segments during the targeting phase. The serving is triggered by the client and has one gateway: the app engine to be post-processed and pushed back to the caller system.

Key takeaways

If retail companies want to use ML, the possibilities are endless. They can use ML to optimize their business and increase their return on investment (ROI). From the data collected via loyalty programs, retail companies can create customized targeted campaigns, e.g. involving promotional offers. The process is straightforward and involves many quantitative steps, such as demand forecasting of product categories and customer segmentation. Such intermediary steps can be re-used in a standalone manner, for instance for further analytical tasks.