The Challenges and Solutions for Sales Forecasting in eCommerce
By: Gene Tyndall, President, MonarchFx
David Lengacher, Head of Data Science, MonarchFx
It is commonly known that forecasting sales is complex and that retailers, wholesalers, and brands have struggled with achieving accuracy since the beginning. Consumer buying behavior is non-scientific. Despite algorithmic and statistical advances in recent years, as well as progress in data science and artificial intelligence (AI), high degrees of accuracy are not always achieved.
The rapid growth of eCommerce has exacerbated these challenges and Sales and Operations Planning (S&OP) processes have not adapted well. For example, SKU-level forecasts by location are proving to be even more challenging.
This article addresses the new challenges and suggests solutions that we believe are insightful. In fact, MonarchFx has configured best-in-class forecasting software to support the eCommerce principle of stock ‘flow’.
Below are the unique challenges of forecasting sales for eCommerce. Several also exist in forecasting store sales and other channels; their applicability in eCommerce is proving to be somewhat unique.
- Statistical forecasts are based on historical data. Therefore, they are not sufficiently representative of the casual factors that influence demand going forward.This problem is especially true in online sales, when SKU-level data is needed.Analyzing patterns and creating baseline forecasts are often the most we can hope for.
- Business actions that impact demand are not always collaborated within the organization, creating a lack of visibility within the company. New product introductions, demand generation initiatives, and business actions can sometimes come as a surprise to demand planners.
- Companies are slow to develop digitalization. Therefore, S&OP processes remain fragmented. This results in time delays, unmatched supply-demand, and excess or insufficient inventories.
- Data management and analytics are mostly restricted to ‘offline work’ and are not integrated within the sales forecasting process. ‘Descriptive analytics’ with its product attribute-based assessment, is one level of information. ‘Predictive analytics’ using more complex algorithms to better predict outcomes, is rarely integrated. Nor is ‘prescriptive analytics’ to provide suggested corrective action.
- Forecasting the sales of ‘eaches’ or items at the SKU level is considerably more difficult than forecasting at the product category level, which occurs in stores or other bulk sales channels. Consumer online buying behavior is centered on multiple factors that are not scientific.
- The demand generation initiatives mentioned above impact online buying significantly but are not evaluated properly in the short-term cycles. Actions such as new products, flash sales, mark-downs, BOGO deals, volume discounts, advertising, and information on social media can influence SKU-level sales almost immediately. This phenomenon contrasts with traditional forecasting plans and methods that project macro demand over 30 to 90 day cycles.
- Markets change rapidly in the eCommerce world where digital technologies are time and data dependent. The traditional S&OP process occurs monthly and reaches decisions that are 90 to 120 days out. The online world needs stock flow planning to be done at least weekly and for two to four week cycles.
The combination of these challenges and their regular reoccurrence cause significant issues for business management. Unless online sales forecasting is a focus of the business separate from traditional demand planning, forecast errors in eCommerce will severely impact operations costs, sales revenues, operating capital, service, and even fixed capital.
The most effective solutions available today for meeting these challenges can be categorized into two classes, process and technology. These can be addressed separately but need to operate congruently.
- The Process
There are three essential components of a process transformation for eCommerce sales forecasting:
- First, the S&OP process must be transformed to include replenishment planning for stores, customers, and fulfillment of online orders. These may operate in different time boxes, however, they both require effective sales forecasts. The monthly S&OP cadence can be retained for traditional channels, but a weekly sub-process must be established for online order fulfillment.
- Second, the process must provide near real-time visibility and integration of the several causal factors like new products, flash sales, discounts, etc. All items should be on a weekly calendar and made visible to anyone involved in demand planning or sales forecasting.
- Third, decision-making must be agile and timely. eCommerce action speeds are five times faster than traditional channels. This means that deep analytical details cannot be inserted into all decisions or choices. One quote addressing this is, “we make decisions for eCommerce sales forecasting on emotion, then justify them later on logic.” While this approach carries risks, the eCommerce world requires that this style of decision-making be in place for high-priority ‘now decisions’.
We recognize that process improvement can be complex and highly dependent on change management. We do highlight the needs for change. Tompkins consulting applies proven methods to design the needed changes for eCommerce, guiding the change management to put them in place and sustain their effectiveness.
- The Technology
Technology is constantly advancing faster than companies or most humans can embrace and use properly. AI is the latest such advancement with astonishing potential for planning both repetitive and periodic actions. Several other new technologies also exist that enable sales forecasting improvements.
We recognize the need for meaningful software that performs at a high level of accuracy in sales forecasting for the retail, wholesale, and brand segments. We also understand the importance of correct advanced algorithms that are user-friendly for ease-of-application. Moreover, such a software needs to be applicable at the SKU-level that can be derived by location and adaptable to multiple demand generation events.
During a wide search for a software platform with these capacities, we discovered Vanguard Software Corporation. Vanguard’s software suite is used at several dozen Fortune corporations and has been recognized by Gartner Associates. We have partnered with Vanguard to configure a platform with these needed eCommerce forecasting capabilities. This solution is Distributed Inventory Flow Forecasting (DIFF).
Below highlights a few of the functions and features of DIFF:
- The DIFF forecasting models are tested and proven by many successful companies that use the system. There are 32 models embedded in the model and the algorithms can be user-defined or automatically determined.
- DIFF forecasts across configurable hierarchies (SKU/category/brand). Some sellers have thousands of SKUs. DIFF initially builds forecasts for each SKU by location then aggregates them to generate higher levels for categories and/or brands. The system can also independently forecast category or brand demands and reconcile them against SKU levels. Forecasts are fully reconciled at each level of aggregation. Aggregations sometime distort forecasts, hiding meaningful projections at the disaggregated level. DIFF reconciles these at each level of roll-up. This is important in eCommerce.
- DIFF applies patterns of demand to new SKUs or supersessions. Similar to macro forecasting, eCommerce demand is influenced by several events that affect the buying of goods. DIFF can readily handle SKU introductions, discontinuations, and supersessions. This happens by overlaying demand patterns from existing SKUs. This helps with seasonality and product life cycles. Product life management systems are helpful in brand management. However, the sales forecasting engine must account for the impact on short-term demand.
- DIFF can also integrate other events that impact demand. These can include competitor actions, promos, flash sales, etc. These may also include supply chain or transportation disruptions and weather events. Integrating these into the forecasts improves accuracy and continuous learning.
- DIFF generates custom replenishment advisory schedules. Most sales forecasting models are disconnected from the supply operations. Therefore, analysts must filter and transfer the demand forecasts in order to buy, re-stock, or replenish inventories at selected locations. The DIFF demand forecasting engine feeds the inventory optimization engine, to maximize the probability of achieving desired fill rates. This is explained in our subsequent article on inventory optimization along with further DIFF optimizing features.
While forecasting for eCommerce sales presents a myriad of challenges, advances in technology have paved the way for new solutions to assist with these processes. The DIFF solution we have applied at MonarchFx to help our sellers deal with the complexities and variability inherent to forecasting online ordering is already showing impressive results.
While this article focuses solely on demand forecasting challenges and solutions, further information on the DIFF solution is available from our MonarchFx technology development team and on our Tompkins consulting capabilities in process reinvention for eCommerce.