A High-Dimensional Choice Model for Online Retailing

Online retailers are facing an increasing variety of product choices and diversified consumer decision journeys. To improve many operations decisions for online retailers, such as demand forecasting and price management, an important first step is to achieve a realistic understanding of the substitution patterns among a large number of products offered in the complex online environment. Although classic choice models offer an elegant framework for estimating substitution patterns among competing options, they have very limited applicability and performance in complex settings with many products. We provide a solution by developing a high-dimensional choice model that not only scales up more easily but also allows for flexible substitution patterns. We leverage consumer clickstream data, and combine econometric and machine learning (graphical lasso, in particular) methods to learn the substitution patterns among a large number of products. We show our method offers significantly better in- and out-of-sample demand forecasts as well as cross-product elasticities in various synthetic datasets and in a real-world empirical setting. For example, in both settings, our model reduces the out-of-sample mean absolute percentage error (MAPE) by approximately 30%--40% compared to classical models (e.g., the IID and the random coefficient Probit models). More importantly, our method can be used to enhance many business decisions such as assortment planning, inventory management and pricing decisions. In particular, we illustrate with a counter-factual pricing experiment that our model recommends better price points which increase total revenue by more than 5% compared to classical models.