Online Display Advertising Causal Attribution and Evaluation

Author(s): Barajas Zamora, Joel | Advisor(s): Akella, Ram | Abstract: The allocation of a given budget to online display advertising as a marketing channel has motivated the development of statistical methods to measure its effectiveness. Recent studies show that display advertising often triggers online users to search for more information on products. Eventually, many of these users convert at the advertiser’s website. A key challenge is to measure the effectiveness of display advertising when users are exposed to multiple unknown advertising channels.We develop a time series approach based on Dynamic Linear Models (DLM), to estimate the impact of ad impressions on the daily number of commercial actions when no user tracking is possible. This method uses aggregate observational data post-campaign and does not require an experimental set-up. We incorporate persistence of campaign effects and account for outliers in the time series without pre-defined thresholds. We analyze user conversions for 2,885 campaigns and 1,251 products during six months for model selection.The current industry practice measures the campaign causal effect on online conversions by running a randomized experiment focused on the ad exposures (using placebo ads). This method ignores other campaign components, including user targeting in marketplaces with competitor effects. We propose a novel randomized design to estimate campaign and ad attribution in marketplaces. We determine the effect on the targeted users using the Potential Outcomes Causal Model and Principal Stratification. We analyze the impact of 2 performance-based (CPA) and 1 Cost-Per-Impression (CPM) campaigns with 20M+ users for each campaign. We estimate a non-zero CPM campaign presence in the marketplace effect (currently ignored by industry). Evidence suggests that CPA campaigns incentivize targeting of users who buy regardless of the ad (always-buyers).We propose a user targeting simulator that leverages data from campaign randomized experiments. Based on the response of 37 million visiting users (targeted and non-targeted) and their demographic user features, we simulate different user targeting policies. We provide evidence that the standard conversion optimization policy shows similar effectiveness to that of uniform targeting, and is significantly inferior to other causally optimized targeting policies. These results challenge the standard practice of targeting users with the highest conversion probability.To guide the user targeting to optimize causally generated conversions, we analyze the campaign on the conversion probability of the users who click on the ad as a behavioral feature. We show that designing a randomized experiment to evaluate this effect is infeasible, and propose a method to estimate the local effect on the clicker conversions. Based on two large-scale randomized experiments, performed for 7.16 and 22.7 million users, a pessimistic analysis shows a minimum increase of the effect on the clicker conversion probability of 75% with respect to the non-clickers. This evidence contradicts a recent belief that clicks are not indicative of campaign success.