Measuring the Return to Online Advertising: Estimation and Inference of Endogenous Treatment Effects

In this paper we aim to conduct inference on the “lift” effect generated by an online advertisement display: specifically we want to analyze if the presence of the brand ad among the advertisements on the page increases the overall number of consumer clicks on that page. A distinctive feature of online advertising is that the ad displays are highly targeted—the advertising platform evaluates the (unconditional) probability of each consumer clicking on a given ad, which leads to a higher probability of displaying the ads that have a higher a priori estimated probability of click. As a result, inferring thecausal effect of the ad display on the page clicks by a given consumer from typical observational data is difficult. To address this we propose a multi-step estimator that focuses on the tails of the consumer distribution to estimate the true causal effect of an ad display. This “identification at infinity” approach alleviates the need for independent experimental randomization but results in nonstandard asymptotic theory, motivating our novel inference method. To validate our results, we use a set of large-scale randomized controlled experiments that Microsoft has run on its advertising platform. Our dataset has a large number of observations and a large number of variables and we employ LASSO to perform variable selection. Providing a basis for comparison with our estimates, we use a study conducted by Microsoft with approximately 9.3 million search sessions focusing on consumer click behavior across search result pages of a major search engine. Randomized experiments indicate that displaying a brand advertisement increases the probability of visiting the advertiser’s website by about 2.27 percentage points relative to a baseline visit rate of roughly 78 percent. Our non-experimental estimates exhibit broadly similar patterns to those obtained from randomized controlled trials, suggesting that the proposed observational estimator can recover qualitatively comparable treatment effects in large-scale advertising data.