A multi-cell experimental design to recover policy relevant treatment effects, with an application to online advertising

Experiments are an important tool to measure the impacts of interventions. However, in experimental settings with one-sided noncompliance, extant empirical approaches may not produce the estimands a decision-maker needs to solve their problem. For example, these experimental designs are common in di...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-04
Main Authors: Waisman, Caio, Gordon, Brett R
Format: Article
Language:English
Subjects:
Advertising
Decision making
Design of experiments
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Experiments are an important tool to measure the impacts of interventions. However, in experimental settings with one-sided noncompliance, extant empirical approaches may not produce the estimands a decision-maker needs to solve their problem. For example, these experimental designs are common in digital advertising settings, but typical methods do not yield effects that inform the intensive margin -- how much should be spent or how many consumers should be reached with a campaign. We propose a solution that combines a novel multi-cell experimental design with modern estimation techniques that enables decision-makers to recover enough information to solve problems with an intensive margin. Our design is straightforward to implement. Using data from advertising experiments at Facebook, we demonstrate our approach outperforms standard techniques in recovering treatment effect parameters. Through a simple advertising reach decision problem, we show that our approach generates better decisions relative to standard techniques.
ISSN:2331-8422