Meta-experiments: Improving experimentation through experimentation

A/B testing is widexly used in the industry to optimize customer facing websites. Many companies employ experimentation specialists to facilitate and improve the process of A/B testing. Here, we present the application of A/B testing to this improvement effort itself, by running experiments on the e...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-06
Main Author: Müller, Melanie J I
Format: Article
Language:English
Subjects:
Experimentation
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A/B testing is widexly used in the industry to optimize customer facing websites. Many companies employ experimentation specialists to facilitate and improve the process of A/B testing. Here, we present the application of A/B testing to this improvement effort itself, by running experiments on the experimentation process, which we call 'meta-experiments'. We discuss the challenges of this approach using the example of one of our meta-experiments, which helped experimenters to run more sufficiently powered A/B tests. We also point out the benefits of 'dog fooding' for the experimentation specialists when running their own experiments.
ISSN:2331-8422