Effective Interpretation of Bucket Testing Results through Big Data Analytics

Bucket testing is a common practice in the internet industry, where new features and services are tested by exposing them to a randomly selected small subset of users. However, in this simple version of bucket testing, since a very small fraction of the total users are selected through uniform indep...

Full description

Saved in:
Bibliographic Details
Main Authors: Das, Ariyam, Ranganath, Harish Siddapura
Format: Conference Proceeding
Language:English
Subjects:
big data
bucket testing
classification
sampling
user attributes
Data handling
Data storage systems
Information management
Measurement
Sociology
Statistics
Testing
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Bucket testing is a common practice in the internet industry, where new features and services are tested by exposing them to a randomly selected small subset of users. However, in this simple version of bucket testing, since a very small fraction of the total users are selected through uniform independent sampling of the population, the samples chosen, at times, do not adequately serve as a reasonable statistical proxy for the total population. This may lead to erroneous interpretation of the bucket testing results, particularly for online sites having large audiences with varying demographics and preferences. In this work, we present a novel algorithmic framework that addresses this challenge and provides an efficient and more accurate interpretation of the bucket testing results by analyzing the big audience data and factoring in the nature of the overall population in terms of the different user attributes. We demonstrate the effectiveness of our algorithm through the data obtained from real experiments conducted on Yahoo's bucket testing platform.
ISSN:2379-7703
DOI:10.1109/BigData.Congress.2013.79