Finance, Knowledge Discovery and Metrics

Variance and significance in large-scale online services

The Unofficial Google Data Science Blog

JANUARY 14, 2016

The LSOS may do this by exposing a random group of users to the new design and compare them to a control group, and then analyze the effect on important user engagement metrics, such as bounce rate, time to first action, or number of experiences deemed positive. In addition to a suitable metric, we must also choose our experimental unit.

Experimentation

Experimentation Statistics Metrics Measurement

LSOS experiments: how I learned to stop worrying and love the variability

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

Variance reduction through conditioning Suppose, as an LSOS experimenter, you find that your key metric varies a lot by country and time of day. And since the metric average is different in each hour of day, this is a source of variation in measuring the experimental effect. Obviously, this doesn’t have to be true.

Experimentation

Experimentation Metrics Statistics Measurement

Data Leaders Brief

Variance and significance in large-scale online services

LSOS experiments: how I learned to stop worrying and love the variability

Webinars

Stay Connected