Big Data, Knowledge Discovery and Metrics

Variance and significance in large-scale online services

The Unofficial Google Data Science Blog

JANUARY 14, 2016

by AMIR NAJMI Running live experiments on large-scale online services (LSOS) is an important aspect of data science. But the fact that a service could have millions of users and billions of interactions gives rise to both big data and methods which are effective with big data.

Experimentation

Experimentation Statistics Metrics Measurement

LSOS experiments: how I learned to stop worrying and love the variability

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

Variance reduction through conditioning Suppose, as an LSOS experimenter, you find that your key metric varies a lot by country and time of day. And since the metric average is different in each hour of day, this is a source of variation in measuring the experimental effect. Obviously, this doesn’t have to be true.

Experimentation

Experimentation Metrics Statistics Measurement

Data Leaders Brief

Variance and significance in large-scale online services

LSOS experiments: how I learned to stop worrying and love the variability

Webinars

Stay Connected