Finance, Knowledge Discovery, Metrics and Statistics

Variance and significance in large-scale online services

The Unofficial Google Data Science Blog

JANUARY 14, 2016

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. Because individual observations have so little information, statistical significance remains important to assess.

Experimentation

Experimentation Statistics Metrics Measurement

LSOS experiments: how I learned to stop worrying and love the variability

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

In this post we explore why some standard statistical techniques to reduce variance are often ineffective in this “data-rich, information-poor” realm. Despite a very large number of experimental units, the experiments conducted by LSOS cannot presume statistical significance of all effects they deem practically significant.

Experimentation

Experimentation Metrics Statistics Measurement

Data Leaders Brief

Variance and significance in large-scale online services

LSOS experiments: how I learned to stop worrying and love the variability

Webinars

Stay Connected