The Unofficial Google Data Science Blog

OCTOBER 7, 2015

A/B testing is used widely in information technology companies to guide product development and improvements. accounting for effects "orthogonal" to the randomization used in experimentation. The first thing you’ll want to do is to run your test for a long time with fixed experimental units, in our case cookies.

Modeling 61

Modeling Experimentation Knowledge Discovery Testing 61

The Unofficial Google Data Science Blog

JANUARY 14, 2016

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. Because individual observations have so little information, statistical significance remains important to assess.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

The Unofficial Google Data Science Blog

JULY 22, 2020

Instead, we focus on the case where an experimenter has decided to run a full traffic ramp-up experiment and wants to use the data from all of the epochs in the analysis. When there are changing assignment weights and time-based confounders, this complication must be considered either in the analysis or the experimental design.

Experimentation 95

Experimentation Statistics Testing Strategy 95

Ontotext

MAY 18, 2020

Ever since Hippocrates founded his school of medicine in ancient Greece some 2,500 years ago, writes Hannah Fry in her book Hello World: Being Human in the Age of Algorithms , what has been fundamental to healthcare (as she calls it “the fight to keep us healthy”) was observation, experimentation and the analysis of data.

Knowledge Discovery 72

Knowledge Discovery Experimentation Data-driven Metadata 72

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

In this post we explore why some standard statistical techniques to reduce variance are often ineffective in this “data-rich, information-poor” realm. Despite a very large number of experimental units, the experiments conducted by LSOS cannot presume statistical significance of all effects they deem practically significant.

Experimentation 40

Experimentation Metrics Statistics Measurement 40