The Unofficial Google Data Science Blog

JANUARY 14, 2016

by AMIR NAJMI Running live experiments on large-scale online services (LSOS) is an important aspect of data science. We must therefore maintain statistical rigor in quantifying experimental uncertainty. In this post we explore how and why we can be “ data-rich but information-poor ”.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

The Unofficial Google Data Science Blog

JULY 22, 2020

For this reason we don’t report uncertainty measures or statistical significance in the results of the simulation. From a Bayesian perspective, one can combine joint posterior samples for $E[Y_i | T_i=t, E_i=j]$ and $P(E_i=j)$, which provides a measure of uncertainty around the estimate. 2] Scott, Steven L. 2015): 37-45. [3]

Experimentation 95

Experimentation Statistics Testing Strategy 95

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

The result is that experimenters can’t afford to be sloppy about quantifying uncertainty. These typically result in smaller estimation uncertainty and tighter interval estimates. We previously went into some detail as to why observations in an LSOS have particularly high coefficient of variation (CV).

Experimentation 40

Experimentation Metrics Statistics Measurement 40