The Unofficial Google Data Science Blog

JULY 22, 2020

One reason to do ramp-up is to mitigate the risk of never before seen arms. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. For example, imagine a fantasy football site is considering displaying advanced player statistics.

Experimentation 95

Experimentation Statistics Testing Strategy 95

Occam's Razor

MARCH 23, 2010

For big success you'll need to have a Multiplicity strategy: So when you step back and realize at the minimum you'll also have to use one Voice of Customer tool (for qualitative analysis), one Experimentation tool and (if you want to be great) one Competitive Intelligence tool… do you still want to have two clickstream tools?

Analytics 118

Analytics Consulting Metrics Experimentation 118

The Unofficial Google Data Science Blog

OCTOBER 12, 2017

Yet when we use these tools to explore data and look for anomalies or interesting features, we are implicitly formulating and testing hypotheses after we have observed the outcomes. We must correct for multiple hypothesis tests. Make experimentation cheap and understand the cost of bad decisions. We ought not dredge our data.

Experimentation 61

Experimentation Testing Statistics Metrics 61