Remove 2012 Remove Experimentation Remove Modeling Remove Risk
article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

Experiments, Parameters and Models At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well. That is true generally, not just in these experiments — spreading measurements out is generally better, if the straight-line model is a priori correct.

article thumbnail

To Balance or Not to Balance?

The Unofficial Google Data Science Blog

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. The choice of space $cal F$ (sometimes called the model ) and loss function $L$ explicitly defines the estimation problem. This is often referred to as the positivity assumption.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Estimating causal effects using geo experiments

The Unofficial Google Data Science Blog

A geo experiment is an experiment where the experimental units are defined by geographic regions. The expected precision of our inferences can be computed by simulating possible experimental outcomes. Further, there is the risk that the increased ad spend will be less productive due to diminishing returns (e.g.,

article thumbnail

Unintentional data

The Unofficial Google Data Science Blog

We data scientists now have access to tools that allow us to run a large numbers of experiments, and then to slice experimental populations by any combination of dimensions collected. Make experimentation cheap and understand the cost of bad decisions. This leads to the proliferation of post hoc hypotheses.