article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

To find optimal values of two parameters experimentally, the obvious strategy would be to experiment with and update them in separate, sequential stages. Our experimentation platform supports this kind of grouped-experiments analysis, which allows us to see rough summaries of our designed experiments without much work.

article thumbnail

To Balance or Not to Balance?

The Unofficial Google Data Science Blog

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. In fact, Hainmueller (2012) show that entropy balancing is equivalent to estimating the weights as a log-linear model of the covariate functions $c_j(X)$. 2012): 25-46.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Estimating causal effects using geo experiments

The Unofficial Google Data Science Blog

A geo experiment is an experiment where the experimental units are defined by geographic regions. The expected precision of our inferences can be computed by simulating possible experimental outcomes. Further, there is the risk that the increased ad spend will be less productive due to diminishing returns (e.g.,

article thumbnail

Unintentional data

The Unofficial Google Data Science Blog

We data scientists now have access to tools that allow us to run a large numbers of experiments, and then to slice experimental populations by any combination of dimensions collected. Make experimentation cheap and understand the cost of bad decisions. This leads to the proliferation of post hoc hypotheses.