article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

It is also a sound strategy when experimenting with several parameters at the same time. To find optimal values of two parameters experimentally, the obvious strategy would be to experiment with and update them in separate, sequential stages. (And sometimes even if it is not[1].)

article thumbnail

To Balance or Not to Balance?

The Unofficial Google Data Science Blog

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. In fact, Hainmueller (2012) show that entropy balancing is equivalent to estimating the weights as a log-linear model of the covariate functions $c_j(X)$. 2012): 25-46.

article thumbnail

Unintentional data

The Unofficial Google Data Science Blog

We data scientists now have access to tools that allow us to run a large numbers of experiments, and then to slice experimental populations by any combination of dimensions collected. Make experimentation cheap and understand the cost of bad decisions. This leads to the proliferation of post hoc hypotheses. Consider your loss function.