article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Experiments, Parameters and Models At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well.

article thumbnail

To Balance or Not to Balance?

The Unofficial Google Data Science Blog

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. In fact, Hainmueller (2012) show that entropy balancing is equivalent to estimating the weights as a log-linear model of the covariate functions $c_j(X)$.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Estimating causal effects using geo experiments

The Unofficial Google Data Science Blog

A geo experiment is an experiment where the experimental units are defined by geographic regions. This means it is possible to specify exactly in which geos an ad campaign will be served – and to observe the ad spend and the response metric at the geo level. They are non-overlapping geo-targetable regions. by turning campaigns off).

article thumbnail

Unintentional data

The Unofficial Google Data Science Blog

With more features come more potential post hoc hypotheses about what is driving metrics of interest, and more opportunity for exploratory analysis. We data scientists now have access to tools that allow us to run a large numbers of experiments, and then to slice experimental populations by any combination of dimensions collected.