Remove 2012 Remove Experimentation Remove Measurement Remove Optimization
article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Taking measurements at parameter settings further from control parameter settings leads to a lower variance estimate of the slope of the line relating the metric to the parameter.

article thumbnail

To Balance or Not to Balance?

The Unofficial Google Data Science Blog

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. It should be noted that inverse probability weighting is not generally optimal (i.e., This is often referred to as the positivity assumption. the curse of dimensionality).

article thumbnail

Unintentional data

The Unofficial Google Data Science Blog

We data scientists now have access to tools that allow us to run a large numbers of experiments, and then to slice experimental populations by any combination of dimensions collected. Make experimentation cheap and understand the cost of bad decisions. This leads to the proliferation of post hoc hypotheses. What is to be done?