article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

If $Y$ at that point is (statistically and practically) significantly better than our current operating point, and that point is deemed acceptable, we update the system parameters to this better value. And we can keep repeating this approach, relying on intuition and luck. Why experiment with several parameters concurrently?

article thumbnail

To Balance or Not to Balance?

The Unofficial Google Data Science Blog

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. Identification We now discuss formally the statistical problem of causal inference. We start by describing the problem using standard statistical notation.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Defining data science in 2018

Data Science and Beyond

I got my first data science job in 2012, the year Harvard Business Review announced data scientist to be the sexiest job of the 21st century. Two years later, I published a post on my then-favourite definition of data science , as the intersection between software engineering and statistics. Things have changed considerably since 2012.

article thumbnail

Estimating causal effects using geo experiments

The Unofficial Google Data Science Blog

A geo experiment is an experiment where the experimental units are defined by geographic regions. Statistical power is traditionally given in terms of a probability function, but often a more intuitive way of describing power is by stating the expected precision of our estimates. They are non-overlapping geo-targetable regions.

article thumbnail

Unintentional data

The Unofficial Google Data Science Blog

1]" Statistics, as a discipline, was largely developed in a small data world. More people than ever are using statistical analysis packages and dashboards, explicitly or more often implicitly, to develop and test hypotheses. Data was expensive to gather, and therefore decisions to collect data were generally well-considered.