Remove 2007 Remove Experimentation Remove Risk Remove Uncertainty
article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

Crucially, it takes into account the uncertainty inherent in our experiments. To find optimal values of two parameters experimentally, the obvious strategy would be to experiment with and update them in separate, sequential stages. In this section we’ll discuss how we approach these two kinds of uncertainty with QCQP.

article thumbnail

Changing assignment weights with time-based confounders

The Unofficial Google Data Science Blog

One reason to do ramp-up is to mitigate the risk of never before seen arms. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. For example, imagine a fantasy football site is considering displaying advanced player statistics.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

The trinity of errors in applying confidence intervals: An exploration using Statsmodels

O'Reilly on Data

Because of this trifecta of errors, we need dynamic models that quantify the uncertainty inherent in our financial estimates and predictions. Practitioners in all social sciences, especially financial economics, use confidence intervals to quantify the uncertainty in their estimates and predictions. and an error term ??

article thumbnail

Estimating causal effects using geo experiments

The Unofficial Google Data Science Blog

A geo experiment is an experiment where the experimental units are defined by geographic regions. The expected precision of our inferences can be computed by simulating possible experimental outcomes. Further, there is the risk that the increased ad spend will be less productive due to diminishing returns (e.g.,