Remove 2015 Remove Experimentation Remove Measurement Remove Statistics
article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Taking measurements at parameter settings further from control parameter settings leads to a lower variance estimate of the slope of the line relating the metric to the parameter.

article thumbnail

Changing assignment weights with time-based confounders

The Unofficial Google Data Science Blog

For example, imagine a fantasy football site is considering displaying advanced player statistics. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. One reason to do ramp-up is to mitigate the risk of never before seen arms.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

To Balance or Not to Balance?

The Unofficial Google Data Science Blog

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. Identification We now discuss formally the statistical problem of causal inference. We start by describing the problem using standard statistical notation.

article thumbnail

The 2015 Digital Marketing Rule Book. Change or Perish.

Occam's Razor

Most companies are astonishingly blasé about data and possibilities of measurement. " Sad, unimaginative measurements of their sad, unimaginative campaigns. AND you can have analysis of your risk in almost real time to get an early read and in a few days with statistical significance! One of my biggest learnings?

Marketing 140
article thumbnail

Estimating causal effects using geo experiments

The Unofficial Google Data Science Blog

It is important that we can measure the effect of these offline conversions as well. Panel studies make it possible to measure user behavior along with the exposure to ads and other online elements. Let's take a look at larger groups of individuals whose aggregate behavior we can measure. days or weeks).

article thumbnail

The trinity of errors in applying confidence intervals: An exploration using Statsmodels

O'Reilly on Data

We develop an ordinary least squares (OLS) linear regression model of equity returns using Statsmodels, a Python statistical package, to illustrate these three error types. CI theory was developed around 1937 by Jerzy Neyman, a mathematician and one of the principal architects of modern statistics.