Remove 2007 Remove Optimization Remove Statistics Remove Testing
article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

If $Y$ at that point is (statistically and practically) significantly better than our current operating point, and that point is deemed acceptable, we update the system parameters to this better value. In isolation, the $x_1$-system is optimal: changing $x_1$ and leaving the $x_2$ at 0 will decrease system performance.

article thumbnail

To Balance or Not to Balance?

The Unofficial Google Data Science Blog

A naïve way to solve this problem would be to compare the proportion of buyers between the exposed and unexposed groups, using a simple test for equality of means. Identification We now discuss formally the statistical problem of causal inference. We start by describing the problem using standard statistical notation.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Measuring Incrementality: Controlled Experiments to the Rescue!

Occam's Razor

We have to do Search Engine Optimization. You need people with deep skills in Scientific Method , Design of Experiments , and Statistical Analysis. Then they isolated regions of the country (by city, zip, state, dma pick your fave) into test and control regions. The nice thing is that you can also test that!

article thumbnail

Knowledge

Occam's Razor

The Awesome Power of Visualization 2 -> Death and Taxes 2007. Five Reasons And Awesome Testing Ideas. Lab Usability Testing: What, Why, How Much. Build A Great Web Experimentation & Testing Program. Experimentation and Testing: A Primer. Tip #9: Leverage Statistical Control Limits. Got Surveys?

KPI 124
article thumbnail

The Lean Analytics Cycle: Metrics > Hypothesis > Experiment > Act

Occam's Razor

Sometimes, we escape the clutches of this sub optimal existence and do pick good metrics or engage in simple A/B testing. You're choosing only one metric because you want to optimize it. Testing out a new feature. Identify, hypothesize, test, react. You don’t need a beautiful beast to go out and test.

Metrics 156
article thumbnail

Changing assignment weights with time-based confounders

The Unofficial Google Data Science Blog

For example, imagine a fantasy football site is considering displaying advanced player statistics. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. We offer two examples where this may be the case.

article thumbnail

Time Series with R

Domino Data Lab

A big part of statistics, particularly for financial and econometric data, is analyzing time series, data that are autocorrelated over time. Fortunately, the forecast package has a number of functions to make working with time series data easier, including determining the optimal number of diffs. 2007-01-04 34.50 2007-01-05 33.96