The Unofficial Google Data Science Blog

APRIL 23, 2024

If $Y$ at that point is (statistically and practically) significantly better than our current operating point, and that point is deemed acceptable, we update the system parameters to this better value. And we can keep repeating this approach, relying on intuition and luck. Why experiment with several parameters concurrently?

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

The Unofficial Google Data Science Blog

JULY 22, 2020

For example, imagine a fantasy football site is considering displaying advanced player statistics. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. One reason to do ramp-up is to mitigate the risk of never before seen arms.

Experimentation 95

Experimentation Statistics Testing Strategy 95

Occam's Razor

JANUARY 9, 2012

AND you can have analysis of your risk in almost real time to get an early read and in a few days with statistical significance! Allocate some of your aforementioned 15% budget to experimentation and testing. If you partake in analysis , let that be at the intersection of custom reports and advanced segments. Got your own?

Marketing 140

Marketing B2B Measurement Risk 140

The Unofficial Google Data Science Blog

MAY 31, 2016

A geo experiment is an experiment where the experimental units are defined by geographic regions. Statistical power is traditionally given in terms of a probability function, but often a more intuitive way of describing power is by stating the expected precision of our estimates. They are non-overlapping geo-targetable regions.

Advertising 42

Advertising Testing Sales Statistics 42