Remove Definition Remove Experimentation Remove Metrics Remove Statistics
article thumbnail

The Lean Analytics Cycle: Metrics > Hypothesis > Experiment > Act

Occam's Razor

To win in business you need to follow this process: Metrics > Hypothesis > Experiment > Act. We are far too enamored with data collection and reporting the standard metrics we love because others love them because someone else said they were nice so many years ago. That metric is tied to a KPI.

Metrics 156
article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Experiments, Parameters and Models At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Variance and significance in large-scale online services

The Unofficial Google Data Science Blog

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.

article thumbnail

Experimentation and Testing: A Primer

Occam's Razor

This post is a primer on the delightful world of testing and experimentation (A/B, Multivariate, and a new term from me: Experience Testing). Experimentation and testing help us figure out we are wrong, quickly and repeatedly and if you think about it that is a great thing for our customers, and for our employers. Counter claims?

article thumbnail

Knowledge

Occam's Razor

" ~ Web Metrics: "What is a KPI? " + Standard Metrics Revisited Series. "Engagement" Is Not A Metric, It's An Excuse. Defining a "Master Metric", + a Framework to Gain a Competitive Advantage in Web Analytics. Build A Great Web Experimentation & Testing Program.

KPI 124
article thumbnail

Uncertainties: Statistical, Representational, Interventional

The Unofficial Google Data Science Blog

Some of that uncertainty is the result of statistical inference, i.e., using a finite sample of observations for estimation. But there are other kinds of uncertainty, at least as important, that are not statistical in nature. Among these, only statistical uncertainty has formal recognition.

article thumbnail

The Impact Matrix | A Digital Analytics Strategic Framework

Occam's Razor

work (collection, processing, reporting, analysis), processes, org structure, governance models, last-mile gaps , metrics ladders of awesomeness , and… so… much… more. Remember, tools, work, processes, org structure, governance models, last-mile gaps, metrics ladders of awesomeness, and… so… much… more. The Implications of Complexity.