Remove Experimentation Remove Reference Remove Statistics Remove Uncertainty
article thumbnail

What you need to know about product management for AI

O'Reilly on Data

All you need to know for now is that machine learning uses statistical techniques to give computer systems the ability to “learn” by being trained on existing data. Machine learning adds uncertainty. Underneath this uncertainty lies further uncertainty in the development process itself.

article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

If $Y$ at that point is (statistically and practically) significantly better than our current operating point, and that point is deemed acceptable, we update the system parameters to this better value. Crucially, it takes into account the uncertainty inherent in our experiments. Why experiment with several parameters concurrently?

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Variance and significance in large-scale online services

The Unofficial Google Data Science Blog

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.

article thumbnail

Getting ready for artificial general intelligence with examples

IBM Big Data Hub

AGI, sometimes referred to as strong AI , is the science-fiction version of artificial intelligence (AI), where artificial machine intelligence achieves human-level learning, perception and cognitive flexibility. The AGI would need to handle uncertainty and make decisions with incomplete information.

article thumbnail

Changing assignment weights with time-based confounders

The Unofficial Google Data Science Blog

This post considers a common design for an OCE where a user may be randomly assigned an arm on their first visit during the experiment, with assignment weights referring to the proportion that are randomly assigned to each arm. For example, imagine a fantasy football site is considering displaying advanced player statistics.

article thumbnail

The Lean Analytics Cycle: Metrics > Hypothesis > Experiment > Act

Occam's Razor

The word hypothesis means a lot of different things, but in this context I like this definition from Wikipedia the best: People refer to a trial solution to a problem as a hypothesis, often called an "educated guess”, because it provides a suggested solution based on the evidence. The result? The graph is impressive, right?

Metrics 156
article thumbnail

LSOS experiments: how I learned to stop worrying and love the variability

The Unofficial Google Data Science Blog

In this post we explore why some standard statistical techniques to reduce variance are often ineffective in this “data-rich, information-poor” realm. Despite a very large number of experimental units, the experiments conducted by LSOS cannot presume statistical significance of all effects they deem practically significant.