article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

If $Y$ at that point is (statistically and practically) significantly better than our current operating point, and that point is deemed acceptable, we update the system parameters to this better value. Crucially, it takes into account the uncertainty inherent in our experiments. Why experiment with several parameters concurrently?

article thumbnail

Uncertainties: Statistical, Representational, Interventional

The Unofficial Google Data Science Blog

by AMIR NAJMI & MUKUND SUNDARARAJAN Data science is about decision making under uncertainty. Some of that uncertainty is the result of statistical inference, i.e., using a finite sample of observations for estimation. But there are other kinds of uncertainty, at least as important, that are not statistical in nature.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

What you need to know about product management for AI

O'Reilly on Data

All you need to know for now is that machine learning uses statistical techniques to give computer systems the ability to “learn” by being trained on existing data. Machine learning adds uncertainty. Underneath this uncertainty lies further uncertainty in the development process itself.

article thumbnail

Changing assignment weights with time-based confounders

The Unofficial Google Data Science Blog

For example, imagine a fantasy football site is considering displaying advanced player statistics. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. One reason to do ramp-up is to mitigate the risk of never before seen arms.

article thumbnail

13 IT resolutions for 2024

CIO Business Intelligence

CIOs are readying for another demanding year, anticipating that artificial intelligence, economic uncertainty, business demands, and expectations for ever-increasing levels of speed will all be in play for 2024. He plans to scale his company’s experimental generative AI initiatives “and evolve into an AI-native enterprise” in 2024.

IT 144
article thumbnail

Belcorp reimagines R&D with AI

CIO Business Intelligence

These circumstances have induced uncertainty across our entire business value chain,” says Venkat Gopalan, chief digital, data and technology officer, Belcorp. “As To address the challenges, the company has leveraged a combination of computer vision, neural networks, NLP, and fuzzy logic.

article thumbnail

Variance and significance in large-scale online services

The Unofficial Google Data Science Blog

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.