The Unofficial Google Data Science Blog

APRIL 23, 2024

Crucially, it takes into account the uncertainty inherent in our experiments. Experiments, Parameters and Models At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well. The data showed us that metrics are not exactly straight-line functions of parameters.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

Rocket-Powered Data Science

FEBRUARY 15, 2023

Third, any commitment to a disruptive technology (including data-intensive and AI implementations) must start with a business strategy. Those F’s are: Fragility, Friction, and FUD (Fear, Uncertainty, Doubt). These changes may include requirements drift, data drift, model drift, or concept drift.

Strategy 289

Strategy Experimentation Uncertainty Machine Learning 289

The Unofficial Google Data Science Blog

DECEMBER 14, 2021

by AMIR NAJMI & MUKUND SUNDARARAJAN Data science is about decision making under uncertainty. Some of that uncertainty is the result of statistical inference, i.e., using a finite sample of observations for estimation. But there are other kinds of uncertainty, at least as important, that are not statistical in nature.

Uncertainty 56

Uncertainty Statistics Measurement Cost-Benefit 56

O'Reilly on Data

OCTOBER 13, 2020

This distinction assumes a slightly different definition of debugging than is often used in software development. From a technical perspective, it is entirely possible for ML systems to function on wildly different data. Proper AI product monitoring is essential to this outcome. I/O validation.

Management 244

Management Metrics Machine Learning Modeling 244

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Without large amounts of labeled training data solving most AI problems is not possible.

Management 110

Management Machine Learning Experimentation Metrics 110

Occam's Razor

APRIL 8, 2013

We are far too enamored with data collection and reporting the standard metrics we love because others love them because someone else said they were nice so many years ago. It helps you to amplify what’s proven to work, throw away what isn’t, and tweak the goal-posts when data indicates that they may be in the wrong place.

Metrics 156

Metrics KPI Analytics Key Performance Indicator 156

The Unofficial Google Data Science Blog

OCTOBER 21, 2015

by NIALL CARDIN, OMKAR MURALIDHARAN, and AMIR NAJMI When working with complex systems or phenomena, the data scientist must often operate with incomplete and provisional understanding, even as she works to advance the state of knowledge. There has been debate as to whether the term “data science” is necessary. Some don’t see the point.

Slice and Dice 45

Slice and Dice Experimentation Data-driven Data Science 45