Rocket-Powered Data Science

FEBRUARY 15, 2023

3) How do we get started, when, who will be involved, and what are the targeted benefits, results, outcomes, and consequences (including risks)? Those F’s are: Fragility, Friction, and FUD (Fear, Uncertainty, Doubt). These changes may include requirements drift, data drift, model drift, or concept drift. Test early and often.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

Domino Data Lab

MAY 15, 2019

This Domino Data Science Field Note covers Pete Skomoroch ’s recent Strata London talk. Pete indicates, in both his November 2018 and Strata London talks, that ML requires a more experimental approach than traditional software engineering. These steps also reflect the experimental nature of ML product management.

Machine Learning 111

Machine Learning Management Experimentation Uncertainty 111

The Unofficial Google Data Science Blog

APRIL 23, 2024

Crucially, it takes into account the uncertainty inherent in our experiments. Experiments, Parameters and Models At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well. It is a big picture approach, worthy of your consideration.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

IBM Big Data Hub

APRIL 18, 2024

While leaders have some reservations about the benefits of current AI, organizations are actively investing in gen AI deployment, significantly increasing budgets, expanding use cases, and transitioning projects from experimentation to production. The AGI would need to handle uncertainty and make decisions with incomplete information.

Cost-Benefit 93

Cost-Benefit Modeling Manufacturing Interactive 93

The Unofficial Google Data Science Blog

JANUARY 14, 2016

by AMIR NAJMI Running live experiments on large-scale online services (LSOS) is an important aspect of data science. We must therefore maintain statistical rigor in quantifying experimental uncertainty. In this post we explore how and why we can be “ data-rich but information-poor ”.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

The Unofficial Google Data Science Blog

JULY 22, 2020

One reason to do ramp-up is to mitigate the risk of never before seen arms. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. An option that can sometimes work is to use only the data from users assigned in the final epoch.

Experimentation 95

Experimentation Statistics Testing Strategy 95

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. They have the foundations of data infrastructure. Yet, this challenge is not insurmountable.

Management 110

Management Machine Learning Experimentation Metrics 110