Rocket-Powered Data Science

FEBRUARY 15, 2023

Third, any commitment to a disruptive technology (including data-intensive and AI implementations) must start with a business strategy. Those F’s are: Fragility, Friction, and FUD (Fear, Uncertainty, Doubt). These changes may include requirements drift, data drift, model drift, or concept drift.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

O'Reilly on Data

MARCH 31, 2020

AI products are automated systems that collect and learn from data to make user-facing decisions. All you need to know for now is that machine learning uses statistical techniques to give computer systems the ability to “learn” by being trained on existing data. Machine learning adds uncertainty.

Management 332

Management Machine Learning Experimentation Metrics 332

The Unofficial Google Data Science Blog

APRIL 23, 2024

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Crucially, it takes into account the uncertainty inherent in our experiments. Figure 2: Spreading measurements out makes estimates of model (slope of line) more accurate.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

The Unofficial Google Data Science Blog

DECEMBER 14, 2021

by AMIR NAJMI & MUKUND SUNDARARAJAN Data science is about decision making under uncertainty. Some of that uncertainty is the result of statistical inference, i.e., using a finite sample of observations for estimation. But there are other kinds of uncertainty, at least as important, that are not statistical in nature.

Uncertainty 56

Uncertainty Statistics Measurement Cost-Benefit 56

Occam's Razor

APRIL 8, 2013

We are far too enamored with data collection and reporting the standard metrics we love because others love them because someone else said they were nice so many years ago. First, you figure out what you want to improve; then you create an experiment; then you run the experiment; then you measure the results and decide what to do.

Metrics 156

Metrics KPI Analytics Key Performance Indicator 156

O'Reilly on Data

OCTOBER 13, 2020

From a technical perspective, it is entirely possible for ML systems to function on wildly different data. For example, you can ask an ML model to make an inference on data taken from a distribution very different from what it was trained on—but that, of course, results in unpredictable and often undesired performance. I/O validation.

Management 244

Management Metrics Machine Learning Modeling 244

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Without large amounts of labeled training data solving most AI problems is not possible.

Management 110

Management Machine Learning Experimentation Metrics 110