The Unofficial Google Data Science Blog

APRIL 19, 2021

by LEE RICHARDSON & TAYLOR POSPISIL Calibrated models make probabilistic predictions that match real world probabilities. Calibration applies in many applications, and hence the practicing data scientist must understand this useful tool. Why calibration matters What are the consequences of miscalibrated models? a sigmoid).

Modeling 122

Modeling IT Metrics Testing 122

Domino Data Lab

MAY 15, 2019

This Domino Data Science Field Note covers Pete Skomoroch ’s recent Strata London talk. Over the years, I have listened to data scientists and machine learning (ML) researchers relay various pain points and challenges that impede their work. Addressing the Uncertainty that ML Adds to Product Roadmaps.

Machine Learning 111

Machine Learning Management Experimentation Uncertainty 111

CIO Business Intelligence

NOVEMBER 14, 2022

A DSS supports the management, operations, and planning levels of an organization in making better decisions by assessing the significance of uncertainties and the tradeoffs involved in making one decision over another. According to Gartner, the goal is to design, model, align, execute, monitor, and tune decision models and processes.

Data mining 129

Data mining Data-driven Statistics OLAP 129

O'Reilly on Data

OCTOBER 13, 2020

Similarly, in “ Building Machine Learning Powered Applications: Going from Idea to Product ,” Emmanuel Ameisen states: “Indeed, exposing a model to users in production comes with a set of challenges that mirrors the ones that come with debugging a model.”. Debugging AI Products.

Management 244

Management Metrics Machine Learning Modeling 244

IBM Big Data Hub

APRIL 18, 2024

While these large language model (LLM) technologies might seem like it sometimes, it’s important to understand that they are not the thinking machines promised by science fiction. Most experts categorize it as a powerful, but narrow AI model. A key trend is the adoption of multiple models in production.

Cost-Benefit 94

Cost-Benefit Modeling Manufacturing Interactive 94

CIO Business Intelligence

JUNE 28, 2022

Philosophers and economists may argue about the quality of the metaphor, but there’s no doubt that organizing and analyzing data is a vital endeavor for any enterprise looking to deliver on the promise of data-driven decision-making. And to do so, a solid data management strategy is key.

Management 132

Management Internet of Things Statistics Data-driven 132

Data Science and Beyond

JANUARY 7, 2019

Therefore, bootstrapping has been promoted as an easy way of modelling uncertainty to hackers who don’t have much statistical knowledge. Confidence intervals are a common way of quantifying the uncertainty in an estimate of a population parameter. Don’t compare confidence intervals visually.

Statistics 77

Statistics Uncertainty Testing Modeling 77

Insight

FEBRUARY 19, 2020

Using variability in machine learning predictions as a proxy for risk can help studio executives and producers decide whether or not to green light a film project Photo by Kyle Smith on Unsplash Originally posted on Toward Data Science. I held out 20% of this as a test set and used the remainder for training and validation.

Risk 67

Risk ROI Modeling Metrics 67

The Unofficial Google Data Science Blog

MARCH 31, 2016

KUEHNEL, and ALI NASIRI AMINI In this post, we give a brief introduction to random effects models, and discuss some of their uses. Through simulation we illustrate issues with model fitting techniques that depend on matrix factorization. Random effects models are a useful tool for both exploratory analyses and prediction problems.

Modeling 40

Modeling Statistics Advertising Testing 40

The Unofficial Google Data Science Blog

JULY 18, 2023

Editor's note : The relationship between reliability and validity are somewhat analogous to that between the notions of statistical uncertainty and representational uncertainty introduced in an earlier post. But for more complicated metrics like xRR, our preference is to bootstrap when measuring uncertainty.

Measurement 52

Measurement Metrics Uncertainty Slice and Dice 52

The Unofficial Google Data Science Blog

JULY 22, 2020

Another reason to use ramp-up is to test if a website's infrastructure can handle deploying a new arm to all of its users. The website wants to make sure they have the infrastructure to handle the feature while testing if engagement increases enough to justify the infrastructure. We offer two examples where this may be the case.

Experimentation 95

Experimentation Statistics Testing Strategy 95

Domino Data Lab

JUNE 23, 2019

As a result, Skomoroch advocates getting “designers and data scientists, machine learning folks together and using real data and prototyping and testing” as quickly as possible. They lack probably what a lot of folks in this room have in terms of domain expertise around data, machine learning, and how these models work.

Management 110

Management Machine Learning Experimentation Metrics 110

The Unofficial Google Data Science Blog

JANUARY 14, 2016

by AMIR NAJMI Running live experiments on large-scale online services (LSOS) is an important aspect of data science. We must therefore maintain statistical rigor in quantifying experimental uncertainty. In this post we explore how and why we can be “ data-rich but information-poor ”.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

The Unofficial Google Data Science Blog

MAY 31, 2016

Similarly, we could test the effectiveness of a search ad compared to showing only organic search results. Structure of a geo experiment A typical geo experiment consists of two distinct time periods: pretest and test. After the test period finishes, the campaigns in the treatment group are reset to their original configurations.

Advertising 42

Advertising Testing Sales Statistics 42

The Unofficial Google Data Science Blog

APRIL 16, 2019

Modern products often don't start with set-in-stone business models because starting and scaling costs are low. Such decisions involve an actual hypothesis test on specific metrics (e.g. This is a good use of existing experiment infrastructure, but it is not really an experiment as in hypothesis testing.

Experimentation 68

Experimentation Sales Metrics Measurement 68

The Unofficial Google Data Science Blog

OCTOBER 21, 2015

Our post describes how we arrived at recent changes to design principles for the Google search page, and thus highlights aspects of a data scientist’s role which involve practicing the scientific method. There has been debate as to whether the term “data science” is necessary. Some don’t see the point.

Slice and Dice 45

Slice and Dice Experimentation Data-driven Data Science 45

Cloudera

NOVEMBER 18, 2021

By adopting a custom developed application based on the Cloudera ecosystem, Carrefour has combined the legacy systems into one platform which provides access to customer data in a single data lake. Learn more about the Cloudera Data Impact Awards and see past winners!

Data Lake 74

Data Lake Cost-Benefit Digital Transformation Risk 74