The Unofficial Google Data Science Blog

JANUARY 14, 2016

by AMIR NAJMI Running live experiments on large-scale online services (LSOS) is an important aspect of data science. We must therefore maintain statistical rigor in quantifying experimental uncertainty. In this post we explore how and why we can be “ data-rich but information-poor ”.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

Cloudera

APRIL 9, 2021

To effectively leverage their predictive capabilities and maximize time-to-value these companies need an ML infrastructure that allows them to quickly move models from data pipelines, to experimentation and into the business. A/B testing). model packaging, deployment and serving. model monitoring. The ML Use Case.

Machine Learning 98

Machine Learning Forecasting Manufacturing Predictive Analytics 98

Sisense

JANUARY 21, 2020

One of the simplest ways to start exploring your data is to aggregate the metrics you are interested in by their relevant dimensions. In general, it is not possible to give a rule of thumb about when data should be partitioned or combined. A new drug promising to reduce the risk of heart attack was tested with two groups.

Testing 104

Testing Risk Data-driven Statistics 104

The Unofficial Google Data Science Blog

OCTOBER 7, 2015

A/B testing is used widely in information technology companies to guide product development and improvements. For questions as disparate as website design and UI, prediction algorithms, or user flows within apps, live traffic tests help developers understand what works well for users and the business, and what doesn’t.

Modeling 61

Modeling Experimentation Knowledge Discovery Testing 61

Cloudera

FEBRUARY 16, 2022

In the case of CDP Public Cloud, this includes virtual networking constructs and the data lake as provided by a combination of a Cloudera Shared Data Experience (SDX) and the underlying cloud storage. Each project consists of a declarative series of steps or operations that define the data science workflow.

Machine Learning 81

Machine Learning Modeling Metadata Recreation/Entertainment 81

DataRobot Blog

JANUARY 10, 2023

ML model builders spend a ton of time running multiple experiments in a data science notebook environment before moving the well-tested and robust models from those experiments to a secure, production-grade environment for general consumption. 42% of data scientists are solo practitioners or on teams of five or fewer people.

Experimentation 52

Experimentation Machine Learning Data Science Modeling 52

O'Reilly on Data

MAY 14, 2020

AI PMs should enter feature development and experimentation phases only after deciding what problem they want to solve as precisely as possible, and placing the problem into one of these categories. Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model.

Management 363

Management Experimentation B2B Machine Learning 363

Insight

MARCH 12, 2020

With breaking this bottleneck in mind, I’ve used my time as an Insight Data Science Fellow to build the AIgent, a web-based neural net to connect writers to representation. I tested several different flavors of BERT for use as synopsis classifiers before settling on the DistilBERT model from Hugging Face. What happened next?

Modeling 59

Modeling Metadata Publishing Sales 59

DataKitchen

APRIL 13, 2021

Testing and Data Observability. We have also included vendors for the specific use cases of ModelOps, MLOps, DataGovOps and DataSecOps which apply DataOps principles to machine learning, AI, data governance, and data security operations. . Genie — Distributed big data orchestration service by Netflix.

Testing 300

Testing Machine Learning Consulting Data Quality 300

O'Reilly on Data

OCTOBER 13, 2020

In Bringing an AI Product to Market , we distinguished the debugging phase of product development from pre-deployment evaluation and testing. During testing and evaluation, application performance is important, but not critical to success. require not only disclosure, but also monitored testing. Debugging AI Products.

Management 244

Management Metrics Machine Learning Modeling 244

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Another pattern that I’ve seen in good PMs is that they’re very metric-driven. Transcript.

Management 110

Management Machine Learning Experimentation Metrics 110

The Unofficial Google Data Science Blog

JUNE 30, 2016

By IVAN DIAZ & JOSEPH KELLY Determining the causal effects of an action—which we call treatment—on an outcome of interest is at the heart of many data analysis efforts. In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects.

Statistics 44

Statistics Optimization Modeling Experimentation 44

The Unofficial Google Data Science Blog

APRIL 16, 2019

by MICHAEL FORTE Large-scale live experimentation is a big part of online product development. This means a small and growing product has to use experimentation differently and very carefully. This blog post is about experimentation in this regime. Such decisions involve an actual hypothesis test on specific metrics (e.g.

Experimentation 68

Experimentation Sales Metrics Measurement 68

The Unofficial Google Data Science Blog

JULY 31, 2016

At the end of the semester, you administer a proficiency test to the students and compare test performance across the groups. For example, you might compare the average test score among students exposed to the new curriculum to the average score among the controls. Sounds straightforward, right?

Experimentation 42

Experimentation Testing Measurement Metrics 42

The Unofficial Google Data Science Blog

DECEMBER 14, 2021

by AMIR NAJMI & MUKUND SUNDARARAJAN Data science is about decision making under uncertainty. This blog post introduces the notions of representational uncertainty and interventional uncertainty to paint a fuller picture of what the practicing data scientist is up against. Vignette: Data Science at fluff.ai

Uncertainty 56

Uncertainty Statistics Measurement Cost-Benefit 56

Domino Data Lab

MAY 8, 2019

The lens of reductionism and an overemphasis on engineering becomes an Achilles heel for data science work. Instead, consider a “full stack” tracing from the point of data collection all the way out through inference. back to the structure of the dataset. Let’s look through some antidotes. Ergo, less interpretable.

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

The Unofficial Google Data Science Blog

MAY 31, 2016

Similarly, we could test the effectiveness of a search ad compared to showing only organic search results. A geo experiment is an experiment where the experimental units are defined by geographic regions. In other words, iROAS is the slope of a curve of the response metric plotted against the underlying advertising spend.

Advertising 42

Advertising Testing Sales Statistics 42

The Unofficial Google Data Science Blog

OCTOBER 21, 2015

Our post describes how we arrived at recent changes to design principles for the Google search page, and thus highlights aspects of a data scientist’s role which involve practicing the scientific method. There has been debate as to whether the term “data science” is necessary. Some don’t see the point.

Slice and Dice 45

Slice and Dice Experimentation Data-driven Data Science 45

Domino Data Lab

AUGUST 26, 2019

This control prevents overfitting and reduction in predictive accuracy on new test data. Models are at the heart of of data science and increasing predictive accuracy of models is a core expectation upon industry. algorithm leaf_size metric metric_params n_jobs n_neighbors p weights. In [1]: # setup.

Testing 79

Testing Modeling Machine Learning Metrics 79

The Unofficial Google Data Science Blog

MARCH 22, 2018

But what if users don't immediately uptake the new experimental version? Background At Google, experimentation is an invaluable tool for making decisions and inference about new products and features. Naturally, this issue is of particular concern to us in the Play Data Science team.

Experimentation 62

Experimentation Measurement Modeling Statistics 62

Domino Data Lab

AUGUST 22, 2019

Data scientists and researchers require an extensive array of techniques, packages, and tools to accelerate core work flow tasks including prepping, processing, and analyzing data. Utilizing NLP helps researchers and data scientists complete core tasks faster. Relative to extrinsic evaluations, intrinsic tests are quick.

Deep Learning 111

Deep Learning Modeling Metrics Testing 111

datapine

JANUARY 31, 2022

The companies that are most successful at marketing in both B2C and B2B are using data and online BI tools to craft hyper-specific campaigns that reach out to targeted prospects with a curated message. Everything is being tested, and then the campaigns that succeed get more money put into them, while the others aren’t repeated.

Business Intelligence 214

Business Intelligence Analytics Cost-Benefit ROI 214

DataRobot Blog

MARCH 7, 2023

DataRobot on Azure accelerates the machine learning lifecycle with advanced capabilities for rapid experimentation across new data sources and multiple problem types. The capability to rapidly build an AI-powered organization with industry-specific solutions and expertise.

Data-driven 59

Data-driven Machine Learning Experimentation Data Lake 59

The Unofficial Google Data Science Blog

OCTOBER 12, 2017

1]" Statistics, as a discipline, was largely developed in a small data world. 1: The Himalaya mountain range Much of the data available to us in big data settings is large not only in the number of observations, but also in the number of features. We must correct for multiple hypothesis tests.

Experimentation 61

Experimentation Testing Statistics Metrics 61

IBM Big Data Hub

SEPTEMBER 25, 2023

There may be inaccuracy because of sampling, but it allows users to discover new viewpoints within the data. If the exploratory work needs to move on to testing and production, they can plan appropriately. Uber chose Presto for the flexibility it provides with compute separated from data storage.

OLAP 94

OLAP Data Lake Data-driven Snapshot 94