DataRobot Blog

OCTOBER 18, 2022

Monitoring and Managing AI Projects with Model Observability. Model Observability – the ability to track key health and service metrics for models in production – remains a top priority for AI-enabled organizations. These accelerators are specifically designed to help organizations accelerate from data to results.

Big Data 96

Big Data Forecasting Experimentation Metrics 96

CIO Business Intelligence

APRIL 2, 2024

Joanne Friedman, PhD, CEO, and principal of smart manufacturing at Connektedminds, says orchestrating success in digital transformation requires a symphony of integration across disciplines : “CIOs face the challenge of harmonizing diverse disciplines like design thinking, product management, agile methodologies, and data science experimentation.

Management 140

Management Digital Transformation Testing Experimentation 140

DataRobot Blog

FEBRUARY 28, 2023

Integrating different systems, data sources, and technologies within an ecosystem can be difficult and time-consuming, leading to inefficiencies, data silos, broken machine learning models, and locked ROI. Model Deployment Customers need the flexibility to deploy models into different environments.

Data Processing 72

Data Processing Experimentation Machine Learning Modeling 72

The Unofficial Google Data Science Blog

APRIL 23, 2024

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Experiments, Parameters and Models At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

CIO Business Intelligence

MARCH 21, 2022

What is a data scientist? Data scientists are analytical data experts who use data science to discover insights from massive amounts of structured and unstructured data to help shape or meet specific business needs and goals. Data scientist salary. Semi-structured data falls between the two.

Unstructured Data 137

Unstructured Data Data Analytics Analytics Structured Data 137

Cloudera

APRIL 9, 2021

The third video in the series highlighted Reporting and Data Visualization. Specifically, we’ll focus on training Machine Learning (ML) models to forecast ECC part production demand across all of its factories. This integration is key in assuring that models evolve with the data – to avoid, for example, model drift.

Machine Learning 98

Machine Learning Forecasting Manufacturing Predictive Analytics 98

The Unofficial Google Data Science Blog

OCTOBER 7, 2015

by HENNING HOHNHOLD, DEIRDRE O'BRIEN, and DIANE TANG In this post we discuss the challenges in measuring and modeling the long-term effect of ads on user behavior. We describe experiment designs which have proven effective for us and discuss the subtleties of trying to generalize the results via modeling.

Modeling 61

Modeling Experimentation Knowledge Discovery Testing 61

Domino Data Lab

DECEMBER 4, 2020

Having calculated AUC/AUMC, we can further derive a number of useful metrics like: Total clearance of the drug from plasma. NCA doesn’t require the assumption of a specific compartmental model for either drug or metabolite; it is instead assumption-free and therefore easily automated [1]. Mean residence time.

Metrics 59

Metrics Data Science Knowledge Discovery Measurement 59

Insight

MARCH 12, 2020

With breaking this bottleneck in mind, I’ve used my time as an Insight Data Science Fellow to build the AIgent, a web-based neural net to connect writers to representation. The AIgent was built with BERT, Google’s state-of-the-art language model. More relevant to the AIgent is Google’s BERT model, a task-agnostic (i.e.

Modeling 59

Modeling Metadata Publishing Sales 59

The Unofficial Google Data Science Blog

JANUARY 14, 2016

by AMIR NAJMI Running live experiments on large-scale online services (LSOS) is an important aspect of data science. We must therefore maintain statistical rigor in quantifying experimental uncertainty. In this post we explore how and why we can be “ data-rich but information-poor ”.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

DataRobot

JUNE 23, 2021

As they strive to improve models, data scientists continually try new approaches to refine their predictions. To help data scientists experiment faster, DataRobot has added Composable ML to automated machine learning. Composable ML then lets you add new types of feature engineering or build entirely new models.

Machine Learning 52

Machine Learning Modeling Experimentation Visualization 52

DataRobot Blog

JANUARY 10, 2023

Most, if not all, machine learning (ML) models in production today were born in notebooks before they were put into production. 42% of data scientists are solo practitioners or on teams of five or fewer people. 42% of data scientists are solo practitioners or on teams of five or fewer people. Deep Dive into DataRobot Notebooks.

Experimentation 52

Experimentation Machine Learning Data Science Modeling 52

The Unofficial Google Data Science Blog

JANUARY 16, 2018

Experimentation on networks A/B testing is a standard method of measuring the effect of changes by randomizing samples into different treatment groups. However, the downside of using a larger unit of randomization is that we lose experimental power. We now describe a generative model for how effects might propagate through the network.

Testing 58

Testing Experimentation Measurement Modeling 58

O'Reilly on Data

MAY 14, 2020

AI PMs should enter feature development and experimentation phases only after deciding what problem they want to solve as precisely as possible, and placing the problem into one of these categories. Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model.

Management 363

Management Experimentation B2B Machine Learning 363

Business Over Broadway

MAY 22, 2019

According to Gartner, companies need to adopt these practices: build culture of collaboration and experimentation; start with a 3-way partnership among executives leading digital initiative, line of business and IT. Remember that digital transformation is about transforming your business and operating models with technology.

Machine Learning 40

Machine Learning Technology Digital Transformation Data Science 40

O'Reilly on Data

OCTOBER 13, 2020

Similarly, in “ Building Machine Learning Powered Applications: Going from Idea to Product ,” Emmanuel Ameisen states: “Indeed, exposing a model to users in production comes with a set of challenges that mirrors the ones that come with debugging a model.”.

Management 244

Management Metrics Machine Learning Modeling 244

DataKitchen

APRIL 13, 2021

DataOps needs a directed graph-based workflow that contains all the data access, integration, model and visualization steps in the data analytic production process. It orchestrates complex pipelines, toolchains, and tests across teams, locations, and data centers. Meta-Orchestration . DevOps Infrastructure Tools.

Testing 307

Testing Machine Learning Consulting Data Quality 307

Domino Data Lab

JUNE 10, 2020

Today, we announced the latest release of Domino’s data science platform which represents a big step forward for enterprise data science teams. I’m also proud to announce an exciting new product: Domino Model Monitor (DMM). Domino’s best-in-class Workbench is now even more powerful for data scientists.

Data Science 82

Data Science Enterprise Experimentation Modeling 82

Rocket-Powered Data Science

JULY 6, 2021

2) MLOps became the expected norm in machine learning and data science projects. MLOps takes the modeling, algorithms, and data wrangling out of the experimental “one off” phase and moves the best models into deployment and sustained operational phase.

Data-driven 279

Data-driven Machine Learning IoT Deep Learning 279

The Unofficial Google Data Science Blog

JUNE 30, 2016

By IVAN DIAZ & JOSEPH KELLY Determining the causal effects of an action—which we call treatment—on an outcome of interest is at the heart of many data analysis efforts. In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects.

Statistics 44

Statistics Optimization Modeling Experimentation 44

Domino Data Lab

FEBRUARY 4, 2019

In Paco Nathan ‘s latest column, he explores the role of curiosity in data science work as well as Rev 2 , an upcoming summit for data science leaders. Welcome back to our monthly series about data science. and dig into details about where science meets rhetoric in data science.

Data Science 66

Data Science Experimentation Data-driven Machine Learning 66

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Another pattern that I’ve seen in good PMs is that they’re very metric-driven.

Management 110

Management Machine Learning Experimentation Metrics 110

Domino Data Lab

AUGUST 22, 2019

The excerpt covers how to create word vectors and utilize them as an input into a deep learning model. While the field of computational linguistics, or Natural Language Processing (NLP), has been around for decades, the increased interest in and use of deep learning models has also propelled applications of NLP forward within industry.

Deep Learning 111

Deep Learning Modeling Metrics Testing 111

Domino Data Lab

MAY 8, 2019

Paco Nathan’s latest article features several emerging threads adjacent to model interpretability. I’ve been out themespotting and this month’s article features several emerging threads adjacent to the interpretability of machine learning models. Machine learning model interpretability. Introduction. 2018-06-21).

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

The Unofficial Google Data Science Blog

DECEMBER 14, 2021

by AMIR NAJMI & MUKUND SUNDARARAJAN Data science is about decision making under uncertainty. This blog post introduces the notions of representational uncertainty and interventional uncertainty to paint a fuller picture of what the practicing data scientist is up against. Vignette: Data Science at fluff.ai

Uncertainty 56

Uncertainty Statistics Measurement Cost-Benefit 56

The Unofficial Google Data Science Blog

APRIL 16, 2019

by MICHAEL FORTE Large-scale live experimentation is a big part of online product development. This means a small and growing product has to use experimentation differently and very carefully. This blog post is about experimentation in this regime. Such decisions involve an actual hypothesis test on specific metrics (e.g.

Experimentation 68

Experimentation Sales Metrics Measurement 68

The Unofficial Google Data Science Blog

JULY 31, 2016

To figure this out, let's consider an appropriate experimental design. In other words, the teacher is our second kind of unit, the unit of experimentation. This type of experimental design is known as a group-randomized or cluster-randomized trial. When analyzing the outcome measure (e.g.,

Experimentation 42

Experimentation Testing Measurement Metrics 42

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

Despite a very large number of experimental units, the experiments conducted by LSOS cannot presume statistical significance of all effects they deem practically significant. The result is that experimenters can’t afford to be sloppy about quantifying uncertainty. In statistics, such segments are often called “blocks” or “strata”.

Experimentation 40

Experimentation Metrics Statistics Measurement 40

The Unofficial Google Data Science Blog

MARCH 22, 2018

But what if users don't immediately uptake the new experimental version? Background At Google, experimentation is an invaluable tool for making decisions and inference about new products and features. Naturally, this issue is of particular concern to us in the Play Data Science team.

Experimentation 62

Experimentation Measurement Modeling Statistics 62

The Unofficial Google Data Science Blog

MAY 31, 2016

A geo experiment is an experiment where the experimental units are defined by geographic regions. This means it is possible to specify exactly in which geos an ad campaign will be served – and to observe the ad spend and the response metric at the geo level. The noisier the data, the higher the standard error.

Advertising 42

Advertising Testing Sales Statistics 42

The Unofficial Google Data Science Blog

OCTOBER 21, 2015

Our post describes how we arrived at recent changes to design principles for the Google search page, and thus highlights aspects of a data scientist’s role which involve practicing the scientific method. There has been debate as to whether the term “data science” is necessary. Some don’t see the point.

Slice and Dice 45

Slice and Dice Experimentation Data-driven Data Science 45

Domino Data Lab

AUGUST 26, 2019

Data scientists, machine learning (ML) researchers, and business stakeholders have a high-stakes investment in the predictive accuracy of models. Data scientists and researchers ascertain predictive accuracy of models using different techniques, methodologies, and settings, including model parameters and hyperparameters.

Testing 79

Testing Modeling Machine Learning Metrics 79

DataRobot Blog

MARCH 7, 2023

Organizations that want to prove the value of AI by developing, deploying, and managing machine learning models at scale can now do so quickly using the DataRobot AI Platform on Microsoft Azure. Models trained in DataRobot can also be easily deployed to Azure Machine Learning, allowing users to host models easier in a secure way.

Data-driven 59

Data-driven Machine Learning Experimentation Data Lake 59

datapine

JANUARY 31, 2022

It’s all about using data to get a clearer understanding of reality so that your company can make more strategically sound decisions (instead of relying only on gut instinct or corporate inertia). Ultimately, business intelligence and analytics are about much more than the technology used to gather and analyze data.

Business Intelligence 214

Business Intelligence Analytics Cost-Benefit ROI 214

DataRobot

APRIL 29, 2021

This list includes: Rachik Laouar is Head of Data Science for the Adecco Group. Rachik is working to transform that company’s products through data analytics and AI and will be speaking on the topic, Executive Track: Turning an Industry Upside Down. . Eric Weber is Head of Experimentation And Metrics for Yelp.

Machine Learning 59

Machine Learning Experimentation Data-driven Data Science 59

The Unofficial Google Data Science Blog

OCTOBER 12, 2017

1: The Himalaya mountain range Much of the data available to us in big data settings is large not only in the number of observations, but also in the number of features. With more features come more potential post hoc hypotheses about what is driving metrics of interest, and more opportunity for exploratory analysis.

Experimentation 61

Experimentation Testing Statistics Metrics 61

IBM Big Data Hub

SEPTEMBER 25, 2023

Uber’s DNA as an analytics company At its core, Uber’s business model is deceptively simple: connect a customer at point A to their destination at point B. For traditional analytics, they are bringing data discipline to their use of Presto. They ingest data in snapshots from operational systems.

OLAP 92

OLAP Data Lake Data-driven Snapshot 92