DataRobot Blog

DECEMBER 6, 2022

Other organizations are just discovering how to apply AI to accelerate experimentation time frames and find the best models to produce results. See how to use DataRobot Model Observability to track service, drift, prediction data, training data, and custom metrics in order to keep models and predictions relevant in a fast-changing world.

Experimentation 145

Experimentation Forecasting Data-driven Machine Learning 145

O'Reilly on Data

MARCH 31, 2020

Because it’s so different from traditional software development, where the risks are more or less well-known and predictable, AI rewards people and companies that are willing to take intelligent risks, and that have (or can develop) an experimental culture. Even if a product is feasible, that’s not the same as product-market fit.

Management 332

Management Machine Learning Experimentation Metrics 332

Smart Data Collective

JULY 9, 2023

Understanding E-commerce Conversion Rates There are a number of metrics that data-driven e-commerce companies need to focus on. It is a crucial metric that provides priceless information about your website’s ability to transform visitors into paying customers. Some of the most important is conversion rates.

Optimization 72

Optimization Data Analytics Analytics Data-driven 72

AWS Big Data

FEBRUARY 12, 2024

A series of materialized view refreshes are used to calculate metrics, after which the incremental data from S3 is loaded into Redshift. Gupshup chose Aurora PostgreSQL as the operational reporting layer due to its anticipated increase in concurrency and cost-effectiveness for queries that retrieve only precalculated metrics.

Data Warehouse 98

Data Warehouse Analytics Snapshot Cost-Benefit 98

CIO Business Intelligence

APRIL 12, 2024

Computer Vision also gives insights about customer traffic in stores, including metrics on conversion rate and sales numbers, thus helping the company decide what products to stock, how to display them, and how to arrange products across the store, Allison adds.

Experimentation 90

Experimentation Sales Digital Transformation Metrics 90

Occam's Razor

FEBRUARY 7, 2022

Since you're reading a blog on advanced analytics, I'm going to assume that you have been exposed to the magical and amazing awesomeness of experimentation and testing. And yet, chances are you really don’t know anyone directly who uses experimentation as a part of their regular business practice. Wah wah wah waaah.

Experimentation 101

Experimentation Testing Optimization Marketing 101

CIO Business Intelligence

NOVEMBER 15, 2022

Experiment with the “highly visible and highly hyped”: Gartner repeatedly pointed out that organisations that innovate during tough economic times “stay ahead of the pack”, with Mesaglio in particular calling for such experimentation to be public and visible.

Digital Transformation 138

Digital Transformation Experimentation Risk Metrics 138

IBM Big Data Hub

DECEMBER 4, 2023

And the abundance of data available for training models has opened up vast possibilities for experimentation and learning. Generative AI can also help developers improve their skills as they deal with more complex tasks. At this exciting frontier, it’s essential that developers adopt generative AI solutions that are right for them.

Experimentation 67

Experimentation Modeling Metrics Machine Learning 67

Occam's Razor

MAY 22, 2006

This post is a primer on the delightful world of testing and experimentation (A/B, Multivariate, and a new term from me: Experience Testing). Experimentation and testing help us figure out we are wrong, quickly and repeatedly and if you think about it that is a great thing for our customers, and for our employers. Counter claims?

Experimentation 103

Experimentation Testing Optimization Measurement 103

The Unofficial Google Data Science Blog

APRIL 16, 2019

by MICHAEL FORTE Large-scale live experimentation is a big part of online product development. This means a small and growing product has to use experimentation differently and very carefully. This blog post is about experimentation in this regime. Such decisions involve an actual hypothesis test on specific metrics (e.g.

Experimentation 68

Experimentation Sales Metrics Measurement 68

AWS Big Data

AUGUST 21, 2023

It similarly codes the query as a vector and then uses a distance metric to find nearby vectors in the multi-dimensional space. The algorithm for finding nearby vectors is called kNN (k Nearest Neighbors). Of course, production-quality search experiences use many more techniques to improve results.

Data Processing 91

Data Processing Experimentation Visualization Metrics 91

Aryng

APRIL 25, 2022

” – Ronald Fisher Experimentation is a powerful tool for businesses to innovate and test new ideas, but few seem to be using this tool right. […] The post How to use experiments to find your way to success appeared first on Aryng's Blog. He can perhaps say what the experiment died of.”

Experimentation 52

Experimentation Consulting Testing Metrics 52

CIO Business Intelligence

APRIL 2, 2024

Joanne Friedman, PhD, CEO, and principal of smart manufacturing at Connektedminds, says orchestrating success in digital transformation requires a symphony of integration across disciplines : “CIOs face the challenge of harmonizing diverse disciplines like design thinking, product management, agile methodologies, and data science experimentation.

Management 119

Management Digital Transformation Testing Experimentation 119

CIO Business Intelligence

NOVEMBER 13, 2023

Foster a culture of innovation: Digital transformation requires innovation and experimentation, and thus a culture for embracing new technologies and ideas. This involves setting up metrics and KPIs and regularly reviewing them to identify areas for improvement.

Digital Transformation 139

Digital Transformation Cost-Benefit Experimentation Measurement 139

CIO Business Intelligence

FEBRUARY 21, 2024

We developed a model to predict student outcomes based on metrics from historical evidence,” he says. “We We’ve done a lot of experimentation on these adaptive tools that use AI,” says Ventimiglia. We ran a pilot with data scientists and other specialists on campus, and using the model, they were able to make very accurate predictions.

Recreation/Entertainment 125

Recreation/Entertainment Experimentation Modeling Data-driven 125

DataRobot Blog

OCTOBER 18, 2022

Model Observability – the ability to track key health and service metrics for models in production – remains a top priority for AI-enabled organizations. Learn more about the DataRobot AI Cloud and the ability to accelerate your experimentation and production timelines. DataRobot Booth at Big Data & AI Toronto 2022.

Big Data 96

Big Data Forecasting Experimentation Metrics 96

Sisense

MARCH 6, 2019

Gartner chose to group the rest of the keynote into three main messages according to the following categories: Here are some of the highlights as presented for each of them: Data Driven – “Adopt an Experimental Mindset”. At Sisense we’ve been preaching for BI prototyping and experimentation for quite a while now.

Data Analytics 75

Data Analytics Analytics Experimentation Data-driven 75

Occam's Razor

MAY 15, 2006

There is a lot of "buzz" around "buzzy" metrics such as brand value / brand impact, blog-pulse , to name a couple. IMHO these "buzzy" metrics might be a sub optimal use of time/resources if we don't first have a hard core understanding of customer satisfaction and task completion on our websites.

Metrics 100

Metrics Experimentation Testing Interactive 100

CIO Business Intelligence

DECEMBER 14, 2023

Reining in the rugged and remote Through rapid cycles of discovery and experimentation, Mathur and De Bernardi’s cross-functional team devised a solution that leaned heavily on additive manufacturing, more commonly known as “3D printing.”

Manufacturing 73

Manufacturing Digital Transformation Experimentation Cost-Benefit 73

DataRobot Blog

JANUARY 10, 2023

Data science teams of all sizes need a productive, collaborative method for rapid AI experimentation. This flexibility allows you to import your local code into the DataRobot platform and continue further experimentation using the combination of DataRobot Notebooks with: Deep integrations with DataRobot comprehensive APIs.

Experimentation 52

Experimentation Machine Learning Data Science Modeling 52

DataRobot Blog

FEBRUARY 28, 2023

They can enjoy a hosted experience with code snippets, versioning, and simple environment management for rapid AI experimentation. When the training process is complete, DataRobot will recommend the best-performing model for production based on the selected metric and provide an explanation. Learn more about DataRobot hosted notebooks.

Data Processing 72

Data Processing Experimentation Machine Learning Modeling 72

DataRobot

JUNE 10, 2021

Traditionally, experimentation and observation was the only way to understand the physical-chemical properties of the molecule. DataRobot also provides per-label metrics so that metrics per class can be compared. Below are the per-label metrics provided by DataRobot for model evaluation purposes.

Metrics 52

Metrics Machine Learning Visualization Experimentation 52

CIO Business Intelligence

NOVEMBER 21, 2023

Short-term plans should specify which departments should experiment, toward which business outcomes and opportunities, as well as what tools can be used, what data can be involved, and where they should report their experimental learnings.

IT 130

IT Risk Cost-Benefit Risk Management 130

The Unofficial Google Data Science Blog

JANUARY 14, 2016

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

Corinium

JUNE 6, 2019

It is also important to have a strong test and learn culture to encourage rapid experimentation. Measure user adoption and engagement metrics to not just understand products take-up, but also to enhance the overall product propositions. What is the most common mistake people make around data?

Insurance 250

Insurance Analytics Forecasting Deep Learning 250

Rocket-Powered Data Science

JULY 6, 2021

MLOps takes the modeling, algorithms, and data wrangling out of the experimental “one off” phase and moves the best models into deployment and sustained operational phase. However, it is far from perfect, since it certainly does not have reasoning skills, and it also loses its “train of thought” after several paragraphs (e.g.,

Data-driven 279

Data-driven Machine Learning IoT Deep Learning 279

IBM Big Data Hub

MARCH 4, 2024

This culture encourages experimentation and expertise growth. For example, by using compliance control scanning of terraform templates to fail provisioning if controls are not met. An AI+ enterprise also recognizes that alongside the necessary tools, fostering a culture that embraces AI and trains talent is crucial.

Enterprise 104

Enterprise ROI Risk Strategy 104

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Another pattern that I’ve seen in good PMs is that they’re very metric-driven.

Management 110

Management Machine Learning Experimentation Metrics 110

Occam's Razor

JULY 24, 2018

work (collection, processing, reporting, analysis), processes, org structure, governance models, last-mile gaps , metrics ladders of awesomeness , and… so… much… more. Remember, tools, work, processes, org structure, governance models, last-mile gaps, metrics ladders of awesomeness, and… so… much… more. The Implications of Complexity.

Analytics 93

Analytics Metrics Strategy Measurement 93

AWS Big Data

JULY 20, 2023

Additionally, partition evolution enables experimentation with various partitioning strategies to optimize cost and performance without requiring a rewrite of the table’s data every time. Orca monitored the cluster status and resource usage of Amazon EMR by utilizing the available metrics through Amazon CloudWatch.

Data Lake 86

Data Lake Analytics Snapshot Optimization 86

The Unofficial Google Data Science Blog

OCTOBER 7, 2015

Nevertheless, A/B testing has challenges and blind spots, such as: the difficulty of identifying suitable metrics that give "works well" a measurable meaning. accounting for effects "orthogonal" to the randomization used in experimentation. accounting for effects "orthogonal" to the randomization used in experimentation.

Modeling 61

Modeling Experimentation Knowledge Discovery Testing 61

CIO Business Intelligence

MAY 16, 2023

It surpasses blockchain and metaverse projects, which are viewed as experimental or in the pilot stage, especially by established enterprises. AI surpassed other technologies in conversations about innovation The research underscores that AI is leading the way in accelerating innovation.

Advertising 85

Advertising Recreation/Entertainment Experimentation Enterprise 85

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

Despite a very large number of experimental units, the experiments conducted by LSOS cannot presume statistical significance of all effects they deem practically significant. The result is that experimenters can’t afford to be sloppy about quantifying uncertainty. In statistics, such segments are often called “blocks” or “strata”.

Experimentation 40

Experimentation Metrics Statistics Measurement 40

Domino Data Lab

SEPTEMBER 10, 2020

This means many projects get stuck in endless research and experimentation. Identify metrics that measure this variability. Existing Metrics: Foot traffic (footfall) monitoring of total customer count in store based on time series. Problem statements should ultimately: Include a problem definition. The total number of staff.

Modeling 101

Modeling IT Data Science Experimentation 101

The Unofficial Google Data Science Blog

JANUARY 16, 2018

Experimentation on networks A/B testing is a standard method of measuring the effect of changes by randomizing samples into different treatment groups. However, the downside of using a larger unit of randomization is that we lose experimental power. Consider the case where experiment metrics are evaluated at the per-user level.

Testing 58

Testing Experimentation Measurement Modeling 58

The Unofficial Google Data Science Blog

OCTOBER 12, 2017

With more features come more potential post hoc hypotheses about what is driving metrics of interest, and more opportunity for exploratory analysis. We data scientists now have access to tools that allow us to run a large numbers of experiments, and then to slice experimental populations by any combination of dimensions collected.

Experimentation 61

Experimentation Testing Statistics Metrics 61

Domino Data Lab

DECEMBER 4, 2020

Having calculated AUC/AUMC, we can further derive a number of useful metrics like: Total clearance of the drug from plasma. Domino Lab supports both interactive and batch experimentation with all popular IDEs and notebooks (Jupyter, RStudio, SAS, Zeppelin, etc.). The area under the first moment curve would respectively be. cl_f = NCA.cl(pain_nca)

Metrics 59

Metrics Data Science Knowledge Discovery Measurement 59