Occam's Razor

MAY 22, 2006

This post is a primer on the delightful world of testing and experimentation (A/B, Multivariate, and a new term from me: Experience Testing). Experimentation and testing help us figure out we are wrong, quickly and repeatedly and if you think about it that is a great thing for our customers, and for our employers.

Experimentation 103

Experimentation Testing Optimization Measurement 103

IBM Big Data Hub

APRIL 18, 2024

LLMs like ChatGPT are trained on massive amounts of text data, allowing them to recognize patterns and statistical relationships within language. ” AGI analyzes relevant code, generates a draft function with comments explaining its logic and allows the programmer to review, optimize and integrate it.

Cost-Benefit 88

Cost-Benefit Modeling Manufacturing Interactive 88

Occam's Razor

APRIL 8, 2013

Sometimes, we escape the clutches of this sub optimal existence and do pick good metrics or engage in simple A/B testing. First, you figure out what you want to improve; then you create an experiment; then you run the experiment; then you measure the results and decide what to do. Measure and decide what to do.

Metrics 156

Metrics KPI Analytics Key Performance Indicator 156

Occam's Razor

AUGUST 22, 2011

Key To Your Digital Success: Web Analytics Measurement Model. " Measuring Incrementality: Controlled Experiments to the Rescue! Barriers To An Effective Web Measurement Strategy [+ Solutions!]. Measuring Online Engagement: What Role Does Web Analytics Play? "Engagement" How Do I Measure Success?

KPI 124

KPI Metrics Measurement ROI 124

Insight

APRIL 28, 2020

You’ll often see the name “data challenge” used when the take-home assignment involves machine learning or statistics or “coding challenge” when the focus is on evaluating a candidate’s software engineering skills. provide an opportunity to measure both. Length: Highly Variable.

Data-driven 53

Data-driven Data Science Experimentation Measurement 53

Domino Data Lab

OCTOBER 2, 2019

This article covers causal relationships and includes a chapter excerpt from the book Machine Learning in Production: Developing and Optimizing Data Science Workflows and Applications by Andrew Kelleher and Adam Kelleher. As data science work is experimental and probabilistic in nature, data scientists are often faced with making inferences.

Machine Learning 68

Machine Learning Measurement Modeling Testing 68

Occam's Razor

JULY 24, 2018

To name a few: Digital Marketing & Measurement Model | Analytics Ecosystem | Web Analytics 2.0. During a discussion around planning for measurement, a peer was struggling with a unique collection of challenges. You see more digital metrics because digital is more measurable. The ability to optimize landing pages.

Analytics 93

Analytics Metrics Strategy Measurement 93

The Unofficial Google Data Science Blog

JUNE 30, 2016

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. Identification We now discuss formally the statistical problem of causal inference. We start by describing the problem using standard statistical notation.

Statistics 44

Statistics Optimization Modeling Experimentation 44

The Unofficial Google Data Science Blog

JULY 22, 2020

For example, imagine a fantasy football site is considering displaying advanced player statistics. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. One reason to do ramp-up is to mitigate the risk of never before seen arms.

Experimentation 95

Experimentation Statistics Testing Strategy 95

The Unofficial Google Data Science Blog

APRIL 16, 2019

by MICHAEL FORTE Large-scale live experimentation is a big part of online product development. This means a small and growing product has to use experimentation differently and very carefully. This blog post is about experimentation in this regime. But these are not usually amenable to A/B experimentation.

Experimentation 68

Experimentation Sales Metrics Measurement 68

Occam's Razor

JUNE 6, 2007

Experimentation & Testing (A/B, Multivariate, you name it). What's the optimal organization structure (and who should own web analytics!)? The book takes a stand on issues, makes choices and cuts through the fog/FUD, in an attempt to make your life a tiny bit easier. It is a book about Web Analytics 2.0. Clicks and outcomes.

Analytics 60

Analytics Metrics Marketing Dashboards 60

O'Reilly on Data

MARCH 23, 2023

Many of these go slightly (but not very far) beyond your initial expectations: you can ask it to generate a list of terms for search engine optimization, you can ask it to generate a reading list on topics that you’re interested in. It was not optimized to provide correct responses. It has helped to write a book.

IT 262

IT Modeling Testing Risk 262

Corinium

APRIL 25, 2019

For example, P&C insurance strives to understand its customers and households better through data, to provide better customer service and anticipate insurance needs, as well as accurately measure risks. Life insurance needs accurate data on consumer health, age and other metrics of risk.

Insurance 150

Insurance Risk IoT Cost-Benefit 150

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. These measurement-obsessed companies have an advantage when it comes to AI.

Management 110

Management Machine Learning Experimentation Metrics 110

Domino Data Lab

MAY 8, 2019

Visualizations are vital in data science work, with the caveat that the information that they convey may be 4-5 layers of abstraction away from the actual business process being measured. measure the subjects’ ability to trust the models’ results. Use of influence functions goes back to the 1970s in robust statistics.

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

Occam's Razor

JANUARY 9, 2012

Most companies are astonishingly blasé about data and possibilities of measurement. " Sad, unimaginative measurements of their sad, unimaginative campaigns. All while constantly optimizing your portfolio via controlled experiments. I told 20 people that Nikon's site is slow and profoundly sub-optimal on mobile.

Marketing 140

Marketing B2B Measurement Risk 140

Occam's Razor

MARCH 23, 2010

Part of it is fueled by a vocal minority genuinely upset that 10 years on we are still not a statistically powered bunch doing complicated analysis that is shifting paradigms. Part of it fueled by some Consultants. I suppose the rational is: self preservation before all else. This is sad. Even a little frustrating. Likely not.

Analytics 118

Analytics Consulting Metrics Experimentation 118

Cloudera

NOVEMBER 25, 2020

And soon also sensor measures, and possibly video or audio data with the increased use of device technology and telemedicine in medical care. How can he make it easy to see statistics, and do calculations, on discovered commonalities, across structured and unstructured data? Innovate on serviceability and optimize utilization.

Data Warehouse 57

Data Warehouse Unstructured Data Analytics Visualization 57

Occam's Razor

FEBRUARY 8, 2021

From observing behavior closely, and from my own experimentation and failure, I've noticed consistent patterns in what great employees do and great bosses do. They find the external author of the statistical algorithm I want them to use, and ask them for guidance. They are there to nudge your career in an optimal direction.

Strategy 132

Strategy Statistics Cost-Benefit Marketing 132

Domino Data Lab

AUGUST 26, 2019

When hyperparameters are also used for optimization and tuned for a specific dataset, they also impact predictive accuracy. Searching is a very general approach to solving or optimizing problems. I will continue using parameters (which are optimized during training) and hyperparameters (which are tuned via cross-validation).

Testing 79

Testing Modeling Machine Learning Metrics 79

The Unofficial Google Data Science Blog

OCTOBER 12, 2017

1]" Statistics, as a discipline, was largely developed in a small data world. More people than ever are using statistical analysis packages and dashboards, explicitly or more often implicitly, to develop and test hypotheses. Data was expensive to gather, and therefore decisions to collect data were generally well-considered.

Experimentation 61

Experimentation Testing Statistics Metrics 61