Occam's Razor

MAY 22, 2006

This post is a primer on the delightful world of testing and experimentation (A/B, Multivariate, and a new term from me: Experience Testing). Experimentation and testing help us figure out we are wrong, quickly and repeatedly and if you think about it that is a great thing for our customers, and for our employers. Counter claims?

Experimentation 103

Experimentation Testing Optimization Measurement 103

IBM Big Data Hub

MAY 17, 2024

” Given the statistics—82% of surveyed respondents in a 2023 Statista study cited managing cloud spend as a significant challenge—it’s a legitimate concern. Teams are comfortable with experimentation and skilled in using data to inform business decisions.

Modeling 91

Modeling Cost-Benefit Optimization Digital Transformation 91

DataRobot Blog

JULY 28, 2022

In this blog post, we would like to present some examples of actual cases in which noise reduction had a significant effect in real-world applications, and in which powerful features were obtained. The first baseline model we created used spectrograms of speech waveform data, statistical features, and spectrogram images.

Data Science 72

Data Science Machine Learning Statistics Modeling 72

DataRobot Blog

JANUARY 17, 2023

Not actually being a machine learning problem: Value-at-Risk modeling is the classic example here—VaR isn’t a prediction of anything, it’s a statistical summation of simulation results. As discussed, we massively accelerate that process of experimentation.

Machine Learning 87

Machine Learning Marketing Experimentation Modeling 87

The Unofficial Google Data Science Blog

JANUARY 14, 2016

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

The Unofficial Google Data Science Blog

APRIL 16, 2019

by MICHAEL FORTE Large-scale live experimentation is a big part of online product development. In fact, this blog has published posts on this very topic. This means a small and growing product has to use experimentation differently and very carefully. This blog post is about experimentation in this regime.

Experimentation 68

Experimentation Sales Metrics Measurement 68

Sisense

JANUARY 21, 2020

This is an example of Simpon’s paradox , a statistical phenomenon in which a trend that is present when data is put into groups reverses or disappears when the data is combined. It’s time to introduce a new statistical term. Everyone wants to get more out of their data, but how exactly to do that can leave you scratching your head.

Testing 104

Testing Risk Data-driven Statistics 104

Ontotext

FEBRUARY 14, 2024

As a statistical model, LLM inherently is random. Experimentation is important, but be explicit when you do. So, we’ve learned our lesson. AI will solve it We’ve seen a massive increase in interest in LLMs and ‘AI’ But LLM without guide rails could lead to unbridled hallucinations. Start with “why?”

Recreation/Entertainment 105

Recreation/Entertainment Data Integration Modeling Data Quality 105

Octopai

MAY 2, 2022

Common elements of DataOps strategies include: Collaboration between data managers, developers and consumers A development environment conducive to experimentation Rapid deployment and iteration Automated testing Very low error rates. Issue detected?

Manufacturing 87

Manufacturing Testing Experimentation Data Quality 87

DataRobot Blog

OCTOBER 27, 2022

You might think that overall, the model’s features drifted relatively little in production, but in reality, the model’s drift statistics might be fluctuating quite a bit up and down. Prior to using DataRobot, the customer was conducting tedious experimentation to track and investigate drift patterns.

Data Science 52

Data Science Experimentation Visualization Modeling 52

Paul DeBeasi

JANUARY 30, 2019

Machine learning projects are inherently different from traditional IT projects in that they are significantly more heuristic and experimental, requiring skills spanning multiple domains, including statistical analysis, data analysis and application development. Four Options for Integrating Machine Learning with IoT.

Machine Learning 56

Machine Learning IoT Experimentation Statistics 56

The Unofficial Google Data Science Blog

OCTOBER 12, 2017

1]" Statistics, as a discipline, was largely developed in a small data world. More people than ever are using statistical analysis packages and dashboards, explicitly or more often implicitly, to develop and test hypotheses. Data was expensive to gather, and therefore decisions to collect data were generally well-considered.

Experimentation 61

Experimentation Testing Statistics Metrics 61

IBM Big Data Hub

SEPTEMBER 25, 2023

This blog takes you on a journey into the world of Uber’s analytics and the critical role that Presto, the open source SQL query engine, plays in driving their success. Data Exploration and Innovation: The flexibility of Presto has encouraged data exploration and experimentation at Uber. Enterprise Management Associates (EMA).

OLAP 88

OLAP Data Lake Data-driven Snapshot 88

Smarten

MAY 11, 2017

Advanced Data Discovery ensures data democratization by enabling users to drastically reduce the time and cost of analysis and experimentation. Plug n’ Play Predictive Analysis enables business users to explore power of predictive analytics without indepth understanding of statistics and data science.

Experimentation 49

Experimentation Visualization Business Intelligence Predictive Analytics 49

CIO Business Intelligence

DECEMBER 6, 2023

Confusing and badly written content created by generative AI is already showing up in business contexts, with conference biographies, blog posts and slide decks that might sound impressive but make no sense being signed off by managers who should know better. Not all generative AI customers have got that message.

IT 125

IT Risk Management Interactive Modeling 125

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

In this post we explore why some standard statistical techniques to reduce variance are often ineffective in this “data-rich, information-poor” realm. Despite a very large number of experimental units, the experiments conducted by LSOS cannot presume statistical significance of all effects they deem practically significant.

Experimentation 40

Experimentation Metrics Statistics Measurement 40

Occam's Razor

AUGUST 22, 2011

Here you'll find all my blog posts categorized into a structure that will hopefully make it easy for you to discover new content, find answers to your questions, or simply wallow in some excellent analytics narratives. Blogging Experience Articles. + Podcast: Measuring Rich Media (Ajax, Flash / Flex, RSS & Blogs).

KPI 124

KPI Metrics Measurement ROI 124

The Unofficial Google Data Science Blog

MARCH 22, 2018

But what if users don't immediately uptake the new experimental version? This blog post provides details for how we can make inferences without waiting for complete uptake. This blog post provides details for how we can make inferences without waiting for complete uptake. What if their uptake rate is not uniform?

Experimentation 62

Experimentation Measurement Modeling Statistics 62

The Unofficial Google Data Science Blog

JANUARY 16, 2018

Experimentation on networks A/B testing is a standard method of measuring the effect of changes by randomizing samples into different treatment groups. However, the downside of using a larger unit of randomization is that we lose experimental power. This simulation is based on the actual user network of GCP.

Testing 58

Testing Experimentation Measurement Modeling 58

O'Reilly on Data

DECEMBER 9, 2019

Recall from my previous blog post that all financial models are at the mercy of the Trinity of Errors , namely: errors in model specifications, errors in model parameter estimates, and errors resulting from the failure of a model to adapt to structural changes in its environment. The interval [-a, a] is called a 90% confidence interval.

Statistics 27

Statistics Uncertainty Risk Marketing 27

IBM Big Data Hub

APRIL 18, 2024

LLMs like ChatGPT are trained on massive amounts of text data, allowing them to recognize patterns and statistical relationships within language. Meet watsonx Explore AI topics The post Getting ready for artificial general intelligence with examples appeared first on IBM Blog.

Cost-Benefit 91

Cost-Benefit Modeling Manufacturing Interactive 91

The Unofficial Google Data Science Blog

JULY 31, 2016

This blog post explores how this problem arises in applications at Google, and how we can 'mind our units' through analysis and logging. To figure this out, let's consider an appropriate experimental design. In other words, the teacher is our second kind of unit, the unit of experimentation. However, this is not always the case.

Experimentation 42

Experimentation Testing Measurement Metrics 42

Cloudera

NOVEMBER 25, 2020

How can he make it easy to see statistics, and do calculations, on discovered commonalities, across structured and unstructured data? It would enable faster experimentation with easy, protected, and governed access to a variety of data. Stay tuned for the next blog post that will dive deeper into this topic!

Data Warehouse 58

Data Warehouse Unstructured Data Analytics Visualization 58

Alation

SEPTEMBER 28, 2022

Technical environments and IDEs must be disposable so that experimental costs can be kept to a minimum. Quality must be monitored continuously to catch unexpected variation cases and produce statistics on its operation. Subscribe to Alation's Blog. Disposable environments. Simplicity. Analytics is manufacturing. Author Bio.

Cost-Benefit 52

Cost-Benefit Data Quality Manufacturing Testing 52

Ontotext

APRIL 3, 2024

Nimit Mehta: I think that 2024 is going to be a buckle-down year, but, at the same time, we’ll see a rapid explosion of experimentation. These are not statistical inferences. So, this is a big driver for the outcome because when you are saving money for the business, you can measure it and see its value. What is a customer?

Technology 59

Technology Risk Digital Transformation Measurement 59

The Unofficial Google Data Science Blog

JUNE 30, 2016

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. Identification We now discuss formally the statistical problem of causal inference. We start by describing the problem using standard statistical notation.

Statistics 44

Statistics Optimization Modeling Experimentation 44

Occam's Razor

JUNE 6, 2011

Update: This blog post is overwhelmingly for those who want to become Analysts ("Business" in the matrix above). So in addition to becoming good at Omniture, Google Analytics, Baidu Analytics , pick one other tool from the Experimentation, Voice of Customer, Competitive Intelligence buckets of Web Analytics 2.0. Pick two blogs.

Analytics 125

Analytics Consulting Statistics Marketing 125

O'Reilly on Data

MARCH 23, 2023

And it can look up an author and make statistical observations about their interests. ChatGPT offers users a paid account that costs $20/month, which is good enough for experimenters, though there is a limit on the number of requests you can make. Personal conversation, though he may also have said this in his blog.

IT 258

IT Modeling Testing Risk 258

The Unofficial Google Data Science Blog

MAY 31, 2016

A geo experiment is an experiment where the experimental units are defined by geographic regions. Statistical power is traditionally given in terms of a probability function, but often a more intuitive way of describing power is by stating the expected precision of our estimates. They are non-overlapping geo-targetable regions.

Advertising 42

Advertising Testing Sales Statistics 42

Domino Data Lab

MAY 8, 2019

Finale Doshi-Velez, Been Kim (2017-02-28) ; see also the Domino blog article about TCAV. They also require advanced skills in statistics, experimental design, causal inference, and so on – more than most data science teams will have. Use of influence functions goes back to the 1970s in robust statistics. 2018-06-21).

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

Occam's Razor

MARCH 23, 2010

Part of it is fueled by a vocal minority genuinely upset that 10 years on we are still not a statistically powered bunch doing complicated analysis that is shifting paradigms. In this blog post let me share with you some ground truths from my own humble experience. Part of it fueled by some Consultants. Don't fall for the FUD.

Analytics 118

Analytics Consulting Metrics Experimentation 118

The Unofficial Google Data Science Blog

OCTOBER 21, 2015

We’re not going to engage in this debate but in this blog post we do focus on science. It is important to make clear distinctions among each of these, and to advance the state of knowledge through concerted observation, modeling and experimentation. Note also that this account does not involve ambiguity due to statistical uncertainty.

Slice and Dice 45

Slice and Dice Experimentation Data-driven Data Science 45

Darkhorse

NOVEMBER 8, 2016

Strip plots in the flesh: NPR looks at school funding The LA Times on pop music’s evolution Eurostat’s key European statistics Jittered Strip Plot. Choosing the right number of bins can have an impact on how any of these charts look, but a bit of experimentation usually leads to a reasonable answer. SUBSCRIBE TO OUR BLOG.

Visualization 72

Visualization Experimentation Measurement Statistics 72

Occam's Razor

JANUARY 9, 2012

You've probably seen these rules sneak into my blog posts. With 55k RSS Subscribers and 110k Visits a month, this blog could make a few dollars with AdSense or Display ads or annoying interstitials offers. I've talked about the long tail on this blog, especially in context for Search. Via incredible daily utility.

Marketing 140

Marketing B2B Measurement Risk 140

Occam's Razor

SEPTEMBER 13, 2016

Like this blog, the newsletter is geared towards being instantly actionable (rather than just theory-smart, which is pretty cool too). Hypothesis development and design of experimentation. Ok, maybe statistical modeling smells like an analytical skill. Pattern recognition and understanding trends. Strategic thinking skills.

Modeling 127

Modeling Reporting Strategy Consulting 127

Occam's Razor

JUNE 4, 2018

I mean developing and inserting a subtle collection of gentle nudges that can help increase the conversion rate by a statistically significant amount. Yes, that additional $6 mil is not as hard to accomplished for an imaginative focused team – in fact you can get that from implementing half of the recommendations in this blog post.

Strategy 124

Strategy Cost-Benefit Testing Sales 124

Occam's Razor

JUNE 6, 2007

It has been such an amazing journey to write the book, and for it to come up almost exactly a year after I started this blog. Experimentation & Testing (A/B, Multivariate, you name it). The book, like this blog, rips up that definition and provides a expanded and more realistic business focused world view. There I said it.

Analytics 60

Analytics Metrics Marketing Dashboards 60