Occam's Razor

NOVEMBER 17, 2008

There is a tendency to think experimentation and testing is optional. Just don't fall for their bashing of all other vendors or their silly claims, false, of "superiority" in terms of running 19 billion combinations of tests or the bonus feature of helping you into your underwear each morning. And I meant every word of it.

Testing 112

Testing Experimentation Risk Optimization 112

Sisense

JANUARY 21, 2020

One of the simplest ways to start exploring your data is to aggregate the metrics you are interested in by their relevant dimensions. A new drug promising to reduce the risk of heart attack was tested with two groups. When the data is combined, it seems that the drug reduces the risk of getting a heart attack.

Testing 104

Testing Risk Data-driven Statistics 104

CIO Business Intelligence

MAY 16, 2023

It surpasses blockchain and metaverse projects, which are viewed as experimental or in the pilot stage, especially by established enterprises. Information governance enables enterprises to achieve strategic goals, mitigate risk, and reduce costs. Conversations suggest that AI is already transforming most major industries.

Advertising 87

Advertising Recreation/Entertainment Experimentation Enterprise 87

AWS Big Data

MAY 2, 2023

If your updates to a dataset triggers multiple subsequent DAGs, then you can use the Airflow metric max_active_tasks_per_dag to control the parallelism of the consumer DAG and reduce the chance of overloading the system. Test the feature To test this feature, run the producer DAG. Removal of experimental Smart Sensors.

Testing 100

Testing Experimentation Management Metadata 100

The Unofficial Google Data Science Blog

OCTOBER 12, 2017

With more features come more potential post hoc hypotheses about what is driving metrics of interest, and more opportunity for exploratory analysis. Yet when we use these tools to explore data and look for anomalies or interesting features, we are implicitly formulating and testing hypotheses after we have observed the outcomes.

Experimentation 61

Experimentation Testing Statistics Metrics 61

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Another pattern that I’ve seen in good PMs is that they’re very metric-driven. Transcript.

Management 110

Management Machine Learning Experimentation Metrics 110

The Unofficial Google Data Science Blog

JANUARY 14, 2016

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

CIO Business Intelligence

MAY 1, 2024

Pilots can offer value beyond just experimentation, of course. McKinsey reports that industrial design teams using LLM-powered summaries of user research and AI-generated images for ideation and experimentation sometimes see a reduction upward of 70% in product development cycle times. Now nearly half of code suggestions are accepted.

Cost-Benefit 140

Cost-Benefit Metrics Insurance Measurement 140

CIO Business Intelligence

JUNE 27, 2022

Tyson: You’re coming up to two years since you added Sentry’s ability to monitor for performance which has some pretty fine-grained metrics in terms of identifying where bottlenecks are located in code. We rely heavily on automated testing. Tyson: That belief in your vision when it’s tested—that is tough! I thought, really?!

Software 99

Software Experimentation Testing Marketing 99

DataRobot Blog

JANUARY 10, 2023

ML model builders spend a ton of time running multiple experiments in a data science notebook environment before moving the well-tested and robust models from those experiments to a secure, production-grade environment for general consumption. Capabilities Beyond Classic Jupyter for End-to-end Experimentation. Auto-scale compute.

Experimentation 52

Experimentation Machine Learning Data Science Modeling 52

Occam's Razor

MARCH 23, 2010

Because every tool uses its own sweet metrics definitions, cookie rules, session start and end rules and so much more. Yet case studies in some sense reduced risk, even if they were simply over blown marketing fluff written by the vendor. Usually at least a test. There is such little risk to actually trying. Likely not.

Analytics 118

Analytics Consulting Metrics Experimentation 118

Domino Data Lab

AUGUST 26, 2019

This control prevents overfitting and reduction in predictive accuracy on new test data. If we don’t like the result of testing after training, we can also select an entirely different learning machine with different internals. algorithm leaf_size metric metric_params n_jobs n_neighbors p weights.

Testing 79

Testing Modeling Machine Learning Metrics 79

Domino Data Lab

MAY 8, 2019

That’s a risk in case, say, legislators – who don’t understand the nuances of machine learning – attempt to define a single meaning of the word interpret. Given how so much of IT gets driven by concerns about risks and costs, in practice auditability tops the list for many business stakeholders. Ergo, less interpretable.

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

O'Reilly on Data

MAY 14, 2020

AI PMs should enter feature development and experimentation phases only after deciding what problem they want to solve as precisely as possible, and placing the problem into one of these categories. Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model.

Management 363

Management Experimentation B2B Machine Learning 363

The Unofficial Google Data Science Blog

MAY 31, 2016

Similarly, we could test the effectiveness of a search ad compared to showing only organic search results. A geo experiment is an experiment where the experimental units are defined by geographic regions. In other words, iROAS is the slope of a curve of the response metric plotted against the underlying advertising spend.

Advertising 42

Advertising Testing Sales Statistics 42

Occam's Razor

JULY 6, 2016

Yes, I worry that Analysts, and Marketers, are spending too much time with their head buried in custom reports and advance segments and smart calculated metrics and strategic or tactical dashboards. Lab usability testing and online surveys both provide great strategies to obsess about user centric design. United Breaks Hearts.

Testing 86

Testing Marketing Experimentation Cost-Benefit 86

CIO Business Intelligence

MARCH 5, 2024

Many other platforms, such as Coveo’s Relative Generative Answering , Quickbase AI , and LaunchDarkly’s Product Experimentation , have embedded virtual assistant capabilities but don’t brand them copilots. Today, top AI-assistant capabilities delivering results include generating code, test cases, and documentation.

Cost-Benefit 115

Cost-Benefit Advertising Experimentation Software 115

Occam's Razor

OCTOBER 23, 2006

Start with measuring these Outcomes metrics (revenue, leads, profit margins, improved product mix, number of new customers etc). Get competitive data (we are at x% of zz metric and our competition is at x+9% of zz metric). Great for a couple months and then you lose the audience. 6 Reporting is not Analysis. Your Choice? .

Data-driven 117

Data-driven Metrics KPI Reporting 117

CIO Business Intelligence

JULY 11, 2023

For example, a good result in a single clinical trial may be enough to consider an experimental treatment or follow-on trial but not enough to change the standard of care for all patients with a specific disease. A provider should be able to show a customer or a regulator the test suite that was used to validate each version of the model.

Modeling 71

Modeling Testing Experimentation Risk Management 71

Occam's Razor

JANUARY 9, 2012

If you are doing lame stuff, why try harder in an analytics context by asking for Economic Value or Visitor Loyalty or Conversation Rate or a thousand other super powerful and insightful metrics ? We don't take risk and try things, imaginative (possibly glorious) things, because we believe the price of failure is so high.

Marketing 140

Marketing B2B Measurement Risk 140

DataRobot Blog

MARCH 16, 2023

Why model-driven AI falls short of delivering value Teams that just focus model performance using model-centric and data-centric ML risk missing the big picture business context. New Snowflake integrations and the SAP joint solution have tightened the data to experimentation to deployment loop.

Experimentation 98

Experimentation Data-driven Modeling Metadata 98

CIO Business Intelligence

APRIL 18, 2022

They run the risk of miscommunication and misaligned business, technology, and operational strategy across the CXO team. They also advise communicating the dashboard’s value consistently since that will drive effective dashboard use, both to increase adoption and to improve company performance on key dashboard metrics, the brief says.

Digital Transformation 145

Digital Transformation Dashboards Cost-Benefit Consulting 145

CIO Business Intelligence

FEBRUARY 3, 2023

He says that IT remains largely in-house across helpdesk, data analytics, cybersecurity and development, bar small pockets of outsourced capability for software development and testing, and suggests that business growth hasn’t been the only challenge—not least in the days after Queen Elizabeth II’s death last September.

IT 52

IT Experimentation Scorecard Digital Transformation 52

The Unofficial Google Data Science Blog

JUNE 30, 2016

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. A naïve way to solve this problem would be to compare the proportion of buyers between the exposed and unexposed groups, using a simple test for equality of means.

Statistics 44

Statistics Optimization Modeling Experimentation 44

Cloudera

JUNE 6, 2019

In this case, an individual with different types of workloads may have access to multiple clusters; the admin trades increased risk of a noisy neighbor for better infrastructure efficiency. 3) By workload priority. A third strategy splits clusters based on the overall priority of the workloads running on those clusters. Cloudera Manager 6.2

Metadata 85

Metadata Data Lake Optimization Strategy 85

DataRobot Blog

MARCH 7, 2023

DataRobot on Azure accelerates the machine learning lifecycle with advanced capabilities for rapid experimentation across new data sources and multiple problem types. The capability to rapidly build an AI-powered organization with industry-specific solutions and expertise. The DataRobot AI Platform is the next generation of AI.

Data-driven 59

Data-driven Machine Learning Experimentation Data Lake 59

IBM Big Data Hub

SEPTEMBER 25, 2023

Large, untested workloads run the risk of hogging all the resources. If the exploratory work needs to move on to testing and production, they can plan appropriately. Data Exploration and Innovation: The flexibility of Presto has encouraged data exploration and experimentation at Uber. This way, the queries run much faster.

OLAP 94

OLAP Data Lake Data-driven Snapshot 94