O'Reilly on Data

MARCH 31, 2020

Because it’s so different from traditional software development, where the risks are more or less well-known and predictable, AI rewards people and companies that are willing to take intelligent risks, and that have (or can develop) an experimental culture. And you, as the product manager, are caught between them.

Management 333

Management Machine Learning Experimentation Metrics 333

CIO Business Intelligence

JUNE 28, 2023

For all of generative AI’s allure, large enterprises are taking their time, many outright banning tools like ChatGPT over concerns of accuracy, data protection, and the risk of regulatory backlash. Experimentation with a use case driven approach. By that measure, you will indeed have done better than you thought.

Enterprise 52

Enterprise Experimentation Risk Data-driven 52

CIO Business Intelligence

SEPTEMBER 12, 2022

Prioritize time for experimentation. One instance of how that exploration led to real business benefits was with the application of machine learning to predict optimal product formulation using a set of desired consumer benefits. Here, they and others share seven ways to create and nurture a culture of innovation.

Experimentation 133

Experimentation Consulting Technology Machine Learning 133

CIO Business Intelligence

OCTOBER 9, 2023

Hyatt’s experimental mindset and listen-first approach are heavily applied to IT’s pursuit of innovation, he says. That’s a pitfall, and it is critical to always have a very comprehensive risk assessment practice in place,” Renganathan says. “We That meant having to let “the urgent things interrupt the important things,” he says.

Strategy 137

Strategy Technology Experimentation Testing 137

IBM Big Data Hub

APRIL 18, 2024

While leaders have some reservations about the benefits of current AI, organizations are actively investing in gen AI deployment, significantly increasing budgets, expanding use cases, and transitioning projects from experimentation to production. AGI wouldn’t just perceive its surroundings; it would understand them.

Cost-Benefit 90

Cost-Benefit Modeling Manufacturing Interactive 90

Ontotext

NOVEMBER 2, 2023

The data itself is stored in a way that is not optimal for extracting insight. And while there is a great deal of experimentation underway, most organizations have only scratched the surface in a use-case-by-use-case fashion. This is partly because integrating and moving data is not the only problem.

IT 69

IT Cost-Benefit Data-driven Technology 69

CIO Business Intelligence

MAY 1, 2024

Pilots can offer value beyond just experimentation, of course. McKinsey reports that industrial design teams using LLM-powered summaries of user research and AI-generated images for ideation and experimentation sometimes see a reduction upward of 70% in product development cycle times. What are you measuring?

Cost-Benefit 143

Cost-Benefit Metrics Insurance Measurement 143

O'Reilly on Data

MAY 14, 2020

AI PMs should enter feature development and experimentation phases only after deciding what problem they want to solve as precisely as possible, and placing the problem into one of these categories. Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model.

Management 363

Management Experimentation B2B Machine Learning 363

The Unofficial Google Data Science Blog

JULY 22, 2020

One reason to do ramp-up is to mitigate the risk of never before seen arms. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. For example, imagine a fantasy football site is considering displaying advanced player statistics.

Experimentation 95

Experimentation Statistics Testing Strategy 95

O'Reilly on Data

SEPTEMBER 15, 2020

Technical sophistication: Sophistication measures a team’s ability to use advanced tools and techniques (e.g., Technical competence: Competence measures a team’s ability to successfully deliver on initiatives and projects. Technical competence results in reduced risk and uncertainty.

Cost-Benefit 294

Cost-Benefit Advertising ROI Machine Learning 294

IBM Big Data Hub

OCTOBER 20, 2023

This unified experience optimizes the process of developing and deploying ML models by streamlining workflows for increased efficiency. Decision optimization: Streamline the selection and deployment of optimization models and enable the creation of dashboards to share results, enhance collaboration and recommend optimal action plans.

Machine Learning 72

Machine Learning Manufacturing Cost-Benefit Deep Learning 72

CIO Business Intelligence

JULY 7, 2023

As more individuals use browser-based apps to get their work done, IT leaders need to provide seamless access to corporate apps and tools while minimizing security risks. A security-by-design culture incorporates security measures deeply into the design and development of systems, rather than treating them as an afterthought.

Measurement 122

Measurement Experimentation Risk Enterprise 122

The Unofficial Google Data Science Blog

JUNE 30, 2016

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. It should be noted that inverse probability weighting is not generally optimal (i.e., This is often referred to as the positivity assumption. the curse of dimensionality).

Statistics 44

Statistics Optimization Modeling Experimentation 44

Cloudera

APRIL 26, 2023

These include data recovery service, quota management, node harvesting, optimizing TCO, and more. Better innovation , first by enabling end users to adopt new features faster for better insights, and second, by allowing developers to run experimental workloads without risking production stability, fostering a culture of innovation.

Cost-Benefit 64

Cost-Benefit Optimization Experimentation Machine Learning 64

Corinium

APRIL 25, 2019

For example, P&C insurance strives to understand its customers and households better through data, to provide better customer service and anticipate insurance needs, as well as accurately measure risks. Life insurance needs accurate data on consumer health, age and other metrics of risk. Now, there is a data risk here.

Insurance 150

Insurance Risk IoT Cost-Benefit 150

O'Reilly on Data

MARCH 23, 2023

What is it, how does it work, what can it do, and what are the risks of using it? Many of these go slightly (but not very far) beyond your initial expectations: you can ask it to generate a list of terms for search engine optimization, you can ask it to generate a reading list on topics that you’re interested in. What Are the Risks?

IT 261

IT Modeling Testing Risk 261

CIO Business Intelligence

JUNE 6, 2023

CIOs should have plans for the upsides and also develop risk mitigation strategies. CIOs should define a specialized co-creation model , optimized for collaborating with startups, that can be very different from their in-house development strategies and partnerships with larger service providers.

Key Performance Indicator 106

Key Performance Indicator Cost-Benefit Enterprise Experimentation 106

Domino Data Lab

FEBRUARY 4, 2019

Spoiler alert: a research field called curiosity-driven learning is emerging at the nexis of experimental cognitive psychology and industry use cases for machine learning, particularly in gaming AI. The ability to measure results (risk-reducing evidence). Legal concerns, risk, compliance. Lack of reproducible workflows.

Data Science 66

Data Science Experimentation Data-driven Machine Learning 66

Domino Data Lab

MAY 8, 2019

That’s a risk in case, say, legislators – who don’t understand the nuances of machine learning – attempt to define a single meaning of the word interpret. Visualizations are vital in data science work, with the caveat that the information that they convey may be 4-5 layers of abstraction away from the actual business process being measured.

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. These measurement-obsessed companies have an advantage when it comes to AI.

Management 110

Management Machine Learning Experimentation Metrics 110

Cloudera

JUNE 6, 2019

This strategy works well for managing internal chargebacks, limiting the impact of less sophisticated users on more experienced users, and overall encouraging individuals to think about and optimize their jobs and queries now that they have a smaller (but dedicated) cluster. 2) By workload type. 3) By workload priority.

Metadata 87

Metadata Data Lake Optimization Strategy 87

Occam's Razor

JANUARY 9, 2012

Most companies are astonishingly blasé about data and possibilities of measurement. " Sad, unimaginative measurements of their sad, unimaginative campaigns. All while constantly optimizing your portfolio via controlled experiments. I told 20 people that Nikon's site is slow and profoundly sub-optimal on mobile.

Marketing 140

Marketing B2B Measurement Risk 140

Occam's Razor

MARCH 23, 2010

My problem with these mistruths and FUD is that they result in a ton of practitioners and companies making profoundly sub optimal choices, which in turn results in not just much longer slogs but also spectacular career implosions and the entire web analytics industry suffering. This is sad. Even a little frustrating. Likely not.

Analytics 118

Analytics Consulting Metrics Experimentation 118

Cloudera

NOVEMBER 25, 2020

And soon also sensor measures, and possibly video or audio data with the increased use of device technology and telemedicine in medical care. Innovate on serviceability and optimize utilization. It would enable faster experimentation with easy, protected, and governed access to a variety of data.

Data Warehouse 59

Data Warehouse Unstructured Data Analytics Visualization 59

Occam's Razor

OCTOBER 23, 2006

Start with measuring these Outcomes metrics (revenue, leads, profit margins, improved product mix, number of new customers etc). Be incessantly focussed on your company customers and dragging their voice to the table (for example via experimentation and testing or via open ended survey questions). 6 Reporting is not Analysis.

Data-driven 117

Data-driven Metrics KPI Reporting 117

Domino Data Lab

AUGUST 26, 2019

When hyperparameters are also used for optimization and tuned for a specific dataset, they also impact predictive accuracy. Searching is a very general approach to solving or optimizing problems. I will continue using parameters (which are optimized during training) and hyperparameters (which are tuned via cross-validation).

Testing 79

Testing Modeling Machine Learning Metrics 79

The Unofficial Google Data Science Blog

OCTOBER 12, 2017

We data scientists now have access to tools that allow us to run a large numbers of experiments, and then to slice experimental populations by any combination of dimensions collected. Make experimentation cheap and understand the cost of bad decisions. This leads to the proliferation of post hoc hypotheses.

Experimentation 61

Experimentation Testing Statistics Metrics 61

Sisense

NOVEMBER 23, 2020

This is particularly relevant when looking for new opportunities and new growth, which requires looking at optimizations not needed in the past. As Scott explains, there is also the risk of data scientists being seen as the solvers of all business problems, and they become overloaded with irrelevant questions. Right tools/open source.

Analytics 98

Analytics Data-driven Marketing Reporting 98

O'Reilly on Data

MAY 11, 2020

The Entertainment” is not the result of algorithms, business incentives and product managers optimizing for engagement metrics. Television only lacked the immediate feedback that comes with clicks, tracking cookies, tracking pixels, online experimentation, machine learning, and “agile” product cycles.

Modeling 189

Modeling Advertising Recreation/Entertainment Metrics 189