CIO Business Intelligence

JUNE 15, 2023

Despite headlines warning that artificial intelligence poses a profound risk to society , workers are curious, optimistic, and confident about the arrival of AI in the enterprise, and becoming more so with time, according to a recent survey by Boston Consulting Group (BCG). For many, their feelings are based on sound experience.

Management 115

Management Experimentation Optimization Enterprise 115

CIO Business Intelligence

OCTOBER 5, 2023

Sandeep Davé knows the value of experimentation as well as anyone. As chief digital and technology officer at CBRE, Davé recognized early that the commercial real estate industry was ripe for AI and machine learning enhancements, and he and his team have tested countless use cases across the enterprise ever since.

Experimentation 129

Experimentation Strategy Machine Learning Interactive 129

CIO Business Intelligence

DECEMBER 19, 2023

If CIOs don’t improve conversions from pilot to production, they may find their investors losing patience in the process and culture of experimentation. And while static application security testing (SAST) was the top-rated tool for usefulness by 82% of respondents, only 28% claim these tools are used on at least 75% of their code base.

Digital Transformation 143

Digital Transformation Unstructured Data Machine Learning Risk Management 143

Decision Management Solutions

MARCH 26, 2020

We’ve been blogging recently on Decision Optimization. The Customer Journey to Decision Optimization. Those trying to improve and optimize their decisions report various challenges. Some approaches have never been tried on certain segments – higher risk customers might never have been targeted with price reductions, say.

Experimentation 72

Experimentation Recreation/Entertainment Optimization Testing 72

CIO Business Intelligence

JUNE 28, 2023

As Belcorp considered the difficulties it faced, the R&D division noted it could significantly expedite time-to-market and increase productivity in its product development process if it could shorten the timeframes of the experimental and testing phases in the R&D labs.

Digital Transformation 101

Digital Transformation Cost-Benefit Informatics Data mining 101

CIO Business Intelligence

NOVEMBER 7, 2022

A developing playbook of best practices for data science teams covers the development process and technologies for building and testing machine learning models. Have business leaders defined realistic success criteria and areas of low-risk experimentation? Are data science teams set up for success?

Machine Learning 128

Machine Learning Data Science Experimentation IoT 128

Occam's Razor

NOVEMBER 17, 2008

There is a tendency to think experimentation and testing is optional. You can start for free with a superb tool: Google's Website Optimizer. For example I am quite fond of the fact that with Offermatica you can "trigger" tests based on behavior. I cannot recommend enough the wisdom of starting with a A/B test.

Testing 112

Testing Experimentation Risk Optimization 112

CIO Business Intelligence

JANUARY 9, 2024

Many organizations know that commercially available, “off-the-shelf” generative AI models don’t work well in enterprise settings because of significant data access and security risks. We’re using our own databases, testing against our own needs, and building around specific problem sets.

Enterprise 118

Enterprise Experimentation Modeling Data-driven 118

The Unofficial Google Data Science Blog

JULY 22, 2020

One reason to do ramp-up is to mitigate the risk of never before seen arms. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. For example, imagine a fantasy football site is considering displaying advanced player statistics.

Experimentation 95

Experimentation Statistics Testing Strategy 95

O'Reilly on Data

SEPTEMBER 15, 2020

Technical competence results in reduced risk and uncertainty. AI initiatives may also require significant considerations for governance, compliance, ethics, cost, and risk. Likewise, AI doesn’t inherently optimize supply chains, detect diseases, drive cars, augment human intelligence, or tailor promotions to different market segments.

Cost-Benefit 293

Cost-Benefit Advertising ROI Machine Learning 293

CIO Business Intelligence

OCTOBER 18, 2023

But if there are any stop signs ahead regarding risks and regulations around generative AI, most enterprise CIOs are blowing past them, with plans to deploy an abundance of gen AI applications within the next two years if not already. in concert with Microsoft’s AI-optimized Azure platform.

Risk 141

Risk Manufacturing Enterprise Technology 141

IBM Big Data Hub

APRIL 18, 2024

While leaders have some reservations about the benefits of current AI, organizations are actively investing in gen AI deployment, significantly increasing budgets, expanding use cases, and transitioning projects from experimentation to production. AGI wouldn’t just perceive its surroundings; it would understand them.

Cost-Benefit 97

Cost-Benefit Modeling Manufacturing Interactive 97

CIO Business Intelligence

MARCH 7, 2023

Oliver Wittmaier, CIO and product owner at DB SYSTEL GmbH DB SYSTEL GmbH Content generation is also an area of particular interest to Michal Cenkl, director of innovation and experimentation at Mitre Corp. “I Michal Cenkl, director of innovation and experimentation, Mitre Corp. Mitre Corp. The technology also needs human oversight.

Enterprise 116

Enterprise IT Experimentation Unstructured Data 116

CIO Business Intelligence

OCTOBER 9, 2023

Hyatt’s experimental mindset and listen-first approach are heavily applied to IT’s pursuit of innovation, he says. In the rush to innovate, teams can forget the basics, such as insufficient testing, so constant monitoring is crucial to ensure you’re providing value, adds TFCU’s Renganathan. He learned that the hard way.

Strategy 137

Strategy Technology Experimentation Testing 137

IBM Big Data Hub

DECEMBER 14, 2023

It is very easy to turn a terraform template into a deployable architecture and open the door to cost optimization, compliance management, self-service, standardization and repeatability. The extension of the CIO office’s dedicated data center to the IBM public cloud allowed for experimentation and growth with zero risk.

Experimentation 74

Experimentation Testing Software Management 74

O'Reilly on Data

MARCH 31, 2020

This has serious implications for software testing, versioning, deployment, and other core development processes. The need for an experimental culture implies that machine learning is currently better suited to the consumer space than it is to enterprise companies. And you, as the product manager, are caught between them.

Management 332

Management Machine Learning Experimentation Metrics 332

AWS Big Data

JULY 3, 2023

This helps traders determine the potential profitability of a strategy and identify any risks associated with it, enabling them to optimize it for better performance. Solution overview To set up and test this experiment, we complete the following high-level steps: Create an S3 bucket. Load the dataset into Amazon S3.

Snapshot 82

Snapshot Data Lake Testing Strategy 82

The Unofficial Google Data Science Blog

OCTOBER 12, 2017

Yet when we use these tools to explore data and look for anomalies or interesting features, we are implicitly formulating and testing hypotheses after we have observed the outcomes. We must correct for multiple hypothesis tests. Make experimentation cheap and understand the cost of bad decisions. We ought not dredge our data.

Experimentation 61

Experimentation Testing Statistics Metrics 61

FineReport

DECEMBER 11, 2019

Effective use of big data helps companies analyze critical information more accurately, ultimately improving operational efficiency, reducing costs, reducing risk, accelerating innovation, and increasing revenue. – Data Divination: Big Data Strategies.

Big Data 98

Big Data Statistics Experimentation Data Strategy 98

CIO Business Intelligence

JUNE 27, 2022

We rely heavily on automated testing. It’s not perfect by any means, and we are continuously breaking our own product, but it’s optimized for shipping new features to customers as quickly as we can. We knew we had a great product, with a number of paying customers and a fair approach to pricing so churn risk was low.

Software 113

Software Experimentation Testing Marketing 113

DataRobot Blog

JANUARY 10, 2023

ML model builders spend a ton of time running multiple experiments in a data science notebook environment before moving the well-tested and robust models from those experiments to a secure, production-grade environment for general consumption. Capabilities Beyond Classic Jupyter for End-to-end Experimentation. Auto-scale compute.

Experimentation 52

Experimentation Machine Learning Data Science Modeling 52

CIO Business Intelligence

MAY 1, 2024

Pilots can offer value beyond just experimentation, of course. McKinsey reports that industrial design teams using LLM-powered summaries of user research and AI-generated images for ideation and experimentation sometimes see a reduction upward of 70% in product development cycle times.

Cost-Benefit 143

Cost-Benefit Metrics Insurance Measurement 143

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Yet, this challenge is not insurmountable. for what is and isn’t possible) to address these challenges. Transcript.

Management 110

Management Machine Learning Experimentation Metrics 110

CIO Business Intelligence

APRIL 24, 2024

That includes many technologies based on machine learning, such as sales forecasting, lead scoring and qualification, pricing optimization, and customer sentiment analysis. We’re mostly still optimizing our sales and marketing processes with CRM tools,” he says. A CFO would just say to wait and see what the risks are,” he says.

Sales 128

Sales Finance Experimentation Cost-Benefit 128

Domino Data Lab

MAY 8, 2019

That’s a risk in case, say, legislators – who don’t understand the nuances of machine learning – attempt to define a single meaning of the word interpret. Given how so much of IT gets driven by concerns about risks and costs, in practice auditability tops the list for many business stakeholders. Ergo, less interpretable.

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

Domino Data Lab

JULY 2, 2019

SQL optimization provides helpful analogies, given how SQL queries get translated into query graphs internally , then the real smarts of a SQL engine work over that graph. The query graph provides metadata that gets leveraged for optimizations at multiple layers of the relational database stack. SQL and Spark. That’s good stuff.

Metadata 105

Metadata Machine Learning Data Science Data-driven 105

Domino Data Lab

AUGUST 26, 2019

This control prevents overfitting and reduction in predictive accuracy on new test data. When hyperparameters are also used for optimization and tuned for a specific dataset, they also impact predictive accuracy. Searching is a very general approach to solving or optimizing problems. We need some sort of well-defined process?

Testing 79

Testing Modeling Machine Learning Metrics 79

Occam's Razor

MARCH 23, 2010

My problem with these mistruths and FUD is that they result in a ton of practitioners and companies making profoundly sub optimal choices, which in turn results in not just much longer slogs but also spectacular career implosions and the entire web analytics industry suffering. Usually at least a test. This is sad. Likely not.

Analytics 118

Analytics Consulting Metrics Experimentation 118

CIO Business Intelligence

MARCH 5, 2024

Many other platforms, such as Coveo’s Relative Generative Answering , Quickbase AI , and LaunchDarkly’s Product Experimentation , have embedded virtual assistant capabilities but don’t brand them copilots. Today, top AI-assistant capabilities delivering results include generating code, test cases, and documentation.

Cost-Benefit 124

Cost-Benefit Advertising Experimentation Software 124

CIO Business Intelligence

MARCH 7, 2023

But the faster transition often caused underperforming apps, greater security risks, higher costs, and fewer business outcomes, forcing IT to address these issues before starting app modernizations. Release an updated data viz, then automate a regression test. billion by 2028 , rising at a market growth of 20.3%

Digital Transformation 145

Digital Transformation Data Governance Experimentation Machine Learning 145

Occam's Razor

JANUARY 9, 2012

I've discovered that if we can just get them to imagine a better existence, undertake serious risks, experiment with new better ideas, and spend money executing them… they will ask for more robust measurement! All while constantly optimizing your portfolio via controlled experiments. AND you can control for risk!

Marketing 140

Marketing B2B Measurement Risk 140

O'Reilly on Data

MARCH 23, 2023

What is it, how does it work, what can it do, and what are the risks of using it? Many of these go slightly (but not very far) beyond your initial expectations: you can ask it to generate a list of terms for search engine optimization, you can ask it to generate a reading list on topics that you’re interested in.

IT 262

IT Modeling Testing Risk 262

The Unofficial Google Data Science Blog

JUNE 30, 2016

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. A naïve way to solve this problem would be to compare the proportion of buyers between the exposed and unexposed groups, using a simple test for equality of means.

Statistics 44

Statistics Optimization Modeling Experimentation 44

Occam's Razor

OCTOBER 23, 2006

Be incessantly focussed on your company customers and dragging their voice to the table (for example via experimentation and testing or via open ended survey questions). Web analytics should be owned by a business function, optimally the one that owns the web strategy (not the web site, web strategy). Very easy to do.

Data-driven 117

Data-driven Metrics KPI Reporting 117

CIO Business Intelligence

APRIL 5, 2023

Midjourney, ChatGPT, Bing AI Chat, and other AI tools that make generative AI accessible have unleashed a flood of ideas, experimentation and creativity. Testing is another area that tends to get neglected, so automated unit test generation will help you get much broader test coverage.

Modeling 109

Modeling Testing Cost-Benefit Experimentation 109

Cloudera

JUNE 6, 2019

This strategy works well for managing internal chargebacks, limiting the impact of less sophisticated users on more experienced users, and overall encouraging individuals to think about and optimize their jobs and queries now that they have a smaller (but dedicated) cluster. 2) By workload type. 3) By workload priority.

Metadata 86

Metadata Data Lake Optimization Strategy 86

DataRobot

JANUARY 20, 2022

— Collaborating via Snowflake Data Cloud and DataRobot AI Cloud Platform will enable multiple organizations to build a community movement where experimentation, innovation, and creativity flourish. Specialized teams from DataRobot and Snowflake will enable ICSs to mitigate data governance and model bias risk with confidence.

Data-driven 145

Data-driven Experimentation Modeling Predictive Modeling 145