CIO Business Intelligence

OCTOBER 5, 2023

Sandeep Davé knows the value of experimentation as well as anyone. As chief digital and technology officer at CBRE, Davé recognized early that the commercial real estate industry was ripe for AI and machine learning enhancements, and he and his team have tested countless use cases across the enterprise ever since.

Experimentation 125

Experimentation Strategy Machine Learning Interactive 125

Cloudera

FEBRUARY 16, 2022

In this example, the Machine Learning (ML) model struggles to differentiate between a chihuahua and a muffin. Will the model correctly determine it is a muffin or get confused and think it is a chihuahua? The extent to which we can predict how the model will classify an image given a change input (e.g. Model Visibility.

Machine Learning 84

Machine Learning Modeling Metadata Recreation/Entertainment 84

Dataiku

MAY 23, 2023

Once a data science project has progressed through the stages of data cleaning and preparation, analysis and experimentation, modeling, testing, and evaluation, it reaches a critical point.

Data Science 52

Data Science Experimentation Machine Learning Testing 52

Decision Management Solutions

DECEMBER 17, 2021

In the recent McKinsey article discussing designing next-generation credit-decisioning models they outlined four best practices for automated credit-decisioning models for banks as they continue their digital transformations. Digital lending based on high-performance credit-decisioning models, says McKinsey, lead to: Increased revenue.

Modeling 52

Modeling Experimentation Visualization Risk 52

Occam's Razor

MAY 22, 2006

This post is a primer on the delightful world of testing and experimentation (A/B, Multivariate, and a new term from me: Experience Testing). Experimentation and testing help us figure out we are wrong, quickly and repeatedly and if you think about it that is a great thing for our customers, and for our employers.

Experimentation 103

Experimentation Testing Optimization Measurement 103

CIO Business Intelligence

APRIL 2, 2024

Joanne Friedman, PhD, CEO, and principal of smart manufacturing at Connektedminds, says orchestrating success in digital transformation requires a symphony of integration across disciplines : “CIOs face the challenge of harmonizing diverse disciplines like design thinking, product management, agile methodologies, and data science experimentation.

Management 125

Management Digital Transformation Testing Experimentation 125

The Unofficial Google Data Science Blog

JANUARY 16, 2018

We present data from Google Cloud Platform (GCP) as an example of how we use A/B testing when users are connected. Experimentation on networks A/B testing is a standard method of measuring the effect of changes by randomizing samples into different treatment groups. This simulation is based on the actual user network of GCP.

Testing 58

Testing Experimentation Measurement Modeling 58

Domino Data Lab

AUGUST 6, 2021

Model Risk Management is about reducing bad consequences of decisions caused by trusting incorrect or misused model outputs. Systematically enabling model development and production deployment at scale entails use of an Enterprise MLOps platform, which addresses the full lifecycle including Model Risk Management.

Risk Management 64

Risk Management Risk Modeling Enterprise 64

CIO Business Intelligence

APRIL 11, 2024

I can’t speak for the entire country but I know that the requirement to adopt a distance learning delivery model at the beginning of the pandemic for all levels of education ranging from K-12 to university studies forced a rapid deployment of technological solutions.

Technology 85

Technology Digital Transformation Experimentation Testing 85

DataKitchen

JANUARY 20, 2021

Develop/execute regression testing . Test data management and other functions provided ‘as a service’ . The center of excellence (COE) model leverages the DataOps team to solve real-world challenges. Examples of technologies that can be delivered ‘as a service’ include: Source code control repository. Deploy to production.

Metrics 243

Metrics Experimentation Measurement Testing 243

CIO Business Intelligence

JUNE 14, 2023

The certification focuses on the seven domains of the analytics process: business problem framing, analytics problem framing, data, methodology selection, model building, deployment, and lifecycle management. Organization: AWS Price: US$300 How to prepare: Amazon offers free exam guides, sample questions, practice tests, and digital training.

Big Data 126

Big Data Data Analytics Analytics Predictive Modeling 126

Dataiku

MARCH 25, 2024

This scenario is not science fiction but a glimpse into the capabilities of Multimodal Large Language Models (M-LLMs), where the convergence of various modalities extends the landscape of AI. But instead, a machine seamlessly identifies the scene and its location, provides a detailed description, and even suggests nearby attractions.

Visualization 132

Visualization Modeling Experimentation Testing 132

CIO Business Intelligence

NOVEMBER 7, 2022

A developing playbook of best practices for data science teams covers the development process and technologies for building and testing machine learning models. CIOs and CDOs should lead ModelOps and oversee the lifecycle Leaders can review and address issues if the data science teams struggle to develop models.

Machine Learning 128

Machine Learning Experimentation Data Science IoT 128

CIO Business Intelligence

JANUARY 9, 2024

That quote aptly describes what Dell Technologies and Intel are doing to help our enterprise customers quickly, effectively, and securely deploy generative AI and large language models (LLMs).Many We’re using our own databases, testing against our own needs, and building around specific problem sets.

Enterprise 112

Enterprise Experimentation Modeling Data-driven 112

CIO Business Intelligence

DECEMBER 19, 2023

Many technology investments are merely transitionary, taking something done today and upgrading it to a better capability without necessarily transforming the business or operating model. If CIOs don’t improve conversions from pilot to production, they may find their investors losing patience in the process and culture of experimentation.

Digital Transformation 142

Digital Transformation Unstructured Data Machine Learning Risk Management 142

Domino Data Lab

NOVEMBER 14, 2019

Model interpretability continues to spark public discourse among industry. We have covered model interpretability previously, including a proposed definition of machine learning (ML) interpretability. Yet there are tradeoffs to consider when selecting a model. Errors like these may occur when the model is being constructed.

Modeling 71

Modeling Machine Learning Data Science Experimentation 71

Cloudera

OCTOBER 18, 2023

In fact, some of the insights presented in this blog have been assisted by the power of large language models (LLMs), highlighting the synergy between human expertise and AI-driven insights. Testing: Generative AI can be used to generate test cases automatically, reducing the time and effort required for developers to test their code.

Cost-Benefit 78

Cost-Benefit Testing Machine Learning Optimization 78

The Unofficial Google Data Science Blog

OCTOBER 7, 2015

by HENNING HOHNHOLD, DEIRDRE O'BRIEN, and DIANE TANG In this post we discuss the challenges in measuring and modeling the long-term effect of ads on user behavior. We describe experiment designs which have proven effective for us and discuss the subtleties of trying to generalize the results via modeling.

Modeling 61

Modeling Experimentation Knowledge Discovery Testing 61

Ontotext

FEBRUARY 14, 2024

Ivory tower modeling We’ve seen too many models developed by isolated ontologists that don’t survive the first battle with the data. There’s a famous saying by a statistician, George Box, “All models are wrong, but some are useful.” ” So, how do you know whether your model is useful?

Recreation/Entertainment 105

Recreation/Entertainment Data Integration Modeling Data Quality 105

CIO Business Intelligence

FEBRUARY 21, 2024

For example, top researchers at Florida State University are now developing innovative large language models (LLMs) to help advance research in areas like material science and healthcare — going beyond gen AI used by the general public. We developed a model to predict student outcomes based on metrics from historical evidence,” he says. “We

Recreation/Entertainment 130

Recreation/Entertainment Experimentation Modeling Data-driven 130

CIO Business Intelligence

MARCH 10, 2022

Fujitsu remains very much interested in the mainframe market, with a new model still on its roadmap for 2024, and a move under way to “shift its mainframes and UNIX servers to the cloud, gradually enhancing its existing business systems to optimize the experience for its end-users.”

Sales 127

Sales Testing Experimentation Data Processing 127

CIO Business Intelligence

JUNE 28, 2023

Belcorp operates under a direct sales model in 14 countries. As Belcorp considered the difficulties it faced, the R&D division noted it could significantly expedite time-to-market and increase productivity in its product development process if it could shorten the timeframes of the experimental and testing phases in the R&D labs.

Digital Transformation 92

Digital Transformation Cost-Benefit Informatics Data mining 92

Domino Data Lab

MAY 15, 2019

Unfortunately, a common challenge that many industry people face includes battling “ the model myth ,” or the perception that because their work includes code and data, their work “should” be treated like software engineering. These steps also reflect the experimental nature of ML product management.

Machine Learning 111

Machine Learning Management Experimentation Uncertainty 111

CIO Business Intelligence

MARCH 7, 2023

OpenAI’s text-generating ChatGPT, along with its image generation cousin DALL-E, are the most prominent among a series of large language models, also known as generative language models or generative AI, that have captured the public’s imagination over the last year. And, he says, using generative AI for coding has worked well.

Enterprise 114

Enterprise IT Experimentation Unstructured Data 114

O'Reilly on Data

MARCH 31, 2020

Instead of writing code with hard-coded algorithms and rules that always behave in a predictable manner, ML engineers collect a large number of examples of input and output pairs and use them as training data for their models. This has serious implications for software testing, versioning, deployment, and other core development processes.

Management 332

Management Machine Learning Experimentation Metrics 332

IBM Big Data Hub

APRIL 18, 2024

While these large language model (LLM) technologies might seem like it sometimes, it’s important to understand that they are not the thinking machines promised by science fiction. Most experts categorize it as a powerful, but narrow AI model. A key trend is the adoption of multiple models in production.

Cost-Benefit 82

Cost-Benefit Modeling Manufacturing Interactive 82

Smarten

MAY 17, 2023

Create a Sandbox Environment If you are in the business of technology, you probably have a robust sandbox environment in which you can use new tools and leverage opportunities with test data to understand the value of tools like Azure OpenAI, GPT 3.x

Digital Transformation 52

Digital Transformation Experimentation Risk Testing 52

DataKitchen

OCTOBER 5, 2021

DataOps enables: Rapid experimentation and innovation for the fastest delivery of new insights to customers. The data scientist’s products are models and segmentations. Instead of focusing on a narrowly defined task with minimal testing and feedback, DataOps focuses on adding value. Create tests. What is DataOps.

Testing 157

Testing Dashboards Measurement Experimentation 157

CIO Business Intelligence

SEPTEMBER 6, 2023

Since ChatGPT’s release in November of 2022, there have been countless conversations on the impact of similar large language models. The use of AI-generated code is still in an experimental phase for many organizations due to numerous uncertainties such as its impact on security, data privacy, copyright, and more.

Software 118

Software Experimentation Risk Data Lake 118

CIO Business Intelligence

JULY 6, 2022

Where cloud is most effective: One area I think cloud is just going to be really effective is any area which involves experimentation and has a high opportunity cost. Because when you can experiment, you can potentially enter a new business quickly, test an idea. I just completely removed the opportunity cost. And you hear this a lot.

Experimentation 92

Experimentation Strategy Testing Risk 92

Corinium

JUNE 6, 2019

Fractal’s recommendation is to take an incremental, test and learn approach to analytics to fully demonstrate the program value before making larger capital investments. It is also important to have a strong test and learn culture to encourage rapid experimentation. What is the most common mistake people make around data?

Insurance 250

Insurance Analytics Forecasting Deep Learning 250

AWS Big Data

AUGUST 21, 2023

For the demo, we’re using the Amazon Titan foundation model hosted on Amazon Bedrock for embeddings, with no fine tuning. If you are interested in exploring the multi-modal model, please reach out to your AWS specialist. With OpenSearch’s Search Comparison Tool , you can compare the different approaches.

Data Processing 92

Data Processing Experimentation Visualization Metrics 92

DataRobot

SEPTEMBER 22, 2021

You have to pick the right model to meet expected performance goals. Train it, test it, tune it. You have to know how to take your model into production, manage it over time, enable the business to consume predictions from it, and use it to make the right decisions. You have to make your AI explainable and trustworthy.

Experimentation 85

Experimentation Testing Modeling Machine Learning 85

CIO Business Intelligence

NOVEMBER 13, 2023

Transformational leaders must ensure their organizations have the expertise to integrate new technologies effectively and the follow-through to test and troubleshoot thoroughly before going live. Leaders must clearly define what they want to achieve through digital transformation and how they plan to do it.

Digital Transformation 141

Digital Transformation Cost-Benefit Experimentation Measurement 141

Smart Data Collective

FEBRUARY 4, 2020

Current R&D Models Provide Diminishing Returns. Phase 0 is the first to involve human testing. Phase I involves dialing-in the proper dosage and further testing in a larger patient pool. Now, picture the same process using heuristic models, machine vision, and artificial intelligence. Some of the pieces are missing.

Experimentation 99

Experimentation Testing Modeling Strategy 99

IBM Big Data Hub

NOVEMBER 3, 2023

Knowing this, we designed IBM Event Automation to make event processing easy with a no-code approach to Apache Flink It gives you the ability to quickly test new ideas, reuse events to expand into new use cases, and help accelerate your time to value.”

Cost-Benefit 78

Cost-Benefit Experimentation Testing Digital Transformation 78

Domino Data Lab

JUNE 13, 2019

Recently, Chhavi Yadav (NYU) and Leon Bottou (Facebook AI Research and NYU) indicated in their paper, “ Cold Case: The Lost MNIST Digits ”, how they reconstructed the MNIST (Modified National Institute of Standards and Technology) dataset and added 50,000 samples to the test set for a total of 60,000 samples. Did they overfit the test set?

Testing 83

Testing Data Science Experimentation Metadata 83