O'Reilly on Data

MARCH 31, 2020

The model outputs produced by the same code will vary with changes to things like the size of the training data (number of labeled examples), network training parameters, and training run time. This has serious implications for software testing, versioning, deployment, and other core development processes.

Management 339

Management Machine Learning Experimentation Metrics 339

DataKitchen

SEPTEMBER 21, 2023

In the context of Data in Place, validating data quality automatically with Business Domain Tests is imperative for ensuring the trustworthiness of your data assets. Moreover, advanced metrics like Percentage Regional Sales Growth can provide nuanced insights into business performance. What is Data in Use?

Testing 169

Testing Data Quality Predictive Modeling Metrics 169

AWS Big Data

NOVEMBER 16, 2023

Building a starter version of anything can often be straightforward, but building something with enterprise-grade scale, security, resiliency, and performance typically requires knowledge and adherence to battle-tested best practices, and using the right tools and features in the right scenario. system implemented with Amazon Redshift.

Enterprise 83

Enterprise Data Warehouse Snapshot Cost-Benefit 83

Alation

OCTOBER 18, 2022

Alation is pleased to be named a dbt Metrics Partner and to announce the start of a partnership with dbt, which will bring dbt data into the Alation data catalog. In the modern data stack, dbt is a key tool to make data ready for analysis. Data Transformation in the Modern Data Stack.

Metadata 52

Metadata Metrics Recreation/Entertainment Data Quality 52

IBM Big Data Hub

JULY 28, 2022

Metadata enrichment is about scaling the onboarding of new data into a governed data landscape by taking data and applying the appropriate business terms, data classes and quality assessments so it can be discovered, governed and utilized effectively. Scalability and elasticity.

Metadata 56

Metadata Data Quality Machine Learning Statistics 56

DataKitchen

OCTOBER 11, 2021

The data engineer then emails the BI Team, who refreshes a Tableau dashboard. Figure 1: Example data pipeline with manual processes. There are no automated tests , so errors frequently pass through the pipeline. The pipeline has automated tests at each step, making sure that each step completes successfully.

Testing 152

Testing Metadata Dashboards Statistics 152

O'Reilly on Data

JUNE 19, 2019

In a previous post , we noted some key attributes that distinguish a machine learning project: Unlike traditional software where the goal is to meet a functional specification, in ML the goal is to optimize a metric. Quality depends not just on code, but also on data, tuning, regular updates, and retraining.

Modeling 196

Modeling Machine Learning Testing Metrics 196

Cloudera

JUNE 29, 2023

To ensure the stability of the US financial system, the implementation of advanced liquidity risk models and stress testing using (MI/AI) could potentially serve as a protective measure. However, because most institutions lack a modern data architecture , they struggle to manage, integrate and analyze financial data at pace.

Data Architecture 85

Data Architecture Risk Management Risk Management 85

AWS Big Data

FEBRUARY 1, 2023

We also used AWS Lambda for data processing. To further optimize and improve the developer velocity for our data consumers, we added Amazon DynamoDB as a metadata store for different data sources landing in the data lake. Clients access this data store with an API’s.

Optimization 52

Optimization Forecasting Data Lake Metadata 52

Alation

MAY 31, 2023

American Family Insurance: Governance by Design – Not as an Afterthought Who: Anil Kumar Kunden , Information Standards, Governance and Quality Specialist at AmFam Group When: Wednesday, June 7, at 2:45 PM Why attend: Learn how to automate and accelerate data pipeline creation and maintenance with data governance, AKA metadata normalization.

Data Governance 52

Data Governance Insurance Metadata Data-driven 52

Alation

JANUARY 25, 2022

As a reminder, here’s Gartner’s definition of data fabric: “A design concept that serves as an integrated layer (fabric) of data and connecting processes. In this blog, we will focus on the “integrated layer” part of this definition by examining each of the key layers of a comprehensive data fabric in more detail.

Metadata 52

Metadata IT Metrics Data-driven 52

DataKitchen

JUNE 24, 2021

GE formed its Digital League to create a data culture. One of the keys for our success was really focusing that effort on what our key business initiatives were and what sorts of metrics mattered most to our customers. Chapin also mentioned that measuring cycle time and benchmarking metrics upfront was absolutely critical. “It

ROI 211

ROI Metrics Measurement Cost-Benefit 211

datapine

JANUARY 31, 2022

The companies that are most successful at marketing in both B2C and B2B are using data and online BI tools to craft hyper-specific campaigns that reach out to targeted prospects with a curated message. Everything is being tested, and then the campaigns that succeed get more money put into them, while the others aren’t repeated.

Business Intelligence 214

Business Intelligence Analytics Cost-Benefit ROI 214

Alation

FEBRUARY 17, 2023

Reichental describes data governance as the overarching layer that empowers people to manage data well ; as such, it is focused on roles & responsibilities, policies, definitions, metrics, and the lifecycle of the data. In this way, data governance is the business or process side. Here’s an example.

Data Governance 52

Data Governance Data Quality Metadata Cost-Benefit 52

Alation

AUGUST 26, 2021

What Is Data Intelligence? Data intelligence is a system to deliver trustworthy, reliable data. It includes intelligence about data, or metadata. IDC coined the term, stating, “data intelligence helps organizations answer six fundamental questions about data.” Yet finding data is just the beginning.

Metadata 64

Metadata Data Governance Dashboards Software 64

DataKitchen

JANUARY 23, 2023

DataOps Observability includes monitoring and testing the data pipeline, data quality, data testing, and alerting. Data testing can be done through various methods, such as data profiling, Statistical Process Control, and quality checks. Are problems with data tests?

Testing 130

Testing Data Governance Data Quality Data-driven 130

AWS Big Data

AUGUST 14, 2023

Goals of DPPM The goals of DPPM can be summarized as follows: Protect value – DPPM protects the value of the organizational data strategy by developing, implementing, and enforcing frameworks to measure the contribution of data products to organizational goals in objective terms. Service Validation and Testing X X.

Management 78

Management Risk Measurement Optimization 78

Andrew White

JANUARY 11, 2021

Do you have an example of how an organization improved data literacy in a really practical useful way? Storytelling is a nice one to use early on to test the approach. I believe the focus would align to governance policies that are in focus: quality, privacy, security, retention, access and ethics, etc. Yes, and no.

Data Analytics 52

Data Analytics Analytics Data-driven Finance 52