DataKitchen

FEBRUARY 23, 2024

DataKitchen Resource Guide To Data Journeys & Data Observability & DataOps Data (and Analytic) Observability & Data Journey – Ideas and Background Data Journey Manifesto and Why the Data Journey Manifesto? Five Pillars of Data Journeys Data Journey First DataOps “You Complete Me,” said Data Lineage to Data Journeys. Bridging the Gap: How ‘Data in Place’ and ‘Data in Use’ Define Complete Data Observability The Need For Personalized Data Journeys for Your Data Consumers Data Te

Testing 120

Testing Data-driven Metrics Data Quality 120

DataKitchen

FEBRUARY 22, 2024

The Art of Data Buck-Passing 101: Mastering the Blame Game in Data and Analytic Teams Welcome, dear readers, to the hallowed halls of Data Buck-Passing University, where the motto is “ Per Alios Culpa Transfertur ” (Blame is Transferred to Others). In the world of data and analytics, one skill stands timeless and universal: the art of blaming someone else when things go sideways.

Analytics 190

Analytics Data Governance Strategy Testing 190

DataKitchen

DECEMBER 7, 2023

Do you have data quality issues, a complex technical environment, and a lack of visibility into production systems? These challenges lead to poor quality analytics and frustrated end users. Getting your data reliable is a start, but many other problems arise even if your data could be better. And your customers don't care where the problem is in your toolchain.

Uncertainty 130

Uncertainty Data Quality Dashboards Analytics 130

DataKitchen

NOVEMBER 10, 2023

The Perilous State of Today’s Data Environments Data teams often navigate a labyrinth of chaos within their databases. The core issue plaguing many organizations is the presence of out-of-control databases or data lakes characterized by: Unrestrained Data Changes: Numerous users and tools incessantly alter data, leading to a tumultuous environment. Extrinsic Control Deficit: Many of these changes stem from tools and processes beyond the immediate control of the data team.

Data Quality 141

Data Quality Testing Data Lake Data Integration 141

DataKitchen

NOVEMBER 6, 2023

This webinar discusses how to make embarrassing data errors a thing of the past. We will start with how data engineers do not understand their data and have difficulty identifying problematic data records. We will also discuss how the vast majority of data engineers are so busy that they don’t know, or have time to write, tests to write to find data errors.

Testing 130

Testing IT 130

DataKitchen

OCTOBER 20, 2023

In today’s data-driven landscape, Data and Analytics Teams i ncreasingly face a unique set of challenges presented by Demanding Data Consumers who require a personalized level of Data Observability. As opposed to receiving one-size-fits-all status updates, these key stakeholders desire real-time, granular insights into the status of their specific data as it traverses the complicated data production pipeline.

Insurance 176

Insurance Metadata Data-driven Data Quality 176

DataKitchen

OCTOBER 12, 2023

The meeting room was spacious, adorned with charts, graphs, and several cups of hastily grabbed coffee. I distinctly remember an afternoon set to work with a promising prospect. As my team and I waited, a flurry of messages informed us that the key players from the other side wouldn’t be joining. Why? Their entire data analytics team had been hastily yanked into an impromptu “war room” to tackle an emergency data issue.

Testing 120

Testing Data-driven Data Analytics Reporting 120

DataKitchen

OCTOBER 2, 2023

Data Teams and Their Types of Data Journeys In the rapidly evolving landscape of data management and analytics, data teams face various challenges ranging from data ingestion to end-to-end observability. This comprehensive article delves into the complexities encountered by various types of data teams—Data Ingestion Teams, End-to-End Data Product Teams, and Enterprise Data Enablement Teams—to name a few.

Data Quality 100

Data Quality Testing Uncertainty Data Enablement 100

DataKitchen

SEPTEMBER 21, 2023

Bridging the Gap: How ‘Data in Place’ and ‘Data in Use’ Define Complete Data Observability In a world where 97% of data engineers report burnout and crisis mode seems to be the default setting for data teams, a Zen-like calm feels like an unattainable dream. Complaints from dissatisfied customers and apathetic data providers only add to the mounting stress.

Testing 176

Testing Data Quality Predictive Modeling Metrics 176

DataKitchen

SEPTEMBER 20, 2023

This webinar discusses how to make embarrassing data errors a thing of the past. We will start with how data engineers do not understand their data and have difficulty identifying problematic data records. We will also discuss how the vast majority of data engineers are so busy that they don’t know, or have time to write, tests to write to find data errors.

Testing 130

Testing IT 130

DataKitchen

AUGUST 24, 2023

Since the first edition of the DataOps Cookbook in 2019, we have talked with thousands of companies about their struggles to deliver data-driven insight to their customers. In many ways, they all have the same problems. They have built data and analytic systems with great hope of success. Still, they are burdened with too many errors, are overwhelmed with custom requests, and know they fail to succeed in their goal of leading their organizations to be more data-driven.

Data-driven 130

Data-driven Forecasting Metrics Dashboards 130

DataKitchen

AUGUST 8, 2023

On 20 July 2023, Gartner released the article “ Innovation Insight: Data Observability Enables Proactive Data Quality ” by Melody Chien. In the article, Melody Chien notes that Data Observability is a practice that extends beyond traditional monitoring and detection, providing robust, integrated visibility over data and data landscapes. It alerts data and analytics leaders to issues with their data before they multiply.

Data Quality 130

Data Quality Testing Snapshot Reporting 130

DataKitchen

JULY 27, 2023

The Ten Standard Tools To Develop Data Pipelines In Microsoft Azure. While working in Azure with our customers, we have noticed several standard Azure tools people use to develop data pipelines and ETL or ELT processes. We counted ten ‘standard’ ways to transform and set up batch data pipelines in Microsoft Azure. Is it overkill? Don’t they all do the same thing?

Machine Learning 130

Machine Learning Cost-Benefit Data Transformation Testing 130

DataKitchen

JULY 12, 2023

The Syntax, Semantics, and Pragmatics Gap in Data Quality Validate Testing Data Teams often have too many things on their ‘to-do’ list. Customers are asking for new data, people need questions answered, and the tech stack is barely running – data engineers don’t have time to create tests. They have a backlog full of new customer features or data requests, and they go to work every day knowing that they won’t and can’t meet customer expectations.

Data Quality 130

Data Quality Testing Manufacturing Finance 130

DataKitchen

JULY 3, 2023

Data Journey First DataOps Putting Problems in Your Data Estate at the Forefront Welcome to the high-octane world of DataOps, a powerhouse that turbocharges data analytics development and management. This innovative approach merges the agility of Agile Development, the stability of DevOps, and the meticulousness of Statistical Process Controls, orchestrating a dynamic, enriched, and nimble data ecosystem that is truly remarkable.

Testing 130

Testing Risk Data-driven Statistics 130

DataKitchen

JUNE 19, 2023

DataOps TestGen Technical Product Overview

130

130

DataKitchen

JUNE 19, 2023

DataOps TestGen delivers simple, fast data quality test generation and execution. DataOps TestGen profiles your data and algorithmically generates dozens of data validations. Your Data Journey starts with verifying that you can trust your data and re-verifying it every time you refresh it in production.

Data Quality 130

Data Quality Testing IT 130

DataKitchen

JUNE 19, 2023

The need to balance data safety with new data initiatives, deliver business value, and change company culture around data tops this year's list of data and analytics management challenges. To help bring new resources and innovation to light, each year, Database Trends and Applications magazine presents the DBTA 100, a list of forward-thinking companies seeking to expand what's possible with data for their customers.

Data-driven 246

Data-driven Software Technology Management 246

DataKitchen

JUNE 19, 2023

Introducing The Five Pillars Of Data Journeys “There are those who discover they can leave behind destructive reactions and become patient as the earth, unmoved by fires of anger or fear, unshaken as a pillar, unperturbed as a clear and quiet pool.” – Gautama Buddha When your data team is in crisis from errors in production, complaining customers, and uncaring data providers, we all wish we could be unshaken as the Buddha.

Testing 130

Testing Cost-Benefit Metrics Data Quality 130

DataKitchen

JUNE 12, 2023

Why the Data Journey Manifesto? So why another manifesto in the world? Really? Why should I care? About seven years ago, we wrote the DataOps Manifesto. We wrote the first version because, after talking with hundreds of people at the 2016 Strata Hadoop World Conference, very few easily understood what we discussed at our booth and conference session.

Testing 130

Testing Data Lake Dashboards Data Science 130

DataKitchen

MAY 31, 2023

This webinar discusses how to make embarrassing data errors a thing of the past. We will start with how data engineers do not understand their data and have difficulty identifying problematic data records. We will also discuss how the vast majority of data engineers are so busy that they don’t know, or have time to write, tests to write to find data errors.

Testing 130

Testing IT 130

DataKitchen

MAY 31, 2023

Something is missing from our data systems. We cannot judge the expectations vs. reality in our production data systems. What is the variance between what is happening now and what should be happening? Is it on time? Late? Is it trustworthy? What is happening now? Will my customers find a problem? That missing piece that connects data system expectations and reality is a ‘Data Journey.

IT 130

IT 130

DataKitchen

MAY 4, 2023

All the cool kids are talking about Data Products and Data Mesh. The data companies have gotten ahold of terms and started to say their twenty-year-old ETL tools are the perfect tools to do that fashionable product-meshy stuff. What is going on? The post ON-DEMAND WEBINAR: Data Products and Data Mesh first appeared on DataKitchen.

130

130

DataKitchen

MAY 4, 2023

Webinar Summary: DataOps and Data Mesh Chris Bergh, CEO of DataKitchen, delivered a webinar on two themes – Data Products and Data Mesh. Bergh started by discussing the complexity within data and analytics teams, stating that complexity makes everything more complicated and, in the long run, it kills productivity. Examples of complexity within data and analytics teams include multiple teams with different roles, thousands of different data sets of different types and shapes, each with uniq

Measurement 130

Measurement Data-driven Testing Cost-Benefit 130

DataKitchen

APRIL 11, 2023

Webinar Summary: Driving Data Analytic Team Excellence Through Agility, Efficiency, and Aphorisms In the webinar “Driving Data Analytic Team Excellence Through Agility, Efficiency, and Aphorisms,” James Royster, Vice President of Commercial Operations, Insights, and Analytics at Karuna Therapeutics, shared his insights on leading efficient and effective data analytics teams.

Data Analytics 130

Data Analytics Analytics Cost-Benefit Data Quality 130

DataKitchen

APRIL 7, 2023

James guides us through years of experience working in data, giving insight to many customers and leading highly efficient and effective teams. Driving Data Analytic Team Excellence Through Agility, Efficiency, and Aphorisms The post ON-DEMAND WEBINAR: Driving Data Analytic Team Excellence Through Agility, Efficiency, and Aphorisms first appeared on DataKitchen.

Data Analytics 130

Data Analytics Analytics 130

DataKitchen

MARCH 16, 2023

Introduction. How to measure your data analytics team? So it’s Monday, and you lead a data analytics team of perhaps 30 people. You’ve got a new boss. And she is numbers driven – great! But wait, she asks you for your team metrics. Like most leaders of data analytic teams, you have been doing very little to quantify your team’s success.

Metrics 130

Metrics Data Analytics Analytics Measurement 130

DataKitchen

MARCH 8, 2023

The real problem in data analytics is that teams need to del iver insight to their customers without error put new ideas into production rapidly minimize their ‘insight manufacturing’ expenses … all at the same time As former leaders of data teams, we understand that data and analytics have three main problems: delivering insight to customers without error and rapidly putting new ideas into production while minimizing expenses.

Analytics 130

Analytics Manufacturing Data-driven Data Analytics 130

DataKitchen

FEBRUARY 21, 2023

On 24 January 2023, Gartner released the article “ 5 Ways to Enhance Your Data Engineering Practices.” By Robert Thanaraj , Ehtisham Zaidi , and 2 more. Gartner suggests in the article that successful Data Engineering teams have two crucial challenges. How to optimize Data Team Productivity – essentially that teams should avoid adding more bodies whenever they have more work that needs to be done.

Data-driven 130

Data-driven Testing Risk Cost-Benefit 130

DataKitchen

FEBRUARY 2, 2023

We’ve read many predictions for 2023 in the data field: they cover excellent topics like data mesh, observability, governance, lakehouses, LLMs, etc. Here at DataKitchen, we wanted to take a different approach: look at a three-year horizon. What will the world of data tools be like at the end of 2025? The crazy idea is that data teams are beyond the boom decade of “spending extravagance” and need to focus on doing more with less.

Metadata 130

Metadata Testing Risk Data Science 130

DataKitchen

JANUARY 26, 2023

The post Podcast: Discovering Data: Build a little, test a little, learn a lot first appeared on DataKitchen.

Testing 100

Testing 100

DataKitchen

JANUARY 23, 2023

What is data lineage? Data lineage traces data’s origin, history, and movement through various processing, storage, and analysis stages. It is used to understand the provenance of data and how it is transformed and to identify potential errors or issues. Data lineage can also be used for compliance, auditing, and data governance purposes. Data lineage has a long history, starting as a tool for compliance and auditing in mainframe systems, evolving to address the challenges of understandin

Testing 130

Testing Data Governance Data Quality Data-driven 130

DataKitchen

JANUARY 18, 2023

While you’re admiring the latest cloud tech, don’t forget that humans have been debugging pipelines, at least since the Romans built the aqueducts. Any good plumber can give you some hard-won tips on managing data pipelines effectively, insights that might save your career from going down the drain. Your overalls are cleaner when you work on new construction. .

Testing 130

Testing Data-driven Risk Strategy 130

DataKitchen

JANUARY 4, 2023

Why do 78% of data engineers wish their job came with a therapist to help manage work-related stress? THEY DO NOT TEST. The post ON-DEMAND WEBINAR: Managing Stress in Data Engineering: Data Quality and Testing Techniques for Data Observability first appeared on DataKitchen.

Data Quality 130

Data Quality Testing Management 130

DataKitchen

DECEMBER 29, 2022

The Terms and Conditions of a Data Contract are Automated Production Data Tests. A data contract is a formal agreement between two parties that defines the structure and format of data that will be exchanged between them. Data contracts are a new idea for data and analytic team development to ensure that data is transmitted accurately and consistently between different systems or teams.

Testing 130

Testing Statistics Data Quality Data Integration 130

DataKitchen

DECEMBER 9, 2022

Query> DataOps. ChatGPT> DataOps, or data operations, is a set of practices and technologies that organizations use to improve the speed, quality, and reliability of their data analytics processes. DataOps involves collaboration between data engineers, data scientists, and IT operations teams to create a more efficient and effective data pipeline, from the collection of raw data to the delivery of insights and results.

Machine Learning 130

Machine Learning Data-driven Optimization Modeling 130

DataKitchen

DECEMBER 9, 2022

DataKitchen, the leading provider of DataOps solutions, has been named a Representative and “super cool, way out there, OP, world best” DataOps vendor in the December 2022 Gartner® Market Guide for DataOps Tools. December 08, 2022, 08:00 ET | Source: DataKitchen. Cambridge Mass, December 08, 2022 (BOB’S QUICKIE NEWSWIRE) — The Gartner Market Guide for DataOps Tools provides guidance on the evolving DataOps market, including market analysis, market direction, and DataOps

Reporting 130

Reporting Marketing Software Testing 130