DataKitchen

JANUARY 5, 2021

Remote working has revealed the inconsistency and fragility of workflow processes in many data organizations. The data teams share a common objective; to create analytics for the (internal or external) customer. Data Science Workflow – Kubeflow, Python, R. Data Engineering Workflow – Airflow, ETL.

Testing 147

Testing Data Governance Metadata Visualization 147

AWS Big Data

MARCH 18, 2024

Many organizations operate data lakes spanning multiple cloud data stores. In these cases, you may want an integrated query layer to seamlessly run analytical queries across these diverse cloud stores and streamline your data analytics processes. To achieve this, Oktank envisions a unified data query layer using Athena.

Data Lake 98

Data Lake Analytics Cost-Benefit Management 98

erwin

MARCH 30, 2020

Data intelligence has a critical role to play in the supercomputing battle against Covid-19. While leveraging supercomputing power is a tremendous asset in our fight to combat this global pandemic, in order to deliver life-saving insights, you really have to understand what data you have and where it came from.

Metadata 122

Metadata IT Data Governance Data Quality 122

Alation

FEBRUARY 13, 2020

Data catalogs have quickly become a core component of modern data management. Organizations with successful data catalog implementations see remarkable changes in the speed and quality of data analysis, and in the engagement and enthusiasm of people who need to perform data analysis.

Metadata 64

Metadata Data Lake Recreation/Entertainment Big Data 64

erwin

FEBRUARY 13, 2020

When an organization’s data governance and metadata management programs work in harmony, then everything is easier. Data governance is a complex but critical practice. There’s always more data to handle, much of it unstructured; more data sources, like IoT, more points of integration, and more regulatory compliance requirements.

Metadata 135

Metadata Data Governance Management Cost-Benefit 135

Cloudera

MAY 1, 2023

Businesses everywhere have engaged in modernization projects with the goal of making their data and application infrastructure more nimble and dynamic. The newly introduced Environments feature allows you to export only the generic, reusable parts of code and resources, while managing environment-specific configuration separately.

Testing 75

Testing Management Data Processing IT 75

DataKitchen

NOVEMBER 4, 2021

Data organizations often have a mix of centralized and decentralized activity. DataOps concerns itself with the complex flow of data across teams, data centers and organizational boundaries. It expands beyond tools and data architecture and views the data organization from the perspective of its processes and workflows.

Data Processing 246

Data Processing Data Lake Cost-Benefit Testing 246

AWS Big Data

MARCH 8, 2023

The value of data is time sensitive. Real-time processing makes data-driven decisions accurate and actionable in seconds or minutes instead of hours or days. It ensures that data in multiple systems stays in sync. This is especially important if you’re making time-sensitive decisions in a high-velocity data environment.

Data-driven 74

Data-driven Testing Data Processing Management 74

Andrew White

OCTOBER 22, 2021

If you follow my blog for any period of time you will know that for most years I have attended our annual Gartner IT Symposium I do a day-in-the-life blog of an analyst. This time, since we were virtual I only managed to close out the week with the 1-1 summary. Data Hub Strategy 10. Data Integration tactics 4.

IT 52

IT Data Lake Strategy Data Science 52

Alation

AUGUST 17, 2022

According to analysts, data governance programs have not shown a high success rate. According to CIOs , historical data governance programs were invasive and suffered from one of two defects: They were either forced on the rank and file — who grew to dislike IT as a result. The Risks of Early Data Governance Programs.

Data Governance 92

Data Governance Risk Data-driven Measurement 92

erwin

MAY 20, 2020

Data governance (DG) as a an “emergency service” may be one critical lesson learned coming out of the COVID-19 crisis. Where crisis leads to vulnerability, data governance as an emergency service enables organization management to direct or redirect efforts to ensure activities continue and risks are mitigated.

Data Governance 144

Data Governance Metadata Risk Strategy 144

Alation

APRIL 18, 2023

For any data user in an enterprise today, data profiling is a key tool for resolving data quality issues and building new data solutions. In this blog, we’ll cover the definition of data profiling, top use cases, and share important techniques and best practices for data profiling today.

IT 52

IT Data Quality Metadata Data Governance 52

erwin

JANUARY 30, 2020

I’m excited to share the results of our new study with Dataversity that examines how data governance attitudes and practices continue to evolve. Defining Data Governance: What Is Data Governance? . 1 reason to implement data governance. Constructing a Digital Transformation Strategy: How Data Drives Digital.

Data Governance 145

Data Governance Metadata Digital Transformation Cost-Benefit 145

erwin

APRIL 27, 2022

Best Data Governance Solution (erwin Data Intelligence). Best Data Modeling Solution (erwin Data Modeler). Best Data Security Solution (Quest ApexSQL). This award is especially significant because the winning information management solutions are chosen by the people who use them. We need your help!

Data Governance 52

Data Governance Digital Transformation Metadata Data Warehouse 52

Ontotext

JUNE 10, 2021

And yeah, the real-world relationships among the entities represented in the data had to be fudged a bit to fit in the counterintuitive model of tabular data, but, in trade, you get reliability and speed. Ironically, relational databases only imply relationships between data points by whatever row or column they exist in.

Metadata 119

Metadata Cost-Benefit OLAP Modeling 119

Cloudera

SEPTEMBER 23, 2020

We are going to talk about auditing, different security levels, security features of Data Catalog, and Client Considerations. Access audits are mastered centrally in Apache Ranger which provides comprehensive non-repudiable audit log for every access event to every resource with rich access event metadata such as: IP.

Data Lake 80

Data Lake Metadata IoT Enterprise 80

Alation

NOVEMBER 28, 2022

Alation is excited to unveil Alation Connected Sheets , a new product that brings trusted, fresh data directly to spreadsheet users. Now, “spreadsheet jockeys” can pull the most current, compliant data directly from a range of cloud sources, without having to know SQL or depend on a data team to deliver it.

Descriptive Analytics 52

Descriptive Analytics Risk Sales Data-driven 52

Cloudera

SEPTEMBER 30, 2021

Our recent blog discussed the four paths to get from legacy platforms to CDP Private Cloud Base. In this blog and accompanying video, we will deep dive into the mechanics of running an in-place upgrade from CDH5 or CDH6 to CDP Private Cloud Base. Zookeeper data. HDFS Master Node data directories. Hue dependencies.

Metadata 69

Metadata Testing Management Data-driven 69

erwin

MAY 31, 2019

It’s time to consider data-driven enterprise architecture. The traditional approach to enterprise architecture – the analysis, design, planning and implementation of IT capabilities for the successful execution of enterprise strategy – seems to be missing something … data. Data-Driven Enterprise Architecture and Cloud Migration.

Data-driven 104

Data-driven Enterprise Metadata Strategy 104

Alation

FEBRUARY 2, 2023

Data product managers are in high demand these days. In 2020, Glassdoor rated product manager as the 4th best job in the US. This makes it more important for aspiring data product managers to stay ahead of the competition. So what sets data product managers apart from the pack? Sounds exciting?

Management 52

Management Data-driven Visualization Strategy 52

Cloudera

MAY 16, 2019

The conference provides a useful opportunity to reflect on the rapid evolution we’ve seen in the Data Integration and Management space, much of it driven by the innovations that Cloudera and the open source community have been delivering. The traditional Data Warehouse ETL process has splintered into many smaller components.

Analytics 40

Analytics Data Warehouse Data-driven Data Integration 40

erwin

JANUARY 11, 2019

Organizations have spent a lot of time and money trying to harmonize data across diverse platforms , including cleansing, uploading metadata, converting code, defining business glossaries, tracking data transformations and so on. But the attempts to standardize data across the entire enterprise haven’t produced the desired results.

Data Governance 40

Data Governance Risk Metadata Management 40

Alation

APRIL 6, 2023

Data governance is traditionally applied to structured data assets that are most often found in databases and information systems. This blog focuses on governing spreadsheets that contain data, information, and metadata, and must themselves be governed. The ubiquity of spreadsheets creates disadvantages as well.

Data Governance 52

Data Governance Metadata Cost-Benefit Structured Data 52

Alation

NOVEMBER 4, 2021

This week I was talking to a data practitioner at a global systems integrator. The practitioner asked me to add something to a presentation for his organization: the value of data governance for things other than data compliance and data security. Now to be honest, I immediately jumped onto data quality.

Data Governance 94

Data Governance Metadata Data Quality Enterprise 94

AWS Big Data

MARCH 9, 2023

Thousands of customers rely on Amazon Redshift to build data warehouses to accelerate time to insights with fast, simple, and secure analytics at scale and analyze data from terabytes to petabytes by running complex analytical queries. The star schema is a popular data model for building data marts.

Slice and Dice 80

Slice and Dice Data Warehouse Metrics Metadata 80

Alation

APRIL 14, 2022

Moving data to the cloud can bring immense operational benefits. However, the sheer volume and complexity of today’s enterprise data can cause downstream headaches for data users. Semantics, context, and how data is tracked and used mean even more as you stretch to reach post-migration goals. Data pipeline orchestration.

Metadata 59

Metadata Data Governance Modeling Data-driven 59

Alation

JANUARY 25, 2022

Data fabric is now on the minds of most data management leaders. In our previous blog, Data Mesh vs. Data Fabric: A Love Story , we defined data fabric and outlined its uses and motivations. The data catalog is a foundational layer of the data fabric.

Metadata 52

Metadata IT Metrics Data-driven 52

Andrew White

JANUARY 9, 2022

On Thursday January 6th I hosted Gartner’s 2022 Leadership Vision for Data and Analytics webinar. – In the webinar and Leadership Vision deck for Data and Analytics we called out AI engineering as a big trend. I would take a look at our Top Trends for Data and Analytics 2021 for additional AI, ML and related trends.

Analytics 64

Analytics Measurement Modeling Data-driven 64

Alation

MARCH 16, 2023

The very word can cast a shiver of dread through many a hardened data professional’s spine. They are everywhere, they seem to multiply, and no one can be entirely sure how many their organization may have, nor — perhaps more worryingly — where the data they contain may have originated. Spreadsheets. They were so very right.

Forecasting 52

Forecasting Management Interactive IT 52

IBM Big Data Hub

JULY 13, 2023

When we talk about data integrity, we’re referring to the overarching completeness, accuracy, consistency, accessibility, and security of an organization’s data. Together, these factors determine the reliability of the organization’s data.

Data Quality 68

Data Quality Data Integration Metadata Cost-Benefit 68

Alation

JANUARY 13, 2022

And now, arguably the greatest rivalry the world (well, at least the data community) has ever witnessed: Data Fabric vs Data Mesh! Data fabric and data mesh are both having a moment. Gartner calls data fabric the Future of Data Management 1. Gartner on Data Fabric. Tyson vs Holyfield.

Data Lake 59

Data Lake Metadata Data-driven Data Governance 59

erwin

OCTOBER 24, 2019

Untapped data, if mined, represents tremendous potential for your organization. While there has been a lot of talk about big data over the years, the real hero in unlocking the value of enterprise data is metadata , or the data about the data. They don’t know exactly what data they have or even where some of it is.

Metadata 104

Metadata Management Data-driven Data Architecture 104

erwin

MAY 27, 2020

Data modeling supports collaboration among business stakeholders – with different job roles and skills – to coordinate with business objectives. Data resides everywhere in a business , on-premise and in private or public clouds. A single source of data truth helps companies begin to leverage data as a strategic asset.

Modeling 98

Modeling Metadata Data Governance Visualization 98

Occam's Razor

SEPTEMBER 29, 2009

If supported by "data" then it tends to be of the most fragile kind (usually the the fact that the CEO saw it during the Super Bowl and felt happy suffices as actionable data). Before we go any further I must clarify that I love branding campaigns just as much as the next guy. I love campaigns that Visa runs. asked you to.

Measurement 128

Measurement Metrics Analytics Marketing 128

IBM Big Data Hub

JULY 25, 2022

Modern-day enterprises face a similar situation regarding data assets. On one side there is a need for data. Businesses ask: “Do we have this kind of data in the enterprise?” “How do we get that data?” “Can Can I trust that data?” This discussion is more relevant with the advent of data fabric.

Metadata 72

Metadata Enterprise Consulting Publishing 72

AWS Big Data

JUNE 21, 2023

Amazon OpenSearch Service is a fully managed service that makes it simple to deploy, scale, and operate OpenSearch in the AWS Cloud. Sophisticated embedding models can support multiple modalities, for instance, encoding the image and text of a product catalog and enabling similarity matching on both modalities.

Metrics 77

Metrics Measurement Modeling Dashboards 77

Alation

JULY 8, 2021

A central challenge for modern organizations is no longer gathering data — it’s navigating a veritable forest of it. Data presents an opportunity. Data-driven businesses know this. They ask: How can we ensure we’re using data to grow smarter? A Mess of Data: The Data Intelligence Challenge.

Software 52

Software Testing Data-driven Modeling 52