Cloudera

MARCH 2, 2023

Recently, we announced enhanced multi-function analytics support in Cloudera Data Platform (CDP) with Apache Iceberg. Iceberg is a high-performance open table format for huge analytic data sets. This enables you to maximize utilization of streaming data at scale. The Catalog Type should be set to Hive.

Snapshot 113

Snapshot Data Processing Metadata Management 113

AWS Big Data

NOVEMBER 10, 2023

Apache Spark is a widely-used open source distributed processing system renowned for handling large-scale data workloads. Amazon Redshift offers seamless integration with Apache Spark, allowing you to easily access your Redshift data on both Amazon Redshift provisioned clusters and Amazon Redshift Serverless.

Data Processing 91

Data Processing Data Lake Data Warehouse Optimization 91

DataKitchen

AUGUST 9, 2021

Below is our third post (3 of 5) on combining data mesh with DataOps to foster greater innovation while addressing the challenges of a decentralized architecture. We’ve talked about data mesh in organizational terms (see our first post, “ What is a Data Mesh? ”) and how team structure supports agility. Source: Thoughtworks.

Testing 246

Testing Data Lake Metadata Publishing 246

Cloudera

MARCH 14, 2023

We just announced the general availability of Cloudera DataFlow Designer , bringing self-service data flow development to all CDP Public Cloud customers. In this blog post we will put these capabilities in context and dive deeper into how the built-in, end-to-end data flow life cycle enables self-service data pipeline development.

Testing 81

Testing Publishing Metadata Interactive 81

erwin

JUNE 26, 2020

If you’re serious about a data-driven strategy , you’re going to need a data catalog. Organizations need a data catalog because it enables them to create a seamless way for employees to access and consume data and business assets in an organized manner. Three Types of Metadata in a Data Catalog.

Metadata 132

Metadata Cost-Benefit Measurement Data-driven 132

AWS Big Data

AUGUST 3, 2023

Today’s modern data lakes span multiple accounts, AWS Regions, and lines of business in organizations. It’s important that their data solution gives them the ability to share and access data securely and safely across Regions. A resource link is a Data Catalog object that is a link to a database or table.

Data Lake 84

Data Lake Metadata Management Data Processing 84

AWS Big Data

MARCH 26, 2024

We are delighted to announce the release of the Data Analytics Lens. The lens consists of a lens whitepaper and an AWS-created lens available in the Lens Catalog of the AWS Well-Architected Tool. What’s new in the Data Analytics Lens? For more information on AWS Well-Architected Lenses, refer to AWS Well-Architected.

Data Analytics 55

Data Analytics Analytics Big Data Data Lake 55

Octopai

APRIL 24, 2023

This new native integration enhances our data lineage solution by providing seamless integration with one of the most powerful cloud-based data warehouses, benefiting data teams and enabling support for a broader range of data lineage, discovery, and catalog.

Cost-Benefit 96

Cost-Benefit Data-driven Data Warehouse Data Governance 96

Ontotext

APRIL 10, 2024

The data ecosystem today is crowded with dazzling buzzwords, all fighting for investment dollars. A survey in 2021 found that a data company was being funded every 45 minutes. Data ecosystems have become jungles and in spite of all the technology, data teams are struggling to create a modern data experience.

Metadata 52

Metadata Data Lake Data Warehouse Data Quality 52

AWS Big Data

FEBRUARY 29, 2024

Data is your generative AI differentiator, and a successful generative AI implementation depends on a robust data strategy incorporating a comprehensive data governance approach. Data governance is a critical building block across all these approaches, and we see two emerging areas of focus.

Data Governance 93

Data Governance Unstructured Data Metadata Data Lake 93

CIO Business Intelligence

DECEMBER 19, 2023

IT service delivery enables an organization to give end users access to essential IT services by designing, developing, and deploying key technology resources, including applications and data. Develop a structured service catalog and adapt Achieving service delivery excellence is not a one-size-fits-all process.

IT 138

IT Testing Metrics Optimization 138

AWS Big Data

OCTOBER 12, 2023

At launch, OpenSearch Serverless supported increasing capacity automatically in response to growing data sizes. Scaling overview Consider an ecommerce website that uses OpenSearch Serverless as a backend search engine to host its product catalog. In the following figure, an index has four shards to handle the product catalog.

Data Processing 86

Data Processing Testing Management Analytics 86

Smart Data Collective

JULY 15, 2021

A data catalog serves the same purpose. By using metadata (or short descriptions), data catalogs help companies gather, organize, retrieve, and manage information. You can think of a data catalog as an enhanced Access database or library card catalog system. What Does a Data Catalog Do?

Metadata 127

Metadata IT Data-driven Data Quality 127

Alation

DECEMBER 6, 2022

Today I’m thrilled to share that we’ve added two new, seasoned leaders to the Alation Product Team. Diby and Jonathan boast a wealth of relevant experience, having led growth at scale at both public and private data management and enterprise SaaS companies, including Salesforce, Confluent, Informatica, and Cloudflare.

Metadata 52

Metadata Enterprise Data Integration Data Quality 52

Alation

MAY 12, 2023

So when leading software review site TrustRadius announced that we had won their “Top Rated” awards in Data Catalog , Data Collaboration, Data Governance , and Metadata Management we were thrilled, but not surprised, since usability has been core to Alation’s product DNA since day 1. What does “Top Rated” mean?

Metadata 52

Metadata Data Governance Data-driven Sales 52

AWS Big Data

MARCH 29, 2024

QuickSight makes it straightforward for business users to visualize data in interactive dashboards and reports. You can slice data by different dimensions like job name, see anomalies, and share reports securely across your organization. With these insights, teams have the visibility to make data integration pipelines more efficient.

Metrics 103

Metrics Visualization Dashboards Interactive 103

AWS Big Data

JUNE 15, 2023

In today’s world, customers manage vast amounts of data in their Amazon Simple Storage Service (Amazon S3) data lakes, which requires convoluted data pipelines to continuously understand the changes in the data layout and make them available to consuming systems. Delete the CloudFormation stack you created.

Data Lake 86

Data Lake Metadata Cost-Benefit Management 86

Alation

JULY 15, 2021

Many data catalog initiatives fail. How can prospective buyers ensure they partner with the right catalog to drive success? According to the latest report from Eckerson Group, Deep Dive on Data Catalogs , shoppers must match the goals of their organizations to the capabilities of their chosen catalog.

Reporting 98

Reporting Data Governance Recreation/Entertainment Metadata 98

CIO Business Intelligence

SEPTEMBER 6, 2022

When Steve Pimblett joined The Very Group in October 2020 as chief data officer, reporting to the conglomerate’s CIO, his task was to help the enterprise uncover value in its rich data heritage. Very grew from the successive mergers of a number of mail order catalog companies, the oldest dating back to the 1890s.

IT 97

IT Forecasting Data Lake Enterprise 97

Alation

JULY 26, 2021

How do you initiate change within a system containing many thousands of people and millions of bytes of data? During my time as a data specialist at American Family Insurance, it became clear that we had to move away from the way things had been done in the past. So you can probably imagine: The company manages a lot of data.

Insurance 98

Insurance Digital Transformation Enterprise Data Governance 98

Alation

JANUARY 10, 2023

And implementing a data catalog is no different. From helping analysts search and find the data they need to promoting data as an asset, there are many ways data catalogs can add value to your organisation. Three reasons companies adopt a data catalog. Upgrading an existing catalog.

ROI 52

ROI Data Lake Strategy Data-driven 52

AWS Big Data

SEPTEMBER 26, 2023

This is a guest post co-written with Tyler Middleton, Experian Senior Partner Marketing Manager, and Jay Rakhe, Experian Group Product Manager. As the data privacy landscape continues to evolve, companies are increasingly seeking ways to collect and manage data while protecting privacy and intellectual property.

Advertising 76

Advertising Data-driven Marketing Interactive 76

Alation

JULY 19, 2021

Metadata is information about data. A clothing catalog or dictionary are both examples of metadata repositories. Indeed, a popular online catalog, like Amazon, offers rich metadata around products to guide shoppers: ratings, reviews, and product details are all examples of metadata. What Is Metadata?

Metadata 105

Metadata Management Data Governance Machine Learning 105

AWS Big Data

NOVEMBER 17, 2023

DynamoDB offers built-in security, continuous backups, automated multi-Region replication, in-memory caching, and data import and export tools. The scalability and flexible data schema of DynamoDB make it well-suited for a variety of use cases. Data stored in DynamoDB is the basis for valuable business intelligence (BI) insights.

Visualization 81

Visualization Metadata Testing Internet of Things 81

Octopai

JUNE 27, 2022

This data catalog is helping boost our bottom line.”. Your data team spends more time doing high-level analysis than it does searching for relevant datasets. Your most technologically-challenged business user searches the data catalog at least once a week – without coming to you for help! But don’t worry.

Management 72

Management Metadata Visualization Data Governance 72

AWS Big Data

NOVEMBER 17, 2023

Athena provides a simplified, flexible way to analyze petabytes of data where it lives. You can analyze data or build applications from an Amazon Simple Storage Service (Amazon S3) data lake and 30 data sources, including on-premises data sources or other cloud systems using SQL or Python.

Optimization 110

Optimization Statistics Metadata Data Lake 110

AWS Big Data

APRIL 3, 2024

Today, we are pleased to announce that Amazon DataZone is now able to present data quality information for data assets. Other organizations monitor the quality of their data through third-party solutions. Amazon DataZone now integrates directly with AWS Glue to display data quality scores for AWS Glue Data Catalog assets.

Data Quality 87

Data Quality Visualization Metadata Metrics 87

Alation

JANUARY 20, 2022

Several weeks ago (prior to the Omicron wave), I got to attend my first conference in roughly two years: Dataversity’s Data Quality and Information Quality Conference. Ryan Doupe, Chief Data Officer of American Fidelity, held a thought-provoking session that resonated with me. Instead, data quality rules promote awareness and trust.

Data Quality 94

Data Quality Data Governance Metrics Statistics 94

CIO Business Intelligence

OCTOBER 16, 2023

Specifically, we’re looking at whether AI will make employees more productive or satisfied in their jobs. AI can unlock developer productivity by allowing teams to bridge the knowledge and change context to gain new information and solve issues faster.

Software 128

Software Data-driven Modeling Machine Learning 128

AWS Big Data

JUNE 6, 2023

Data is the lifeblood of modern businesses. In today’s data-driven world, companies rely on data to make informed decisions, gain a competitive edge, and provide exceptional customer experiences. However, not all data is created equal. AWS Glue Data Quality measures and monitors the quality of your dataset.

Data Quality 67

Data Quality Data Lake Visualization Data-driven 67

Cloudera

DECEMBER 9, 2022

Cloudera has been providing enterprise support for Apache NiFi since 2015, helping hundreds of organizations take control of their data movement pipelines on premises and in the public cloud. Developers need to onboard new data sources, chain multiple data transformation steps together, and explore data as it travels through the flow.

Testing 96

Testing Cost-Benefit Interactive Visualization 96

AWS Big Data

APRIL 25, 2024

Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is a managed orchestration service for Apache Airflow that you can use to set up and operate data pipelines in the cloud at scale. By isolating workloads with specific security requirements or compliance needs, organizations can maintain the highest levels of data privacy and security.

Metadata 59

Metadata Data Processing Management Testing 59

Cloudera

JUNE 7, 2021

Since then, we have been working hard to expose the full power of Apache Flink SQL and the existing Data Warehousing tools in CDP to combine it into a state-of-the-art real-time analytics platform. Flink SQL DDL and Catalog support. Catalog Integrations. This step significantly speeds up query development and data exploration.

Analytics 101

Analytics Management IT 101

AWS Big Data

JUNE 6, 2023

We are excited to announce the General Availability of AWS Glue Data Quality. Our journey started by working backward from our customers who create, manage, and operate data lakes and data warehouses for analytics and machine learning. To make confident business decisions, the underlying data needs to be accurate and recent.

Data Quality 93

Data Quality Statistics Data Lake Visualization 93

AWS Big Data

JANUARY 31, 2023

Data governance is the collection of policies, processes, and systems that organizations use to ensure the quality and appropriate handling of their data throughout its lifecycle for the purpose of generating business value.

Data Lake 80

Data Lake Data Governance Data Architecture Machine Learning 80

IBM Big Data Hub

DECEMBER 16, 2022

An enterprise data catalog does all that a library inventory system does – namely streamlining data discovery and access across data sources – and a lot more. For example, data catalogs have evolved to deliver governance capabilities like managing data quality and data privacy and compliance.

Metadata 52

Metadata Data Quality Data-driven Data Governance 52