AWS Big Data

MARCH 10, 2023

Since the deluge of big data over a decade ago, many organizations have learned to build applications to process and analyze petabytes of data. Data lakes have served as a central repository to store structured and unstructured data at any scale and in various formats.

Data Lake 98

Data Lake Sales Data Warehouse Snapshot 98

AWS Big Data

JANUARY 31, 2023

In this post, we are excited to summarize the features that the AWS Glue Data Catalog, AWS Glue crawler, and Lake Formation teams delivered in 2022. Whether you are a data platform builder, data engineer, data scientist, or any technology leader interested in data lake solutions, this post is for you.

Data Lake 78

Data Lake Data Governance Data Architecture Machine Learning 78

AWS Big Data

APRIL 19, 2023

Independent data products often only have value if you can connect them, join them, and correlate them to create a higher order data product that creates additional insights. A modern data architecture is critical in order to become a data-driven organization.

Technology 78

Technology Data-driven Machine Learning Sales 78

AWS Big Data

APRIL 3, 2024

If the asset has AWS Glue Data Quality enabled, you can now quickly visualize the data quality score directly in the catalog search pane. By selecting the corresponding asset, you can understand its content through the readme, glossary terms , and technical and business metadata.

Data Quality 85

Data Quality Visualization Metadata Metrics 85

AWS Big Data

MARCH 26, 2024

A Gartner Marketing survey found only 14% of organizations have successfully implemented a C360 solution, due to lack of consensus on what a 360-degree view means, challenges with data quality, and lack of cross-functional governance structure for customer data. Then, you transform this data into a concise format.

Data Strategy 66

Data Strategy Strategy Data Warehouse Prescriptive Analytics 66

TDAN

JUNE 6, 2023

You would think that after knocking around in semantics and knowledge graphs for over two decades I’d have had a pretty good idea about Knowledge Management, but it turns out I didn’t. I think in the rare event the term came up I internally conflated it with Knowledge Graphs and moved on. The first tap […]

Management 97

Management IT Data Architecture Metadata 97

AWS Big Data

OCTOBER 25, 2023

But most important of all, the assumed dormant value in the unstructured data is a question mark, which can only be answered after these sophisticated techniques have been applied. Therefore, there is a need to being able to analyze and extract value from the data economically and flexibly. The solution integrates data in three tiers.

Unstructured Data 115

Unstructured Data Metadata Management Analytics 115

AWS Big Data

SEPTEMBER 13, 2023

The Analytics specialty practice of AWS Professional Services (AWS ProServe) helps customers across the globe with modern data architecture implementations on the AWS Cloud. The File Manager Lambda function consumes those messages, parses the metadata, and inserts the metadata to the DynamoDB table odpf_file_tracker.

Data Lake 87

Data Lake Data Processing Metadata Snapshot 87

TDAN

JUNE 21, 2022

Twenty-five years ago today, I published the first issue of The Data Administration Newsletter. It only took a few months to recognize that there was an audience for an “online” publication focused on data administration. […].

Publishing 78

Publishing Data Quality Data Architecture IT 78

TDAN

SEPTEMBER 15, 2020

In the world of data, automation plays a well-honed role in rapidly developing modern data estates. Before we proceed any further, let’s establish an understanding about the purpose of a corporate data estate. A data estate is the technical architecture and enterprise infrastructure that enables organizations to […].

Enterprise 85

Enterprise Data Architecture Metadata Data Strategy 85

TDAN

MAY 31, 2022

One very influential factor that can potentially undermine your data and document strategies is the natural and emotional reactions of people when things change. It is common to take great care in the selection and implementation of new technology.

Testing 52

Testing Interactive Strategy Software 52

TDAN

AUGUST 6, 2019

Deep learning, as defined by MathWorks, is a system of artificial intelligence that is built around learning by example. Multiple industries have already understood the benefits that deep learning brings to their operational capabilities.

Deep Learning 81

Deep Learning Technology Data Architecture Metadata 81

TDAN

MAY 14, 2019

Cross-Agency Priority (CAP) Goal #2 is “leverage data as a strategic asset to grow the economy, increase the effectiveness of the Federal Government, facilitate oversight, and promote transparency.” The President’s Management Agenda (PMA) lays out a long-term vision for modernizing the Federal Government.

Management 40

Management Data Quality Data Architecture Metadata 40

TDAN

NOVEMBER 5, 2019

This series of columns will focus on many things including the nature of a graph database, how we evolved to a graph database, how a graph database is different from a relational database, two classes of graph databases – property and semantic, how property and semantic graph databases differ, use cases, challenges, how to map […].

Metadata 40

Metadata Data Strategy Data Architecture Business Intelligence 40

IBM Big Data Hub

AUGUST 4, 2023

Today, the way businesses use data is much more fluid; data literate employees use data across hundreds of apps, analyze data for better decision-making, and access data from numerous locations. It uses knowledge graphs, semantics and AI/ML technology to discover patterns in various types of metadata.

Data Architecture 64

Data Architecture Data Lake Machine Learning Data Governance 64

Alation

FEBRUARY 20, 2020

The next stops on the MLDC World Tour include Data Transparency in Washington, Gartner Symposium/ITxpo in Orlando, Teradata Analytics Universe in Las Vegas, Tableau in New Orleans, Big Data LDN in London, TDWI in Orlando and Forrester Data Strategy & Insights in Orlando, again. Data Catalogs Are the New Black.

Machine Learning 52

Machine Learning Metadata Reporting Data-driven 52

AWS Big Data

JULY 14, 2023

These inputs reinforced the need of a unified data strategy across the FinOps teams. We decided to build a scalable data management product that is based on the best practices of modern data architecture. This global catalog captures new or updated partitions from the data producer AWS Glue Data Catalogs.

Finance 79

Finance Metadata Big Data Recreation/Entertainment 79