AWS Big Data

MARCH 4, 2024

Apache Iceberg manages these schema changes in a backward-compatible way through its innovative metadata table evolution architecture. With Lake Formation, you can manage fine-grained access control for your data lake data on Amazon S3 and its metadata in the Data Catalog. Iceberg maintains the table state in metadata files.

Snapshot 111

Snapshot Data Lake Metadata Recreation/Entertainment 111

AWS Big Data

NOVEMBER 17, 2023

Starting today, the Athena SQL engine uses a cost-based optimizer (CBO), a new feature that uses table and column statistics stored in the AWS Glue Data Catalog as part of the table’s metadata. By using these statistics, CBO improves query run plans and boosts the performance of queries run in Athena.

Optimization 112

Optimization Statistics Metadata Data Lake 112

Domino Data Lab

JUNE 2, 2019

It also represents part of the current focus for Project Jupyter : adding support for collaboration, enhanced security, projects as top-level entities, data registry, metadata management, and telemetry about usage. my answer was almost immediate: Daniel Kahneman.

Data-driven 110

Data-driven Data Science Machine Learning Modeling 110

Alation

APRIL 12, 2022

In 2002, Capital One became the first company to appoint a Chief Data Officer (CDO). Through serving as a centralized conduit for discovering and requesting access to data, a data catalog provides CDOs and their data governance teams with information and metadata to determine which people should see and access what data.

Data Quality 52

Data Quality Data Governance Metrics Data Strategy 52