2005, Metadata, Snapshot and Testing

Use Apache Iceberg in a data lake to support incremental data processing

AWS Big Data

MARCH 2, 2023

Apache Iceberg is an open table format for very large analytic datasets, which captures metadata information on the state of datasets as they evolve and change over time. Apache Iceberg addresses customer needs by capturing rich metadata information about the dataset at the time the individual data files are created.

Data Lake

Data Lake Data Processing Metadata Snapshot

Modernize a legacy real-time analytics application with Amazon Managed Service for Apache Flink

AWS Big Data

OCTOBER 11, 2023

Frequent materialized view refreshes on top of constantly changing base tables due to streamed data can lead to snapshot isolation errors. The second streaming data source constitutes metadata information about the call center organization and agents that gets refreshed throughout the day. We use two datasets in this post.

Management

Management Metadata Analytics Dashboards

Data Leaders Brief

Use Apache Iceberg in a data lake to support incremental data processing

Modernize a legacy real-time analytics application with Amazon Managed Service for Apache Flink

Webinars

Stay Connected