article thumbnail

Use Apache Iceberg in your data lake with Amazon S3, AWS Glue, and Snowflake

AWS Big Data

Manage your Iceberg table with AWS Glue You can use AWS Glue to ingest, catalog, transform, and manage the data on Amazon Simple Storage Service (Amazon S3). With AWS Glue, you can discover and connect to more than 70 diverse data sources and manage your data in a centralized data catalog.

article thumbnail

Break data silos and stream your CDC data with Amazon Redshift streaming and Amazon MSK

AWS Big Data

Using Amazon MSK, we securely stream data with a fully managed, highly available Apache Kafka service. Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

article thumbnail

Simplify AWS Glue job orchestration and monitoring with Amazon MWAA

AWS Big Data

In these scenarios, customers looking for a serverless data integration offering use AWS Glue as a core component for processing and cataloging data. NONE" else (datetime.today() - timedelta(days=1)).strftime("%Y-%m-%d")