AWS Big Data

APRIL 3, 2024

Manage your Iceberg table with AWS Glue You can use AWS Glue to ingest, catalog, transform, and manage the data on Amazon Simple Storage Service (Amazon S3). With AWS Glue, you can discover and connect to more than 70 diverse data sources and manage your data in a centralized data catalog.

Data Lake 92

Data Lake Snapshot Metadata Data Architecture 92

AWS Big Data

DECEMBER 13, 2023

A CDC-based approach captures the data changes and makes them available in data warehouses for further analytics in real-time. usually a data warehouse) needs to reflect those changes in near real-time. This post showcases how to use streaming ingestion to bring data to Amazon Redshift.

Data Warehouse 95

Data Warehouse Snapshot Data Processing Management 95

AWS Big Data

NOVEMBER 10, 2023

This integration expands the possibilities for AWS analytics and machine learning (ML) solutions, making the data warehouse accessible to a broader range of applications. These tables are then joined with tables from the Enterprise Data Lake (EDL) at runtime.

Data Processing 89

Data Processing Data Lake Data Warehouse Optimization 89

AWS Big Data

OCTOBER 19, 2023

Data lakes and data warehouses are two of the most important data storage and management technologies in a modern data architecture. Data lakes store all of an organization’s data, regardless of its format or structure. Various data stores are supported in AWS Glue; for example, AWS Glue 4.0

Data Lake 109

Data Lake Data Warehouse Visualization Snapshot 109

AWS Big Data

FEBRUARY 9, 2023

To achieve this, they combine their CRM data with a wealth of information already available in their data warehouse, enterprise systems, or other software as a service (SaaS) applications. One widely used approach is getting the CRM data into your data warehouse and keeping it up to date through frequent data synchronization.

Data Warehouse 73

Data Warehouse Data-driven Snapshot Testing 73

AWS Big Data

NOVEMBER 22, 2023

Apache Hudi is an open table format that brings database and data warehouse capabilities to data lakes. Apache Hudi helps data engineers manage complex challenges, such as managing continuously evolving datasets with transactions while maintaining query performance. He works based in Tokyo, Japan.

Data Lake 89

Data Lake Snapshot Metadata Optimization 89

AWS Big Data

JULY 19, 2023

Amazon Redshift is a fully managed and petabyte-scale cloud data warehouse that is used by tens of thousands of customers to process exabytes of data every day to power their analytics workload. You can structure your data, measure business processes, and get valuable insights quickly can be done by using a dimensional model.

Modeling 66

Modeling Sales Data Warehouse Snapshot 66