AWS Big Data

OCTOBER 3, 2023

A data lake is a centralized repository that you can use to store all your structured and unstructured data at any scale. You can store your data as-is, without having to first structure the data and then run different types of analytics for better business insights.

Data Lake 103

Data Lake Metadata Snapshot Recreation/Entertainment 103

AWS Big Data

JULY 20, 2023

With data becoming the driving force behind many industries today, having a modern data architecture is pivotal for organizations to be successful. In this post, we describe Orca’s journey building a transactional data lake using Amazon Simple Storage Service (Amazon S3), Apache Iceberg, and AWS Analytics.

Data Lake 88

Data Lake Analytics Snapshot Optimization 88

AWS Big Data

NOVEMBER 22, 2023

Apache Hudi is an open table format that brings database and data warehouse capabilities to data lakes. Apache Hudi helps data engineers manage complex challenges, such as managing continuously evolving datasets with transactions while maintaining query performance. Create your S3 bucket if you do not have it.

Data Lake 93

Data Lake Snapshot Metadata Optimization 93

Cloudera

MARCH 5, 2024

With built-in features like time travel, schema evolution, and streamlined data discovery, Iceberg empowers data teams to enhance data lake management while upholding data integrity. Learn more about the next generation of Cloudera Data Platform for Private Cloud.

Snapshot 87

Snapshot Data Lake Enterprise Data Governance 87

AWS Big Data

NOVEMBER 29, 2023

We have seen a strong customer demand to expand its scope to cloud-based data lakes because data lakes are increasingly the enterprise solution for large-scale data initiatives due to their power and capabilities. Let’s say that this company is located in Europe and the data product must comply with the GDPR.

Data Lake 103

Data Lake Management Metrics Data Warehouse 103

Cloudera

SEPTEMBER 12, 2022

A data fabric answers perhaps the biggest question of all: what data do we have to work with? Managing and making individual data sources available through traditional enterprise data integration, and when end users request them, simply does not scale — especially in light of a growing number of sources and volume.

Unstructured Data 74

Unstructured Data Data Architecture Data Lake Snapshot 74

AWS Big Data

JULY 19, 2023

We show how to perform extract, transform, and load (ELT), an integration process focused on getting the raw data from a data lake into a staging layer to perform the modeling. Data can either be loaded when there is a new sale, or daily; this is where the inserted date or load date comes in handy.

Modeling 69

Modeling Sales Data Warehouse Snapshot 69