AWS Big Data

NOVEMBER 1, 2023

Originally published on December 9th, 2022. Amazon Redshift is a fully managed, petabyte scale cloud data warehouse that enables you to analyze large datasets using standard SQL. Amazon Redshift is a cloud-based data warehouse that supports many recovery capabilities to address unforeseen outages and minimize downtime.

Data Warehouse 97

Data Warehouse Snapshot Testing Management 97

Cloudera

APRIL 3, 2023

Cloudera Contributors: Ayush Saxena, Tamas Mate, Simhadri Govindappa Since we announced the general availability of Apache Iceberg in Cloudera Data Platform (CDP), we are excited to see customers testing their analytic workloads on Iceberg. We will publish follow up blogs for other data services. ID, TBL_ICEBERG_PART_2.NAME,

Data Warehouse 81

Data Warehouse Snapshot Metadata Cost-Benefit 81

Cloudera

AUGUST 8, 2022

In June 2022, Cloudera announced the general availability of Apache Iceberg in the Cloudera Data Platform (CDP). The general availability covers Iceberg running within some of the key data services in CDP, including Cloudera Data Warehouse ( CDW ), Cloudera Data Engineering ( CDE ), and Cloudera Machine Learning ( CML ).

Snapshot 91

Snapshot Data Warehouse Machine Learning Cost-Benefit 91

AWS Big Data

APRIL 27, 2023

It supports modern analytical data lake operations such as create table as select (CTAS), upsert and merge, and time travel queries. Athena also supports the ability to create views and perform VACUUM (snapshot expiration) on Apache Iceberg tables to optimize storage and performance. Create a table to point to the CDC data.

Data Lake 76

Data Lake Snapshot Optimization Data Transformation 76

AWS Big Data

MARCH 2, 2023

In early 2022, AWS announced general availability of Athena ACID transactions, powered by Apache Iceberg. Whenever there is an update to the Iceberg table, a new snapshot of the table is created, and the metadata pointer points to the current table metadata file. The snapshot points to the manifest list.

Data Lake 120

Data Lake Data Processing Metadata Snapshot 120

AWS Big Data

NOVEMBER 29, 2023

dbt is an open source, SQL-first templating engine that allows you to write repeatable and extensible data transforms in Python and SQL. dbt is predominantly used by data warehouses (such as Amazon Redshift ) customers who are looking to keep their data transform logic separate from storage and engine.

Data Lake 105

Data Lake Management Metrics Data Warehouse 105

AWS Big Data

MAY 30, 2023

Under Instance configuration , for High Availability , choose Dev or test workload (Single-AZ). Create a Python file called generate-data-for-kds.py : $ python3 generate-data-for-kds.py client("kinesis")) This script puts a Kinesis data stream record every 2 seconds. Choose Create replication instance. mode("append").save(s3_output_folder)

Data Lake 83

Data Lake Data Analytics Analytics Data Processing 83