article thumbnail

Defining Simplicity for Enterprise Software as “a 10 Year Old Can Demo it”

Cloudera

During the development of Operational Database and Replication Manager, I kept telling folks across the team it has to be “so simple that a 10 year old can demo it”. so simple that a 10 year old can demo it”. Watch this: Enterprise Software that is so easy a 10 year old can demo it. Create a snapshot .

article thumbnail

Top 20 most-asked questions about Amazon RDS for Db2 answered

IBM Big Data Hub

AWS ran a live demo to show how to get started in just a few clicks. At what level are snapshot-based backups taken? Also, you can create snapshots, which are user-initiated backups of your instance kept until explicitly deleted. Answer : We refer to snapshots as storage-level backups. Backup and restore 11.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Migrate an existing data lake to a transactional data lake using Apache Iceberg

AWS Big Data

An in-place migration can be performed in either of two ways: Using add_files : This procedure adds existing data files to an existing Iceberg table with a new snapshot that includes the files. Unlike migrate or snapshot, add_files can import files from a specific partition or partitions and doesn’t create a new Iceberg table.

Data Lake 104
article thumbnail

Optimization Strategies for Iceberg Tables

Cloudera

Problem with too many snapshots Everytime a write operation occurs on an Iceberg table, a new snapshot is created. Regularly expiring snapshots is recommended to delete data files that are no longer needed, and to keep the size of table metadata small. You could also change the isolation level to snapshot isolation.

article thumbnail

Materialized Views in Hive for Iceberg Table Format

Cloudera

Subsequently, these snapshot IDs are used to determine the delta changes that should be applied to the materialized view rows. Hive does this by asking the Iceberg library to return only the rows inserted since that table’s last snapshot when the materialized view was last rebuilt/created.

article thumbnail

Interact with Apache Iceberg tables using Amazon Athena and cross account fine-grained permissions using AWS Lake Formation

AWS Big Data

For Target database , enter lf-demo-db. In the Athena query editor, run the following SELECT query on the shared table: SELECT * FROM "lf-demo-db"."consumer_iceberg" In the Athena query editor, run the following SELECT query on the shared table: SELECT * FROM "lf-demo-db"."consumer_iceberg" Choose Create new filter. Choose Save.

article thumbnail

Perform upserts in a data lake using Amazon Athena and Apache Iceberg

AWS Big Data

Athena also supports the ability to create views and perform VACUUM (snapshot expiration) on Apache Iceberg tables to optimize storage and performance. Create a database with the following code: CREATE DATABASE raw_demo; Next, create a folder in an S3 bucket that you can use for this demo. Name this folder sporting_event_full.