article thumbnail

Use Apache Iceberg in your data lake with Amazon S3, AWS Glue, and Snowflake

AWS Big Data

licensed, 100% open-source data table format that helps simplify data processing on large datasets stored in data lakes. Data engineers use Apache Iceberg because it’s fast, efficient, and reliable at any scale and keeps records of how datasets change over time.

article thumbnail

AWS Lake Formation 2022 year in review

AWS Big Data

In this post, we are excited to summarize the features that the AWS Glue Data Catalog, AWS Glue crawler, and Lake Formation teams delivered in 2022. Whether you are a data platform builder, data engineer, data scientist, or any technology leader interested in data lake solutions, this post is for you.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Use Apache Iceberg in a data lake to support incremental data processing

AWS Big Data

Iceberg has become very popular for its support for ACID transactions in data lakes and features like schema and partition evolution, time travel, and rollback. In early 2022, AWS announced general availability of Athena ACID transactions, powered by Apache Iceberg. and later supports the Apache Iceberg framework for data lakes.

Data Lake 114
article thumbnail

Australia’s IT leadership moves 2022

CIO Business Intelligence

Here’s an update on the moves of Australian IT leaders starting from June 2022. Kuret replaces Scott Wall who after four years with BankVic joined Bank Australia as chief transformation officer in June 2022. CIO Australia consistently tracks the moves of IT leaders. Simon Herbert departs from NSW Customer Service. IT Leadership

IT 94
article thumbnail

Building Best-in-Class Enterprise Analytics

Speaker: Anthony Roach, Director of Product Management at Tableau Software, and Jeremiah Morrow, Partner Solution Marketing Director at Dremio

A self-service platform for data exploration and visualization that broadens access to analytic insights. Register now for the webinar on April 21, 2022 at 10:00 am PDT, 12:00 pm EDT to learn how Dremio and Tableau are delivering mission critical BI and interactive analytics on data directly in the data lake.

article thumbnail

Build a real-time GDPR-aligned Apache Iceberg data lake

AWS Big Data

Data lakes are a popular choice for today’s organizations to store their data around their business activities. As a best practice of a data lake design, data should be immutable once stored. A data lake built on AWS uses Amazon Simple Storage Service (Amazon S3) as its primary storage environment.

article thumbnail

Enable business users to analyze large datasets in your data lake with Amazon QuickSight

AWS Big Data

Events and many other security data types are stored in Imperva’s Threat Research Multi-Region data lake. Imperva harnesses data to improve their business outcomes. As part of their solution, they are using Amazon QuickSight to unlock insights from their data.