article thumbnail

Orca Security’s journey to a petabyte-scale data lake with Apache Iceberg and AWS Analytics

AWS Big Data

With data becoming the driving force behind many industries today, having a modern data architecture is pivotal for organizations to be successful. In this post, we describe Orca’s journey building a transactional data lake using Amazon Simple Storage Service (Amazon S3), Apache Iceberg, and AWS Analytics.

article thumbnail

Themes and Conferences per Pacoid, Episode 8

Domino Data Lab

The longer answer is that in the context of machine learning use cases, strong assumptions about data integrity lead to brittle solutions overall. Data coming from machines tends to land (aka, data at rest ) in durable stores such as Amazon S3, then gets consumed by Hadoop, Spark, etc. Email is a good example.