AWS Big Data

OCTOBER 19, 2023

Data lakes and data warehouses are two of the most important data storage and management technologies in a modern data architecture. Data lakes store all of an organization’s data, regardless of its format or structure. Delta Lake doesn’t have a specific concept for incremental queries.

Data Lake 111

Data Lake Data Warehouse Visualization Snapshot 111

Domino Data Lab

JANUARY 11, 2021

Data Science works best with a high degree of data granularity when the data offers the closest possible representation of what happened during actual events – as in financial transactions, medical consultations or marketing campaign results. Domino Data Lab is the system-of-record for enterprise data science teams.

Recreation/Entertainment 103

Recreation/Entertainment Data Science Data Warehouse Modeling 103

AWS Big Data

FEBRUARY 12, 2024

About Redshift and some relevant features for the use case Amazon Redshift is a fully managed, petabyte-scale, massively parallel data warehouse that offers simple operations and high performance. It makes it fast, simple, and cost-effective to analyze all your data using standard SQL and your existing business intelligence (BI) tools.

Data Warehouse 100

Data Warehouse Analytics Snapshot Cost-Benefit 100

O'Reilly on Data

OCTOBER 19, 2021

It has far-reaching implications as to how such applications should be developed and by whom: ML applications are directly exposed to the constantly changing real world through data, whereas traditional software operates in a simplified, static, abstract world which is directly constructed by the developer. This approach is not novel.

IT 346

IT Testing Experimentation Software 346

AWS Big Data

JANUARY 24, 2023

This post explains how to create a design that automatically backs up Amazon Simple Storage Service (Amazon S3), the AWS Glue Data Catalog, and Lake Formation permissions in different Regions and provides backup and restore options for disaster recovery. He specializes in migrating enterprise data warehouses to AWS Modern Data Architecture.

Data Architecture 64

Data Architecture Metadata Data Lake Snapshot 64

AWS Big Data

MAY 24, 2023

The following examples are also available in the sample notebook in the aws-samples GitHub repo for quick experimentation. You can use either the AWS Glue Data Catalog (recommended) or a Hive catalog for Iceberg tables. In that case, we have to query the table with the snapshot-id corresponding to the deleted row.

Data Lake 81

Data Lake Snapshot Metadata Optimization 81

Corinium

APRIL 25, 2019

Then when there is a breach, it comes as a shock, “wow, I didn’t even know that application had access to so much sensitive data”. Step One in any data security program should first be to discover and classify datasets that are sensitive, and know where that data is, and understand who really needs it to do their jobs.

Insurance 150

Insurance Risk IoT Cost-Benefit 150