AWS Big Data

JANUARY 8, 2024

This is the first post to a blog series that offers common architectural patterns in building real-time data streaming infrastructures using Kinesis Data Streams for a wide range of use cases. Refer to Amazon Kinesis Data Streams integrations for additional details.

Analytics 108

Analytics IoT Data-driven Snapshot 108

AWS Big Data

MARCH 25, 2024

Furthermore, data events are filtered, enriched, and transformed to a consumable format using a stream processor. The result is made available to the application by querying the latest snapshot. For more information, refer to Notions of Time: Event Time and Processing Time. For more information, refer to Dynamic Tables.

Data Lake 86

Data Lake Unstructured Data Management Modeling 86

AWS Big Data

APRIL 3, 2024

By analyzing the historical report snapshot, you can identify areas for improvement, implement changes, and measure the effectiveness of those changes. For instructions, refer to Amazon DataZone quickstart with AWS Glue data. To learn more about Amazon DataZone, refer to the Amazon DataZone User Guide.

Data Quality 85

Data Quality Visualization Metadata Metrics 85

AWS Big Data

JANUARY 24, 2023

This means that if data is moved from a bucket in the source Region to another bucket in the target Region, the data access permissions need to be reapplied in the target Region. AWS Glue Data Catalog The AWS Glue Data Catalog is a central repository of metadata about data stored in your data lake.

Data Architecture 64

Data Architecture Metadata Data Lake Snapshot 64

AWS Big Data

OCTOBER 18, 2023

Many customers migrate their data warehousing workloads to Amazon Redshift and benefit from the rich capabilities it offers, such as the following: Amazon Redshift seamlessly integrates with broader data, analytics, and AI or machine learning (ML) services on AWS , enabling you to choose the right tool for the right job.

Analytics 92

Analytics Data Warehouse Testing Dashboards 92

AWS Big Data

MARCH 28, 2023

This post is designed to be implemented for a real customer use case, where you get full snapshot data on a daily basis. Over the years, he has helped multiple customers on data platform transformations across industry verticals. His core area of expertise include Technology Strategy, Data Analytics, and Data Science.

Data Lake 68

Data Lake Testing Snapshot Sales 68

AWS Big Data

AUGUST 2, 2023

Using existing analytics tools such as Amazon Athena and Amazon QuickSight an organization can gain insight into its estimated carbon footprint. The data architecture diagram below shows an example of how you could use AWS services to calculate and visualize an organization’s estimated carbon footprint.

Data Lake 68

Data Lake Measurement Visualization Data Architecture 68