AWS Big Data

JUNE 23, 2023

Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, straightforward to use, and makes it simple for anyone with SQL skills to quickly analyze large-scale datasets in multiple Regions. The following code provides a SQL example for a view creation.

Data Lake 66

Data Lake Cost-Benefit Dashboards Data Warehouse 66

Cloudera

NOVEMBER 10, 2021

For big data platforms like Cloudera’s stack that are used by multiple business units with many users, upgrading even minor versions must be a well-planned activity to reduce the impact to users and business. This blog post provides CDH users with a quick overview of Ranger as a Sentry replacement for Hadoop SQL policies in CDP.

Data Lake 78

Data Lake Management Metadata Modeling 78

AWS Big Data

FEBRUARY 24, 2023

The solution workflow consists of the following steps: Streaming data is generated in JSON format using the KDG template and inserted into Kinesis Data Streams. On the Athena console, create an Athena workgroup named demoworkgroup for SQL queries. To learn more about AWS integrations, refer to Iceberg AWS integrations.

Data Lake 79

Data Lake Metadata Testing Data Warehouse 79

AWS Big Data

APRIL 26, 2023

This access to the metadata supports the need for updating visuals based on changes as well as automating row and column level security ensuring customer data is properly governed. Enables a hub-and-spoke model for Data Access in multiple AWS accounts in a data mesh fashion.

Metadata 52

Metadata Dashboards Visualization Consulting 52

Jet Global

SEPTEMBER 4, 2020

The SQL query language used to extract data for reporting could also potentially be used to insert, update, or delete records from the database. Instead of summarizing tens of thousands of rows of data at runtime, reports could be produced very quickly because the heavy lifting had already been done. Data Entities.

Data Lake 88

Data Lake OLAP Data Warehouse Unstructured Data 88

AWS Big Data

SEPTEMBER 22, 2023

Amazon Redshift has added many features to enhance analytical processing like ROLLUP, CUBE and GROUPING SETS , which were demonstrated in the post Simplify Online Analytical Processing (OLAP) queries in Amazon Redshift using new SQL constructs such as ROLLUP, CUBE, and GROUPING SETS.

Data Warehouse 100

Data Warehouse OLAP Risk Online Analytical Processing 100

AWS Big Data

MARCH 9, 2023

Dimensions provide answers to exploratory business questions by allowing end-users to slice and dice data in a variety of ways using familiar SQL commands. SCD population challenge Populating an SCD dimension table involves merging data from multiple source tables, which are usually normalized.

Slice and Dice 77

Slice and Dice Data Warehouse Metrics Metadata 77

Cloudera

APRIL 3, 2023

The new feature this blog post is aiming to discuss about Iceberg V2 format (version 2), as the Iceberg table specification explains, the V1 format aimed to support large analytic data tables, while V2 aimed to add row level deletes and updates. eliminating the deleted rows from the output).

Data Warehouse 83

Data Warehouse Snapshot Metadata Cost-Benefit 83

AWS Big Data

OCTOBER 19, 2023

We also discuss the benefits of incremental loading and the techniques for implementing the architecture using AWS Glue , which is a serverless, scalable data integration service that helps you discover, prepare, move, and integrate data from multiple sources. AWS Glue supports the Redshift MERGE SQL command within its data integration jobs.

Data Lake 109

Data Lake Data Warehouse Visualization Snapshot 109

AWS Big Data

MAY 24, 2023

Apache Iceberg is an open table format for large datasets in Amazon Simple Storage Service (Amazon S3) and provides fast query performance over large tables, atomic commits, concurrent writes, and SQL-compatible table evolution. It does this while it scales in the background to handle the increased request rate. and below). and Spark 3.3.1.

Data Lake 80

Data Lake Snapshot Metadata Optimization 80

AWS Big Data

FEBRUARY 19, 2024

In this post, we discuss when you should consider using multiple warehouses to write to the same databases, explain how multi-warehouse writes through data sharing works, and walk you through an example on how to use multiple warehouses to write to the same database. We do not ingest data into the af_customer table.

Cost-Benefit 82

Cost-Benefit Data Warehouse Marketing Interactive 82

AWS Big Data

JUNE 6, 2023

AWS Glue is a serverless data integration service that makes it simple to discover, prepare, and combine data for analytics, machine learning (ML), and application development. You can use AWS Glue to create, run, and monitor data integration and ETL (extract, transform, and load) pipelines and catalog your assets across multiple data stores.

Data Quality 71

Data Quality Data Lake Data-driven Metrics 71

AWS Big Data

JUNE 9, 2023

This data is then projected into analytics services such as data warehouses, search systems, stream processors, query editors, notebooks, and machine learning (ML) models through direct access, real-time, and batch workflows.

Data Lake 110

Data Lake Metadata Optimization Statistics 110

AWS Big Data

FEBRUARY 22, 2023

hours just to ingest data, which was hundreds of thousands of rows. The CDC records consist of all inserts, updates, and deletes from the source database. The files are transformed to Parquet columnar format with snappy compression and table partitioning to optimize SQL queries from Athena.

Data Lake 52

Data Lake Dashboards Cost-Benefit Metadata 52

Cloudera

OCTOBER 25, 2022

The data lakehouse is gaining in popularity because it enables a single platform for all your enterprise data with the flexibility to run any analytic and machine learning (ML) use case. It allows users to rapidly ingest data and run self-service analytics and machine learning.

Data Processing 52

Data Processing Data Lake Cost-Benefit Risk 52

AWS Big Data

SEPTEMBER 21, 2023

Amazon Redshift provides numerous features such as role-based access control (RBAC), row-level security (RLS), column-level security (CLS), and dynamic data masking to facilitate the secure use of data. If you’re familiar with SQL Notebooks , you can download this SQL notebook for the demonstration and import it to quickly get started.

Data Warehouse 81

Data Warehouse Interactive Data Architecture Data Lake 81

AWS Big Data

APRIL 5, 2023

Amazon Redshift provides role-based access control, row-level security, column-level security, and dynamic data masking, along with other database security features to enable organizations to enforce fine-grained data security. You may optionally choose to test it with your own SQL client or business intelligence tools.

Data Warehouse 73

Data Warehouse Management Business Intelligence Testing 73

Jet Global

JUNE 13, 2022

Cloud enterprise resource planning (ERP) software is ideal for a variety of applications, including managing multiple departments and CRM integration. Like Microsoft’s out-of-the-box offerings, they still contain a similar reporting environment with a short learning curve. Interested in Multiple Companies/Databases Consolidation.

Reporting 52

Reporting Data Lake Sales Operational Reporting 52