AWS Big Data

OCTOBER 3, 2023

A data lake is a centralized repository that you can use to store all your structured and unstructured data at any scale. You can store your data as-is, without having to first structure the data and then run different types of analytics for better business insights.

Data Lake 100

Data Lake Metadata Snapshot Recreation/Entertainment 100

AWS Big Data

MARCH 10, 2023

Since the deluge of big data over a decade ago, many organizations have learned to build applications to process and analyze petabytes of data. Data lakes have served as a central repository to store structured and unstructured data at any scale and in various formats.

Data Lake 100

Data Lake Sales Data Warehouse Snapshot 100

AWS Big Data

MARCH 7, 2024

At the same time, they need to optimize operational costs to unlock the value of this data for timely insights and do so with a consistent performance. With this massive data growth, data proliferation across your data stores, data warehouse, and data lakes can become equally challenging.

Data Lake 111

Data Lake Analytics Dashboards Metrics 111

AWS Big Data

OCTOBER 17, 2023

Tens of thousands of customers use Amazon Redshift to gain business insights from their data. With Amazon Redshift, you can use standard SQL to query data across your data warehouse, operational data stores, and data lake. After you install the data extraction agent, register it in AWS SCT.

Data Warehouse 83

Data Warehouse Data Processing Data Lake Management 83

AWS Big Data

MARCH 29, 2024

Typically, you have multiple accounts to manage and run resources for your data pipeline. Looking at the Skewness Job per Job visualization, there was spike on November 1, 2023. Visualize and analyze with AWS Glue job observability metrics Let’s use the dashboard to make AWS Glue usage more performant. Let’s drill down into details.

Metrics 103

Metrics Visualization Dashboards Interactive 103

Cloudera

JUNE 5, 2023

On May 3, 2023, Cloudera kicked off a contest called “Best in Flow” for NiFi developers to compete to build the best data pipelines. RK built some simple flows to pull streaming data into Google Cloud Storage and Snowflake. This blog is to congratulate our winner and review the top submissions.

Testing 73

Testing Data Lake Data Processing IT 73

CIO Business Intelligence

FEBRUARY 20, 2024

Data also needs to be sorted, annotated and labelled in order to meet the requirements of generative AI. No wonder CIO’s 2023 AI Priorities study found that data integration was the number one concern for IT leaders around generative AI integration, above security and privacy and the user experience.

Cost-Benefit 115

Cost-Benefit Data Lake Data Warehouse Data Integration 115