AWS Big Data

NOVEMBER 17, 2023

You can analyze data or build applications from an Amazon Simple Storage Service (Amazon S3) data lake and 30 data sources, including on-premises data sources or other cloud systems using SQL or Python. Let’s discuss some of the cost-based optimization techniques that contributed to improved query performance.

Optimization 115

Optimization Statistics Metadata Data Lake 115

AWS Big Data

JANUARY 24, 2024

AWS-powered data lakes, supported by the unmatched availability of Amazon Simple Storage Service (Amazon S3), can handle the scale, agility, and flexibility required to combine different data and analytics approaches. The timestamp clause lets us travel back without altering current data.

Snapshot 103

Snapshot Data Lake Metadata Optimization 103

CIO Business Intelligence

OCTOBER 21, 2022

When Curt Garner became Chipotle’s first CIO in 2015, the only technology used for online restaurant ordering was, “believe it or not,” a fax machine, he says. The system provides “demand-based cooking and ingredient preparation forecasts to optimize throughput and freshness while minimizing food waste,” according to the company.

Digital Transformation 145

Digital Transformation Data Lake Forecasting Technology 145

Cloudera

JUNE 2, 2022

Since 2015, the Cloudera DataFlow team has been helping the largest enterprise organizations in the world adopt Apache NiFi as their enterprise standard data movement tool. Every organization on the hybrid cloud journey needs the ability to take control of their data flows from origination through all points of consumption.

Enterprise 99

Enterprise Data Lake Data Collection Data-driven 99

CIO Business Intelligence

JULY 13, 2022

Since 2015, the Cloudera DataFlow team has been helping the largest enterprise organizations in the world adopt Apache NiFi as their enterprise standard data movement tool. Every organization on the hybrid cloud journey needs the ability to take control of their data flows from origination through all points of consumption.

Enterprise 88

Enterprise Data Lake Data Collection Data-driven 88

AWS Big Data

JUNE 6, 2023

In this post we showcase how we used AWS Glue to move siloed digital analytics data, with inconsistent arrival times, to AWS S3 (our Data Lake) and our central data warehouse (DWH), Snowflake. AWS Glue gave us a cost-efficient option to migrate the data and we further optimized storage cost by pruning cold data.

Analytics 56

Analytics Data Lake Testing Optimization 56

AWS Big Data

JULY 28, 2023

Recently, Athena added support for creating and querying views on federated data sources to bring greater flexibility and ease of use to use cases such as interactive analysis and business intelligence reporting. Big Data Architect on Amazon Athena. Let’s dive into the solution. Pathik Shah is a Sr.

Big Data 82

Big Data Data Architecture Data Lake Interactive 82