Remove apache-kafka-deployments-and-systems-reliability-part-1
article thumbnail

Apache Kafka Deployments and Systems Reliability – Part 1

Cloudera

There are many ways that Apache Kafka has been deployed in the field. In our Kafka Summit 2021 presentation, we took a brief overview of many different configurations that have been observed to date. Serial and Parallel Systems Reliability . Serial Systems Reliability.

article thumbnail

Create a modern data platform using the Data Build Tool (dbt) in the AWS Cloud

AWS Big Data

As part of their cloud modernization initiative, they sought to migrate and modernize their legacy data platform. As part of their cloud modernization initiative, they sought to migrate and modernize their legacy data platform. This separation further simplifies data management and enhances the system’s overall performance.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Data governance beyond SDX: Adding third party assets to Apache Atlas

Cloudera

While Cloudera Data Platform (CDP) already supports the entire data lifecycle from ‘Edge to AI’, we at Cloudera are fully aware that enterprises have more systems outside of CDP. Apache Atlas as a fundamental part of SDX. can be part of a superior class, allowing the creation of a tree-like, structured storage for assets.

article thumbnail

Krones real-time production line monitoring with Amazon Managed Service for Apache Flink

AWS Big Data

Production lines are complex systems with lots of possible errors that could stall the line and decrease the production yield. Krones wants to detect the failure as early as possible (sometimes even before it happens) and notify production line operators to increase reliability and output. So how to detect a failure?

article thumbnail

How Klarna Bank AB built real-time decision-making with Amazon Kinesis Data Analytics for Apache Flink

AWS Big Data

This post presents a reference architecture for real-time queries and decision-making on AWS using Amazon Kinesis Data Analytics for Apache Flink. In addition, we explain why the Klarna Decision Tooling team selected Kinesis Data Analytics for Apache Flink for their first real-time decision query service.

article thumbnail

How SOCAR handles large IoT data with Amazon MSK and Amazon ElastiCache for Redis

AWS Big Data

In this post, we provide a detailed overview of streaming messages with Amazon Managed Streaming for Apache Kafka (Amazon MSK) and Amazon ElastiCache for Redis , covering technical aspects and design considerations that are essential for achieving optimal results. SOCAR has deployed in-car devices that capture data using AWS IoT Core.

IoT 103
article thumbnail

Reliable Data Exchange with the Outbox Pattern and Cloudera DiM

Cloudera

In this post, I will demonstrate how to use the Cloudera Data Platform (CDP) and its streaming solutions to set up reliable data exchange in modern applications between high-scale microservices, and ensure that the internal state will stay consistent even under the highest load. Introduction Many modern application designs are event-driven.