AWS Big Data

APRIL 2, 2024

You can use Amazon Data Firehose to aggregate and deliver log events from your applications and services captured in Amazon CloudWatch Logs to your Amazon Simple Storage Service (Amazon S3) bucket and Splunk destinations, for use cases such as data analytics, security analysis, application troubleshooting etc.

Metadata 90

Metadata Marketing Analytics Data Transformation 90

Cloudera

MAY 30, 2024

The financial services industry is undergoing a significant transformation, driven by the need for data-driven insights, digital transformation, and compliance with evolving regulations.

Data Analytics 52

Data Analytics Risk Management Digital Transformation Analytics 52

AWS Big Data

MARCH 6, 2024

Part 1 of this two-part series described how to build a pseudonymization service that converts plain text data attributes into a pseudonym or vice versa. A centralized pseudonymization service provides a unique and universally recognized architecture for generating pseudonyms.

Metrics 98

Metrics Statistics Testing Data Lake 98

AWS Big Data

MARCH 13, 2024

Many organizations around the world rely on the use of physical assets, such as vehicles, to deliver a service to their end-customers. By combining historical vehicle location data with information from other sources, the company can devise empirical approaches for better decision-making.

Analytics 99

Analytics IoT Metadata Internet of Things 99

AWS Big Data

JANUARY 12, 2024

Many organizations, small and large, are working to migrate and modernize their analytics workloads on Amazon Web Services (AWS). Leadership and development teams can spend more time optimizing current solutions and even experimenting with new use cases, rather than maintaining the current infrastructure.

Data Lake 98

Data Lake Cost-Benefit Visualization Structured Data 98

CIO Business Intelligence

OCTOBER 30, 2023

Whether you’re looking to deliver a new product release, fix an issue, or enhance a service, the longer you make customers wait, the worse for your business. By employing Value Stream Management (VSM), executives gain the visibility to align teams, workflows, and investments around one key aspect: customer value.

Management 144

Management Digital Transformation Cost-Benefit ROI 144

AWS Big Data

MARCH 7, 2024

Organizations often need to manage a high volume of data that is growing at an extraordinary rate. At the same time, they need to optimize operational costs to unlock the value of this data for timely insights and do so with a consistent performance. We think of this concept as inside-out data movement. Example Corp.

Data Lake 117

Data Lake Analytics Dashboards Metrics 117

Cloudera

SEPTEMBER 26, 2023

Organizations increasingly rely on streaming data sources not only to bring data into the enterprise but also to perform streaming analytics that accelerate the process of being able to get value from the data early in its lifecycle.

Technology 92

Technology IT Data-driven Data Architecture 92

AWS Big Data

APRIL 3, 2024

Businesses are constantly evolving, and data leaders are challenged every day to meet new requirements. Customers are using AWS and Snowflake to develop purpose-built data architectures that provide the performance required for modern analytics and artificial intelligence (AI) use cases.

Data Lake 97

Data Lake Snapshot Metadata Data Architecture 97

CIO Business Intelligence

MARCH 4, 2024

Artificial intelligence (AI) is delivering rapid change for Australian business by raising customers’ expectations, generating new competitive challenges, and creating opportunities for new products and services. The integration of AI into Tungsten Automation’s service offering is organised around three key pillars.

Cost-Benefit 139

Cost-Benefit Predictive Analytics Data-driven Finance 139

AWS Big Data

MARCH 25, 2024

Amazon MSK Connect is a fully managed service for Apache Kafka Connect. With a few clicks, MSK Connect allows you to deploy connectors that move data between Apache Kafka and external systems. Together, these new capabilities make it straightforward to manage your MSK Connect resources and automate deployments through CI/CD pipelines.

Management 87

Management Metadata Reporting Big Data 87

AWS Big Data

JANUARY 10, 2024

Thousands of developers use Apache Flink to build streaming applications to transform and analyze data in real time. Apache Flink is an open source framework and engine for processing data streams. We show you how to scale by using metrics such as CPU, memory, backpressure, or any custom metric of your choice.

Metrics 102

Metrics Management Snapshot IT 102

O'Reilly on Data

FEBRUARY 4, 2019

In a recent survey , we explored how companies were adjusting to the growing importance of machine learning and analytics, while also preparing for the explosion in the number of data sources. You can find full results from the survey in the free report “Evolving Data Infrastructure”.).

Machine Learning 276

Machine Learning Enterprise IoT Big Data 276

AWS Big Data

MARCH 14, 2024

to Amazon Managed Streaming for Apache Kafka (Amazon MSK) running version 2.6.2. For us, Kafka’s high-performance distributed log system excels in handling massive data streams, making it indispensable for seamless communication. The unwavering reliability of Kafka aligns with our commitment to data integrity.

Management 111

Management Insurance Optimization Strategy 111

AWS Big Data

DECEMBER 13, 2023

Data loses value over time. Traditionally, customers used batch-based approaches for data movement from operational systems to analytical systems. A batch-based approach can introduce latency in data movement and reduce the value of data for analytics. Batch load can run once or several times a day.

Data Warehouse 101

Data Warehouse Snapshot Data Processing Management 101

AWS Big Data

MARCH 21, 2024

The volume of data generated globally continues to surge, from gaming, retail, and finance, to manufacturing, healthcare, and travel. Organizations are looking for more ways to quickly use the constant inflow of data to innovate for their businesses and customers. Apache Kafka is a popular choice for these real-time streaming needs.

Data Lake 100

Data Lake Management Modeling Optimization 100

AWS Big Data

MARCH 29, 2024

QuickSight makes it straightforward for business users to visualize data in interactive dashboards and reports. We walk through ingesting CloudWatch metrics into QuickSight using a CloudWatch metric stream and QuickSight SPICE. With these insights, teams have the visibility to make data integration pipelines more efficient.

Metrics 109

Metrics Visualization Dashboards Interactive 109

IBM Big Data Hub

SEPTEMBER 29, 2023

In a survey run by IDC, a leading provider of global IT research and advice, 43% of technology leaders indicated that they were “planning to deliver innovative digital products and services at a faster pace.” To show this with an example, consider one stream of business events sourced from an order management system.

Data-driven 105

Data-driven Uncertainty Cost-Benefit Marketing 105

AWS Big Data

JANUARY 2, 2024

Kinesis Agent is a standalone Java software application that offers a straightforward way to collect and send data to Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose. The agent continuously monitors a set of files and sends new data to the desired destination. southeast-2.amazonaws.com",

Testing 98

Testing Data Processing Metrics Publishing 98

Smart Data Collective

DECEMBER 1, 2022

The world of big data is constantly changing and evolving, and 2021 is no different. As we look ahead to 2022, there are four key trends that organizations should be aware of when it comes to big data: cloud computing, artificial intelligence, automated streaming analytics, and edge computing.

Big Data 141

Big Data Internet of Things Data-driven IoT 141

AWS Big Data

AUGUST 2, 2023

This post was co-written by Sheel Saket, Senior Data Science Manager at FIS, and Rupesh Tiwari, Senior Architect at Amazon Web Services. FIS , a leading technology and services provider, has encountered these very challenges. In their quest for a solution, they teamed up with AWS to tackle these obstacles head-on.

Visualization 86

Visualization Insurance Cost-Benefit Finance 86

AWS Big Data

NOVEMBER 9, 2023

Building a data platform involves various approaches, each with its unique blend of complexities and solutions. In this post, we delve into a case study for a retail use case, exploring how the Data Build Tool (dbt) was used effectively within an AWS environment to build a high-performing, efficient, and modern data platform.

Data Warehouse 109

Data Warehouse Testing Data Quality Reporting 109

AWS Big Data

MARCH 18, 2024

Apache Flink is an open source distributed processing engine, offering powerful programming interfaces for both stream and batch processing, with first-class support for stateful processing and event time semantics. New connectors Before we dive into the new functionalities of Apache Flink available with version 1.18.1,

Management 104

Management Snapshot Broadcasting Optimization 104

CIO Business Intelligence

APRIL 25, 2022

After launching industry-specific data lakehouses for the retail, financial services and healthcare sectors over the past three months, Databricks is releasing a solution targeting the media and the entertainment (M&E) sector. Partner solutions to boost functionality, adoption.

Recreation/Entertainment 120

Recreation/Entertainment Data Lake Data Warehouse Unstructured Data 120

AWS Big Data

AUGUST 3, 2023

With the rapid growth of technology, more and more data volume is coming in many different formats—structured, semi-structured, and unstructured. Data analytics on operational data at near-real time is becoming a common need. a new version of AWS Glue that accelerates data integration workloads in AWS.

Data Lake 85

Data Lake Visualization Dashboards Insurance 85

CIO Business Intelligence

APRIL 12, 2023

By George Trujillo, Principal Data Strategist, DataStax I recently had a conversation with a senior executive who had just landed at a new organization. He had been trying to gather new data insights but was frustrated at how long it was taking. Sound familiar?) It isn’t easy.

Machine Learning 145

Machine Learning Cost-Benefit Data-driven Strategy 145

AWS Big Data

OCTOBER 25, 2023

As the scale and complexity of microservices and distributed applications continues to expand, customers are seeking guidance for building cost-efficient infrastructure supporting operational analytics use cases. Operational analytics is a popular use case with Amazon OpenSearch Service.

Analytics 124

Analytics Data Processing Optimization Metrics 124

Smart Data Collective

AUGUST 30, 2022

Companies spent over $405 billion on cloud services last year. Cloud computing has found its way into many business scenarios and is a relatively new concept for businesses. Data streaming. Cloud computing ( [link] ) can be used to support real-time data streams for better business decision-making.

Technology 133

Technology Enterprise Cost-Benefit Testing 133

CIO Business Intelligence

MAY 8, 2024

In the age of big data, where information is generated at an unprecedented rate, the ability to integrate and manage diverse data sources has become a critical business imperative. Traditional data integration methods are often cumbersome, time-consuming, and unable to keep up with the rapidly evolving data landscape.

Data Integration 100

Data Integration IoT Cost-Benefit Data Quality 100

IBM Big Data Hub

FEBRUARY 12, 2024

Recognizing the need to harness real-time data, businesses are increasingly turning to event-driven architecture (EDA) as a strategic approach to stay ahead of the curve. This trend grows stronger as organizations realize the benefits that come from the power of real-time data streaming.

Data-driven 81

Data-driven Cost-Benefit Uncertainty Technology 81

IBM Big Data Hub

JANUARY 8, 2024

This approach then enables them to build new responsive applications.  Stage 2—Broader adoption Increased awareness across IT organizations leads to a transition to standardized methods of creating an event backbone that caters to both existing and new event-driven projects across multiple teams.

Data-driven 76

Data-driven Management Advertising Enterprise 76

AWS Big Data

NOVEMBER 28, 2023

As customers become more data driven and use data as a source of competitive advantage, they want to easily run analytics on their data to better understand their core business drivers to grow sales, reduce costs, and optimize their businesses. ETL is the process data engineers use to combine data from different sources.

Data Warehouse 95

Data Warehouse Data-driven Machine Learning B2B 95

AWS Big Data

FEBRUARY 8, 2024

AWS Glue is a serverless data integrating service that you can use to catalog data and prepare for analytics. With AWS Glue, you can discover your data, develop scripts to transform sources into targets, and schedule and run extract, transform, and load (ETL) jobs in a serverless environment.

Consulting 59

Consulting Testing Data Integration Data Analytics 59

CIO Business Intelligence

FEBRUARY 14, 2023

By George Trujillo, Principal Data Strategist, DataStax Increased operational efficiencies at airports. Titanium Intelligent Solutions, a global SaaS IoT organization, even saved one customer over 15% in energy costs across 50 distribution centers , thanks in large part to AI. Instant reactions to fraudulent activities at banks.

Data Strategy 116

Data Strategy Strategy Machine Learning Data-driven 116

CIO Business Intelligence

APRIL 25, 2024

And that’s all before considering the need to fuel new AI initiatives , which can push cloud costs up further. For CIOs who may need to customize their cloud cost information streams or manage a complex cloud estate, do-it-yourself cloud cost management may be the way to go.

Management 125

Management Optimization Strategy Enterprise 125

AWS Big Data

JUNE 1, 2023

Apache Flink is a framework and distributed processing engine for stateful computations over data streams. Amazon Kinesis Data Analytics for Apache Flink is a fully managed service that enables you to use an Apache Flink application to process streaming data.

Deep Learning 66

Deep Learning Data Analytics Analytics Machine Learning 66

AWS Big Data

MARCH 27, 2024

Amazon Redshift is a fast, scalable, and fully managed cloud data warehouse that allows you to process and run your complex SQL analytics workloads on structured and semi-structured data. The scope should contain inputs and data points on the current architecture as well as the target architecture.

Testing 102

Testing Data Warehouse Metrics Cost-Benefit 102