AWS Big Data

JANUARY 8, 2024

This is the first post to a blog series that offers common architectural patterns in building real-time data streaming infrastructures using Kinesis Data Streams for a wide range of use cases. In this post, we will review the common architectural patterns of two use cases: Time Series Data Analysis and Event Driven Microservices.

Analytics 115

Analytics IoT Data-driven Snapshot 115

AWS Big Data

MARCH 7, 2023

This post provides guidance on how to build scalable analytical solutions for gaming industry use cases using Amazon Redshift Serverless. Flexible and easy to use – The solutions should provide less restrictive, easy-to-access, and ready-to-use data. Data hubs and data lakes can coexist in an organization, complementing each other.

Analytics 73

Analytics Data Warehouse Data Lake Metadata 73

AWS Big Data

JUNE 29, 2023

Amazon Kinesis Data Analytics makes it easy to transform and analyze streaming data in real time. In this post, we discuss why AWS recommends moving from Kinesis Data Analytics for SQL Applications to Amazon Kinesis Data Analytics for Apache Flink to take advantage of Apache Flink’s advanced streaming capabilities.

Data Analytics 76

Data Analytics Analytics IoT Data Lake 76

AWS Big Data

MAY 30, 2023

Customers have been using data warehousing solutions to perform their traditional analytics tasks. Traditional batch ingestion and processing pipelines that involve operations such as data cleaning and joining with reference data are straightforward to create and cost-efficient to maintain. options(**additional_options).mode("append").save(s3_output_folder)

Data Lake 82

Data Lake Data Analytics Analytics Data Processing 82

AWS Big Data

OCTOBER 3, 2023

In our previous post Improve operational efficiencies of Apache Iceberg tables built on Amazon S3 data lakes , we discussed how you can implement solutions to improve operational efficiencies of your Amazon Simple Storage Service (Amazon S3) data lake that is using the Apache Iceberg open table format and running on the Amazon EMR big data platform.

Optimization 111

Optimization Snapshot Data Lake Metadata 111

CIO Business Intelligence

JUNE 20, 2022

Otis One’s cloud-native platform is built on Microsoft Azure and taps into a Snowflake data lake. IoT sensors send elevator data to the cloud platform, where analytics are applied to support business operations, including reporting, data visualization, and predictive modeling.

Internet of Things 104

Internet of Things IoT Manufacturing Data Lake 104

BizAcuity

NOVEMBER 22, 2022

In today’s world that is largely data-driven, organizations depend on data for their success and survival, and therefore need robust, scalable data architecture to handle their data needs. This typically requires a data warehouse for analytics needs that is able to ingest and handle real time data of huge volumes.

Data Warehouse 98

Data Warehouse Cost-Benefit Data Lake Internet of Things 98

CIO Business Intelligence

OCTOBER 1, 2022

The partners say they will create the future of digital manufacturing by leveraging the industrial internet of things (IIoT), digital twin , data, and AI to bring products to consumers faster and increase customer satisfaction, all while improving productivity and reducing costs. Smart manufacturing at scale is a challenge.

Manufacturing 142

Manufacturing Digital Transformation IoT Internet of Things 142

Cloudera

AUGUST 18, 2021

For those models to produce meaningful outcomes, organizations need a well-defined data lifecycle management process that addresses the complexities of capturing, analyzing, and acting on data. If the data goes into a data lake before analysis, extracting it can get pretty complex and time-consuming.

Data Lake 88

Data Lake IoT Internet of Things Data-driven 88

Cloudera

JANUARY 30, 2024

According to Gartner , 80 percent of manufacturing CEOs are increasing investments in digital technologies—led by artificial intelligence (AI), Internet of Things (IoT), data, and analytics. Add appropriate contextual data (IT/business data), which is critical in AI analysis of manufacturing data.

Manufacturing 79

Manufacturing Contextual Data IoT Digital Transformation 79

CIO Business Intelligence

MAY 24, 2022

Data has always been fundamental to business, but as organisations continue to move to Cloud based environments coupled with advances in technology like streaming and real-time analytics, building a data driven business is one of the keys to success. There are many attributes a data-driven organisation possesses.

Data-driven 75

Data-driven Data Lake Data Warehouse Cost-Benefit 75

CIO Business Intelligence

APRIL 11, 2022

billion connected Internet of Things (IoT) devices by 2025, generating almost 80 billion zettabytes of data at the edge. This next manifestation of centralized data strategy emanates from past experiences with trying to coalesce the enterprise around a large-scale monolithic data lake. over last year.

Optimization 52

Optimization Data Lake Data Strategy Internet of Things 52

BizAcuity

APRIL 1, 2023

Introduction In today’s world that is largely data-driven, organizations depend on data for their success and survival, and therefore need robust, scalable data architecture to handle their data needs. For this reason, Snowflake is often the cloud-native data warehouse of choice. So, parallelism is not guaranteed.

Data Warehouse 52

Data Warehouse Cost-Benefit Data Lake Internet of Things 52

David Menninger's Analyst Perspectives

MAY 13, 2019

Organizations now must store, process and use data of significantly greater volume and variety than in the past.

Business Analytics 180

Business Analytics Big Data Analytics Enterprise 180

Sisense

JUNE 11, 2019

Does your data come in at high speeds and change rapidly? Those are all Big Data challenges that traditional analytics and BI platforms just can’t adequately handle. First off, IoT, the Internet of Things. The Internet has always, technically, been on “things”. are all things. What’s Next?

Big Data 73

Big Data Slice and Dice IoT Data Lake 73

Cloudera

AUGUST 27, 2020

To do this Manulife’s in-house data team built an Enterprise Data Lake (EDL) — a robust, enterprise-wide, data backend supporting digital connection, report automation, and AI & advanced analytics development. The two examples above are the perfect embodiment of why the connected data lifecycle matters.

Internet of Things 100

Internet of Things Data Lake Uncertainty IoT 100

AWS Big Data

APRIL 11, 2023

Such a solution should use the latest technologies, including Internet of Things (IoT) sensors, cloud computing, and machine learning (ML), to provide accurate, timely, and actionable data. However, analyzing large volumes of data can be a time-consuming and resource-intensive task. This is where Athena come in.

Forecasting 78

Forecasting Management IoT Data-driven 78

CIO Business Intelligence

SEPTEMBER 8, 2022

Most organizations understand the profound impact that data is having on modern business. In Foundry’s 2022 Data & Analytics Study , 88% of IT decision-makers agree that data collection and analysis have the potential to fundamentally change their business models over the next three years.

Cost-Benefit 101

Cost-Benefit Internet of Things Data-driven Data Lake 101

David Menninger's Analyst Perspectives

APRIL 8, 2021

The goal is to understand how to manage the growing volume of data in real time, across all sources and platforms, and use it to inform, streamline and transform internal operations. However, cloud adoption means living with a mix of on-premises and multiple cloud-based systems in a hybrid computing environment.

Management 248

Management Digital Transformation Business Intelligence Internet of Things 248

CIO Business Intelligence

JULY 17, 2023

It’s about possessing meaningful data that helps make decisions around product launches or product discontinuations, because we have information at the product and region level, as well as margins, profitability, transport costs, and so on. How is Havmor leveraging emerging technologies such as cloud, internet of things (IoT), and AI?

IT 80

IT Digital Transformation IoT Internet of Things 80

CIO Business Intelligence

APRIL 6, 2022

For decades organizations chased the Holy Grail of a centralized data warehouse/lake strategy to support business intelligence and advanced analytics. billion connected Internet of Things (IoT) devices by 2025, generating almost 80 billion zettabytes of data at the edge. over 2021.

Data Strategy 52

Data Strategy Strategy Internet of Things Data Warehouse 52

AWS Big Data

NOVEMBER 14, 2023

Ten years ago, we launched Amazon Kinesis Data Streams , the first cloud-native serverless streaming data service, to serve as the backbone for companies, to move data across system boundaries, breaking data silos. Next, let’s go back to the NHL use case where they combine IoT, data streaming, and machine learning.

IoT 55

IoT Data-driven Data Lake Data Strategy 55

Sisense

JANUARY 13, 2020

In Moving Parts , we explore the unique data and analytics challenges manufacturing companies face every day. The world of data in modern manufacturing. From a practical perspective, the computerization and automation of manufacturing hugely increase the data that companies acquire.

Manufacturing 52

Manufacturing Data Lake Big Data Data Warehouse 52

Sanjeev Mohan

FEBRUARY 28, 2018

I finished my trip at the Gartner Data & Analytics Summit in Sydney which exceeded over 1,000 attendees and became our biggest D&A Summit in the APAC region so far. The popularity of the Summit was not a surprise to me as I witnessed a pent-up surge in our clients’ and prospects’ desires to learn more about data related topics.

Management 52

Management Internet of Things Data Lake IoT 52

AWS Big Data

DECEMBER 13, 2023

Traditionally, customers used batch-based approaches for data movement from operational systems to analytical systems. A batch-based approach can introduce latency in data movement and reduce the value of data for analytics. usually a data warehouse) needs to reflect those changes in near real-time.

Data Warehouse 100

Data Warehouse Snapshot Data Processing Management 100

In(tegrate) the Clouds

AUGUST 4, 2016

In this post I wanted to share a few points made recently in a TDWI institute interview with SnapLogic founder and CEO Gaurav Dhillon when he was asked: What are some of the most interesting trends you’re seeing in the BI, analytics, and data warehousing space? Instead, one would ship the function to the data and return results.

IoT 45

IoT Internet of Things Data Lake Data Warehouse 45

BizAcuity

MARCH 1, 2019

There is a coherent overlap between the Internet of Things and Artificial Intelligence. IoT is basically an exchange of data or information in a connected or interconnected environment. At the backend, based on the data collected, data is stored in data lakes. Evolution of Internet of Things.

IoT 56

IoT Internet of Things Big Data Data-driven 56

Cloudera

FEBRUARY 4, 2019

One of the most promising technology areas in this merger that already had a high growth potential and is poised for even more growth is the Data-in-Motion platform called Hortonworks DataFlow (HDF). Process millions of real-time messages per second to feed into your data lake or for immediate streaming analytics.

IoT 73

IoT Prescriptive Analytics Digital Transformation Internet of Things 73

Sisense

OCTOBER 6, 2020

When these systems connect with external groups — customers, subscribers, shareholders, stakeholders — even more data is generated, collected, and exchanged. The result, as Sisense CEO Amir Orad wrote , is that every company is now a data company. The challenge comes when the data becomes huge and fast-changing.

Statistics 98

Statistics Unstructured Data Data-driven Visualization 98

Corinium

APRIL 25, 2019

Ahead of the Chief Data Analytics Officers & Influencers, Insurance event we caught up with Dominic Sartorio, Senior Vice President for Products & Development, Protegrity to discuss how the industry is evolving. And more recently, we have also seen innovation with IOT (Internet Of Things).

Insurance 150

Insurance Risk IoT Cost-Benefit 150

Cloudera

JANUARY 22, 2019

Forrester describes Big Data Fabric as, “A unified, trusted, and comprehensive view of business data produced by orchestrating data sources automatically, intelligently, and securely, then preparing and processing them in big data platforms such as Hadoop and Apache Spark, data lakes, in-memory, and NoSQL.”.

Big Data 44

Big Data Data Lake Internet of Things Enterprise 44

In(tegrate) the Clouds

DECEMBER 21, 2015

From AWS Aurora and Redshift for database management and data warehousing, to AWS GovCloud, which brings public cloud options to US government agencies, AWS continues to set the cloud computing standard for enterprise IT organizations and independent software vendors (ISVs). 2016 will be the year of the data lake.

Technology 45

Technology Internet of Things Digital Transformation Software 45

Sisense

MAY 11, 2021

You can’t talk about data analytics without talking about data modeling. These two functions are nearly inseparable as we move further into a world of analytics that blends sources of varying volume, variety, veracity, and velocity. But this was only the tip of the analytics iceberg.

Modeling 84

Modeling Big Data IoT Data Warehouse 84

AWS Big Data

JUNE 12, 2023

Organizations across the world are increasingly relying on streaming data, and there is a growing need for real-time data analytics, considering the growing velocity and volume of data being collected. During his leisure time, he prioritizes spending time with his family. Subramanya Vajiraya is a Sr.

Management 81

Management Metadata Testing Internet of Things 81

Data Virtualization

DECEMBER 1, 2023

Reading Time: 3 minutes In previous articles in this series, I examined the challenges facing energy utilities and how data and analytics can help to overcome them. In earlier posts, I discussed the challenges of decarbonization and electrification. In this post, I’m focusing on.

Data Integration 40

Data Integration Management Internet of Things Analytics 40

Alation

AUGUST 2, 2022

Customer centricity requires modernized data and IT infrastructures. Too often, companies manage data in spreadsheets or individual databases. This means that you’re likely missing valuable insights that could be gleaned from data lakes and data analytics. Customer Data Privacy And Security.

Strategy 52

Strategy Cost-Benefit Metrics Data Lake 52

Alation

FEBRUARY 20, 2020

Today, CDOs in a wide range of industries have a mechanism for empowering their organizations to leverage data. As data initiatives mature, the Alation data catalog is becoming central to an expanding set of use cases. Governing Data Lakes to Find Opportunities for Customers.

Data-driven 52

Data-driven Internet of Things Data Lake Strategy 52