Cloudera

DECEMBER 21, 2021

New in 2021. Figure 2 – CDE product launch highlights in 2021. Early on in 2021 we expanded our APIs to support pipelines using a new job type — Airflow. This enabled new use-cases with customers that were using a mix of Spark and Hive to perform data transformations. . Test Drive CDP Pubic Cloud.

Snapshot 115

Snapshot Data-driven Optimization Management 115

DataKitchen

OCTOBER 5, 2021

Too many data organizations run data operations like a hundred-year-old car factory. While car companies lowered costs using mass production, companies in 2021 put data engineers and data scientists on the assembly line. That’s the state of data analytics today. . Their product is the data.

Testing 152

Testing Dashboards Measurement Experimentation 152

AWS Big Data

NOVEMBER 9, 2023

A modern data platform entails maintaining data across multiple layers, targeting diverse platform capabilities like high performance, ease of development, cost-effectiveness, and DataOps features such as CI/CD, lineage, and unit testing. It does this by helping teams handle the T in ETL (extract, transform, and load) processes.

Data Warehouse 108

Data Warehouse Testing Data Quality Reporting 108

Cloudera

DECEMBER 9, 2022

In 2021 we launched Cloudera DataFlow for the Public Cloud (CDF-PC) , addressing operational challenges that administrators face when running NiFi flows in production environments. Developers need to onboard new data sources, chain multiple data transformation steps together, and explore data as it travels through the flow.

Testing 95

Testing Cost-Benefit Interactive Visualization 95

AWS Big Data

MARCH 13, 2024

You can also use the data transformation feature of Data Firehose to invoke a Lambda function to perform data transformation in batches. This solution includes a Lambda function that continuously updates the Amazon Location tracker with simulated location data from fictitious journeys.

Analytics 97

Analytics IoT Metadata Internet of Things 97

CIO Business Intelligence

JUNE 13, 2022

1 priority within the CIO function is cybersecurity strategies, up from the second spot in 2021. Angel-Johnson says she, too, has a heightened level of concern around security issues and more specifically data protection. I thought I was hired for digital transformation but what is really needed is a data transformation,” she says.

IT 144

IT Digital Transformation Strategy Internet of Things 144

AWS Big Data

APRIL 25, 2024

Alternatively, you can use AWS Glue for Apache Spark, which provides built-in support for bucketing configurations during the data transformation process. AWS Glue allows you to define bucketing parameters, such as the number of buckets and the columns to bucket on, providing an optimized data layout for efficient querying with Athena.

Optimization 66

Optimization Data Lake Cost-Benefit Reporting 66