Cloudera

DECEMBER 21, 2021

This enabled new use-cases with customers that were using a mix of Spark and Hive to perform data transformations. . Secondly, instead of being tied to the embedded Airflow within CDE, we wanted any customer using Airflow (even outside of CDE) to tap into the CDP platform, that’s why we published our Cloudera provider package.

Snapshot 115

Snapshot Data-driven Optimization Management 115

Cloudera

JULY 21, 2022

Cloudera Data Warehouse). Efficient batch data processing. Complex data transformations. Triton Digital, for example, uses Rill to deploy self-serve reporting for hundreds of digital media publishers with little or no training. Apache Hive. Large-scale high throughput analytics. Joins and subqueries . Apache Druid.

Metrics 82

Metrics Slice and Dice Data Warehouse Dashboards 82

AWS Big Data

MARCH 13, 2024

Developers can use the support in Amazon Location Service for publishing device position updates to Amazon EventBridge to build a near-real-time data pipeline that stores locations of tracked assets in Amazon Simple Storage Service (Amazon S3). You can test this solution yourself using the AWS Samples GitHub repository.

Analytics 91

Analytics IoT Metadata Internet of Things 91

Alation

MARCH 22, 2022

This produces end-to-end lineage so business and technology users alike can understand the state of a data lake and/or lake house. They can better understand data transformations, checks, and normalization. They can better grasp the purpose and use for specific data (and improve the pipeline!). Transparency is key.

Metadata 52

Metadata Cost-Benefit Data Quality Data Lake 52

Insight

APRIL 8, 2020

For example, they may give applicants access to an API and ask them to query data that satisfies some criteria, or they may share a large dataset and asking applicants to perform some sort of data transformation. Each submission is run through a series of tests to ensure that the desired output is produced.

Cost-Benefit 96

Cost-Benefit Software Data Warehouse Optimization 96

AWS Big Data

APRIL 25, 2023

The following AWS services are used for data ingestion, processing, and load: Amazon AppFlow is a fully managed integration service that enables you to securely transfer data between SaaS applications like Salesforce, SAP, Marketo, Slack, and ServiceNow, and AWS services like Amazon S3 and Amazon Redshift , in just a few clicks.

Sales 69

Sales Visualization Software Marketing 69

Ontotext

APRIL 4, 2019

Within a large enterprise, there is a huge amount of data accumulated over the years – many decisions have been made and different methods have been tested. Milena Yankova : What we did for the BBC in the previous Olympics was that we helped journalists publish their reports faster. I think artists can relax.

Recreation/Entertainment 76

Recreation/Entertainment Testing Enterprise Knowledge Discovery 76

AWS Big Data

APRIL 12, 2023

We also share a Spark benchmark solution that suits all Amazon EMR deployment options, so you can replicate the process in your environment for your own performance test cases. The solution uses the TPC-DS dataset and unmodified data schema and table relationships, but derives queries from TPC-DS to support the SparkSQL test cases.

Testing 72

Testing Big Data Metadata Optimization 72

AWS Big Data

JUNE 20, 2023

Kinesis Data Firehose is a fully managed service for delivering near-real-time streaming data to various destinations for storage and performing near-real-time analytics. You can perform analytics on VPC flow logs delivered from your VPC using the Kinesis Data Firehose integration with Datadog as a destination.

Dashboards 81

Dashboards Visualization Metrics Data Transformation 81

DataKitchen

OCTOBER 25, 2022

The goal of DataOps Observability is to provide visibility of every journey that data takes from source to customer value across every tool, environment, data store, data and analytic team, and customer so that problems are detected, localized and raised immediately. That data then fills several database tables.

Testing 130

Testing Data-driven Visualization Dashboards 130

AWS Big Data

OCTOBER 31, 2023

Data is decompressed and stored in a different S3 bucket (transformed data can be stored in the same S3 bucket where data was ingested, but for simplicity, we’re using two separate S3 buckets). The transformed data is then made accessible to Snowflake for data analysis. Set the protocol to Email.

Data Processing 86

Data Processing Management Publishing Testing 86

AWS Big Data

OCTOBER 11, 2023

Traditionally, such a legacy call center analytics platform would be built on a relational database that stores data from streaming sources. Data transformations through stored procedures and use of materialized views to curate datasets and generate insights is a known pattern with relational databases.

Management 101

Management Metadata Analytics Dashboards 101

AWS Big Data

JUNE 29, 2023

In this post, we discuss why AWS recommends moving from Kinesis Data Analytics for SQL Applications to Amazon Kinesis Data Analytics for Apache Flink to take advantage of Apache Flink’s advanced streaming capabilities. View the stream data. Transform and enrich the data. Manipulate the data with Python.

Data Analytics 71

Data Analytics Analytics IoT Data Lake 71

AWS Big Data

JULY 27, 2023

DataBrew is a visual data preparation tool that enables you to clean and normalize data without writing any code. The over 200 transformations it provides are now available to be used in an AWS Glue Studio visual job. Now that you have addressed all data quality issues identified on the sample, publish the project as a recipe.

Visualization 79

Visualization Cost-Benefit Data Quality Interactive 79

AWS Big Data

MARCH 3, 2023

Tricentis is the global leader in continuous testing for DevOps, cloud, and enterprise applications. Speed changes everything, and continuous testing across the entire CI/CD lifecycle is the key. Tricentis instills that confidence by providing software tools that enable Agile Continuous Testing (ACT) at scale.

Software 52

Software Data Lake Testing Cost-Benefit 52

Alation

OCTOBER 27, 2022

Few actors in the modern data stack have inspired the enthusiasm and fervent support as dbt. This data transformation tool enables data analysts and engineers to transform, test and document data in the cloud data warehouse. But what does this mean from a practitioner perspective?

Metrics 52

Metrics Dashboards Sales Reporting 52

AWS Big Data

FEBRUARY 6, 2023

For data pipeline orchestration, the Apache Airflow UI is a user-friendly tool that provides detailed views into your data pipeline. When it comes to pipeline health management, each service that your tasks are interacting with could be storing or publishing logs to different locations, such as an S3 bucket or Amazon CloudWatch logs.

Management 81

Management Interactive Metadata Publishing 81

IBM Big Data Hub

JANUARY 18, 2023

For these workloads, data lake vendors usually recommend extracting data into flat files to be used solely for model training and testing purposes. This adds an additional ETL step, making the data even more stale. Data lakehouse was created to solve these problems. Data discoverability.

Data Lake 52

Data Lake Data Warehouse Data-driven Metadata 52

IBM Big Data Hub

JULY 17, 2023

Real-time analytics and BI: Combine data from existing sources with new data to unlock new, faster insights without the cost and complexity of duplicating and moving data across different environments. How you can get started today Test out watsonx.ai and watsonx.data for yourself with our watsonx trial experience.

Machine Learning 68

Machine Learning Data Warehouse Modeling Cost-Benefit 68

AWS Big Data

NOVEMBER 29, 2023

dbt is an open source, SQL-first templating engine that allows you to write repeatable and extensible data transforms in Python and SQL. dbt is predominantly used by data warehouses (such as Amazon Redshift ) customers who are looking to keep their data transform logic separate from storage and engine.

Data Lake 99

Data Lake Management Metrics Data Warehouse 99

DataKitchen

APRIL 26, 2021

GSK had been pursuing DataOps capabilities such as automation, containerization, automated testing and monitoring, and reusability, for several years. DataOps provides the “continuous delivery equivalent for Machine Learning and enables teams to manage the complexities around continuous training, A/B testing, and deploying without downtime.

Measurement 246

Measurement Metrics Data-driven Testing 246

Jet Global

FEBRUARY 23, 2024

This field guide to data mapping will explore how data mapping connects volumes of data for enhanced decision-making. Why Data Mapping is Important Data mapping is a critical element of any data management initiative, such as data integration, data migration, data transformation, data warehousing, or automation.

Data Warehouse 52

Data Warehouse Reporting Data Transformation Sales 52

Jet Global

MAY 1, 2023

Strategic Objective Create a complete, user-friendly view of the data by preparing it for analysis. Requirement Multi-Source Data Blending Data from multiple sources is compiled and the output is a single view, metric, or visualization. Data Transformation and Enrichment Data can be enriched for analysis.

Analytics 52

Analytics Cost-Benefit Visualization Dashboards 52