AWS Big Data

APRIL 16, 2024

Running Apache Airflow at scale puts proportionally greater load on the Airflow metadata database, sometimes leading to CPU and memory issues on the underlying Amazon Relational Database Service (Amazon RDS) cluster. A resource-starved metadata database may lead to dropped connections from your workers, failing tasks prematurely.

Metadata 100

Metadata Metrics Testing Management 100

Rita Sallam

APRIL 2, 2023

We explored these questions and more at our Bake-Offs and Show Floor Showdowns at our Data and Analytics Summit in Orlando with 4,000 of our closest D&A friends and family. The first featured analytics and BI platform Gartner Magic Quadrant leaders while the other showcased high interest data science and machine learning platforms.

Optimization 52

Optimization Machine Learning Insurance Risk 52

Peter James Thomas

AUGUST 18, 2018

That was the Science, here comes the Technology… A Brief Hydrology of Data Lakes. One of the early promises of a Data Lake approach was that – once all relevant data had been ingested – this would be directly leveraged by Data Scientists to derive insight.

Data Lake 63

Data Lake Data Warehouse Data mining Statistics 63

Cloudera

NOVEMBER 13, 2020

Data science experiment result and performance analysis, for example, calculating model lift. While plan time statistics are unreliable, an execution engine that adapts in real-time based on actual data means that the right optimization can be applied dynamically when the query seems to be taking longer than it should.

Optimization 143

Optimization Metadata Statistics Cost-Benefit 143

Cloudera

JUNE 25, 2019

On June 18th, Cloudera provided an exclusive preview of these capabilities, and more, with the introduction of Cloudera Data Platform (CDP), the industry’s first enterprise data cloud. Over 2000 customers and partners joined us in this live webinar featuring a first-look at our upcoming cloud-native CDP services.

Enterprise 87

Enterprise Machine Learning Recreation/Entertainment IoT 87

Sisense

SEPTEMBER 3, 2019

Redshift sort keys allow you to specify in what order the data is stored across your nodes. By using metadata about where the data is stored, it allows the query engine to skip over chunks of data that it knows are not within the bounds of your query’s parameters.

Data Warehouse 78

Data Warehouse OLAP Statistics Cost-Benefit 78

Domino Data Lab

SEPTEMBER 9, 2019

Data science teams in industry must work with lots of text, one of the top four categories of data used in machine learning. That’s excellent for supporting really interesting workflow integrations in data science work. metadata=convention_df["speaker"]? ). category="democrat",?.

Deep Learning 110

Deep Learning Machine Learning Data Science Visualization 110