DataRobot Blog

SEPTEMBER 1, 2022

Now you can aggregate prediction statistics much faster while controlling the governance and security of your sensitive data — no need to submit their entire prediction requests to DataRobot AI Cloud Platform to get data about drift and accuracy monitoring. 1 IDC, MLOps – Where ML Meets DevOps, doc #US48544922, March 2022.

Metrics 145

Metrics Statistics Modeling Data Science 145

AWS Big Data

JULY 3, 2023

With scalable metadata indexing, Apache Iceberg is able to deliver performant queries to a variety of engines such as Spark and Athena by reducing planning time. Apache Iceberg – Apache Iceberg is an open-source table format that is designed to provide efficient, scalable, and secure access to large datasets.

Snapshot 86

Snapshot Data Lake Testing Strategy 86

AWS Big Data

JUNE 9, 2023

Iceberg doesn’t optimize file sizes or run automatic table services (for example, compaction or clustering) when writing, so streaming ingestion will create many small data and metadata files. Using column statistics , Iceberg offers efficient updates on tables that are sorted on a “key” column.

Data Lake 115

Data Lake Metadata Optimization Statistics 115

AWS Big Data

APRIL 12, 2023

release in January 2022, the optimized Spark runtime was 3.5 Metadata store – We use Spark’s in-memory data catalog to store metadata for TPC-DS databases and tables— spark.sql.catalogImplementation is set to the default value in-memory. No statistics are pre-calculated for these tables. As of the Amazon EMR 6.5

Testing 77

Testing Big Data Metadata Optimization 77

Andrew White

JANUARY 9, 2022

On Thursday January 6th I hosted Gartner’s 2022 Leadership Vision for Data and Analytics webinar. Which trends do you see for 2022 in AI & ML technology and tools and tool capabilities? We will publish a new Top Trends for D&A for 2022 in a couple of months. The last item should be updated in 2022.

Analytics 64

Analytics Measurement Modeling Data-driven 64

Alation

JULY 7, 2022

In 2013 I joined American Family Insurance as a metadata analyst. I had always been fascinated by how people find, organize, and access information, so a metadata management role after school was a natural choice. The use cases for metadata are boundless, offering opportunities for innovation in every sector.

Metadata 52

Metadata Data-driven Insurance Statistics 52

AWS Big Data

DECEMBER 12, 2023

Athena uses the AWS Glue Data Catalog to store the table metadata. In our example, we use the latest Parquet files for 2022. ny_taxi_summary.py – This PySpark script processes the green_zone and yellow_zone files to aggregate statistics on taxi rides, grouping data by service zones and location IDs.

Data Processing 92

Data Processing Management Statistics Interactive 92

AWS Big Data

JUNE 15, 2023

Overview of AWS Clean Rooms AWS Clean Rooms was first announced at AWS re:Invent 2022, and is now generally available. Analysis rules – AWS Clean Rooms supports two types of analysis rules: Aggregation – Members can run queries that aggregate statistics using COUNT, SUM, or AVG functions along optional dimensions.

Metadata 92

Metadata Dashboards Risk Statistics 92

Cloudera

SEPTEMBER 15, 2022

The gathering in 2022 marked the sixteenth year for top data and analytics professionals to come to the MIT campus to explore current and future trends. In June 2022, Barr Moses of Monte Carlo expanded on her initial article defining data observability. What’s a trend you see on the horizon? .

Data Lake 81

Data Lake Data Architecture Data-driven Data Warehouse 81

Alation

APRIL 25, 2023

between 2022 and 2029. Whether you deal in customer contact information, website traffic statistics, sales data, or some other type of valuable information, you’ll need to put a framework of policies in place to manage your data seamlessly. Using bad data, or the incorrect data can generate devastating results.

Data Governance 52

Data Governance Marketing Machine Learning Sales 52

Alation

APRIL 25, 2023

between 2022 and 2029. Whether you deal in customer contact information, website traffic statistics, sales data, or some other type of valuable information, you’ll need to put a framework of policies in place to manage your data seamlessly. Using bad data, or the incorrect data can generate devastating results.

Data Governance 40

Data Governance Marketing Machine Learning Sales 40

Sisense

AUGUST 21, 2020

From Forecast to Trends to natural language querying, we are completely transparent about the technology behind and the statistical characteristics of the output. It also converts metadata from being used in auditing, lineage and reporting to powering dynamic systems.”. Trend 2: Decline of the dashboard. Trend 6: Cloud is a given.

Analytics 112

Analytics Machine Learning Dashboards Visualization 112

O'Reilly on Data

MARCH 23, 2023

ChatGPT, or something built on ChatGPT, or something that’s like ChatGPT, has been in the news almost constantly since ChatGPT was opened to the public in November 2022. And it can look up an author and make statistical observations about their interests. O’Reilly, 2022). This example taken from [link].

IT 258

IT Modeling Testing Risk 258