AWS Big Data

NOVEMBER 17, 2023

In our testing, the dataset was stored in Amazon S3 in non-compressed Parquet format and the AWS Glue Data Catalog was used to store metadata for databases and tables. The following graph presents the top 10 queries from the TPC-DS benchmark with the greatest performance improvement.

Optimization 120

Optimization Statistics Metadata Data Lake 120

IBM Big Data Hub

JULY 6, 2023

.” “Data science” was first used as an independent discipline in 2001. The fields have evolved such that to work as a data analyst who views, manages and accesses data, you need to know Structured Query Language (SQL) as well as math, statistics, data visualization (to present the results to stakeholders) and data mining.

Machine Learning 58

Machine Learning Data Science Statistics Deep Learning 58

Domino Data Lab

APRIL 3, 2019

In particular, here’s my Strata SF talk “Overview of Data Governance” presented in article form. That presented an opportunity to learn, putting me in the same position as much of the audience. The on-the-ground reality of DG presents an almost overwhelming array of topics. The presentation layer moved UX into the browser.

Data Governance 84

Data Governance Machine Learning Metadata Big Data 84

Occam's Razor

APRIL 25, 2011

Every presentation I do is customized for the audience in the room. I'm going to present a cluster of what I think are digital "crimes against humanity." Making website iterations based on executive opinions, but not site testing. But maybe the issue is that you (and the Marketers and Leaders.

Marketing 126

Marketing Measurement Metrics Testing 126

O'Reilly on Data

MAY 27, 2020

Under school district policy, each of Audrey’s eleven- and twelve-year old students is tested at least three times a year to determine his or her Lexile, a number between 200 and 1,700 that reflects how well the student can read. They test each student’s grasp of a particular sentence or paragraph—but not of a whole story.

Risk 355

Risk Testing Reporting Measurement 355

Domino Data Lab

MAY 20, 2021

We present the inner workings of the SMOTE algorithm and show a simple “from scratch” implementation of SMOTE. Their tests are performed using C4.5-generated 2002) do not present a rigorous mathematical treatment for this modification, and the suggested median correction appears to be purely empirical-driven.

Machine Learning 136

Machine Learning Metrics Data mining Knowledge Discovery 136

Domino Data Lab

AUGUST 8, 2019

Secondly: some key insights discussed at Sci Foo finally clicked for me—after I’d heard them presented a few times elsewhere. Consider the following timeline: 2001 – Physics grad students are getting hired in quantity by hedge funds to work on Wall St. Across the board, organizations struggle with hiring enough data scientists.

Data Science 71

Data Science Machine Learning Data Governance Statistics 71