AWS Big Data

NOVEMBER 17, 2023

Starting today, the Athena SQL engine uses a cost-based optimizer (CBO), a new feature that uses table and column statistics stored in the AWS Glue Data Catalog as part of the table’s metadata. The following graph presents the top 10 queries from the TPC-DS benchmark with the greatest performance improvement.

Optimization 112

Optimization Statistics Metadata Data Lake 112

Domino Data Lab

APRIL 3, 2019

In particular, here’s my Strata SF talk “Overview of Data Governance” presented in article form. That’s a lot of priorities – especially when you group together closely related items such as data lineage and metadata management which rank nearby. The on-the-ground reality of DG presents an almost overwhelming array of topics.

Data Governance 84

Data Governance Machine Learning Metadata Big Data 84

Domino Data Lab

JULY 22, 2019

Paco Nathan presented, “Data Science, Past & Future” , at Rev. I am honored to be able to present here and thrilled to have been involved in Rev. The presentation layer was about, say, web browsers, right, what you could do in a web browser. You see these drivers involving risk and cost, but also opportunity.

Data Science 109

Data Science Machine Learning Data Governance Modeling 109

Domino Data Lab

AUGUST 8, 2019

Secondly: some key insights discussed at Sci Foo finally clicked for me—after I’d heard them presented a few times elsewhere. The gist is, leveraging metadata about research datasets, projects, publications, etc., The probabilistic nature changes the risks and process required.

Data Science 71

Data Science Machine Learning Data Governance Statistics 71