article thumbnail

Speed up queries with the cost-based optimizer in Amazon Athena

AWS Big Data

Amazon Athena is a serverless, interactive analytics service built on open source frameworks, supporting open table file formats. Starting today, the Athena SQL engine uses a cost-based optimizer (CBO), a new feature that uses table and column statistics stored in the AWS Glue Data Catalog as part of the table’s metadata.

article thumbnail

Themes and Conferences per Pacoid, Episode 8

Domino Data Lab

That’s a lot of priorities – especially when you group together closely related items such as data lineage and metadata management which rank nearby. My read of that narrative arc is that some truly weird tensions showed up circa 2001: Arguably, it’s the heyday of DW+BI. Allows metadata repositories to share and exchange.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Modernize a legacy real-time analytics application with Amazon Managed Service for Apache Flink

AWS Big Data

We introduce you to Amazon Managed Service for Apache Flink Studio and get started querying streaming data interactively using Amazon Kinesis Data Streams. The second streaming data source constitutes metadata information about the call center organization and agents that gets refreshed throughout the day.

article thumbnail

Data Science, Past & Future

Domino Data Lab

By virtue of that, if you take those log files of customers interactions, you aggregate them, then you take that aggregated data, run machine learning models on them, you can produce data products that you feed back into your web apps, and then you get this kind of effect in business. I can point to the year 2001. All righty.

article thumbnail

Generate security insights from Amazon Security Lake data using Amazon OpenSearch Ingestion

AWS Big Data

An example is provided below ocsf-cuid-${/class_uid}-${/metadata/product/name}-${/class_name}-%{yyyy.MM.dd} Complete the following steps to install the index templates and dashboards for your data: Download the component_templates.zip and index_templates.zip files and unzip them on your local device. Set region as us-east-1.

article thumbnail

Themes and Conferences per Pacoid, Episode 12

Domino Data Lab

The gist is, leveraging metadata about research datasets, projects, publications, etc., Consider the following timeline: 2001 – Physics grad students are getting hired in quantity by hedge funds to work on Wall St. then building machine learning models to recommend methods and potential collaborators to scientists.