Octopai

MARCH 31, 2021

Look for the Metadata. In order to perform accurate data lineage mapping, every process in the system that transforms or touches the data must be recorded. This metadata (read: data about your data) is key to tracking your data. Data Lineage by Tagging or Self-Contained Data Lineage.

Metadata 130

Metadata Data Transformation Business Intelligence Reporting 130

AWS Big Data

MARCH 13, 2024

You can also use the data transformation feature of Data Firehose to invoke a Lambda function to perform data transformation in batches. Athena is used to run geospatial queries on the location data stored in the S3 buckets. The firehose table stores raw, unmodified data from the Amazon Location tracker.

Analytics 97

Analytics IoT Metadata Internet of Things 97

Cloudera

DECEMBER 9, 2022

In 2021 we launched Cloudera DataFlow for the Public Cloud (CDF-PC) , addressing operational challenges that administrators face when running NiFi flows in production environments. Developers need to onboard new data sources, chain multiple data transformation steps together, and explore data as it travels through the flow.

Testing 95

Testing Cost-Benefit Interactive Visualization 95

AWS Big Data

APRIL 25, 2024

Alternatively, you can use AWS Glue for Apache Spark, which provides built-in support for bucketing configurations during the data transformation process. AWS Glue allows you to define bucketing parameters, such as the number of buckets and the columns to bucket on, providing an optimized data layout for efficient querying with Athena.

Optimization 66

Optimization Data Lake Cost-Benefit Reporting 66

Octopai

JANUARY 19, 2021

Octopai is the first BI Intelligence Platform in the Industry to Support Azure Data Factory, Providing Full Lineage of Advanced BI Tools. With Octopai’s support and analysis of Azure Data Factory, enterprises can now view complete end-to-end data lineage from Azure Data Factory all the way through to reporting for the first time ever.

Metadata 52

Metadata ROI Optimization Data Quality 52

AWS Big Data

DECEMBER 21, 2023

To ingest the data, smava uses a set of popular third-party customer data platforms complemented by custom scripts. After the data lands in Amazon S3, smava uses the AWS Glue Data Catalog and crawlers to automatically catalog the available data, capture the metadata, and provide an interface that allows querying all data assets.

Data Lake 92

Data Lake Data Warehouse Data-driven B2B 92

Jet Global

MAY 1, 2023

This is in contrast to traditional BI, which extracts insight from data outside of the app. We rely on increasingly mobile technology to comb through massive amounts of data and solve high-value problems. Plus, there is an expectation that tools be visually appealing to boot. Their dashboards were visually stunning.

Analytics 52

Analytics Cost-Benefit Visualization Dashboards 52