AWS Big Data

JULY 20, 2023

Prior to the creation of the data lake, Orca’s data was distributed among various data silos, each owned by a different team with its own data pipelines and technology stack. Moreover, running advanced analytics and ML on disparate data sources proved challenging.

Data Lake 85

Data Lake Analytics Snapshot Optimization 85

O'Reilly on Data

MARCH 31, 2020

But there’s a host of new challenges when it comes to managing AI projects: more unknowns, non-deterministic outcomes, new infrastructures, new processes and new tools. For machine learning systems used in consumer internet companies, models are often continuously retrained many times a day using billions of entirely new input-output pairs.

Management 332

Management Machine Learning Experimentation Metrics 332

Domino Data Lab

OCTOBER 1, 2020

Combining the power of Domino Data Labs with Okera, your data scientists only get access to the columns, rows, and cells allowed, easily removing or redacting sensitive data such as PII and PHI not relevant to training models. So what does this look like? client('s3') obj = s3.get_object(Bucket='clinical-trials',

Enterprise 105

Enterprise Metadata Cost-Benefit Data Processing 105

AWS Big Data

NOVEMBER 16, 2023

As with all AWS services, Amazon Redshift is a customer-obsessed service that recognizes there isn’t a one-size-fits-all for customers when it comes to data models, which is why Amazon Redshift supports multiple data models such as Star Schemas, Snowflake Schemas and Data Vault.

Enterprise 83

Enterprise Data Warehouse Snapshot Cost-Benefit 83

Jet Global

FEBRUARY 23, 2024

Data mapping is essential for integration, migration, and transformation of different data sets; it allows you to improve your data quality by preventing duplications and redundancies in your data fields. Source-to-target mapping integration tasks vary in complexity, depending on data hierarchy and structure.

Data Warehouse 52

Data Warehouse Reporting Data Transformation Sales 52

IBM Big Data Hub

JANUARY 9, 2024

Companies still often accept the risk of using internal data when exploring large language models (LLMs) because this contextual data is what enables LLMs to change from general-purpose to domain-specific knowledge. In the generative AI or traditional AI development cycle, data ingestion serves as the entry point.

Enterprise 81

Enterprise Data Integration Data Quality Contextual Data 81

Andrew White

JANUARY 9, 2022

On Thursday January 6th I hosted Gartner’s 2022 Leadership Vision for Data and Analytics webinar. Data mesh seems to mean a network of systems and hubs connected to help cope with silos. Perhaps a data mesh is what a pure architecture might design. Here is the link to the replay, in case you are interested.

Analytics 64

Analytics Measurement Modeling Data-driven 64

Andrew White

JANUARY 11, 2021

On January 4th I had the pleasure of hosting a webinar. It was titled, The Gartner 2021 Leadership Vision for Data & Analytics Leaders. This was for the Chief Data Officer, or head of data and analytics. Where does the Data Architect role fits in the Operational Model ? Try some gamification?

Data Analytics 52

Data Analytics Analytics Data-driven Finance 52

erwin

APRIL 16, 2024

In this blog, we’ll delve into the critical role of governance and data modeling tools in supporting a seamless data mesh implementation and explore how erwin tools can be used in that role. erwin also provides data governance, metadata management and data lineage software called erwin Data Intelligence by Quest.

Metadata 52

Metadata Data Quality Data Governance Modeling 52

Alation

FEBRUARY 17, 2023

This past week, I had the pleasure of hosting Data Governance for Dummies author Jonathan Reichental for a fireside chat , along with Denise Swanson , Data Governance lead at Alation. Our organization is struggling with the guardrails between the data governance and management concepts.

Data Governance 52

Data Governance Data Quality Metadata Cost-Benefit 52