Alation

SEPTEMBER 7, 2021

In this blog we will discuss how Alation helps minimize risk with active data governance. Governance influences how an organization’s objectives are set and achieved, how risk is monitored and addressed, and how performance is optimized. Organizations that run afoul of such laws risk damaging their reputation.

Data Governance 59

Data Governance Risk Data Quality Dashboards 59

O'Reilly on Data

JULY 28, 2020

These measures are commonly referred to as guardrail metrics , and they ensure that the product analytics aren’t giving decision-makers the wrong signal about what’s actually important to the business. When a measure becomes a target, it ceases to be a good measure ( Goodhart’s Law ). Any metric can and will be abused.

Marketing 362

Marketing Experimentation Metrics Testing 362

Alation

MARCH 14, 2023

All critical data elements (CDEs) should be collated and inventoried with relevant metadata, then classified into relevant categories and curated as we further define below. Store Where individual departments have their own databases for metadata management, data will be siloed, meaning it can’t be shared and used business-wide.

Data Strategy 52

Data Strategy Strategy Metadata Data Quality 52

Octopai

JULY 9, 2020

By adopting automated data lineage and automated metadata tagging, companies have the opportunity to increase their data processing speed. It required banks to develop a data architecture that could support risk-management tools. A key piece of legislation that emerged from that crisis was BCBS-239.

Insurance 86

Insurance Risk Management Machine Learning Metadata 86

erwin

APRIL 15, 2020

This deeper understanding will help us make better decisions about the risks and rewards of using that external data. Ingest to a quarantined area where it can be profiled and measured for quality, completeness, and correctness, and where necessary, cleansed. Data Governance Methods for Data Distancing.

Data Governance 134

Data Governance Metadata Testing Reporting 134

Octopai

DECEMBER 11, 2022

Banks didn’t accurately assess their credit and operational risk and hold enough capital reserves, leading to the Great Recession of 2008-2009. Data lineage and financial risk data compliance. All these models need to be informed by data, with operational risk assessment mandating loss data that goes back 10 years. .

Insurance 52

Insurance Risk Metadata Reporting 52

DataRobot Blog

SEPTEMBER 1, 2022

Imagine yourself as a pilot operating aircraft through a thunderstorm; you have all the dashboards and automated systems that inform you about any risks. But many of the things you need to measure for your business are hyperspecific for your unique problems and opportunities — specific business KPIs or data science secrets.

Metrics 145

Metrics Statistics Modeling Data Science 145

Cloudera

OCTOBER 15, 2021

With FSO, Apache Ozone guarantees atomic directory operations, and renaming or deleting a directory is a simple metadata operation even if the directory has a large set of sub-paths (directories/files) within it. Please refer to Apache Ozone documentation for more details regarding Apache Ozone’s atomicity guarantees. Further Reading.

Testing 87

Testing Measurement Optimization Metadata 87

Alation

OCTOBER 5, 2021

What is data governance and how do you measure success? These goals must also be SMART— Specific, Measurable, Attainable, Relevant, and Time-based. To measure failure vs. success, governance leaders must actively monitor all governance efforts. Data governance is a system for answering core questions about data.

Data Governance 96

Data Governance Strategy Data Quality Metrics 96

erwin

DECEMBER 20, 2018

Whether implemented as preventative measures (risk management and regulation) or proactive endeavors (value creation and ROI), the benefits of a data governance initiative is becoming more apparent. Its value can be hard to demonstrate to those who don’t work directly with data and metadata on a daily basis.

Data Governance 40

Data Governance ROI Risk Management Metadata 40

Domino Data Lab

APRIL 3, 2019

That’s a lot of priorities – especially when you group together closely related items such as data lineage and metadata management which rank nearby. Also, while surveying the literature two key drivers stood out: Risk management is the thin-edge-of-the-wedge ?for Allows metadata repositories to share and exchange.

Data Governance 84

Data Governance Machine Learning Metadata Big Data 84

Laminar Security

SEPTEMBER 18, 2023

Do you know the data risks of your regular backup practices? I’ve previously discussed the risk of shadow data (those data stores you didn’t even know you had), and the team specifically dove into how versioning (a feature used to protect from accidental deletion) can unintentionally create shadow data. How do you combat the risk?

Risk 128

Risk Testing Data Governance Metadata 128

IBM Big Data Hub

APRIL 11, 2024

Compliance with official policies through auditing tools and other measures is merely the final step. In An Overview of Catastrophic AI Risks , the authors identify several mitigations that can be addressed through governance and regulation (in addition to cybersecurity). ” Academic and scientific perspectives are also essential.

Risk 78

Risk Consulting Modeling Data Processing 78

IBM Big Data Hub

FEBRUARY 22, 2024

Defense leaders focused on operationalizing the responsible curation of AI must first agree upon a shared vocabulary—a common culture that guides safe, responsible use of AI—before they implement technological solutions and guardrails that mitigate risk. These are table stakes for the DoD or any government agency.

Metadata 102

Metadata Measurement Risk Modeling 102

AWS Big Data

JULY 28, 2023

It requires careful analysis to identify data dependencies and mitigate any potential risks or disruptions. Data management and governance Addressing the challenges mentioned requires a combination of technical, operational, and legal measures. Tags provide metadata about resources at a glance.

Snapshot 83

Snapshot Metadata Measurement Data Warehouse 83

Alation

JANUARY 19, 2023

Or the data potentially puts them at risk every day. Documentation has always been an afterthought. Improving data intelligence through the automation, distribution, stewardship, and effective use of business and technical processes and metadata will certainly alleviate many of the pain points associated with governing data.

Data Governance 94

Data Governance IT Metadata Recreation/Entertainment 94

Ontotext

MAY 22, 2023

At the heart of such tools is the extraction of fields from forms or specific attributes from documents. Luckily, the text analysis that Ontotext does is focused on tasks that require complex domain knowledge and linking of documents to reference data or master data. LLMs do most of this better and with lower cost of customization.

Management 59

Management Unstructured Data Metadata Cost-Benefit 59

erwin

APRIL 21, 2020

By combining physical system catalogs, critical data elements, and key performance measures with clearly defined product and sales goals, you can manage the effectiveness of your business and ensure you understand what critical systems are for business continuity and measuring corporate performance.

Measurement 97

Measurement Sales Data Governance Metadata 97

erwin

APRIL 21, 2020

As COVID-19 continues to spread, organizations are evaluating and adjusting their operations in terms of both risk management and business continuity. One example is the European Union (EU) Open Data Portal , which is used to document, catalog and govern EU data related to the pandemic.

Measurement 96

Measurement Sales Data Governance Metadata 96

IBM Big Data Hub

JANUARY 9, 2024

Companies still often accept the risk of using internal data when exploring large language models (LLMs) because this contextual data is what enables LLMs to change from general-purpose to domain-specific knowledge. Increased variance: Variance measures consistency.

Enterprise 80

Enterprise Data Integration Data Quality Contextual Data 80

O'Reilly on Data

MARCH 20, 2019

Like many others, I’ve known for some time that machine learning models themselves could pose security risks. An attacker could use an adversarial example attack to grant themselves a large loan or a low insurance premium or to avoid denial of parole based on a high criminal risk score. Newer types of fair and private models (e.g.,

Modeling 221

Modeling Machine Learning Predictive Modeling Consulting 221

erwin

FEBRUARY 4, 2021

While acknowledging that data governance is about more than risk management and regulatory compliance may indicate that companies are more confident in their data, the data governance practice is nonetheless growing in complexity because of more: Data to handle, much of it unstructured.

Data Governance 89

Data Governance Metadata Data Quality Risk Management 89

Ontotext

FEBRUARY 26, 2024

The answers to these foundational questions help you uncover opportunities and detect risks. We bundle these events under the collective term “Risk and Opportunity Events” This post is part of Ontotext’s AI-in-Action initiative aimed to empower data, scientists, architects and engineers to leverage LLMs and other AI models.

Data-driven 59

Data-driven Risk Modeling Risk Management 59

Alation

APRIL 14, 2022

In today’s AI/ML-driven world of data analytics, explainability needs a repository just as much as those doing the explaining need access to metadata, EG, information about the data being used. These factors risk data originating in far-flung environments, where the data structures and semantics are not well understood or documented.

Metadata 59

Metadata Data Governance Modeling Data-driven 59

IBM Big Data Hub

OCTOBER 12, 2023

Among the tasks necessary for internal and external compliance is the ability to report on the metadata of an AI model. Metadata includes details specific to an AI model such as: The AI model’s creation (when it was created, who created it, etc.) This allows for a high degree of transparency and auditability.

Cost-Benefit 83

Cost-Benefit Metadata Data Governance Modeling 83

CIO Business Intelligence

APRIL 7, 2022

KPMG calculates a score for an employee’s risk of attrition, tries to identify a reason for that, and then suggests a remediation. But the AI tool can help measure employee moods. Prior to the pandemic, new hires would read documentation, watch videos, and then sit in a classroom. Previously, Genpact used surveys, he says.

Consulting 145

Consulting Management Interactive Metadata 145

O'Reilly on Data

MARCH 23, 2023

What is it, how does it work, what can it do, and what are the risks of using it? But Transformers have some other important advantages: Transformers don’t require training data to be labeled; that is, you don’t need metadata that specifies what each sentence in the training data means. What Are the Risks?

IT 260

IT Modeling Testing Risk 260

AWS Big Data

JANUARY 22, 2024

Data as a product Treating data as a product entails three key components: the data itself, the metadata, and the associated code and infrastructure. For orchestration, they use the AWS Cloud Development Kit (AWS CDK) for infrastructure as code (IaC) and AWS Glue Data Catalogs for metadata management.

Data-driven 85

Data-driven Advertising Metadata Data Architecture 85

Ontotext

SEPTEMBER 2, 2020

The risk is that the organization creates a valuable asset with years of expertise and experience that is directly relevant to the organization and that valuable asset can one day cross the street to your competitors. Institutional memory loss creates significant and measurable inefficiencies. The Cost of Memory Loss.

Insurance 72

Insurance Metadata Publishing Unstructured Data 72

Andrew White

JANUARY 9, 2022

First, how we measure emissions and carbon footprint is about data design and policy. In other words, D&A plays a key role in the foundational measuring angle. I would have to admit that there are few documents that talk about all the connections across any set of topics. – Yes and no.

Analytics 64

Analytics Measurement Modeling Data-driven 64

Domino Data Lab

AUGUST 8, 2019

The second rule of Foo Camp concerns a practice called “FrieNDA” where people speak candidly and others respect that without either (1) “Oh gosh I must live tweet stream this amazing insight” or (2) anybody having to sign legal documents. The gist is, leveraging metadata about research datasets, projects, publications, etc.,

Data Science 71

Data Science Machine Learning Data Governance Statistics 71

Grooper

OCTOBER 23, 2019

By measuring performance, they drive improvements through necessary process and workflow changes. It simply takes the right resources and a measured approach to get there. For example, business growth, profits, cost and performance optimization, efficiency, compliance, and risk management. And they simply execute better.

Metrics 59

Metrics Measurement Strategy Optimization 59

Anmut

NOVEMBER 3, 2020

Daniel Moody’s classic paper Measuring the Value of Information draws our attention to one of the most important properties of data, referred to by economists as being “anti-rivalrous”. Costs arise from implementing compliance, while risks arise from the possibility of fines and reputational damage if regulations are broken.

Management 52

Management ROI Metadata Data Quality 52

Alation

DECEMBER 15, 2021

This uncovers actionable intelligence, maintains compliance with regulations, and mitigates risks. This includes: Assigning responsibility for implementing the policies and processes, Defining policies for sharing and processing data, Creating processes for naming and storing data, Establishing measurements for keeping data clean and usable.

Data Governance 52

Data Governance Strategy Metadata Data Strategy 52

Sisense

SEPTEMBER 3, 2019

When it comes to picking your node type, while it is easy to see the storage of the Dc2.8xlarge and chose it because it is the largest, if you don’t have the need for that node type, you risk losing out on the benefits of having a distributed database that can process your query in parallel across all of your nodes. Sort & Dist Keys.

Data Warehouse 78

Data Warehouse OLAP Statistics Cost-Benefit 78

AWS Big Data

AUGUST 14, 2023

At the same time, unstructured approaches to data mesh management that don’t have a vision for what types of products should exist and how to ensure they are developed are at high risk of creating the same effect through simple neglect. How do we define “risk” and “value” in the context of data products, and how can we measure this?

Management 79

Management Risk Measurement Optimization 79

Alation

NOVEMBER 22, 2022

Data breach mitigation measures. A well-governed data landscape enables data users in the public sector to better understand the driving forces and needs to support public policy – and measure impact once a change is made. Citizens, companies, and government employees need access to data and documents. Privacy concerns.

Data Governance 52

Data Governance Metadata Data-driven Unstructured Data 52