O'Reilly on Data

JANUARY 29, 2019

In other words, could we see a roadmap for transitioning from legacy cases (perhaps some business intelligence) toward data science practices, and from there into the tooling required for more substantial AI adoption? Data scientists and data engineers are in demand.

Deep Learning 208

Deep Learning Machine Learning Data Science Metadata 208

TDAN

SEPTEMBER 3, 2019

As I write this, I can almost hear you wail “No, no, we don’t have too much metadata, we don’t have nearly enough! We have several projects in flight to expand our use of metadata.” Sorry, I’m going to have to disagree with you there. You are on a fool’s errand that will just provide […].

Metadata 65

Metadata Data Governance Big Data Modeling 65

DataKitchen

JANUARY 5, 2021

Execution of this mission requires the contribution of several groups: data center/IT, data engineering, data science, data visualization, and data governance. Each of the roles mentioned above views the world through a preferred set of tools: Data Center/IT – Servers, storage, software.

Testing 147

Testing Data Governance Metadata Visualization 147

Timo Elliott

JANUARY 6, 2022

If storage costs are escalating in a particular area, you may have found a good source of dark data. Analyze your metadata. If you’ve been properly managing your metadata as part of a broader data governance policy, you can use metadata management explorers to reveal silos of dark data in your landscape.

IT 133

IT Metadata Data-driven Data Governance 133

Alation

AUGUST 30, 2022

We continue to make deep investments in governance, including new capabilities in the Stewardship Workbench, a core part of the Data Governance App. Centralization of metadata. A decade ago, metadata was everywhere. Consequently, useful metadata was unfindable and unusable. Then Alation came along.

Metadata 52

Metadata Data Quality Data Governance Machine Learning 52

Alation

DECEMBER 15, 2022

June 2017: Dresner Advisory Services names Alation the #1 data catalog in its inaugural Data Catalog End-User Market Study. August 2017: Alation debuts as a leader in the Gartner MQ for Metadata Management Solutions. August 2018: Gartner names Alation a 2X Leader in the MQ for Metadata Management Solutions.

Metadata 52

Metadata Data Governance Data Quality Marketing 52

O'Reilly on Data

JANUARY 8, 2019

Beyond investments in narrowing the skills gap, companies are beginning to put processes in place for their data science projects, for example creating analytics centers of excellence that centralize capabilities and share best practices. Automation in data science and data. Burgeoning IoT technologies.

IoT 204

IoT Machine Learning Internet of Things Data Science 204

Alation

FEBRUARY 20, 2020

As data drives more and more of the modern economy, data governance and data management are racing to keep up with an ever-expanding range of requirements, constraints and opportunities. Prior to the Big Data revolution, companies were inward-looking in terms of data. THE NEED FOR METADATA TOOLS.

IT 52

IT Metadata Data Governance Data Warehouse 52

AWS Big Data

MARCH 7, 2023

Analytics reference architecture for gaming organizations In this section, we discuss how gaming organizations can use a data hub architecture to address the analytical needs of an enterprise, which requires the same data at multiple levels of granularity and different formats, and is standardized for faster consumption.

Analytics 68

Analytics Data Warehouse Data Lake Metadata 68

Andrew White

DECEMBER 9, 2019

The outline of the call went as follows: I was taking to a central state agency who was organizing a data governance initiative (in their words) across three other state agencies. All four agencies had reported an independent but identical experience with data governance in the past. An expensive consulting engagement.

Analytics 66

Analytics Data Lake Data Governance Metadata 66

Alation

FEBRUARY 13, 2020

Making datasets easy to find, understand, and access is the purpose of data curation—a purpose that demands well-described datasets. Data curation is a metadata management activity and data catalogs are essential data curation technology. Who Are the Data Curators? What about Data Stewards?

Metadata 52

Metadata Data Warehouse Data Lake Data Governance 52

Domino Data Lab

APRIL 3, 2019

Paco Nathan ‘s latest column dives into data governance. This month’s article features updates from one of the early data conferences of the year, Strata Data Conference – which was held just last week in San Francisco. In particular, here’s my Strata SF talk “Overview of Data Governance” presented in article form.

Data Governance 84

Data Governance Machine Learning Metadata Big Data 84

IBM Big Data Hub

JANUARY 5, 2023

The right data architecture can help your organization improve data quality because it provides the framework that determines how data is collected, transported, stored, secured, used and shared for business intelligence and data science use cases. Perform data quality monitoring based on pre-configured rules.

Data Quality 52

Data Quality Data Architecture Strategy Data Lake 52

Cloudera

FEBRUARY 16, 2022

In the case of CDP Public Cloud, this includes virtual networking constructs and the data lake as provided by a combination of a Cloudera Shared Data Experience (SDX) and the underlying cloud storage. Each project consists of a declarative series of steps or operations that define the data science workflow.

Machine Learning 81

Machine Learning Modeling Metadata Recreation/Entertainment 81

Alation

FEBRUARY 20, 2020

These recommendations organize the data about your data — what the industry for many years has called metadata. And yet instead of storing this metadata in a vault accessible only by IT experts, Alation makes metadata come alive as a living data recommendation engine for the business.

Metadata 52

Metadata Analytics Dashboards Sales 52

Sanjeev Mohan

FEBRUARY 6, 2019

Then in the middle of 2017, a realization set in that we were one year away from GDPR and needed to focus on data governance. I ended up writing two documents on data governance. As you can tell, data governance is a hot topic but an area that many public cloud vendors are weak in.

Data Analytics 49

Data Analytics Analytics Data Governance Experimentation 49

Jen Stirrup

OCTOBER 5, 2019

I’m particularly interested in learning more about metadata management. As I’ve written previously, I believe that metadata management is often overlooked in data science. It’s an important part of obtaining full value from data, and it should be part of the data maturity model.

Metadata 44

Metadata Data Science Risk Modeling 44

AWS Big Data

APRIL 19, 2023

The solution generates a list of data products, product attributes, and the associated probability scores to show join ability. We use Valentine, a data science algorithm for comparing datasets, to improve data product recommendations. The data science algorithm Valentine is an effective tool for this.

Technology 78

Technology Data-driven Machine Learning Sales 78

Domino Data Lab

OCTOBER 1, 2020

This data supports all kinds of use cases within organizations, from helping production analysts understand how production is progressing, to allowing research scientists to look at the results of a set of treatments across different trials and cross-sections of the population.

Enterprise 105

Enterprise Metadata Cost-Benefit Data Processing 105

Cloudera

MAY 12, 2021

Use cases could include but are not limited to: workload analysis and replication, migrating or bursting to cloud, data warehouse optimization, and more. SECURITY AND GOVERNANCE LEADERSHIP. Data for Enterprise AI: E xperian BIS — Improving the accuracy of commercial data aggregation with data science and machine learning.

Digital Transformation 65

Digital Transformation Machine Learning Optimization Data Lake 65

IBM Big Data Hub

MAY 9, 2022

As for data delivery at the “right time” automated data engineering tasks, workload balancing and elastic scaling should provide the needed alacrity for all businesses. Data governance and privacy. Automated metadata generation is essential in order to turn a manual process into one that is better controlled.

Management 57

Management Data Quality Metadata Data Integration 57

Andrew White

JULY 5, 2023

.” This is not much of catalog at all, and is closer to being a D&A governance solution and stewardship solution. When a business analyst or data science leader is building a model, they will often start out by searching a catalog for data. They WOULD have some opportunity to govern the glossary!

Metrics 52

Metrics Metadata Modeling Data Quality 52

O'Reilly on Data

FEBRUARY 11, 2020

They don’t have the resources they need to clean up data quality problems. The building blocks of data governance are often lacking within organizations. These include the basics, such as metadata creation and management, data provenance, data lineage, and other essentials. And that’s just the beginning.

Data Quality 304

Data Quality Metadata Data Governance Publishing 304

Octopai

DECEMBER 12, 2021

What enables you to use all those gigabytes and terabytes of data you’ve collected? Metadata is the pertinent, practical details about data assets: what they are, what to use them for, what to use them with. Without metadata, data is just a heap of numbers and letters collecting dust. Where does metadata come from?

Metadata 93

Metadata Management Data Quality Data Governance 93

AWS Big Data

MARCH 10, 2023

However, as data processing at scale solutions grow, organizations need to build more and more features on top of their data lakes. Additionally, the task of maintaining and managing files in the data lake can be tedious and sometimes complex. The Iceberg table is synced with the AWS Glue Data Catalog.

Data Lake 98

Data Lake Sales Data Warehouse Snapshot 98

DataKitchen

FEBRUARY 2, 2023

A combined, interoperable suite of tools for data team productivity, governance, and security for large and small data teams. Ultimately, there will be an interoperable toolset for running the data team , just like a more focused toolset (ELT/Data Science/BI) for acting upon data.

Metadata 130

Metadata Testing Risk Data Science 130

Octopai

OCTOBER 5, 2021

Data catalog tools that utilize automation will routinely review all metadata within your BI landscape and update your data catalog accordingly. Check to what extent the data catalog software is automated – both for initial catalog creation and for ongoing updates.

Business Intelligence 84

Business Intelligence Software Cost-Benefit Metadata 84

TDAN

NOVEMBER 3, 2020

In the September issue of TDAN.com, Anthony Algmin denounced Data Catalogs as a “1980’s solution to a 2020’s problem.” What is the state of data science today? As I state in my book, The Data Catalog: Sherlock Holmes Sleuthing for Data Analytics and in many articles, 80% (or more) of a data analyst’s […].

Data Science 59

Data Science Data Analytics Analytics Data Quality 59

Peter James Thomas

AUGUST 29, 2018

I have since run and driven transformation in Reference Data, Master Data , KYC [3] , Customer Data, Data Warehousing and more recently Data Lakes and Analytics , constantly building experience and capability in the Data Governance , Quality and data services domains, both inside banks, as a consultant and as a vendor.

Data-driven 61

Data-driven Cost-Benefit Metadata Technology 61

O'Reilly on Data

JULY 28, 2020

There’s a substantial literature about ethics, data, and AI, so rather than repeat that discussion, we’ll leave you with a few resources. Ethics and Data Science is a short book that helps developers think through data problems, and includes a checklist that team members should revisit throughout the process.

Marketing 362

Marketing Experimentation Metrics Testing 362

IBM Big Data Hub

JANUARY 17, 2023

Essential components of a data lakehouse architecture and what makes an open data lakehouse. At the core of a data lakehouse architecture includes the storage, metadata service and the query engine, and typically a data governance component made up of a policy engine and a data dictionary.

Data Lake 52

Data Lake Metadata Data Warehouse Data Governance 52

IBM Big Data Hub

JANUARY 17, 2023

IBM Cloud Pak for Data Express solutions offer clients a simple on ramp to start realizing the business value of a modern architecture. Data governance. The data governance capability of a data fabric focuses on the collection, management and automation of an organization’s data. Start a trial.

Data-driven 57

Data-driven Enterprise Data Governance Prescriptive Analytics 57

Domino Data Lab

AUGUST 8, 2019

Paco Nathan ‘s latest monthly article covers Sci Foo as well as why data science leaders should rethink hiring and training priorities for their data science teams. In this episode I’ll cover themes from Sci Foo and important takeaways that data science teams should be tracking. Introduction.

Data Science 71

Data Science Machine Learning Data Governance Statistics 71

AWS Big Data

SEPTEMBER 25, 2023

With Amazon Redshift , you can run a complex mix of workloads on your data warehouse, such as frequent data loads running alongside business-critical dashboard queries and complex transformation jobs. We also see more and more data science and machine learning (ML) workloads. ID Service class 1–4 Reserved for system use.

Management 91

Management Data Warehouse Metrics Dashboards 91

Domino Data Lab

JULY 2, 2019

In other words, using metadata about data science work to generate code. In this case, code gets generated for data preparation, where so much of the “time and labor” in data science work is concentrated. BTW, videos for Rev2 are up: [link]. On deck this time ’round the Moon: program synthesis.

Metadata 105

Metadata Machine Learning Data Science Data-driven 105

Alation

APRIL 14, 2022

Semantics, context, and how data is tracked and used mean even more as you stretch to reach post-migration goals. This is why, when data moves, it’s imperative for organizations to prioritize data discovery. Data discovery is also critical for data governance , which, when ineffective, can actually hinder organizational growth.

Metadata 59

Metadata Data Governance Modeling Data-driven 59

IBM Big Data Hub

APRIL 27, 2022

Gartner defines a data fabric as “a design concept that serves as an integrated layer of data and connecting processes. The data fabric architectural approach can simplify data access in an organization and facilitate self-service data consumption at scale. 2 “Exposing The Data Mesh Blind Side ” Forrester.

Management 89

Management Metadata Data Architecture Data Lake 89