AWS Big Data

APRIL 2, 2024

In March 2024, we announced the general availability of the generative artificial intelligence (AI) generated data descriptions in Amazon DataZone. In this post, we share what we heard from our customers that led us to add the AI-generated data descriptions and discuss specific customer use cases addressed by this capability.

Metadata 91

Metadata Metrics Data-driven Modeling 91

erwin

APRIL 16, 2020

We’re excited about our recognition as a March 2020 Gartner Peer Insights Customers’ Choice for Metadata Management Solutions. The solutions work in tandem to automate the processes involved in harvesting, integrating, activating and governing enterprise data according to business requirements.

Metadata 52

Metadata Management Data Governance Digital Transformation 52

DataKitchen

OCTOBER 20, 2023

In today’s data-driven landscape, Data and Analytics Teams i ncreasingly face a unique set of challenges presented by Demanding Data Consumers who require a personalized level of Data Observability. Data Observability platforms often need to deliver this level of customization.

Insurance 169

Insurance Metadata Data-driven Data Quality 169

AWS Big Data

OCTOBER 11, 2023

Organizations with legacy, on-premises, near-real-time analytics solutions typically rely on self-managed relational databases as their data store for analytics workloads. Near-real-time streaming analytics captures the value of operational data and metrics to provide new insights to create business opportunities.

Management 107

Management Metadata Analytics Dashboards 107

AWS Big Data

NOVEMBER 17, 2023

DynamoDB offers built-in security, continuous backups, automated multi-Region replication, in-memory caching, and data import and export tools. The scalability and flexible data schema of DynamoDB make it well-suited for a variety of use cases. Data stored in DynamoDB is the basis for valuable business intelligence (BI) insights.

Visualization 85

Visualization Metadata Testing Internet of Things 85

DataKitchen

MAY 4, 2023

Webinar Summary: DataOps and Data Mesh Chris Bergh, CEO of DataKitchen, delivered a webinar on two themes – Data Products and Data Mesh. Bergh started by discussing the complexity within data and analytics teams, stating that complexity makes everything more complicated and, in the long run, it kills productivity.

Measurement 130

Measurement Data-driven Testing Cost-Benefit 130

AWS Big Data

DECEMBER 13, 2023

In addition to using native managed AWS services that BMS didn’t need to worry about upgrading, BMS was looking to offer an ETL service to non-technical business users that could visually compose data transformation workflows and seamlessly run them on the AWS Glue Apache Spark-based serverless data integration engine.

Metadata 91

Metadata Data Lake Visualization Data Transformation 91

erwin

OCTOBER 8, 2020

So if you’re going to move from your data from on-premise legacy data stores and warehouse systems to the cloud, you should do it right the first time. And as you make this transition, you need to understand what data you have, know where it is located, and govern it along the way. Then you must bulk load the legacy data.

Data Governance 145

Data Governance Metadata Testing Data Lake 145

AWS Big Data

FEBRUARY 12, 2024

Public health organizations need access to data insights that they can quickly act upon, especially in times of health emergencies, when data needs to be updated multiple times daily. Instead, they rely on up-to-date dashboards that help them visualize data insights to make informed decisions quickly.

Publishing 93

Publishing Dashboards Metadata Visualization 93

Ontotext

MAY 2, 2024

Data leaders today are faced with an almost impossible challenge. They are expected to understand the entire data landscape and generate business-moving insights while facing the voracious needs of different teams and the constraints of technology architecture and compliance.

Metadata 52

Metadata Data Lake Data-driven Enterprise 52

erwin

JANUARY 9, 2020

Enterprises are trying to manage data chaos. They also face increasing regulatory pressure because of global data regulations , such as the European Union’s General Data Protection Regulation (GDPR) and the new California Consumer Privacy Act (CCPA), that went into effect last week on Jan. GDPR: Key Differences.

Data Governance 144

Data Governance Modeling Metadata Unstructured Data 144

AWS Big Data

JUNE 5, 2023

If you also needed to preserve the history of DAG runs, you had to take a backup of your metadata database and then restore that backup on the newly created environment. Amazon MWAA manages the entire upgrade process, from provisioning new Apache Airflow versions to upgrading the metadata database. or v2.0.2, and higher environment.

Snapshot 79

Snapshot Metadata Testing Data-driven 79

Rocket-Powered Data Science

SEPTEMBER 20, 2021

The way that I explained it to my data science students years ago was like this. They realized that the search results would probably not provide an answer to my question, but the results would simply list websites that included my words on the page or in the metadata tags: “Texas”, “Cows”, “How”, etc. What is a semantic layer?

Data Science 286

Data Science Forecasting Business Intelligence Sales 286

DataKitchen

AUGUST 3, 2021

The data mesh design pattern breaks giant, monolithic enterprise data architectures into subsystems or domains, each managed by a dedicated team. DataOps helps the data mesh deliver greater business agility by enabling decentralized domains to work in concert. . But first, let’s define the data mesh design pattern.

Data Architecture 246

Data Architecture Data Lake Cost-Benefit Data Warehouse 246

Alation

SEPTEMBER 21, 2021

When we work with data, we know where we are starting from, often with a destination in mind, but the path we take is not always the fastest because we may not have all the necessary intelligence to optimize our route. Not only is the data distributed, diverse, and dynamic, but so are the people working with data.

Metadata 52

Metadata Testing Recreation/Entertainment Data-driven 52

AWS Big Data

DECEMBER 18, 2023

AWS Step Functions is a fully managed visual workflow service that enables you to build complex data processing pipelines involving a diverse set of extract, transform, and load (ETL) technologies such as AWS Glue , Amazon EMR , and Amazon Redshift. The data is then cleansed, transformed, and uploaded to Amazon S3 for further processing.

Metadata 123

Metadata Visualization Data Lake Data-driven 123

Octopai

OCTOBER 26, 2022

Data-driven organizations are a bad idea. Using data to drive your organization is wonderful. But data, at best, can only be a powerful vehicle, or a reliable GPS system. Except sometimes we call organizations “data-driven” when really the data is driving them up the wall. And it should be.

Data Quality 52

Data Quality Data Analytics Analytics Manufacturing 52

IBM Big Data Hub

MARCH 25, 2022

To simplify data access and empower users to leverage trusted information, organizations need a better approach that provides better insights and business outcomes faster, without sacrificing data access controls. There are many different approaches, but you’ll want an architecture that can be used regardless of your data estate.

Metadata 107

Metadata Data Quality Data Governance Data Integration 107

erwin

NOVEMBER 6, 2020

Prashant Parikh, erwin’s Senior Vice President of Software Engineering, talks about erwin’s vision to automate every aspect of the data governance journey to increase speed to insights. The clear benefit is that data stewards spend less time building and populating the data governance framework and more time realizing value and ROI from it.

Data Governance 97

Data Governance Metadata Data-driven Visualization 97

AWS Big Data

NOVEMBER 15, 2023

Stream data processing allows you to act on data in real time. Real-time data analytics can help you have on-time and optimized responses while improving the overall customer experience. Data streaming workloads often require data in the stream to be enriched via external sources (such as databases or other data streams).

Testing 92

Testing Optimization Management Metadata 92

O'Reilly on Data

DECEMBER 19, 2018

We need to do more than automate model building with autoML; we need to automate tasks at every stage of the data pipeline. In a previous post , we talked about applications of machine learning (ML) to software development, which included a tour through sample tools in data science and for managing data infrastructure.

Machine Learning 258

Machine Learning Software Metadata Testing 258

AWS Big Data

DECEMBER 21, 2023

This is a guest post co-written by Alex Naumov, Principal Data Architect at smava. smava believes in and takes advantage of data-driven decisions in order to become the market leader. smava believes in and takes advantage of data-driven decisions in order to become the market leader.

Data Lake 91

Data Lake Data Warehouse Data-driven B2B 91

CIO Business Intelligence

NOVEMBER 4, 2022

Rigid requirements to ensure the accuracy of data and veracity of scientific formulas as well as machine learning algorithms and data tools are common in modern laboratories. When Bob McCowan was promoted to CIO at Regeneron Pharmaceuticals in 2018, he had previously run the data center infrastructure for the $81.5

Data Lake 109

Data Lake IT Experimentation Data-driven 109

erwin

JANUARY 24, 2019

Organizations are responsible for governing more data than ever before, making a strong automation framework a necessity. In most companies, an incredible amount of data flows from multiple sources in a variety of formats and is constantly being moved and federated across a changing system landscape. Governing metadata.

Data Governance 71

Data Governance Metadata Data-driven Cost-Benefit 71

datapine

JANUARY 31, 2022

Because things are changing and becoming more competitive in every sector of business, the benefits of business intelligence and proper use of data analytics are key to outperforming the competition. Everything is being tested, and then the campaigns that succeed get more money put into them, while the others aren’t repeated.

Business Intelligence 214

Business Intelligence Analytics Cost-Benefit ROI 214

DataKitchen

JANUARY 23, 2023

What is data lineage? Data lineage traces data’s origin, history, and movement through various processing, storage, and analysis stages. It is used to understand the provenance of data and how it is transformed and to identify potential errors or issues. What about DataOps Observability? How does it compare?

Testing 130

Testing Data Governance Data Quality Data-driven 130

Cloudera

MARCH 14, 2023

We just announced the general availability of Cloudera DataFlow Designer , bringing self-service data flow development to all CDP Public Cloud customers. In this blog post we will put these capabilities in context and dive deeper into how the built-in, end-to-end data flow life cycle enables self-service data pipeline development.

Testing 83

Testing Publishing Metadata Interactive 83

Alation

AUGUST 26, 2021

What Is Data Intelligence? Data intelligence is a system to deliver trustworthy, reliable data. It includes intelligence about data, or metadata. IDC coined the term, stating, “data intelligence helps organizations answer six fundamental questions about data.” Why do we have data?

Metadata 64

Metadata Data Governance Dashboards Software 64

AWS Big Data

NOVEMBER 15, 2023

It seamlessly consolidates data from various data sources within AWS, including AWS Cost Explorer (and forecasting with Cost Explorer ), AWS Trusted Advisor , and AWS Compute Optimizer. Overview of the BMW Cloud Data Hub At the BMW Group, Cloud Data Hub (CDH) is the central platform for managing company-wide data and data solutions.

Dashboards 97

Dashboards Analytics Metadata Data Warehouse 97

DataRobot Blog

MARCH 16, 2023

To deliver on this new approach, one that we are calling Value-Driven AI , we set out to design new and enhanced platform capabilities that enable customers to realize value faster. Best-Practice Compliance and Governance: Businesses need to know that their Data Scientists are delivering models that they can trust and defend over time.

Experimentation 98

Experimentation Data-driven Modeling Metadata 98

AWS Big Data

MAY 16, 2024

Apache Airflow is a popular platform for enterprises looking to orchestrate complex data pipelines and workflows. In this post, we’re excited to introduce two new features that address common customer challenges and unlock new possibilities for building robust, scalable, and flexible data orchestration solutions using Amazon MWAA.

Testing 91

Testing Interactive Metrics Management 91

AWS Big Data

SEPTEMBER 13, 2023

The Analytics specialty practice of AWS Professional Services (AWS ProServe) helps customers across the globe with modern data architecture implementations on the AWS Cloud. In this post, we discuss a common use case in relation to operational data processing and the solution we built using Apache Hudi and AWS Glue.

Data Lake 93

Data Lake Data Processing Metadata Snapshot 93

Cloudera

APRIL 22, 2021

Today’s enterprise data analytics teams are constantly looking to get the best out of their platforms. Storage plays one of the most important roles in the data platforms strategy, it provides the basis for all compute engines and applications to be built on top of it. Separates control and data plane enabling high performance.

Data Lake 100

Data Lake Cost-Benefit Testing Metadata 100

Cloudera

SEPTEMBER 28, 2020

Cloudera delivers an enterprise data cloud that enables companies to build end-to-end data pipelines for hybrid cloud, spanning edge devices to public or private cloud, with integrated security and governance underpinning it to protect customers data. Lineage and chain of custody, advanced data discovery and business glossary.

Testing 131

Testing Metadata Risk Data Science 131

Ontotext

FEBRUARY 12, 2024

Part one of this three-part series discussed the concept of data mesh and explored what it is and why an organization should care. Here, part two provides best practices for data mesh, including practical guidance, challenges, and limitations. Enterprises should identify and adopt specific data mesh elements to achieve velocity.

Data Quality 69

Data Quality Data-driven Data Lake Data Governance 69

O'Reilly on Data

JULY 28, 2020

Product Managers are responsible for the successful development, testing, release, and adoption of a product, and for leading the team that implements those milestones. It’s often difficult for businesses without a mature data or machine learning practice to define and agree on metrics. Agreeing on metrics.

Marketing 361

Marketing Experimentation Metrics Testing 361

Cloudera

OCTOBER 7, 2022

dbt allows data teams to produce trusted data sets for reporting, ML modeling, and operational workflows using SQL, with a simple workflow that follows software engineering best practices like modularity, portability, and continuous integration/continuous development (CI/CD). The Open Data Lakehouse . Introduction.

Data Warehouse 81

Data Warehouse Data Transformation Testing Data Lake 81