AWS Big Data

JUNE 15, 2023

In today’s world, customers manage vast amounts of data in their Amazon Simple Storage Service (Amazon S3) data lakes, which requires convoluted data pipelines to continuously understand the changes in the data layout and make them available to consuming systems.

Data Lake 86

Data Lake Metadata Cost-Benefit Management 86

IBM Big Data Hub

APRIL 24, 2024

An online retailer always gets users’ explicit consent before sharing customer data with its partners. A navigation app anonymizes activity data before analyzing it for travel trends. One cannot overstate the importance of data privacy for businesses today. The user can accept or reject each use of their data individually.

Risk 83

Risk Measurement Data Governance Insurance 83

AWS Big Data

APRIL 24, 2024

EMR Studio is an integrated development environment (IDE) that makes it straightforward for data scientists and data engineers to develop, visualize, and debug analytics applications written in PySpark, Python, and Scala. Make sure you set up resources like your S3 bucket, VPC subnets, and EMR Studio in the same AWS Region.

Interactive 80

Interactive Visualization Big Data Management 80

DataKitchen

AUGUST 3, 2021

The data mesh design pattern breaks giant, monolithic enterprise data architectures into subsystems or domains, each managed by a dedicated team. DataOps helps the data mesh deliver greater business agility by enabling decentralized domains to work in concert. . But first, let’s define the data mesh design pattern.

Data Architecture 246

Data Architecture Data Lake Cost-Benefit Data Warehouse 246

AWS Big Data

MARCH 2, 2023

Iceberg has become very popular for its support for ACID transactions in data lakes and features like schema and partition evolution, time travel, and rollback. and later supports the Apache Iceberg framework for data lakes. Apache Iceberg is designed to support these features on cost-effective petabyte-scale data lakes on Amazon S3.

Data Lake 116

Data Lake Data Processing Metadata Snapshot 116

DataKitchen

JANUARY 5, 2021

Remote working has revealed the inconsistency and fragility of workflow processes in many data organizations. The data teams share a common objective; to create analytics for the (internal or external) customer. Data Science Workflow – Kubeflow, Python, R. Data Engineering Workflow – Airflow, ETL.

Testing 147

Testing Data Governance Metadata Visualization 147

DataKitchen

MAY 4, 2023

Webinar Summary: DataOps and Data Mesh Chris Bergh, CEO of DataKitchen, delivered a webinar on two themes – Data Products and Data Mesh. Bergh started by discussing the complexity within data and analytics teams, stating that complexity makes everything more complicated and, in the long run, it kills productivity.

Measurement 130

Measurement Data-driven Testing Cost-Benefit 130

AWS Big Data

JULY 31, 2023

Data has become an integral part of most companies, and the complexity of data processing is increasing rapidly with the exponential growth in the amount and variety of data. AWS Glue is a serverless data integration and ETL service with the ability to scale on demand.

Sales 52

Sales Data Warehouse Visualization Testing 52

Cloudera

JULY 15, 2021

The release of Cloudera Data Platform (CDP) Private Cloud Base edition provides customers with a next generation hybrid cloud architecture. This unified distribution is a scalable and customizable platform where you can securely run many types of workloads. Cloudera SDX for consistent security and governance across the platform.

Data Processing 96

Data Processing Metadata Testing Management 96

IBM Big Data Hub

DECEMBER 19, 2023

In this stage, the focus is building an enterprise catalog of continuous integration, deployment and testing (CI/CD/CT) and Ops patterns with necessary tooling integrations to automate core development and testing activities. We recommend a 4-stage roadmap for implementation, as shown in the figure below. This helps adoption and acceleration.

Testing 82

Testing Enterprise Modeling Software 82

Cloudera

MARCH 22, 2023

Cloudera Data Platform (CDP) is no different: it’s a hybrid data platform that meets organizations’ needs to get to grips with complex data anywhere, turning it into actionable insight quickly and easily. There are various user roles that all have different questions “why?” That’s where observability comes in.

Metrics 84

Metrics Data Governance Cost-Benefit Dashboards 84

IBM Big Data Hub

JANUARY 8, 2024

In modern enterprises, where operations leave a massive digital footprint, business events allow companies to become more adaptable and able to recognize and respond to opportunities or threats as they occur. Becoming a real-time enterprise Businesses often go on a journey that traverses several stages of maturity when they establish an EDA.

Data-driven 80

Data-driven Management Advertising Enterprise 80

AWS Big Data

APRIL 24, 2023

Building a data lake on Amazon Simple Storage Service (Amazon S3) provides numerous benefits for an organization. However, many use cases, like performing change data capture (CDC) from an upstream relational database to an Amazon S3-based data lake, require handling data at a record level.

Data Lake 78

Data Lake Data Governance Cost-Benefit Machine Learning 78

Alation

AUGUST 17, 2022

According to analysts, data governance programs have not shown a high success rate. According to CIOs , historical data governance programs were invasive and suffered from one of two defects: They were either forced on the rank and file — who grew to dislike IT as a result. The Risks of Early Data Governance Programs.

Data Governance 92

Data Governance Risk Data-driven Measurement 92

AWS Big Data

OCTOBER 3, 2023

In our previous post Improve operational efficiencies of Apache Iceberg tables built on Amazon S3 data lakes , we discussed how you can implement solutions to improve operational efficiencies of your Amazon Simple Storage Service (Amazon S3) data lake that is using the Apache Iceberg open table format and running on the Amazon EMR big data platform.

Optimization 108

Optimization Snapshot Data Lake Metadata 108

Alation

MARCH 14, 2023

Data gathering and use pervades almost every business function these days — and it’s widely acknowledged that businesses with a clear strategy around data are best placed to succeed in competitive, challenging markets such as defence. What is a data strategy? Why is a data strategy important?

Data Strategy 52

Data Strategy Strategy Metadata Data Quality 52

Alation

NOVEMBER 4, 2022

In business, data-based goals tend to be very tangible. Perhaps you want to boost your ROI or CAGR, or reduce the time your analysts spend accessing and leveraging data. In this blog we’ll describe digital transformation: how it can be achieved as well as how it can benefit your business. What data is being used?

Digital Transformation 52

Digital Transformation Cost-Benefit Insurance Machine Learning 52

erwin

JULY 12, 2019

In light of recent, high-profile data breaches, it’s past-time we re-examined strategic data governance and its role in managing regulatory requirements. for alleged violations of the European Union’s General Data Protection Regulation (GDPR). Given this, Oppenheimer & Co. Complexity.

Data Governance 110

Data Governance Management Metadata Risk Management 110

Alation

FEBRUARY 13, 2020

In an earlier blog, I defined a data catalog as “a collection of metadata, combined with data management and search tools, that helps analysts and other data users to find the data that they need, serves as an inventory of available data, and provides information to evaluate fitness data for intended uses.”.

Metadata 52

Metadata Management Data Lake Data Governance 52

Cloudera

JANUARY 26, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata 72

Metadata Data Governance Management Finance 72

erwin

MARCH 30, 2020

Data intelligence has a critical role to play in the supercomputing battle against Covid-19. While leveraging supercomputing power is a tremendous asset in our fight to combat this global pandemic, in order to deliver life-saving insights, you really have to understand what data you have and where it came from.

Metadata 122

Metadata IT Data Governance Data Quality 122

Alation

APRIL 13, 2021

Much of his work focuses on democratising data and breaking down data silos to drive better business outcomes. In this blog, Chris shows how Snowflake and Alation together accelerate data culture. He shows how Texas Mutual Insurance Company has embraced data governance to build trust in data.

Insurance 98

Insurance Data Governance Data-driven Metrics 98

erwin

MAY 20, 2020

Data governance (DG) as a an “emergency service” may be one critical lesson learned coming out of the COVID-19 crisis. Where crisis leads to vulnerability, data governance as an emergency service enables organization management to direct or redirect efforts to ensure activities continue and risks are mitigated.

Data Governance 144

Data Governance Metadata Risk Strategy 144

Cloudera

FEBRUARY 16, 2021

In the previous blog post , we looked at some of the application development concepts for the Cloudera Operational Database (COD). In this blog post, we’ll see how you can use other CDP services with COD. Integrated across the Enterprise Data Lifecycle . Cloudera Data Engineering to ingest bulk data and data from mainframes.

Machine Learning 81

Machine Learning Data Lake Enterprise Data Warehouse 81

AWS Big Data

FEBRUARY 20, 2024

This setup has been a key enabler to make corporate users and groups available inside EMR clusters and define access control policies to control their data access (for example, through the Amazon EMR native Apache Ranger integration ). The following diagram summarizes how the authentication flow works.

Testing 84

Testing Data Processing Interactive Management 84

IBM Big Data Hub

OCTOBER 9, 2023

Artificial intelligence (AI) can help usher in a new era of human resource management, where data analytics, machine learning and automation can work together to save people time and support higher-quality outcomes. These conversations are happening as the workforce is undergoing a major change.

Machine Learning 74

Machine Learning Management Optimization Technology 74

AWS Big Data

MARCH 8, 2023

The value of data is time sensitive. Real-time processing makes data-driven decisions accurate and actionable in seconds or minutes instead of hours or days. It ensures that data in multiple systems stays in sync. This is especially important if you’re making time-sensitive decisions in a high-velocity data environment.

Data-driven 74

Data-driven Testing Data Processing Management 74

Cloudera

MARCH 4, 2024

In this article, we will walk you through the process of implementing fine grained access control for the data governance framework within the Cloudera platform. In a good data governance strategy, it is important to define roles that allow the business to limit the level of access that users can have to their strategic data assets.

Metadata 52

Metadata Data Governance Management Finance 52

Alation

SEPTEMBER 30, 2021

Alation achieves a top-rank for Innovation within the peer group Data Governance Products , according to BARC’s The Data Management Survey 22. Alation was ranked #1 in two KPIs within the Data Governance Products peer group: Innovation and Innovation Power. We use it as a central hub for all things data.

Management 52

Management Data Governance KPI Metadata 52

Alation

NOVEMBER 15, 2021

Most people close to data management are aware of the EDM Council. This highly respected standards & training organization was “created to elevate the practice of data management as a business and operational priority.” The 14 key controls of the Cloud Data Management Framework. Approach & Deliverables.

Management 52

Management Testing Metadata Consulting 52

Alation

MARCH 29, 2023

Alation has secured a Software Licensing Program (SLP) in California and a Department of Information Resources (DIR) in Texas. Alation has secured a Standard Licensing Program (SLP) in California and a Department of Information Resources (DIR) in Texas. What role do partners play in the process? They can trust us.

Data Strategy 52

Data Strategy Strategy Data Governance Sales 52

Alation

NOVEMBER 17, 2022

But what does it mean for an organisation to be truly data-driven? What foundation needs to be in place at the start, and what journey does an organisation need to embrace to benefit from the forensic insights their data can reveal? This requires: Data is embedded into the processes of the organisation.

Data-driven 52

Data-driven Cost-Benefit Business Objectives Technology 52

Alation

APRIL 18, 2023

For any data user in an enterprise today, data profiling is a key tool for resolving data quality issues and building new data solutions. In this blog, we’ll cover the definition of data profiling, top use cases, and share important techniques and best practices for data profiling today.

IT 52

IT Data Quality Metadata Data Governance 52

Alation

JULY 13, 2021

Yet increasing complexity of data makes the old “lift-and-shift” model not just unrealistic, but risky. Businesses with complex data environments need a migration method that takes that complexity into account. The Data Race to the Cloud. This recent cloud migration applies to all who use data. Fern Halper, Ph.D.

Strategy 52

Strategy Data mining Structured Data Data Governance 52

Alation

MARCH 1, 2023

In a world as interconnected as ours, it’s difficult to overstate the value of secure information exchange. Blockchain technology delivers a secure record of transaction, using cryptography to interlink records. How are blockchain organizations tackling data management? What are the goals of your data team?

Data Strategy 52

Data Strategy Strategy Metadata Interactive 52

AWS Big Data

JUNE 29, 2023

Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud that delivers powerful and secure insights on all your data with the best price-performance. With Amazon Redshift, you can analyze your data to derive holistic insights about your business and your customers.

Data Warehouse 59

Data Warehouse Dashboards Testing Visualization 59

Alation

AUGUST 9, 2022

Today, data is more valuable to a company than it’s ever been. It can help you make data-driven decisions which can improve business performance, boost revenue, and improve efficiencies. But in the four years since it came into force, have companies reached their full potential for data integrity? What is data integrity?

Data Integration 52

Data Integration Data Quality Measurement Strategy 52