DataKitchen

OCTOBER 5, 2021

DataOps establishes a process hub that automates data production and analytics development workflows so that the data team is more efficient, innovative and less prone to error. In this blog, we’ll explore the role of the DataOps Engineer in driving the data organization to higher levels of productivity.

Testing 152

Testing Dashboards Measurement Experimentation 152

Cloudera

FEBRUARY 8, 2021

Continuing from my previous blog post about how awesome and easy it is to develop web-based applications backed by Cloudera Operational Database (COD), I started a small project to integrate COD with another CDP cloud experience, Cloudera Machine Learning (CML). . b) Basic data transformation. Go to runner.py and run it.

Machine Learning 77

Machine Learning Statistics Modeling Dashboards 77

Cloudera

DECEMBER 9, 2022

Developers need to onboard new data sources, chain multiple data transformation steps together, and explore data as it travels through the flow. A reimagined visual editor to boost developer productivity and enable self service. Enabling self-service for developers.

Testing 103

Testing Cost-Benefit Interactive Visualization 103

Sisense

FEBRUARY 26, 2020

In Transform to Win , we explore the challenges facing modern companies, diving into their individual digital transformations and the people who drive them. We all know that data is becoming more and more essential for businesses, as the volume of data keeps growing.

Digital Transformation 90

Digital Transformation Analytics Cost-Benefit Data-driven 90

Alation

FEBRUARY 20, 2020

For ease of understanding the differences between all of the them Rita shared this visual, categorizing the vendors: So at least for now, it looks like we’re a self-service data prep vendor, which makes sense. Alation helps analysts find, understand and use their data. Subscribe to Alation's Blog. I hope to see you there!

Scorecard 52

Scorecard ROI Data-driven Visualization 52

Cloudera

AUGUST 8, 2022

The general availability covers Iceberg running within some of the key data services in CDP, including Cloudera Data Warehouse ( CDW ), Cloudera Data Engineering ( CDE ), and Cloudera Machine Learning ( CML ). In this first blog, we shared with you how to use Apache Iceberg in Cloudera Data Platform to build an open lakehouse.

Snapshot 96

Snapshot Data Warehouse Machine Learning Cost-Benefit 96

Sisense

APRIL 10, 2020

Healthy Data is your window into how data is helping these organizations address this crisis. As the rapid spread of COVID-19 continues, data managers around the world are pulling together a wide variety of global data sources to inform governments, the private sector, and the public with the latest on the spread of this disease.

Visualization 105

Visualization Reporting Data Processing Dashboards 105

AWS Big Data

JULY 31, 2023

We create the insert_orders_fact_tbl AWS Glue job manually using AWS Glue Visual Studio. You will see the message “Successfully connected to the data store with connection blog-redshift-connection.” Under Data Catalog in the navigation pane, choose Crawlers. Select Visual with a blank canvas , then choose Create.

Sales 52

Sales Data Warehouse Visualization Testing 52

AWS Big Data

MAY 16, 2023

This blog post is co-written with Steve Alexander at PG&E. Data collection and processing are handled by a third-party smart sensor manufacturer application residing in Amazon Virtual Private Cloud (Amazon VPC) private subnets behind a Network Load Balancer.

Dashboards 79

Dashboards Statistics Data Collection Business Intelligence 79

Cloudera

MARCH 14, 2023

We just announced the general availability of Cloudera DataFlow Designer , bringing self-service data flow development to all CDP Public Cloud customers. In our previous DataFlow Designer blog post , we introduced you to the new user interface and highlighted its key capabilities.

Testing 87

Testing Publishing Metadata Interactive 87

erwin

APRIL 29, 2020

These tools range from enterprise service bus (ESB) products, data integration tools; extract, transform and load (ETL) tools, procedural code, application program interfaces (API)s, file transfer protocol (FTP) processes, and even business intelligence (BI) reports that further aggregate and transform data.

Metadata 111

Metadata Key Performance Indicator Data Governance Data Quality 111

Alation

FEBRUARY 2, 2023

Many are subject matter experts for a particular kind of data, which enables them to spot anomalies in that data quickly, understand the root cause, and resolve the issue. Image Sourced from theproductmanager.com Further, they can quickly create helpful visualizations from the data they analyze. Here is her LinkedIn.

Management 52

Management Data-driven Visualization Strategy 52

Sisense

OCTOBER 15, 2020

We live in a world of data: There’s more of it than ever before, in a ceaselessly expanding array of forms and locations. Dealing with Data is your window into the ways data teams are tackling the challenges of this new world to help their companies and their customers thrive. billion market by 2025.

Dashboards 98

Dashboards IoT Optimization Internet of Things 98

Sisense

MARCH 14, 2021

Attempting to learn more about the role of big data (here taken to datasets of high volume, velocity, and variety) within business intelligence today, can sometimes create more confusion than it alleviates, as vital terms are used interchangeably instead of distinctly.

Big Data 52

Big Data IoT Data Warehouse Data-driven 52

AWS Big Data

OCTOBER 27, 2023

AWS Glue , a serverless data integration and extract, transform, and load (ETL) service, has revolutionized this process, making it more accessible and efficient. AWS Glue eliminates complexities and costs, allowing organizations to perform data integration tasks in minutes, boosting efficiency. Customers can now use AWS Glue 4.0

Analytics 97

Analytics Visualization Data Integration Cost-Benefit 97

DataKitchen

JULY 27, 2023

Here are a few examples that we have seen of how this can be done: Batch ETL with Azure Data Factory and Azure Databricks: In this pattern, Azure Data Factory is used to orchestrate and schedule batch ETL processes. Azure Blob Storage serves as the data lake to store raw data. Azure Machine Learning).

Machine Learning 130

Machine Learning Cost-Benefit Data Transformation Testing 130

Alation

JANUARY 17, 2023

Business Intelligence Tools: Business intelligence (BI) tools are used to visualize your data. You should pick those that allow for easy integration and can create beautiful data visualizations. These help data analysts visualize key insights that can help you make better data-backed decisions.

Data Warehouse 52

Data Warehouse Cost-Benefit Data Transformation Data Science 52

AWS Big Data

OCTOBER 31, 2023

This blog post is co-written with James Sun from Snowflake. Customers rely on data from different sources such as mobile applications, clickstream events from websites, historical data, and more to deduce meaningful patterns to optimize their products, services, and processes. Choose Airflow version 2.6.3. Choose Next.

Data Processing 91

Data Processing Management Publishing Testing 91

Cloudera

AUGUST 17, 2021

When we announced the GA of Cloudera Data Engineering back in September of last year, a key vision we had was to simplify the automation of data transformation pipelines at scale. Typically users need to ingest data, transform it into optimal format with quality checks, and optimize querying of the data by visual analytics tool.

Management 115

Management Cost-Benefit Data Transformation Optimization 115

Alation

OCTOBER 27, 2022

Few actors in the modern data stack have inspired the enthusiasm and fervent support as dbt. This data transformation tool enables data analysts and engineers to transform, test and document data in the cloud data warehouse. Curious to learn how the data catalog can power your data strategy?

Metrics 52

Metrics Dashboards Sales Reporting 52

IBM Big Data Hub

JULY 17, 2023

In this blog, I will cover: What is watsonx.ai? Capabilities within the Prompt Lab include: Summarize: Transform text with domain-specific content into personalized overviews and capture key points (e.g., What capabilities are included in watsonx.ai? What is watsonx.data? What capabilities are included in watsonx.data?

Machine Learning 62

Machine Learning Data Warehouse Modeling Cost-Benefit 62

DataKitchen

OCTOBER 25, 2022

It’s because it’s a hard thing to accomplish when there are so many teams, locales, data sources, pipelines, dependencies, data transformations, models, visualizations, tests, internal customers, and external customers.

Testing 130

Testing Data-driven Visualization Dashboards 130

Sisense

DECEMBER 12, 2019

Building a data-driven business includes choosing the right software and implementing best practices around its use. Every year when budget time rolls around, many organizations find themselves asking the same question: “what are we going to do about our data?” This is a summary article. New year, same questions.

Business Intelligence 88

Business Intelligence Software Data-driven Visualization 88

Cloudera

OCTOBER 14, 2020

Also, such a concept helps admin to visualize the jobs which are scheduled for debugging purposes. YuniKorn, thus empowers Apache Spark to become an enterprise-grade essential platform for users, offering a robust platform for a variety of applications ranging from large scale data transformation to analytics to machine learning.

Machine Learning 119

Machine Learning Management Big Data Optimization 119

Smart Data Collective

OCTOBER 13, 2021

There are countless examples of big data transforming many different industries. It can be used for something as visual as reducing traffic jams, to personalizing products and services, to improving the experience in multiplayer video games. This is something that you can learn more about in just about any technology blog.

Visualization 135

Visualization Cost-Benefit Big Data Prescriptive Analytics 135

AWS Big Data

FEBRUARY 8, 2023

For addressing data quality challenges in Amazon Simple Storage Service (Amazon S3) data lakes and data pipelines, AWS has announced AWS Glue Data Quality (preview). Stored procedures Stored procedures are commonly used to encapsulate logic for data transformation, data validation, and business-specific logic.

Data Quality 78

Data Quality Testing Data Warehouse Unstructured Data 78

AWS Big Data

MARCH 3, 2023

Data ingestion – Steps 1 and 2 use AWS DMS, which connects to the source database and moves full and incremental data (CDC) to Amazon S3 in Parquet format. Data transformation – Steps 3 and 4 represent an EMR Serverless Spark application (Amazon EMR 6.9 For Name , enter emr-delta-blog. For Type , choose Spark.

Data Lake 84

Data Lake Dashboards Metrics Metadata 84

Sisense

MAY 11, 2021

You can’t talk about data analytics without talking about data modeling. The reasons for this are simple: Before you can start analyzing data, huge datasets like data lakes must be modeled or transformed to be usable.

Modeling 84

Modeling Big Data IoT Data Warehouse 84

Occam's Razor

OCTOBER 19, 2010

I was reflecting on that recently and thought it was incredible that in all my years of writing this blog I have never written a blog post, not one single one (!!), My goal is to give you a list of tools that I use in my everyday life as a practitioner (you'll see many of them implemented on this blog). Disclosure].

Analytics 135

Analytics Testing Measurement Optimization 135

AWS Big Data

OCTOBER 5, 2023

Amazon AppFlow , a fully managed data integration service, has been at the forefront of streamlining data transfer between AWS services, software as a service (SaaS) applications, and now Google BigQuery. Select a data source: In Amazon AppFlow , select Google BigQuery as your data source.

Data Warehouse 86

Data Warehouse Machine Learning Data Integration Data-driven 86

AWS Big Data

AUGUST 1, 2023

Provision resources with AWS CloudFormation For the initial setup, you launch an AWS CloudFormation stack to create an S3 bucket to store data, IAM roles for data access, and the AWS Glue crawler and Data Catalog components. This will enable both the CDC steps and the data transformation steps for the Jira data.

Data Lake 87

Data Lake Data Transformation Cost-Benefit Data-driven 87

Cloudera

NOVEMBER 22, 2021

In addition, more data is becoming available for processing / enrichment of existing and new use cases e.g., recently we have experienced a rapid growth in data collection at the edge and an increase in availability of frameworks for processing that data. As a result, alternative data integration technologies (e.g.,

Data Processing 97

Data Processing Data Warehouse Enterprise Visualization 97

DataKitchen

JANUARY 23, 2023

Testing data and analytic systems require a development system with accurate test data, tools, and relevant tool code. Only then can you tell the true impact of a column name change on the data transformations, the models, and the visualization you give to your customers.

Testing 130

Testing Data Governance Data Quality Data-driven 130

Jet Global

MARCH 13, 2024

Between complex data structures, data security questions, and error-prone manual processes, merging data from disparate sources into a single system can quickly turn your routine reporting processes into a stressful and time-consuming ordeal.

Reporting 52

Reporting Finance Data Quality Sales 52