AWS Big Data

JULY 12, 2023

Organizations use their data to extract valuable insights and drive informed business decisions. In this post, we explain how data teams can quickly configure low-latency data pipelines that ingest and model data from a variety of sources, using Etleap’s end-to-end pipelines with Amazon Redshift and dbt.

Data Warehouse 52

Data Warehouse Modeling Dashboards Data Lake 52

AWS Big Data

JULY 28, 2023

Amazon Redshift is a petabyte-scale, enterprise-grade cloud data warehouse service delivering the best price-performance. Today, tens of thousands of customers run business-critical workloads on Amazon Redshift to cost-effectively and quickly analyze their data using standard SQL and existing business intelligence (BI) tools.

Data Lake 66

Data Lake Data Governance Data Warehouse Modeling 66

Cloudera

DECEMBER 21, 2021

Since the release of Cloudera Data Engineering (CDE) more than a year ago , our number one goal was operationalizing Spark pipelines at scale with first class tooling designed to streamline automation and observability. Data pipelines are composed of multiple steps with dependencies and triggers. New in 2021.

Snapshot 116

Snapshot Data-driven Optimization Management 116

AWS Big Data

MARCH 25, 2024

Foundation models (FMs) are large machine learning (ML) models trained on a broad spectrum of unlabeled and generalized datasets. This scale and general-purpose adaptability are what makes FMs different from traditional ML models. FMs are multimodal; they work with different data types such as text, video, audio, and images.

Data Lake 88

Data Lake Unstructured Data Management Modeling 88

AWS Big Data

NOVEMBER 8, 2023

Join us as we delve into the world of real-time streaming data at re:Invent 2023 and discover how you can use real-time streaming data to build new use cases, optimize existing projects and processes, and reimagine what’s possible. High-quality data is not just about accuracy; it’s also about timeliness. Reserve your seat now!

Data-driven 87

Data-driven Data Lake Machine Learning Cost-Benefit 87

DataKitchen

NOVEMBER 20, 2021

Errors are an inherent part of data analytics. The product for a data engineer is the data set. For an analyst, the product is the analysis that they deliver for a data object. For an analyst, the product is the analysis that they deliver for a data object. A DataOps Engineer can make test data available on demand.

Testing 246

Testing Machine Learning Data Warehouse Reporting 246

AWS Big Data

MAY 2, 2023

Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is a managed orchestration service for Apache Airflow that makes it simple to set up and operate end-to-end data pipelines in the cloud at scale. New feature: Data-aware scheduling using datasets With the release of Apache Airflow v2.4.0, and v2.2.2).

Testing 99

Testing Experimentation Management Metadata 99

AWS Big Data

SEPTEMBER 13, 2023

The Analytics specialty practice of AWS Professional Services (AWS ProServe) helps customers across the globe with modern data architecture implementations on the AWS Cloud. In this post, we discuss a common use case in relation to operational data processing and the solution we built using Apache Hudi and AWS Glue.

Data Lake 89

Data Lake Data Processing Metadata Snapshot 89

IBM Big Data Hub

SEPTEMBER 19, 2023

Though you may encounter the terms “data science” and “data analytics” being used interchangeably in conversations or online, they refer to two distinctly different concepts. Meanwhile, data analytics is the act of examining datasets to extract value and find answers to specific questions.

Data Science 82

Data Science Data Analytics Prescriptive Analytics Analytics 82

FineReport

MARCH 24, 2024

Rapid technological advancements and extensive networking have propelled the evolution of data analytics, fundamentally reshaping decision-making practices across various sectors. In this landscape, data analysts assume a pivotal role, tasked with interpreting data to drive informed decision-making.

Statistics 52

Statistics Data mining Visualization Reporting 52

AWS Big Data

NOVEMBER 16, 2023

Amazon Redshift is a popular cloud data warehouse, offering a fully managed cloud-based service that seamlessly integrates with an organization’s Amazon Simple Storage Service (Amazon S3) data lake, real-time streams, machine learning (ML) workflows, transactional workflows, and much more—all while providing up to 7.9x

Enterprise 82

Enterprise Data Warehouse Snapshot Cost-Benefit 82

AWS Big Data

MARCH 7, 2023

It covers how to use a conceptual, logical architecture for some of the most popular gaming industry use cases like event analysis, in-game purchase recommendations, measuring player satisfaction, telemetry data analysis, and more. Flexible and easy to use – The solutions should provide less restrictive, easy-to-access, and ready-to-use data.

Analytics 70

Analytics Data Warehouse Data Lake Metadata 70

Cloudera

JULY 18, 2022

In part 1 of this blog we discussed how Cloudera DataFlow for the Public Cloud (CDF-PC), the universal data distribution service powered by Apache NiFi, can make it easy to acquire data from wherever it originates and move it efficiently to make it available to other applications in a streaming fashion. Data decays!

Analytics 86

Analytics Dashboards Statistics Visualization 86

Cloudera

OCTOBER 6, 2020

The Cloudera Operational Database (COD) is a managed dbPaaS solution available as an experience in Cloudera Data Platform (CDP). It offers multi-modal client access with NoSQL key-value using Apache HBase APIs and relational SQL with JDBC (via Apache Phoenix). All code is in my github repo. Going through The Code.

Data Processing 79

Data Processing Testing Management Modeling 79

AWS Big Data

JULY 3, 2023

Backtesting is a process used in quantitative finance to evaluate trading strategies using historical data. Amazon Simple Storage Service (Amazon S3) is a popular cloud-based object storage service that can be used as the foundation for building a data lake.

Snapshot 82

Snapshot Data Lake Testing Strategy 82

DataKitchen

MAY 10, 2021

Data errors impact decision-making. Data errors infringe on work-life balance. Data errors also affect careers. If you have been in the data profession for any length of time, you probably know what it means to face a mob of stakeholders who are angry about inaccurate or late analytics.

Testing 214

Testing Manufacturing Data Quality Statistics 214

AWS Big Data

DECEMBER 8, 2023

In this post, we share how FanDuel moved from a DC2 nodes architecture to a modern Amazon Redshift architecture, which includes Redshift provisioned clusters using RA3 instances , Amazon Redshift data sharing , and Amazon Redshift Serverless. Their individual, product-specific, and often on-premises data warehouses soon became obsolete.

Data Warehouse 67

Data Warehouse Cost-Benefit KPI Risk 67

AWS Big Data

MARCH 9, 2023

Thousands of customers rely on Amazon Redshift to build data warehouses to accelerate time to insights with fast, simple, and secure analytics at scale and analyze data from terabytes to petabytes by running complex analytical queries. The star schema is a popular data model for building data marts.

Slice and Dice 80

Slice and Dice Data Warehouse Metrics Metadata 80

AWS Big Data

JANUARY 24, 2023

Data abstraction – Decouple and refactor underlying data structure and infrastructure. In this post, you will learn how to build a serverless analytics application using Amazon Redshift Data API and Amazon API Gateway WebSocket and REST APIs. The Data API takes care of managing database connections and buffering data.

Analytics 58

Analytics Data-driven Management Reporting 58

Domino Data Lab

MAY 1, 2019

This Domino Data Science Field Note provides highlights and excerpted slides from Chloe Mawer ’s “ The Ingredients of a Reproducible Machine Learning Model ” talk at a recent WiMLDS meetup. Mawer is a Principal Data Scientist at Lineage Logistics as well as an Adjunct Lecturer at Northwestern University.

Machine Learning 81

Machine Learning Testing Data Science Modeling 81

AWS Big Data

JULY 21, 2023

Data-driven organizations treat data as an asset and use it across different lines of business (LOBs) to drive timely insights and better business decisions. This leads to having data across many instances of data warehouses and data lakes using a modern data architecture in separate AWS accounts.

Data Lake 85

Data Lake Data Warehouse Marketing Management 85

AWS Big Data

JANUARY 22, 2024

The company uses AWS Cloud services to build data-driven products and scale engineering best practices. To ensure a sustainable data platform amid growth and profitability phases, their tech teams adopted a decentralized data mesh architecture.

Data-driven 83

Data-driven Advertising Metadata Data Architecture 83

Smart Data Collective

OCTOBER 14, 2019

From the tech industry to retail and finance, big data is encompassing the world as we know it. More organizations rely on big data to help with decision making and to analyze and explore future trends. Big Data Skillsets. billion allocated for data center systems and $90.2 NoSQL and SQL. billion for IT services.

Big Data 75

Big Data Software Data mining Prescriptive Analytics 75

Smart Data Collective

JUNE 4, 2021

What is data science? Data science is analyzing and predicting data, It is an emerging field. Some of the applications of data science are driverless cars, gaming AI, movie recommendations, and shopping recommendations. Data scientists use algorithms for creating data models. Where to start?

Data Science 104

Data Science Statistics Deep Learning Machine Learning 104

AWS Big Data

APRIL 24, 2023

Building a data lake on Amazon Simple Storage Service (Amazon S3) provides numerous benefits for an organization. However, many use cases, like performing change data capture (CDC) from an upstream relational database to an Amazon S3-based data lake, require handling data at a record level.

Data Lake 79

Data Lake Data Governance Cost-Benefit Machine Learning 79

AWS Big Data

MAY 9, 2023

It allows you to visually compose data transformation workflows using nodes that represent different data handling steps, which later are converted automatically into code to run. AWS Glue Studio recently released 10 more visual transforms to allow creating more advanced jobs in a visual way without coding skills.

Visualization 73

Visualization Marketing Big Data IT 73

Cloudera

AUGUST 10, 2021

Follow your data in object storage on-premises. With Apache Ozone on the Cloudera Data Platform (CDP) , they can implement a scale-out model and build out their next generation storage architecture without sacrificing security, governance and lineage. Ozone stores data as objects which live inside these buckets.

Testing 103

Testing Metadata IT Modeling 103

AWS Big Data

MARCH 26, 2024

This view is used to identify patterns and trends in customer behavior, which can inform data-driven decisions to improve business outcomes. In this post, we discuss how you can use purpose-built AWS services to create an end-to-end data strategy for C360 to unify and govern customer data that address these challenges.

Data Strategy 67

Data Strategy Strategy Data Warehouse Prescriptive Analytics 67

FineReport

APRIL 3, 2024

In today’s data-driven landscape, businesses are leaning more on BI tools , particularly BI dashboard solutions, to enhance decision-making through data visualization. Throughout this article, we’ll explore the importance of BI, data visualization, and dashboard tools in navigating intricate data landscapes.

Dashboards 52

Dashboards Visualization Metrics Interactive 52

Sisense

SEPTEMBER 9, 2019

Mark’s team is constantly adapting to and meeting the challenges of a rapidly evolving business using cloud technologies, real-time analytics, data warehousing, and virtualization. What if we could use this data to focus our resources and deliver better products? Predicting Return Rates On New Products.

Predictive Analytics 75

Predictive Analytics Predictive Modeling Forecasting Data-driven 75

Jen Stirrup

JANUARY 1, 2023

In this post, I’ll share some New Year Resolutions that I think Business Intelligence, Data Engineering, Artificial Intelligence, and Data Science teams should start doing from today. Adopt robust Data Engineering Practices. Consider the ethical and social consequences of your Data Science and AI Applications.

Business Intelligence 62

Business Intelligence Data Science Consulting Software 62

AWS Big Data

OCTOBER 4, 2023

Amazon DataZone enables customers to discover, access, share, and govern data at scale across organizational boundaries, reducing the undifferentiated heavy lifting of making data and analytics tools accessible to everyone in the organization. This is challenging because access to data is managed differently by each of the tools.

Metadata 98

Metadata Data Lake Publishing Data Governance 98

CIO Business Intelligence

MARCH 31, 2023

Business analysts are in high demand, with 24% of Fortune 500 companies currently hiring business analysts across a range of industries, including technology (27%), finance (13%), professional services (10%), and healthcare (5%), according to data from Zippia. Amazon, Capgemini, and IBM.

Business Analysis 88

Business Analysis Testing Business Analytics Statistics 88

Insight

MAY 8, 2019

In an ideal world, data scientists would focus their time on analysis and modeling, but there is often seemingly unavoidable overhead in visualizing and presenting their results. I was inspired to offer an alternative to both of these options at Insight, when I saw how long data science Fellows were taking to deploy their Flask apps.

Data Processing 72

Data Processing Testing Data Science Visualization 72

O'Reilly on Data

MARCH 23, 2023

It can explain code that you don’t understand, including code that has been intentionally obfuscated. It’s important to understand that ChatGPT is not actually a language model. It’s a convenient user interface built around one specific language model, GPT-3.5, The GPT-series LLMs are also called “foundation models.”

IT 260

IT Modeling Testing Risk 260

Sisense

JUNE 12, 2019

At Sisense, our mission is to empower users of all kinds with deep insights from even the most complex data. Soon after, we announced the release of Sisense Hunch which provides the ability to transform even the most massive data sets into a deep neural net which can be placed anywhere, even on an IoT device. AI Throughout.

Analytics 75

Analytics Data-driven Dashboards IoT 75

The Unofficial Google Data Science Blog

SEPTEMBER 9, 2015

by SEAN GERRISH and AMIR NAJMI To deliver the services our users have come to rely upon, Googlers have to process a lot of data — often at web-scale. In this post, we’ll describe some things we have learned about mixing declarative and procedural programing paradigms to simplify MapReduce as used by data scientists.

Data Science 56

Data Science Statistics Testing Metadata 56