Smart Data Collective

FEBRUARY 20, 2022

Whether it’s data management, analytics, or scalability, AWS can be the top-notch solution for any SaaS company. Your SaaS company can store and protect any amount of data using Amazon Simple Storage Service (S3), which is ideal for data lakes, cloud-native applications, and mobile apps. Management.

Cost-Benefit 116

Cost-Benefit Data Lake Software Machine Learning 116

AWS Big Data

DECEMBER 13, 2023

Redshift streaming ingestion provides low latency, high-throughput data ingestion, which enables customers to derive insights in seconds instead of minutes. After that, using materialized-view refresh, you can ingest hundreds of megabytes of data per second. This solution uses Amazon Aurora MySQL hosting the example database salesdb.

Data Warehouse 94

Data Warehouse Snapshot Data Processing Management 94

AWS Big Data

MARCH 6, 2024

The account on the right hosts the pseudonymization service, which you can deploy using the instructions provided in the Part 1 of this series. For an overview of how to build an ACID compliant data lake using Iceberg, refer to Build a high-performance, ACID compliant, evolving data lake using Apache Iceberg on Amazon EMR.

Metrics 89

Metrics Statistics Testing Data Lake 89

Cloudera

MAY 12, 2021

2020 saw us hosting our first ever fully digital Data Impact Awards ceremony, and it certainly was one of the highlights of our year. We saw a record number of entries and incredible examples of how customers were using Cloudera’s platform and services to unlock the power of data. SECURITY AND GOVERNANCE LEADERSHIP.

Digital Transformation 69

Digital Transformation Machine Learning Optimization Data Lake 69

AWS Big Data

JUNE 6, 2023

Building on the success of Amazon EMR on EKS, customers have been running and managing jobs using the emr-containers API, creating EMR virtual clusters , and submitting jobs to the EKS cluster, either through the AWS Command Line Interface (AWS CLI) or Apache Airflow scheduler. Helm 3.7+ , the package manager for Kubernetes.

Optimization 67

Optimization Data Lake Cost-Benefit Management 67

AWS Big Data

MAY 30, 2023

Customers have been using data warehousing solutions to perform their traditional analytics tasks. Traditional batch ingestion and processing pipelines that involve operations such as data cleaning and joining with reference data are straightforward to create and cost-efficient to maintain. options(**additional_options).mode("append").save(s3_output_folder)

Data Lake 76

Data Lake Data Analytics Analytics Data Processing 76

AWS Big Data

JULY 20, 2023

With data becoming the driving force behind many industries today, having a modern data architecture is pivotal for organizations to be successful. In this post, we describe Orca’s journey building a transactional data lake using Amazon Simple Storage Service (Amazon S3), Apache Iceberg, and AWS Analytics.

Data Lake 83

Data Lake Analytics Snapshot Optimization 83

Cloudera

NOVEMBER 17, 2022

We needed a solution to manage our data at scale, to provide greater experiences to our customers. With Cloudera Data Platform, we aim to unlock value faster and offer consistent data security and governance to meet this goal. and primarily served regulatory reporting and internal analytics requirements.

Management 87

Management Data Lake Consulting Unstructured Data 87

DataKitchen

NOVEMBER 29, 2021

Data Gets Meshier. 2022 will bring further momentum behind modular enterprise architectures like data mesh. The data mesh addresses the problems characteristic of large, complex, monolithic data architectures by dividing the system into discrete domains managed by smaller, cross-functional teams.

Testing 245

Testing Data Lake Data Architecture Manufacturing 245

AWS Big Data

APRIL 26, 2024

To enable your workforce users for analytics with fine-grained data access controls and audit data access, you might have to create multiple AWS Identity and Access Management (IAM) roles with different data permissions and map the workforce users to one of those roles.

Analytics 90

Analytics Data Lake Management Enterprise 90

AWS Big Data

MARCH 20, 2023

In this post, we share how Poshmark improved CX and accelerated revenue growth by using a real-time analytics solution. High-level challenge: The need for real-time analytics Previous efforts at Poshmark for improving CX through analytics were based on batch processing of analytics data and using it on a daily basis to improve CX.

Analytics 52

Analytics Slice and Dice Data Processing Data Lake 52

datapine

APRIL 15, 2020

When it comes to implementing and managing a successful BI strategy we have always proclaimed: start small, use the right BI tools , and involve your team. When encouraging these BI best practices what we are really doing is advocating for agile business intelligence and analytics. Top 10 Tips For Agile BI & Analytics Development.

Business Intelligence 276

Business Intelligence Analytics Testing Dashboards 276

AWS Big Data

NOVEMBER 8, 2023

Because Gilead is expanding into biologics and large molecule therapies, and has an ambitious goal of launching 10 innovative therapies by 2030, there is heavy emphasis on using data with AI and machine learning (ML) to accelerate the drug discovery pipeline. This data volume is expected to increase monthly and is fully refreshed each month.

Data Lake 72

Data Lake Data Warehouse Cost-Benefit Optimization 72

CIO Business Intelligence

MARCH 10, 2022

And you can manage the IT and budget in such a way that people are empowered to focus on pancreatic cancer. … Next up: AI and data lake decisions. To that end, UAB’s next step is to tackle big decisions around expanding its AI and data analytics platforms, says Carver, who is not handling the long-term planning alone.

IT 98

IT Data Lake Digital Transformation Data Governance 98

Cloudera

NOVEMBER 18, 2020

Data Impact Achievement Award. United Overseas Bank (UOB), a Singaporean multinational banking organization, is recognized as one of the most excellent and professionally managed financial institutions in Asia. UOB understands that the future is data-driven. Winner: United Overseas Bank. Globe Telecom, Inc. Winner: Telkomsel.

Internet Publishing and Broadcasting 122

Internet Publishing and Broadcasting Data-driven Broadcasting Digital Transformation 122

AWS Big Data

MARCH 26, 2024

AWS provides different services for building data ingestion pipelines: AWS Glue is a serverless data integration service that ingests data in batches from on-premises databases and data stores in the cloud. Streaming data with low latency needs is stored in Amazon Kinesis Data Streams for real-time consumption.

Data Strategy 66

Data Strategy Strategy Data Warehouse Prescriptive Analytics 66

AWS Big Data

OCTOBER 23, 2023

And because Athena is serverless and fully managed, you can run your workloads without worrying about the underlying infrastructure. Before you run these workloads, most customers run SQL queries to interactively extract, filter, join, and aggregate data into a shape that can be used for decision-making, model training, or inference.

Data Lake 97

Data Lake Visualization Optimization Interactive 97

AWS Big Data

JANUARY 22, 2024

The existing monolith and centralized architecture was struggling to meet the growing demands of data consumers. Data engineers were finding it increasingly challenging to maintain and scale the data infrastructure, resulting in data access, data silos, and inefficiencies in data management.

Data-driven 82

Data-driven Advertising Metadata Data Architecture 82

BizAcuity

MAY 10, 2022

2007: Amazon launches SimpleDB, a non-relational (NoSQL) database that allows businesses to cheaply process vast amounts of data with minimal effort. The platform is built on S3 and EC2 using a hosted Hadoop framework. An efficient big data management and storage solution that AWS quickly took advantage of.

Data-driven 52

Data-driven IoT Unstructured Data Data Lake 52

AWS Big Data

JUNE 7, 2023

Cargotec captures terabytes of IoT telemetry data from their machinery operated by numerous customers across the globe. This data needs to be ingested into a data lake, transformed, and made available for analytics, machine learning (ML), and visualization. The job runs in the target account.

Metadata 55

Metadata Data Lake Machine Learning Big Data 55

AWS Big Data

MARCH 3, 2023

Building data lakes from continuously changing transactional data of databases and keeping data lakes up to date is a complex task and can be an operational challenge. You can then apply transformations and store data in Delta format for managing inserts, updates, and deletes.

Data Lake 79

Data Lake Dashboards Metrics Metadata 79

AWS Big Data

DECEMBER 18, 2023

AWS Step Functions is a fully managed visual workflow service that enables you to build complex data processing pipelines involving a diverse set of extract, transform, and load (ETL) technologies such as AWS Glue , Amazon EMR , and Amazon Redshift. There are multiple tables related to customers and order data in the RDS database.

Metadata 117

Metadata Visualization Data Lake Data-driven 117

Sisense

DECEMBER 11, 2019

With data growing at a staggering rate, managing and structuring it is vital to your survival. In this piece, we detail the Israeli debut of Periscope Data. Driving startup growth with the power of data. It’s why Sisense, having merged with Periscope Data in May 2019, chose to host this event in Tel Aviv.

Data Lake 80

Data Lake Big Data Sales Data-driven 80

AWS Big Data

NOVEMBER 8, 2023

Putting your data to work with generative AI – Innovation Talk Thursday, November 30 | 12:30 – 1:30 PM PST | The Venetian Join Mai-Lan Tomsen Bukovec, Vice President, Technology at AWS to learn how you can turn your data lake into a business advantage with generative AI. Reserve your seat now! Reserve your seat now!

Data-driven 85

Data-driven Data Lake Machine Learning Cost-Benefit 85

AWS Big Data

FEBRUARY 27, 2024

In this post, we explore how Bluestone uses AWS services, notably the cloud data warehousing service Amazon Redshift , to implement a cutting-edge data mesh architecture, revolutionizing the way they manage, access, and utilize their data assets. This enables data-driven decision-making across the organization.

Data-driven 90

Data-driven Data Lake Data Quality Data Governance 90

Andrew White

JANUARY 11, 2021

On January 4th I had the pleasure of hosting a webinar. It was titled, The Gartner 2021 Leadership Vision for Data & Analytics Leaders. This was for the Chief Data Officer, or head of data and analytics. How do you think Technology Business Management plays into this strategy? Governance.

Data Analytics 52

Data Analytics Analytics Data-driven Finance 52

AWS Big Data

DECEMBER 21, 2023

To speed up the self-service analytics and foster innovation based on data, a solution was needed to provide ways to allow any team to create data products on their own in a decentralized manner. To create and manage the data products, smava uses Amazon Redshift , a cloud data warehouse.

Data Lake 85

Data Lake Data Warehouse Data-driven B2B 85

IBM Big Data Hub

JUNE 15, 2023

It is comprised of commodity cloud object storage, open data and open table formats, and high-performance open-source query engines. To help organizations scale AI workloads, we recently announced IBM watsonx.data , a data store built on an open data lakehouse architecture and part of the watsonx AI and data platform.

Data Warehouse 99

Data Warehouse Data Lake Optimization Data-driven 99

AWS Big Data

JULY 14, 2023

Consumers prioritized data discoverability, fast data access, low latency, and high accuracy of data. Producers prioritized ownership, governance, access management, and reuse of their datasets. These inputs reinforced the need of a unified data strategy across the FinOps teams.

Finance 79

Finance Metadata Big Data Recreation/Entertainment 79