article thumbnail

Data Modeling 301 for the cloud: data lake and NoSQL data modeling and design

erwin

For NoSQL, data lakes, and data lake houses—data modeling of both structured and unstructured data is somewhat novel and thorny. This blog is an introduction to some advanced NoSQL and data lake database design techniques (while avoiding common pitfalls) is noteworthy. Data Modeling.

article thumbnail

Data Modeling 201 for the cloud: designing databases for data warehouses

erwin

Accordingly, data modelers must embrace some new tricks when designing data warehouses and data marts. Data modeling for the cloud: good database design means “right size” and savings. Figure 1: Pricing for a 4 TB data warehouse in AWS. Data Modeling. So, let go of any old OLTP design.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

The Madness of Data (and analytics) Governance

Andrew White

The client had recently engaged with a well-known consulting company that had recommended a large data catalog effort to collect all enterprise metadata to help identify all data and business issues. Through the use of AI and ML, these new catalogs would find all the data and create a new data model much more quickly then before.

article thumbnail

Azure Data Sources for Data Science and Machine Learning

Jen Stirrup

Recently, I gave a Make Your Data Work Monday webinar on the complexities of the data sources for data science in Azure, and I thought it important enough to turn into an actual post. How can you differentiate the different opportunities to store your data in Azure? The data is also distributed.

article thumbnail

Generative AI: 5 enterprise predictions for AI and security — for 2023, 2024, and beyond

CIO Business Intelligence

The release of intellectual property and non-public information Generative AI tools can make it easy for well-meaning users to leak sensitive and confidential data. Once shared, this data can be fed into the data lakes used to train large language models (LLMs) and can be discovered by other users.

article thumbnail

An A-Z Data Adventure on Cloudera’s Data Platform

Cloudera

Company data exists in the data lake. Data Catalog profilers have been run on existing databases in the Data Lake. A Cloudera Data Warehouse virtual warehouse with Cloudera Data Visualisation enabled exists. Model building. Model training . Model deployment & serving.

article thumbnail

Three Trends for Modernizing Analytics and Data Warehousing in 2019

Cloudera

Business intelligence (BI), an umbrella term coined in 1989 by Howard Dresner, Chief Research Officer at Dresner Advisory Services, refers to the ability of end-users to access and analyze enterprise data. Big data architecture is used to augment different applications, operating alongside or in a discrete fashion with a data warehouse.