O'Reilly on Data

DECEMBER 10, 2019

Karpathy suggests something radically different: with machine learning, we can stop thinking of programming as writing a step of instructions in a programming language like C or Java or Python. Nor are building data pipelines and deploying ML systems well understood. Instead, we can program by example. and Matroid.

Software 261

Software Machine Learning Risk Data-driven 261

FineReport

MARCH 24, 2024

Rapid technological advancements and extensive networking have propelled the evolution of data analytics, fundamentally reshaping decision-making practices across various sectors. In this landscape, data analysts assume a pivotal role, tasked with interpreting data to drive informed decision-making.

Statistics 52

Statistics Data mining Visualization Reporting 52

AWS Big Data

JUNE 5, 2023

AWS SDK for pandas is a popular Python library among data scientists, data engineers, and developers. It simplifies interaction between AWS data and analytics services and pandas DataFrames. In the previous post , we discussed how you can use AWS SDK for pandas to scale your workloads on AWS Glue for Ray.

Measurement 65

Measurement Management Interactive Analytics 65

CIO Business Intelligence

MARCH 31, 2022

There are many practical reasons driving this movement away from the data center as a hallmark of information technology. Cloud service providers continue to roll out amazing products and time-saving services that make it simple to stand up amazing code. It was simpler to just buy my own machine than rewrite the code.

Data Processing 109

Data Processing IT Software Reporting 109

AWS Big Data

JANUARY 24, 2024

AWS-powered data lakes, supported by the unmatched availability of Amazon Simple Storage Service (Amazon S3), can handle the scale, agility, and flexibility required to combine different data and analytics approaches.

Snapshot 98

Snapshot Data Lake Metadata Optimization 98

AWS Big Data

APRIL 3, 2023

Tens of thousands of customers run business-critical workloads on Amazon Redshift , AWS’s fast, petabyte-scale cloud data warehouse delivering the best price-performance. With Amazon Redshift, you can query data across your data warehouse, operational data stores, and data lake using standard SQL.

Data Warehouse 96

Data Warehouse Testing Data Lake Data-driven 96

AWS Big Data

AUGUST 16, 2023

Automated Data Analytics (ADA) on AWS is an AWS solution that enables you to derive meaningful insights from data in a matter of minutes through a simple and intuitive user interface. ADA offers an AWS-native data analytics platform that is ready to use out of the box by data analysts for a variety of use cases.

Data Analytics 74

Data Analytics Analytics Visualization Software 74

IBM Big Data Hub

SEPTEMBER 19, 2023

Though you may encounter the terms “data science” and “data analytics” being used interchangeably in conversations or online, they refer to two distinctly different concepts. Meanwhile, data analytics is the act of examining datasets to extract value and find answers to specific questions.

Data Science 82

Data Science Data Analytics Prescriptive Analytics Analytics 82

AWS Big Data

APRIL 8, 2024

In this post, we share how this new feature helps you simplify the way you use Amazon DataZone to enable secure and governed sharing of your data in the AWS Glue Data Catalog. We also delve into how data producers can share their AWS Glue tables through Amazon DataZone without needing to register them in Lake Formation first.

Finance 76

Finance Sales Publishing Metadata 76

AWS Big Data

FEBRUARY 26, 2024

When you start the process of designing your data model for Amazon Keyspaces, it’s essential to possess a comprehensive understanding of your access patterns, similar to the approach used in other NoSQL databases. It empowers businesses to explore and gain insights from large volumes of data quickly.

Dashboards 61

Dashboards Testing Metrics Optimization 61

AWS Big Data

MARCH 17, 2023

Amazon Athena is a serverless and interactive query service that allows you to easily analyze data in Amazon Simple Storage Service (Amazon S3) and 25-plus data sources, including on-premises data sources or other cloud systems using SQL or Python. H3 divides the globe into equal-sized regular hexagons.

Visualization 91

Visualization Machine Learning Consulting Data Warehouse 91

Domino Data Lab

MAY 1, 2019

This Domino Data Science Field Note provides highlights and excerpted slides from Chloe Mawer ’s “ The Ingredients of a Reproducible Machine Learning Model ” talk at a recent WiMLDS meetup. Mawer is a Principal Data Scientist at Lineage Logistics as well as an Adjunct Lecturer at Northwestern University. Introduction.

Machine Learning 81

Machine Learning Testing Data Science Modeling 81

AWS Big Data

DECEMBER 12, 2023

As data engineering becomes increasingly complex, organizations are looking for new ways to streamline their data processing workflows. Many data engineers today use Apache Airflow to build, schedule, and monitor their data pipelines. You can use standard SQL to interact with data.

Data Processing 87

Data Processing Management Statistics Interactive 87

AWS Big Data

APRIL 24, 2023

Building a data lake on Amazon Simple Storage Service (Amazon S3) provides numerous benefits for an organization. However, many use cases, like performing change data capture (CDC) from an upstream relational database to an Amazon S3-based data lake, require handling data at a record level.

Data Lake 79

Data Lake Data Governance Cost-Benefit Machine Learning 79

Darkhorse

DECEMBER 11, 2017

Analytics and now Data Science are trapped in the middle. A recent HBR article put it at 100% for data science projects. Data Steward – this skillset is alive and well in most organizations. What they should be talking about is how to get the data out more quickly and cleanly. That’s abysmal. He adds scale.

Analytics 80

Analytics Visualization Data Warehouse Data-driven 80

bridgei2i

JANUARY 25, 2022

The Basic Guide to Data Science at the Workplace. The Basic Guide to Data Science at the Workplace. In this podcast episode, AI enthusiasts Anirudh and Janci talk about how sincere curiosity and passion for learning can help anyone understand data science and become an active contributor. And now you’re a data scientist.

Data Science 52

Data Science Machine Learning Data-driven Statistics 52

Ontotext

JANUARY 21, 2021

The Financial Industry Business Ontology (FIBO) is a standard that is being developed and published by the Enterprise Data Management Council that attempts to capture business domain knowledge using sophisticated knowledge representation techniques and linked open data technologies. In this way, FIBO can give meaning to any data (e.g.,

Modeling 72

Modeling Finance Technology Insurance 72

O'Reilly on Data

OCTOBER 19, 2021

As with many burgeoning fields and disciplines, we don’t yet have a shared canonical infrastructure stack or best practices for developing and deploying data-intensive applications. Why: Data Makes It Different. If you peek under the hood of an ML-powered application, these days you will often find a repository of Python code.

IT 346

IT Testing Experimentation Software 346

Insight

NOVEMBER 14, 2019

The prevalent use of online platforms for interaction and the large size of the text data from users’ input makes digesting data increasingly more time-consuming. Data The company shared 180k students’ reflections that, based on the company’s rubric system, were considered high quality (having a strategy/strategies).

Testing 54

Testing Modeling Strategy Interactive 54

AWS Big Data

MAY 3, 2023

In this post, we walk through creating a new PySpark project that analyzes weather data from the NOAA Global Surface Summary of Day open dataset. Deploy the code and dependencies to Amazon Simple Storage Service (Amazon S3). It requires Python version >= 3.7 We’ll use the EMR CLI to do the following: Initialize the project.

Data Processing 80

Data Processing Management Testing IT 80

Domino Data Lab

JULY 22, 2019

Paco Nathan presented, “Data Science, Past & Future” , at Rev. At Rev’s “ Data Science, Past & Future” , Paco Nathan covered contextual insight into some common impactful themes over the decades that also provided a “lens” help data scientists, researchers, and leaders consider the future.

Data Science 109

Data Science Machine Learning Data Governance Modeling 109

AWS Big Data

JULY 6, 2023

Extracting time series on given geographical coordinates from satellite or Numerical Weather Prediction data can be challenging because of the volume of data and of its multidimensional nature (time, latitude, longitude, height, multiple parameters). It has not been specifically designed for heavy data transformation tasks.

Machine Learning 128

Machine Learning Visualization IoT Digital Transformation 128

AWS Big Data

JUNE 29, 2023

Amazon Kinesis Data Analytics makes it easy to transform and analyze streaming data in real time. In this post, we discuss why AWS recommends moving from Kinesis Data Analytics for SQL Applications to Amazon Kinesis Data Analytics for Apache Flink to take advantage of Apache Flink’s advanced streaming capabilities.

Data Analytics 73

Data Analytics Analytics IoT Data Lake 73

AWS Big Data

FEBRUARY 21, 2024

Towards the end of 2022, AWS announced the general availability of real-time streaming ingestion to Amazon Redshift for Amazon Kinesis Data Streams and Amazon Managed Streaming for Apache Kafka (Amazon MSK) , eliminating the need to stage streaming data in Amazon Simple Storage Service (Amazon S3) before ingesting it into Amazon Redshift.

Analytics 105

Analytics Management Data-driven Data Integration 105

AWS Big Data

SEPTEMBER 13, 2023

The Analytics specialty practice of AWS Professional Services (AWS ProServe) helps customers across the globe with modern data architecture implementations on the AWS Cloud. In this post, we discuss a common use case in relation to operational data processing and the solution we built using Apache Hudi and AWS Glue.

Data Lake 89

Data Lake Data Processing Metadata Snapshot 89

Domino Data Lab

JANUARY 11, 2021

Data science is an exciting field, but it can be intimidating to get started, especially for those new to coding. Even for experienced developers and data scientists, the process of developing a model could involve stringing together many steps from many packages, in ways that might not be as elegant or efficient as one might like.

Modeling 145

Modeling Metrics Testing Data Science 145

Sisense

FEBRUARY 20, 2020

Analytics is constantly evolving; as datasets become bigger and more complex, only AI, materialized views, and more sophisticated coding languages will be able to glean insights from them. Sentiment analysis is a classification problem where data teams attempt to predict whether text is positive or negative in tone. Preparing Data.

Testing 93

Testing Modeling Machine Learning Visualization 93

Cloudera

OCTOBER 26, 2022

Data practitioners are at the top of the list of employees who are now able to put more focus on innovation. . Cloudera has seen a lot of opportunity to extend even more time saving benefits specifically to data scientists with the debut of Applied Machine Learning Prototypes (AMPs). What are AMPs and why do they help?

Machine Learning 76

Machine Learning Deep Learning Cost-Benefit Forecasting 76

AWS Big Data

NOVEMBER 21, 2023

This holistic approach not only enhances the customer experience but also offers efficiency gains, ensures data privacy compliance, and drives customer retention and sales growth. RAG allows you to retrieve relevant data from databases or document repositories to provide helpful context to large language models (LLMs).

Modeling 70

Modeling Big Data Data-driven Data Processing 70

Cloudera

NOVEMBER 3, 2021

Have you ever asked a data scientist if they wanted their code to run faster? According to a poll in Kaggle’s State of Machine Learning and Data Science 2020 , A Convolutional Neural Network was the most popular deep learning algorithm used amongst polled individuals, but it was not even in the top 3. In fact only 43.2%

Deep Learning 116

Deep Learning Machine Learning Data Science Cost-Benefit 116

AWS Big Data

FEBRUARY 6, 2023

Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is a managed orchestration service for Apache Airflow that makes it simple to set up and operate end-to-end data pipelines in the cloud at scale. A data pipeline is a set of tasks and processes used to automate the movement and transformation of data between different systems.­

Management 83

Management Interactive Metadata Publishing 83

Domino Data Lab

OCTOBER 23, 2019

It covers questions to consider as well as collecting, prepping and plotting data. Collecting and prepping data are core research tasks. While the most ideal situation is to start a project with clean well-labeled data, the reality is that data scientists spend countless hours on obtaining and prepping data.

Statistics 59

Statistics Machine Learning Testing Modeling 59

Data Science 101

NOVEMBER 8, 2023

You are welcome to follow along and let me know on Threads if you would like to be interviewed. Threads Dev Interviews I am finding developers on Threads and interviewing them, right on Threads. Note: The views in these interviews are personal views and do not represent the interviewee’s employer. and that I was a good learner.

Data Science 59

Data Science Visualization Management Data Processing 59

Occam's Razor

OCTOBER 21, 2013

That of course will mean more referring keyword data will disappear. At the moment it is not clear whether Bing, Baidu, Yandex and others will move to similarly protect users’ search privacy; if and when they do, the result will be loss of even more keyword-level user behavior data. Alternatives For Keyword Data Analysis.

Optimization 154

Optimization Reporting Experimentation Metrics 154

AWS Big Data

SEPTEMBER 25, 2023

This blog post shows how to improve security in a data pipeline architecture based on Amazon Managed Workflows for Apache Airflow (Amazon MWAA) and Amazon Elastic Kubernetes Service (Amazon EKS) by setting up fine-grained permissions, using HashiCorp Terraform for infrastructure as code. The job execution remained on Amazon EKS.

Consulting 94

Consulting Data Processing Software Management 94

AWS Big Data

MAY 9, 2023

It allows you to visually compose data transformation workflows using nodes that represent different data handling steps, which later are converted automatically into code to run. AWS Glue Studio recently released 10 more visual transforms to allow creating more advanced jobs in a visual way without coding skills.

Visualization 73

Visualization Marketing Big Data IT 73

AWS Big Data

MARCH 22, 2023

With this protocol, the attributes from Azure AD are passed along to IAM Identity Center, which inherits the defined attribute for the user’s profile in IAM Identity Center. Amazon QuickSight is cloud-native, scalable business intelligence (BI) service that supports identity federation.

Management 94

Management Metadata Enterprise Testing 94