Remove Data Transformation Remove Machine Learning Remove Testing Remove Workshop
article thumbnail

Amazon EMR on EKS widens the performance gap: Run Apache Spark workloads 5.37 times faster and at 4.3 times lower cost

AWS Big Data

Also, you can run other types of business applications, such as web applications and machine learning (ML) TensorFlow workloads, on the same EKS cluster. We also share a Spark benchmark solution that suits all Amazon EMR deployment options, so you can replicate the process in your environment for your own performance test cases.

Testing 76
article thumbnail

Extract time series from satellite weather data with AWS Lambda

AWS Big Data

It has not been specifically designed for heavy data transformation tasks. Step Functions helps developers use AWS services to build distributed applications, automate processes, orchestrate microservices, and create data and machine learning (ML) pipelines. Note that Lambda is a general purpose serverless engine.

article thumbnail

Improve observability across Amazon MWAA tasks

AWS Big Data

The most common use case for Airflow is ETL (extract, transform, and load). Operationalizing machine learning (ML) is another growing use case, where data has to be transformed and normalized before it can be loaded into an ML model. To run the scripts, refer to the Amazon MWAA analytics workshop.