Remove creating-an-eks-cluster-with-no-manual-coding
article thumbnail

Use Amazon EMR with S3 Access Grants to scale Spark access to Amazon S3

AWS Big Data

Then, we’ll use the AWS CloudFormation template below to create an Amazon EMR on Amazon Elastic Compute Cloud ( Amazon EC2 ) Cluster, an EMR Serverless application and two different job roles. Before we get started on walking through the Amazon EMR and Amazon S3 Access Grants integration, we’ll set up and configure S3 Access Grants.

article thumbnail

Run interactive workloads on Amazon EMR Serverless from Amazon EMR Studio

AWS Big Data

You can now use EMR Serverless applications as the compute, in addition to Amazon EMR on EC2 clusters and Amazon EMR on EKS virtual clusters, to run JupyterLab notebooks from EMR Studio Workspaces. Enter values for AdminPassword and DevPassword and make a note of the passwords you create. Choose Next. Choose Submit.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Improve reliability and reduce costs of your Apache Spark workloads with vertical autoscaling on Amazon EMR on EKS

AWS Big Data

Amazon EMR on Amazon EKS is a deployment option offered by Amazon EMR that enables you to run Apache Spark applications on Amazon Elastic Kubernetes Service (Amazon EKS) in a cost-effective manner. However, tuning these values is a manual process that can be complex and ripe with pitfalls.

Metrics 75
article thumbnail

How to build your own CDN with Kubernetes

Insight

Design and code to deploy a self-hosted content delivery network. In this blog post, I discuss the design and implementation of kubeCDN , a tool designed to simplify geo-replication of Kubernetes clusters in order to deploy services with high availability on a global scale. a self-hosted CDN based on Kubernetes.

article thumbnail

Nexthink scales to trillions of events per day with Amazon MSK

AWS Big Data

Furthermore, the absence of a streaming platform like Kafka created dependencies between teams through tight HTTP/gRPC coupling. In this post, Nexthink shares how Amazon Managed Streaming for Apache Kafka (Amazon MSK) empowered them to achieve massive scale in event processing.