article thumbnail

How SafeGraph built a reliable, efficient, and user-friendly Apache Spark platform with Amazon EMR on Amazon EKS

AWS Big Data

This resulted in engineers choosing various versions of distro, some of which hadn’t been tested with our internal tools. With EMR on EKS, we can keep a new version of Spark distro hidden from users before testing it with our internal toolings and systems and make a formal release. These versions are all exposed to users via their UI.