article thumbnail

Build efficient, cross-Regional, I/O-intensive workloads with Dask on AWS

AWS Big Data

Dataset Variables Disk Size Xarray Dataset Size Region ERA5 2011–2020 (120 netcdf files) 53.5GB 364.1 Chakra Nagarajan is a Principal Machine Learning Prototyping SA with 21 years of experience in machine learning, big data, and high-performance computing. The following table summarizes our dataset details.