...
...
We investigated the most commonly used workflows that can be applied to NASA CUMULUS. We've developed 20 AWS Lambda functions that can enable image services. They can be easily deployed via Serverless Framework.
Recently we learned that Sentinel-2 data are available from AWS Open Data Registry. We wanted to know if we can replicate what Esri did. We tested CoG files for Aerosol Optical Thickness (AOT) with Lambda functions that we've developed for this project. It took only a few days to create an image service out of Sentinel-2 data. This experiment validates that our approach can be easily adapted by NASA DAACs and end-users. It would be great if NASA Earthdata provides an authoritative ArcGIS Image Service, not Esri.
Terra Fusion is NASA ACCESS 2015 project. Terra Fusion is an ultimate test dataset for the existing software stack around netCDF and cloud. Terra Fusion helped us to find issues in Hyrax, THREDDS, GDAL, and ArcGIS. For example, GDAL alone can't handle TerraFusion properly since netCDF swath handling needs improvement. SDT is necessary to read lat/lon and reproject swath to grid. We subsetted MODIS and created an aggregated netCDF. Then, we created a mosaic dataset directly from the aggregated netCDF. Here, there are a lot of technical details but the lesson is that the CF convention alone is not enough to make dataset fully interoperable with GDAL and ArcGIS.
It's worth nothing the two extreme approaches that Terra Fusion and Sentinel-2 took and compare them. What's convenient for hard-core atmospheric data scientists may not work well for general public under different computing environment like cloud. It's time to update NASA Data Producer's guide and DIWG recommendations to addresses the issues when data are put into cloud.
ASDC infused our technology and created several image services in on-prem systems.
...