Vision for Legacy HDF (.he4), SMAP and ICESat-2 migration to the Cloud
Feature Summary:
- L1/L2 - Trajectory Subsetter - SMAP special case
- Gridded outputs to .h5 (Cropped grid, Revive un-merged branch? Not-GeoTIFF as is current)
- GeoTIFF formatter (COG, HGA) - chaining
- Reprojection special case? (chaining)
- L1/L2 - Trajectory Subsetter - general chaining for formats (ICESat-2, SMAP, other?) √
Reformat: Vector Data: Shapefile, KML, NetCDF, multi-file: ASCII
- as micro services, lifted mostly from on-prem implementations (or port/rewrite) - (Note size constraints, e.g., ICESat-2 files)
- NetCDF: CF, metadata upgrade from .h5
- ShapeFile, KML format output (.shp, .kml, .kmz)
- ASCII format output? - rewrite to Python, chaining - single file per variable
- CSV
- Choice of delimiters? (Comma, tab)
- “Row of vars” per time-stamp? (see Tabular ASCII)
- “Time, Name, Value” format
or “Time, Value” per variable (name)? - GDAL standard?
- “Tabular-ASCII” - lifted from on-prem, or rewritten to Python
- Textual layout
- “Row of vars” per time-stamp
- Blocked per group and/or frequency (count)
- OPeNDAP / GDAL (HGA): various ASCII format choices ?
- L3/L4 - HOSS + chaining (ICESat-2, SMAP other?) √ (some reconfig, perhaps tweaks to code?)
(Note will be .nc4 output not originally scoped .h5, but hopefully not significant) - ICESat-2 has some notorious quirkiness - Lower-Left orientation, Edge-Aligned dimension scales
- Possible support for missing dimension-scales - use CF-ish config entry for grid-parameters (SMAP)
- see: GeoTagging for a Geo-Located Grid Array
- Reprojection - TBD (I suspect not HGA, rather PyResample based, Data Reprojection Wizard)
- Reformat - GeoTIFF/COG (HGA?), other standard Cloud formats, Zarr
Some preliminary thoughts:
...