Recommendation Details: NASA data products based on Earth Science observations are typically in HDF, trending to HDF5, while Earth Science modelers generally prefer to produce data in netCDF, trending to netCDF4. It is not possible to make HDF4 files look exactly like netCDF3 files. On the other hand, netCDF4 is built on HDF5 (netCDF4 is essentially a subset of HDF5), and so it is possible to construct HDF5 files that are accessible from the netCDF4 API, which is a tremendous opportunity for interoperability. While using the netCDF4 API ensures this, the recommendation also provides guidance for those using the HDF5 API to ensure netCDF4 interoperability.

A generic HDF5 format should be used in conjunction with netCDF4:

• HDF5 files can be made nearly indistinguishable from netCDF4 files by adding dimension scales to the HDF5 files in a way that mimics netCDF shared dimensions.
• HDF4 has limitations that create difficulty making it netCDF-compliant. For this reason, we discourage use of HDF4 for new NASA Earth Science Data products and recommend use of the more flexible HDF5.
• HDF5 features that are not supported in netCDF4, either completely or at all, include:
  • Fixed size string arrays
  • Compound data types
  • Datasets with more than one unlimited dimension
  • Long double data types
  • Soft links

These features should be avoided to enable interoperability between HDF5 and netCDF4.

• The Unidata Common Data Model (CDM) attempts to merge netCDF, OPeNDAP and HDF5datamodels. Use of these data model specifications will substantially increase interoperability and minimize data processing time and effort.
• If product developers prefer to use HDF-EOS5, then their HDF-EOS5 products should be made netCDF4-compatible and CF-compliant.

Interoperability between NASA produced HDF5 products and netCDF4 APIs should be validated:

• The netCDF4 compatibility of HDF5 files can be tested by ncdump, ncks, the JPL Web-based Metadata Compliance Checker (MCC), or HDF Product Designer (HPD). Note that the enhanced mode/group hierarchy may not be fully supported by some tools.