We assume that port 8080 for Tomcat server is accessible.

Step-by-step guide

  1. Connect to server using arcgis account via ssh.
  2. Download tomcat 8.5.57 and unpack.
  3. Download tds 5.0.0 beta8 WAR file.
  4. Rename it to thredds.war and put it under webapps/
  5. Create apache-tomcat-8.5.57/bin/setenv.sh.

    CONTENT_ROOT=-Dtds.content.root.path=/home/arcgis/apache-tomcat-8.5.57/content
    JAVA_OPTS="$CONTENT_ROOT $JAVA_OPTS"
  6. (Optional) Edit apache-tomcat-8.5.57/content/thredds/threddsCofnig.xml. Increase binLimit for TerraFusion. The unit is mega bytes and default is 500MB. The example below increases 100 times bigger.

    <name>SDT TDS</name>
    <logoAltText>SDT TDS</logoAltText><Opendap>
    <ascLimit>5000</ascLimit>
    <binLimit>50000</binLimit>
    <serverVersion>opendap/3.7</serverVersion>
    </Opendap>
  7. Edit apache-tomcat-8.5.57/content/thredds/catalog.xml

    • Use the following for east region.

        <datasetRoot path="s3-test" location="s3://basicterrafusion" />
        <dataset name="Test Basic TerraFusion on S3" ID="testS3Swath"
          urlPath="s3-test/TERRA_BF_L1B_O53557_20100112014327_F000_V001.h5"
          dataType="Swath"/>
    • Use the following for west region. Please note that cdms3 is a new way to address the bucket in THREDDS.

        <datasetRoot path="s3-test" location="cdms3:terrafusiondatasampler" />
        <dataset name="Test Basic TerraFusion on S3" ID="testS3Swath"
          urlPath="s3-test/P108/TERRA_BF_L1B_O10204_20011118010522_F000_V001.h5"
          dataType="Swath"/>
  8. Create ~/.aws/config file with the default region. We use us-west-2 for official open data.

    [default]
    region=us-west-2

    If you don't set it properly, you'll get the following error message from THREDDS:

    Error {
     code = 500; message = "null (Service: S3, Status Code: 400, Request ID: null)";};
  9. (Re)start tomcat using ~/apache-tomcat-8.5.57/bin/startup.sh.

Screenshot


It may take several minutes to get a response since TerraFusion file is huge (> 32G).

Known Issues

  TDS fails to read a large dataset like MODIS/ASTER. MOPITT is fine. Pydap has an issue in handling large dataset [4].

Reference

  1. https://github.com/Unidata/tds/blob/master/tds/src/test/content/thredds/tds-s3.xml
  2. https://github.com/Unidata/netcdf-java/issues/433
  3. https://www.unidata.ucar.edu/software/tds/current/reference/Performance.html
  4. https://github.com/pydap/pydap/issues/184