Multidimensional Data and Tools Assessment

Working with Multi-Dimensional Data in a GIS

NASA distributes scientific data in a variety of file formats, including Network Common Data Form (netCDF), Hierarchical Data Format (HDF), and Gridded Binary (GRIB). These file formats are optimized for storing multidimensional scientific data and the associated metadata. A Multidimensional Raster can have spatial, temporal, and vertical dimensions. You can think of this as the three dimensions of a cube with x, y, and z coordinates or as four dimensions adding in a temporal component with x, y, z, and t coordinates. GIS workflows have typically dealt only with two-dimensional data; however with the addition of multidimensional tools in many GIS programs, such as ArcGIS or QGIS, there is now support for these multidimensional formats which allows for scientific data management and analysis.

There are two ways to make a mosaic dataset that is multidimensional-aware. You can create a mosaic dataset by adding rasters using HDF, netCDF, or GRIB raster file types which will import dimensional data and variable information into the mosaic dataset. The other option is to build out multidimensional information table in order to generate required muiltdimensional fileds, metadata, and structure to make the dataset muiltidimensional-aware. In order to do this you will need to have a field in your mosaic dataset footprint table that identifies the variable in each raster and there needs to be either a time or height field to define one or more dimensions

Mosaic datasets can manage one or multiple variables in ArcGIS Pro. Whether you should put all variables in one mosaic dataset or create one mosaic dataset per variable depends on the application.

• If your application only uses one variable, or a few independent variables, one mosaic per variable is easy to create, use, and manage.
• If your application involves computing from multiple variables using a raster function template, you need to add all the variables used by the template in one mosaic.
• If you want to serve scientific data and want to minimize the number of services, you can add multiple variables to a mosaic and use the variable selector template to access each variable.

netCDF and HDF

Some netCDF or HDF data stores its geolocation as irregularly spaced arrays. When adding to a mosaic dataset, the data is automatically converted to square pixels for display purposes only. The cell size is estimated, but users can change this in addition to setting an interpolation method. Supported interpolation methods include:

• Nearest Neighbor
• Bilinear
• Linear Tinning
• Natural Neighbor

GRIB

Some GRIB products store variables using a parameter code instead of an actual name. A GRIB TAB file with the extended metadata information is required to serve as an interpreter for the code. The text file—normally provided by the organization who produced the data—has a .TAB extension and contains the parameter code, name, center, subcenter, and the table version that produce the data. For example, NASA’s NLDAS dataset requires a GRIB TAB file to interpret variable code 153.

Create Multidimensional Dataset

Multidimensional Toolbox in ArcGIS Pro

Analysis Tools (Examples to Follow) once access to ArcGIS Pro 2.5

Aggregate Multidimensional Raster(Spatial Analyst) (Image Analyst)- Generates a multidimensional raster dataset by aggregating existing multidimensional raster variables along a specific dimension. Choose an interval using a keyword, a value, or a range of values

This tool produces a multidimensional raster dataset in Cloud Raster Format (CRF).

Find Argument Statistics (Spatial Analyst) (Image Analyst)- Extracts a dimension or band index at which a given statistics for each pixel in a multidimensional or multiband raster.

Generate Multidimensional Anomaly (Spatial Analyst)(Image Analyst)- Computes the anomaly for each slice in a multidimensional raster to generate a multidimensional raster. An anomaly is the deviation of an observation from its standard or mean value

This tool produces a multidimensional raster dataset in Cloud Raster Format (CRF).

Generate Trend Raster (Spatial Analyst) (Image Analyst)- Estimates the trend for each slice in a multidimensional raster to generate a multidimensional raster.

Predict Using Trend Raster (Spatial Analyst) (Image Analyst)- Computes a forecasted multidimensional raster using the output trend raster from the Generate Raster Tool.

Build Multidimensional Info (2.5)- Generates multidimensional metadata in the mosaic dataset, including information regarding variables and dimensions. When you create a mosaic dataset from netCDF, GRIB, or HDF files, the multidimensional information is built into the mosaic dataset, and you can view the information in the dataset properties. The Build Multidimensional Info tool creates multidimensional information for a mosaic dataset that contains images collected at multiple times, depths, or heights, but was not created from netCDF files.

Build Multidimensional Transpose(2.5)- Transposes a multidimensional raster dataset, which divides the multidimensional data along each dimension to optimize performance when accessing pixel values across all slices. The input multidimensional raster must be in Cloud Raster Format (CRF).

Make Multidimensional Raster Layer(2.5)- Creates a raster layer from a multidimensional raster layer by slicing data along defined variables and dimensions. This tool creates a multidimensional raster layer from a multidimensional raster or mosaic dataset by extracting a subset of variables. For example, you have a mosaic dataset that contains 30 years of monthly precipitation data, and you only want to extract data for each January to see how precipitation has changed for that month

 Make OPeNDAP Raster Layer- Creates a raster layer from data stored on an OPeNDAP server. The performance of this tool depends on the speed of the remote OPeNDAP server and the robustness of the Internet connection between the machine executing the tool and the remote server.

Select by Dimension- Updates a netCDF layer display or netCDF table view based on the dimension value. Inputs for this tool can be created using the Make NetCDF Feature Layer, Make NetCDF Raster Layer, Make Mosaic Layer, Make Image Server Layer or Make NetCDF Table View too

Subset Mutidimensional Raster(2.5)- Creates a subset of a multidimensional raster by slicing data along defined variables and dimensions. This tool produces a multidimensional raster dataset in Cloud Raster Format (CRF).

Creating a Cloud Raster Format (CRF) from a Mosaic Dataset

ArcGIS Multidimensional Cloud Raster Format is designed for multdimensional raster storage and analysis. The enhanced CRF format allows for optimized fast slicing operation and transposing for optimal profiling operation.

Users should use a Multidimensional mosaic dataset when you want to aggregate data from multiple files or publish a service with one or more processing templates. Users should convert to a Cloud Raster Format (CRF) to optimize temporal profile option for long time series datasets and for storage in the cloud.

1. Select Analysis > Tools > Copy Raster
2. Select a mosaic dataset in either local or remote database. (e.g., C:\Users\Administrator\Documents\aws_rds_geodb_conn.sde\sdt_db.sdt_user.cer_syn1deg_1hour_terra_aqua_modis)
3. Select output file name (e.g., c:/tmp/crf/ceres.crf)
4. In Properties tab, Check Process as Multidimensional.
5. Check Build Multidimensional Transpose.
6. In Environment tab, select LERCfor compression.
7. Change Width and Height to match raster x and y size. E.g., 360 and 180 for CERES.
8. Upload CRF folder into an S3 bucket.
9. Create S3 connection using Create Cloud Connection tool in Toolbox.
10. Add it into folder in catalog pane.
11. Browse the connection using catalog pane.
12. Select CRF file that you uploaded. Share as web layer in portal as image service.

Trend Analysis for time series imagery

Statistics of multidimensional raster

Subsetting muiltidimensional