You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 54 Next »

Chapter 1: Before You Begin

The NASA-developed Earth Observing System (EOS) Common Metadata Repository (CMR) is a spatial and temporal metadata registry that enables the science community to more easily use and exchange NASA's data and services. The CMR's main objective is to enable broader use of NASA's EOS data. It allows users to more efficiently search and access data and services and increases the potential for interoperability with new tools and services. The value of these resources increases as the potential to exchange and inter-operate increases. The CMR has been working with other organizations to provide their Earth science metadata alongside NASA's for users to search and access. The CMR stores metadata from a variety of science disciplines and domains, including Climate Variability and Change, Carbon Cycle and Ecosystems, Earth Surface and Interior, Atmospheric Composition, Weather, and Water and Energy Cycle. 

NASA's Earth science data has already proven itself to be extremely useful in understanding the planet Earth as an integrated system. To help science communities that need data from multiple organizations and multiple disciplines, the CMR provides a uniform view of NASA's data. It allows users to more efficiently search and access data and services and increases the potential for interoperability with new tools and services. For examples of how NASA's Earth science data is helping scientists understand the complexities of our Earth, visit Sensing our Planet and Other Featured Research Articles at https://earthdata.nasa.gov/

The CMR was designed to increase access to NASA Earth science data and services by providing a system with a machine-to-machine interface, that is, an Application Programming Interface (API). This API facilitates the discovery, online access, and delivery for a Data Partner's data holdings. The CMR Data Partners retain complete control over what metadata are represented in the CMR by means of inserting new metadata, modifying existing metadata, removing old metadata, and controlling access to their metadata. The CMR Client Partners develop client applications that access the CMR API and take advantage of the services made available. These clients, such as Earthdata Search (https://search.earthdata.nasa.gov), CMR open search (https://api.echo.nasa.gov/opensearch), Reverb (https:reverb.echo.nasa.gov), etc. allow end users to discover data which has been registered in the CMR's holdings and can be custom made to meet the needs of a general user audience, or a specific science application.

The CMR Concept and Design

NASA's Earth Science Data and Information System (ESDIS) has built the CMR based on Extensible Markup Language (XML) and Web Service technologies. The CMR interfaces with clients and users through its series of Application Program Interfaces (APIs). The CMR is an open system with published APIs available to the CMR Development and User community. 

CMR System Concept

Internally, the CMR specifies APIs and provides middleware components, including data and service search and access functions, in a layered architecture. The figure above depicts the CMR system context in relation to its public APIs. 

The CMR allows Client Partners to use the API to create their own tailored software systems for their purposes. The API allows clients to search and retrieve metadata including but not limited to collections, granules, and services. All CMR metadata is stored as received by the data partners and can be retrieved by the native specification or another specification as requested by the client.  


Key features of the CMR architecture are:

  • Ease of Partner Participation – Designed to be low-cost and minimally intrusive, the CMR offers a set of standard ways for partners to interface with the system through provided web UIs and a metadata exchange approach that accommodates existing partners and technology.
  • Open System / Published APIs – To accommodate independent CMR clients, CMR uses an open system approach and publishes domain APIs. These APIs are independent of the underlying transport protocols used. CMR communicates using WS-I Basic Profile v1.0 compliant web services for legacy services and RESTful web services for CMR ingest, search, and metadata management.
  • Evolutionary Development – The CMR system is being developed incrementally to allow for insight and feedback during the development cycle. Industry trends are followed and the use of commercial, off-the-shelf (COTS) products is optimized.

Security

The CMR system requires Secure Sockets Layer (SSL)-based communication from Client Applications to the CMR API, but does not require, secure communication from CMR to a Client Partner's service. Internally, the CMR system is protected through a layer of software and hardware control mechanisms to preserve the integrity of CMR's holdings. When configuring data access fulfillment, Data Partners are strongly encouraged to utilize SSL communications.

CMR Capability And Functionality

CMR provides an infrastructure that allows various communities to share tools, services, and metadata. It supports many data access paradigms such as navigation and discovery.  It facilitates data access through appropriate Data Partners, decentralizes end user functionality, and supports interoperability of distributed functions. 
Although this Guide focuses on the needs of Client Partners, support is provided for the following different, nonexclusive types of Partners:

  • Data Partners – Organizations that supply metadata representing their data holdings to the CMR system
  • Client Partners – Organizations that participate by developing software applications to access the Earth science metadata in the CMR system 
  • Service Partners – Organizations that participate by advertising their Earth science-related services to the user community via the CMR, which maintains service descriptions in a Service Catalog (either special services, or services that are available as an option on a selected set of granules/collections) and support the user in accessing those services.

The CMR addresses science user needs through a set of well-defined and open interfaces upon which the user community can build its own client applications. In this way, the CMR supports extendable, flexible user interfaces, allowing industry and the science community to drive the progress of available earth science applications. For more complete information about client applications, refer to the companion piece to this Guide, the CMR Data Partner's Guide.
The CMR approach allows users to build their own user interfaces to the CMR, rather than being limited to the data search and access system provided by NASA. For Data Partners, the system offloads the burden of providing the system resources required for searching and gives users the flexibility to support community-specific services and functionality. The CMR's interoperability features allow all participants to benefit from the distributed development of functions, again reducing dependence on NASA resources.

 

CMR as a Spatially Enabled Metadata Search and Retrieval System

The CMR allows Data Partners to define the spatial extent of a granule or a collection with different spatial constructs (for example: point and polygon). These spatial extents may be in either the Geodetic or Cartesian coordinate systems. Orbital data may also be provided to describe a collection or granules spatial extents. A Client Partner can then construct a search using a point, a line, or a polygon (or multiple polygon) spatial type, and the CMR responds with data whose spatial region intersects the described region. 
The CMR provides services for interacting with its catalog of metadata. Queries can be performed in a number of ways; result contents can be specified, and the resulting data sets can be incrementally accessed so that large return sets can be handled gracefully. The system also supports constructing, submitting, and tracking access requests for the data that the metadata represents. The CMR supports both an embedding of a Uniform Resource Locator (URL) within the metadata for accessing the data (which the client simply accesses via Hypertext Transfer Protocol [HTTP]) or File Transfer Protocol (FTP), and a more complicated data access process in which other options are accommodated.

CMR Benefits to Client Partners

The CMR's open system provides earth science data and services to a large, diverse pool of users, enabling scientific community interaction and collaboration. The Client Partners are benefitted in the following ways:

 

  • Ease of Participation - The primary goal of the CMR is to enable organizations to participate in making their resources and capabilities available to the Earth Science community. To facilitate participation by these organizations, the CMR has:
    • Minimized the number of requirements that a partner must meet to participate.
    • Involved partners in the system's development cycle and requirements definition.
    • Selected metadata insert and update mechanisms based on current standard industry practice (for example, XML) that most databases can generate automatically.
    • Provided mapping capabilities to convert from one XML representation into another.
  • Cost to Field - While aggressive in the capabilities it is targeted to support, the CMR minimizes the Cost to Field by continually evaluating performance and functionality against costs, for example, licensing of Commercial Off-the-Shelf (COTS) applications, amount of custom code required, hardware platform requirements, and complexity of networking and installation.
  • Cost to Operate - Once fielded, the CMR seeks to minimize the cost to operate the system by making it easier to use, thereby minimizing the load on operations staff.

Client Partner Skills

Since the CMR uses platform-independent web service definitions for its API, there are no requirements for a client programming language. All examples in this document use curl; however, the code samples provided could be translated to any web service capable language. 
As a CMR Client Partner, you need to be familiar with basic software development and Service Oriented Architecture (SOA) concepts such as:

  • XML and XML Schema (XSD)
  • Client/Server-based programming (client stubs, remote endpoints, etc.)
  • RESTful client and service communication programming
  • Service-based Application Programmer's Interface (API)

Client Partner Tasks

As a Client Partner who is beginning to integrate with CMR, you should expect to perform the steps below which are detailed in later sections: 

  • Creating and managing user accounts and user access
  • Searching for data
  • Retrieving data
  • Fulfilling orders is a service provided by ECHO. - Place this in the Retrieving section.\

CMR System Environments

There are three CMR Systems that you, as a Client Partner, have access to. A short description of each system is given below. For additional information, refer to the links below;

  • CMR Operations - The CMR Operations environment is the CMR production system - monitored 24/7 - which serves the needs of the Earth Science community. The Data Holdings within this system include those which CMR's Data Partners have made available for use by the Earth Science community.  This environment is updated with enhancements and fixes on a monthly cycle with virtually no down time.
  • CMR UAT (User Acceptance Test) - The UAT environment provides a stable test system which serves the needs of the CMR Data, Client, and Service partners. The Data Holdings within this system include those which CMR's Data Partners have made available for their own testing purposes. This environment is updated with the same enhancements and fixes that are going to operations, but are installed two weeks prior to operations delivery. CMR Partners are encouraged to verify the capabilities when a new release is installed.
  • CMR SIT (System Integration Test) The SIT system has been established in order to facilitate an exchange of ideas and provide early insight into the upcoming capabilities.  Not all capabilities will become operational. There is often very little metadata available in this mode, but it is fully functional. The next operational version is released into this system approximately 1 month before its schedule Operational release date.

Chapter 2: Getting Started

This chapter will discuss

  • creating and managing user accounts
  • CMR session management - creating, using, and deleting tokens to provide authorization

If searching for and retrieving publicly available data is the only desired operation, this section can be skipped and the reader can go straight to Chapter 3.

User Accounts

User accounts are used to get access to restricted data, manage privileges, or to interact with other services and tools provided by the CMR or ECHO. User accounts for the CMR system are created and manage by the EARTHDATA Login (URS) system. If you need an account and don't already have one, please click on EARTHDATA Login to create one. Once created you can always go back to EARTHDATA Login to manage it. If you are part of a Data Provider group or other team the team administrator can set up permissions for you to access their restricted data. If you need special privileges you can always contact the CMR operational team at support@earthdata.nasa.gov and they can help you. 

Creating and Managing CMR Sessions

The CMR uses tokens in request messages - the http call to CMR - to validate per request who the requester is and what privileges they have. For most searches, a token is not needed because the metadata records are open to everyone. When certain metadata records are restricted a token is needed so that privileged users can see and access those records.  A Session is nothing more than a series of requests that use the same token meaning that you can use the same token for many requests before you delete it. All tokens expire at the end of a time period; At the time of this writing the duration is 30 days. Because the token is used to track your session, it must be protected by client applications with the same level of security that you use for your login name and password.

To conduct a session the normal steps are:

  • Create a token
  • Do one or more of the following in any order:
    • Search for records
    • Retrieve records
  • Delete the token 

Create a Token

Information on creating EDL tokens can be found at https://urs.earthdata.nasa.gov/documentation/for_users/user_token

 

Now that the client partner has created a token, they can search and retrieve records, and conduct other functionality through the CMR or ECHO APIs. This functionality is covered in later chapters of this document. Once finished interacting with the CMR the token can be deleted.

Delete the Token

Information on deleting EDL tokens can be found at https://urs.earthdata.nasa.gov/documentation/for_users/user_token


 

Interacting with the CMR

The REST way

You can interact with any CMR-REST resource using guest by simply GETing, POSTing, PUTing or DELETEing the resource. Some of these operations will fail if guest does not have the authority to perform them. In cases, where you want to use a registered user you should acquire a CMR token above and attach it as a header to the request you make. For example,

 

Getting a list of providers with a CMR token
Request headers:
Content-Type: application/xml
CMR-Token: CMR-TOKEN-ID
 
Request:
GET https://api.cmr.nasa.gov/cmr-rest/providers
 
Response headers:
Status Code: 200 OK
 
Response Body:
<?xml version="1.0" encoding="UTF-8"?>
<providers type="array">
  <provider>
   ...
  </provider>
</providers>

 

The SOAP way

Interacting with the rest of the CMR API follows the same pattern as logging in and logging out except that it requires that you pass a valid CMR token to each operation. The following example shows logging in, retrieving the version number of the CMR system and logging back out.

 

Code Listing 4: Getting the CMR Version Number
// Login
String token = authenticationService.login("jdoe", "mypass", new ClientInformation("A Client", "192.168.1.1"), null, null);
// Print out CMR version number.
System.out.println("CMR's version number: " + authenticationService.getCMRVersion(token));
// Logout using token from previous login
authenticationService.logout(token);
token = null;

Chapter 3: Searching for metadata

Client partners can search the CMR for metadata.  Currently clients can search for collection and granule metadata.  In the future clients will also be able to search for metadata describing services, visualizations, parameters (variables), and documents.

CMR Environment URLs

The CMR system has three environments: The Systems Integration Test environment is where the CMR development team tests new functionality. This is the environment that first gets the newest upgrades, but it is the least stable. Once the CMR software has been tested it gets deployed to the User Acceptance Test environment. The environment here is quite stable and it is tested as a system for a couple of weeks before the software is deployed to the operational environment. Client Partners can test their software in either the SIT or UAT environment depending the level of integration and testing. The operational environment is the live system available to users around the world.  All of the examples provided in the rest of the document are using the Systems Integration Test environment.  To run the commands in the other environments just replace the SIT URL with either the UAT or OPS URL to use the API in the respective environments.

CMR Environment

Base API URL

Operational (OPS)

https://cmr.earthdata.nasa.gov

User Acceptance Test (UAT)

https://cmr.uat.earthdata.nasa.gov

Systems Integration Test (SIT)

https://cmr.sit.earthdata.nasa.gov

Headers

Headers are a part of HTTP requests and for the CMR they provide information such as the content of the message (Content-Type), tokens to allow increased privileges (Echo-Token), the format of the data that gets returned (accept), etc. Content-Type is a standard HTTP header that specifies the content type of the body of the request for POST method messages. Search and retrieval requests support the following Content-Types. If the Content-Type is not specified, XML is assumed.

 Content-Type headers

Body formatContent-Type
XMLapplication/xml

JSON

application/json

 

The Echo-Token allows the CMR to know who is making a request. The Token is in the format of XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX.  A token must first be generated as described in the previous section. Once the requester has the token, the token can be placed into the http header for the necessary API calls.

If the caller wishes to control in what format and or specification the data gets returned they can use the Accept header.  The following table lists the valid values.  If this header or an alternative method is not used, the returned outcome will be a reference list of results in XML format.

Accept Headers

Type Received
Accept HeaderValue
Comments
XML

application/xml

returns a reference list of results using the XML format
JSONapplication/jsonreturns an expanded list of results using the JSON format
echo10application/echo10+xmlreturns an expanded list of results in the echo 10 specification using the XML format
isoapplication/iso19115+xmlreturns an expanded list of results in the ISO 19115-2 (MENDS) specification using the XML format
iso19115application/iso19115+xmlreturns an expanded list of results in the ISO 19115-2 (MENDS) specification using the XML format
difapplication/dif+xmlsupported for collections only and returns an expanded list of results in the DIF 9 specification using the XML format
dif10application/dif10+xmlsupported for collections only and returns an expanded list of results in the DIF 10 specification using the XML format
csvtext/csvsupported for granules only and returns an expanded list of results in a comma separated value format
atomapplication/atom+xmlreturns an expanded list of results in the ATOM specification using the XML format
opendataapplication/opendata+jsonsupported for collections only and returns an expanded list of results in the open data specification using the JSON format
kmlapplication/vnd.google-earth.kml+xmlreturns an expanded list of results using the KML specification in the XML format
nativeapplication/metadata+xmlreturns an expanded list of results in their individual native specification using the XML format

For more information about the types please see the CMR API documentation.

Client-Id is another header that allows the client to specify a name.  This helps the CMR operations team monitor query performance per client and it can also make it easier for them to identify your requests if you contact them for assistance.

Following are some examples for using the headers. The purple part of the example will be explained in this section, the rest will be described later:

The following curl command issues a search request with the search parameters contained in a file called searchterms in the current directory. The Content-Type - the specification and format of the searchterms file - is using the json format to specify the search parameters. The accept header states that we want the results as a reference list using the XML format. The Echo-Token header allows the CMR to know who is making the request for authorization purposes. The Client-Id header allows the operations team to monitor performance and allows them to quickly find your requests should you need help.

curl -v -XPOST -H "Content-Type: application/json" -H "Echo-Token: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Accept: application/xml" -H "Client-Id: Test_Team" -d @searchterms -i https://cmr.sit.earthdata.nasa.gov/search/collections

 

In the next example a user is issuing a search request using publicly available data and is returned an expanded list of results. Notice that only the Accept header is needed.

curl -v -H "Accept: application/metadata+xml" -i https://cmr.sit.earthdata.nasa.gov/search/collections

In the next example a user is issuing a search request using publicly available data and the default result reference list.  Notice that no headers are needed.

curl -v -i https://cmr.sit.earthdata.nasa.gov/search/collections

As an alternate to using the Accept header are extensions where the client can use the Type Received name in the query to get the same results. Instead of using "Accept: application/opendata+json", "opendata" can be used at the end of the main query before parameters are specified. 

curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/collections.opendata"

Other examples use the DIF 10 specification and the ISO specification respectively.

curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/collections.dif10"

curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/collections.iso"

To change what is wanted in the request just replace the header as needed per the tables above.

Results

There are several types of results that can be returned:

  1. A reference list of results and 
  2. An expanded list of results

The following is an example of a reference list of results

<results>

    <hits>2215</hits>

    <took>16</took>

    <references>

        <reference>

            <name>100m Digital Elevation Model Data V001</name>

            <id>C1000000803-DEV08</id>

            <location>https://cmr.sit.earthdata.nasa.gov:443/search/concepts/C1000000803-DEV08>

            <revision-id>8</revision-id>

        </reference>

        <reference>

            <name>100m Digital Elevation Model Data V001</name>

            <id>C1000000719-EDF_OPS</id>

            <location>https://cmr.sit.earthdata.nasa.gov:443/search/concepts/C1000000719-EDF_OPS>

            <revision-id>8</revision-id>

        </reference>

        ...

    </references>

</results>

The results specify

  1. how many metadata records were found by the "hits" tag
  2. how long the query took in milliseconds
  3. a list of metadata record results specified by the "reference" tag

With in each reference tag a limited information about the metadata is provided.

  1. The metadata name which corresponds to the UMM Entry Title
  2. The CMR profile or concept ID -  a CMR generated unique ID.  The ID is encoded by a letter of the profile or concept (C for collection, G for granule, S for service), followed by a CMR generated number, followed by a "-" and then followed by the ID of the metadata provider.
  3. The exact CMR location to download the metadata
  4. The latest revision number of the metadata record.

The following is an example of an expanded list of results in the ECHO 10 specification using the XML format.

<results>

    <hits>2215</hits>

    <took>53</took>

    <result concept-id="C1000000803-DEV08"

        format="application/echo10+xml" revision-id="8">

        <Collection>

            <ShortName>DEM_100M</ShortName>

            <VersionId>1</VersionId>

            <InsertTime>2002-04-27T15:27:55.293Z</InsertTime>

            <LastUpdate>2013-10-04T08:49:26.783Z</LastUpdate>

            <LongName>100m Digital Elevation Model Data</LongName>

...

</Collection>

    </result>

    <result concept-id="C1000000719-EDF_OPS"

        format="application/echo10+xml" revision-id="8">

        <Collection>

            <ShortName>DEM_100M</ShortName>

...
</Collection>
...

  </result>

</results>

The results specify

  1. how many metadata records were found by the "hits" tag
  2. how long the query took in milliseconds
  3. a list of metadata record results specified by the "result" tag

Within each result tag three attributes are shown about the metadata record followed by the full metadata record. The attributes display the CMR concept id (profile ID), the specification and format of the metadata, and the revision number of the shown metadata record.  For more information about the result specifications and formats available please see the CMR API documentation.

Searching

There are several ways to search the CMR system all using the RESTful principles:

  1. Using the API calls and parameters with the GET or POST methods
  2. Using a JSON query language with a POST method
  3. Using the Alternative Query Language (AQL)

API calls and parameters GET method

The most popular and preferred way is to use the API calls and parameters with the GET or POST methods. For detailed documentation the API documentation is located at https://cmr.earthdata.nasa.gov/search/site/search_api_docs.html.

The CMR uses jetty as its application server and it is currently set to take roughly 500k characters in the URL. Clients using the Search API with query parameters should be careful not to exceed this limit or they will get an HTTP response of 413 FULL HEAD. If a client expects that the query url could be extra long so that it exceeds 500k characters, they should use the POST method for searching instead of the GET method. First we will describe search using the GET method.

The basic search command is shown in the following example 

curl -v -i "https://cmr.earthdata.nasa.gov/search/collections"

The query returns the first 10 publicly available collection results in a reference list using the XML format.  As described in the header section to see restricted data, if you have the privileges you will need to use a token.

There are search parameters that can be applied to provide more functionality.  They are listed below

  • page_size - The number of results per page - the default is 10. 0 and 2000 are the minimum and maximum values respectively.  For example: page_size=100 shows 100 result records per page if that many results exist.
  • page_num - The page number to return. For example: page_num=1 is the first page of results;  page_num=2 is the second page of results; page_num=10 is the 10th page of results.
  • sort_key - Indicates one or more elements to sort on. For example: sort_key[]=platform  (the brackets "[" and "]" may need to be escaped by using the \ character)
  • pretty - Returns formatted - readable - results if set to true.  For example pretty=true.  For all the returned examples in this document this flag is used.
  • token - Specifies the client token. This is an alternative to using the Echo-Token header.
  • echo-compatible - This is used by systems requiring ECHO results. To get the best use out of CMR Client Partners shouldn't use this parameter.

These search parameters are for collection requests only.

  • include_has_granules - Includes a has-granules tag or attribute in the response so the client knows if the collection encompasses any granules. E.g. include_has_granules=true
  • include_granule_counts - Includes a granule-counts tag or attribute in the response with the number of granules represented by the collection. E.g. include_granule_counts=true
  • include_facets - Includes a list of facets and their counts at the end of the results. This is mainly used for collection search displays. E.g. include_facets=true
  • include_facets with hierarchical_facets - Includes a list of facets preserving the hierarchical order. This is mainly used for collection search displays. E.g include_facets=true&hierarchical_facets=true

 In next example box we will see a set of examples conducting a basic search with using the search parameters just described. The first example shows a client wanting to see 50 metadata references per page. The second example shows 50 metadata references per page using the formatted print.  The third example shows page 2 of 20 results per page showing full records in the echo 10 specification using formatted print. The fourth example issues a request including token and client id headers and does a basic search sorting the results using the platform element.  The request returns page 2 results of 20 results per page showing full records in the ISO 19115 specification using the XML format in a formatted fashion. As one can see the ? character separates the URL from the search parameters and the search parameters are separated by the & character.  The parameters can be in any order. 

curl -v -i "https://cmr.earthdata.nasa.gov/search/collections?page_size=50"

curl -v -i "https://cmr.earthdata.nasa.gov/search/collections?page_size=50&pretty=true"

curl -v -i "https://cmr.earthdata.nasa.gov/search/collections.echo10?page_num=2&page_size=20&pretty=true"

curl -v -i -H "Echo-Tocken: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team"  "https://cmr.earthdata.nasa.gov/search/collections.iso?sort_key\[\]=platform&page_num=2&page_size=20&pretty=true"  

Now to refine our searches we can use another set of search parameters documented in the table below. All of these parameters have the brackets next to them and may need to be escaped (\[\]).  All of CMR time search parameters (temporal, updated_since and revision_date) formats are specified as yyyy-MM-ddTHH:mm:ss.SSSZ format. Where yyyy is year; MM is month; dd is day; T is the date time separator character; HH is the hour; mm is the minute; ss is the second; SSS is the milliseconds (the .SSS can be omitted); and Z specifies Zulu time.  January 2 2000 at 5 seconds and 4 minutes past 3 O'clock in the morning Zulu time is represented as 2000-01-02T03:04:05Z.

 

Search ParametersExampleNotesSupports Pattern OptionSupports Case Insensitivity OptionSupports AND Option (ALL values have to be present within an element)Supports OR Option (ANY value has to be present within an element)
concept_idconcept_id\[\]=C123456-LPDAAC_ECS   NONO
echo_collection_id

echo_collection_id\[\]=C1000000001-CMR_PROV2

uses concept_id  NONO
entry_titleentry_title\[\]=this is a title YESYESNONO
dataset_iddataset_id\[\]=this is a titleuses entry_titleN/AN/ANONO
entry_identry_id\[\]=SHORT_V5   NONO
dif_entry_iddif_entry_id\[\]=SHORT_V5matches either entry_id or associated difs  NONO
archive_centerarchive_center\[\]=SEDAC YESYESNONO
temporal

temporal\[\]=2000-01-01T10:00:00Z,2010-03-10T12:00:00Z,30,60

or

temporal\[\]=2000-01-01T10:00:00Z/P10Y2M10DT2H,30,60

format is: begin datetime, end datetime, period, duration

or: begin datetime/ISO 8601 time interval

One can leave out the begin time or end time or both the period and duration

ranges are inclusive unless otherwise specified

N/A N/ANONO
projectproject\[\]=ESI YES YESYESNO
campaigncampaign\[\]=ESI uses projectYES YESYESNO
updated_sinceupdated_since=2000-01-01T01:00:00Z The time is inclusive.NON/ANONO
revision_daterevision_date\[\]=2000-01-01T01:00:00Z, can specify an inclusive rangeNON/A YESNO
processing_level_idprocessing_level_id\[\]=1B YES YESNONO
platformplatform\[\]=AQUA platform short nameYES YESYESNO
instrumentinstrument\[\]=CERES instrument short nameYES YESYESNO
sensorsensor\[\]=CCD sensor short nameYES YESYESNO
spatial_keywordspatial_keyword\[\]=VA YESYESYESNO
science_keywords    NOYES
two_d_coordinate_system_name     NO
two_d_coordinate_system[name]     NO
collection_data_type     NO
provider     NO
short_name     NO
version     NO
polygon     NO
bounding_box     NO
point     NO
line     NO
keyword     NO


In the following example we wish to find all collection metadata records that contain an AQUA platform and we would like the to see only a formatted reference list of results that contain 20 references.

curl -v -i -H "Echo-Tocken: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team"  "https://cmr.earthdata.nasa.gov/search/collections?platform\[\]=AQUA&page_size=20&pretty=true"

In the next example we wish to find all collection metadata records that contain an AQUA or an AURA platform and we would like the to see only a formatted reference list of results that contain 20 references.

curl -v -i -H "Echo-Tocken: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team"  "https://cmr.earthdata.nasa.gov/search/collections?platform\[\]=AQUA&platform\[\]=AURA&page_size=20&pretty=true"

There are a couple of extra options that certain search parameters have to aid the user.  To use these options the syntax is: options[parameter name][option_key]=value. Parameter name is the name of the search parameter to be affected such as platform. Value is either set to true or false.  Option_key is one of the following:

Option nameDescription
ignore_caseIf "ignore_case" is set to true the search will be case insensitive and if set to false the search will be case sensitive.  The default value is true. E.g. ignore_case=true - the search will match on both AQUA and aqua
patternThis is the wildcard capability. If "pattern" is set to true the CMR will treat '*' as matches zero or more characters and '?' matches any single character. For example: platform[]=AQUA will match only on the value 'AQUA'.  if platform[]=A?U*&options[platform][pattern]=true platforms containing A followed by any alphanumeric character followed by U followed by any number of alphanumeric characters will be found.  So AQUA, ASUBB, ADUSD34H, AUU, etc. will all be found. The pattern option defaults to false.
andIf "and" is set to true and if multiple values are listed for the parameter, the metadata records must contain ALL of these values in order to match. The default is false meaning metadata records that match ANY of the values will match.
orThis option only applies to granule attributes or science-keyword searches. If "or" is set to true, the search will find records that match any of the attributes. The default for this option is false.

The following is an example of using options with the platform search parameter.  This example will find any platform that matches A followed by any number of alphanumeric characters and ends with A. This will find both platforms of AQUA and AURA. 

curl -v -i -H "Echo-Tocken: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team"  "https://cmr.earthdata.nasa.gov/search/collections.echo10?platform\[\]=A*A&options\[platform\]\[pattern\]=true&pretty=true"

This next example demonstrates a user looking to find records that only contain an instrument that matches "HELLO" in uppercase letters.

curl -v -i -H "Echo-Tocken: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team"  "https://cmr.earthdata.nasa.gov/search/collections.echo10?instrument\[\]=HELLO&options\[instrument\]\[ignore-case\]=false&pretty=true"

 

Besides science_keywords, if any of the parameters that are searched are repeated, the metadata records that have ANY of the values will match.  The following example demonstrates that the CMR system will match any metadata record containing either value.

 

curl -v -i -H "Echo-Tocken: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team"  "https://cmr.earthdata.nasa.gov/search/collections.iso?concept_id\[\]=C123456-LPDAAC_ECS&concept_id\[\]=C123457-LPDAAC_ECS&pretty=true"

 

For a complete set of examples using all of the search parameters please see the API documentation: https://cmr.earthdata.nasa.gov/search/site/search_api_docs.html

API calls and parameters POST method

The API using the POST method is the same as with the GET method with the exception being the method used is POST instead of GET and the parameters are in the body of the message without a length constraint instead of existing in the URL string.  Using the curl command the following example shows the query.xml file that contains the query we want to execute followed by the curl search request. In the query.xml file the parameters can be left as one long set or formatted to be more easily read - so long as syntax remains the same. Also notice in this example we did not escape the brackets ([ ])

query.xml:

pretty=true&

page_size=1&

page_num=3&

sort_key[]=platform&platform[]=AQUA&platform[]=AURA&revision_date[]=2015-07-01T01:00:00Z,2016-01-01T01:00:00Z&revision_date[]=2014-01-01T01:00:00Z,2014-06-01T01:00:00Z&temporal[]=2000-01-01T10:00:00Z/2010-03-10T12:00:00Z&include_has_granules=true&include_granule_counts=true&include_facets=true&hierarchical_facets=true


curl -v -XPOST -i -d @query.xml "https://cmr.sit.earthdata.nasa.gov/search/collections.echo10"

Notice in this specific instance that the Content-Type header is not used.  Don't use it, it will cause an error.

JSON query language with a POST method

JSON query language with a POST method

 

 


Alternative Query Language (AQL)

The CMR uses the Alternative Query Language (AQL) for its querying capabilities. (See Chapter 5 for a detailed explanation of the language syntax.) To support potentially large queries, the CMR handles a query Alternative  expression without regard to the details of whether and how it will return query results to the user. A CMR query consists of a query expression specified in XML, starting with <query>. All AQL queries must conform to the AQLQueryLanguage.dtd available on the CMR website. Also, refer to Chapter 5, The Alternative Query Language for additional information about AQL.  The AQL schema specification can be found here: https://cmr.sit.earthdata.nasa.gov/search/site/IIMSAQLQueryLanguage.xsd the DTD can be found here: http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd. All of the examples in this section use JAVA, but the client can write their software in any language they choose provided that they can make RESTful calls to the server.

You may use the optional argument MaxResults to limit the numbers of results returned from a query, which is useful when a query produces more data than can be processed. This also saves processing time for the query and presentation of results. 
You may retrieve the results of a query in several different ways. The ExecuteQuery operation on the Catalog Service allows you to indicate whether you would like the data itself returned or simply the number of hits. When doing so, ensure that the CatalogItemID tag is returned. The string value under this tag is the actual catalog item GUID that you can use to order items from the CMR. You can specify this function in the ResultType argument with one of the following values:

Table 2: Query Return Result Types

Value

Description

RESULTS

Returns the detailed metadata for items that match the query directly in the response. When using this option, you may choose to limit the actual metadata values returned. In addition, the CMR will return a result set identifier (ResultSetGUID) for subsequent retrievals of the results or for paging through the result list. Note that CMR Operations limits the maximum number of items returned to 2,000 at a time. The complete results are stored in your result set which you can retrieve by using the GetQueryResults operation.

RESULT_SET_GUID

Returns the result set guid of the results that are stored on the server. The CMR will generate a result set but will not return any results. You must subsequently retrieve the results using the GetQueryResults operation.

HITS

Returns the number of hits (matches) to the query and a ResultSetGUID for the results stored on the server. The CMR will generate a result set but will not return any results. The number of records may be a statistically determined for large result sets. You must subsequently retrieve the results using the GetQueryResults operation. Hits is a relatively expensive operation therefore if the client only needs to know if some data exists, it is faster to simply query for ITEM_GUIDS with a small iterator size.

ITEM_GUIDS

Returns the Catalog Item GUIDs that match the specified query. Note: No ResultSetGUID is returned since results do not persist in the system.

All the GUIDs of the granules/collections that satisfy the query are returned to the client. It is the client‘s responsibility to request the metadata for each individual granule/collection using the GetCatalogItemMetadata operation discussed later.

Formatting Your Query Results

You can specify a subset of the information in the result set by using different parameters for the operations
ExecuteQuery and GetQueryResults
The following elements are used to specify the format and content of a result set:

Table 3: Result Set Content Elements

Argument
Description
IteratorSize

Specifies the number of results to be returned from a single operation. This does not limit the number of items a query may match (see MaxResults) but limits to 2,000 the number of matching items returned in the result set, starting from the Cursor position.

This field is only used if the result type is set to RESULTS.

Cursor

Specifies the index of the first record to be returned in the result set. For example, a value of 5 will return results starting from the fifth record. If none is specified, it defaults to 1. If you repeat the same query later, use the same Cursor value.

This field is only used if the result type is set to RESULTS.

MetadataAttributes

Specifies which fields of the CMR Metadata you actually want to return. By only requesting the parts of the metadata you are interested in, you can increase query performance substantially. By default, the CMR returns all of the metadata for each item.

This field is only used if the result type is set to RESULTS.


Metadata results are returned as XML that conforms to one of the following DTDs:

Granule metadata conforms to the Granule Results DTD—refer to Appendix F, Results DTDs (also located at: http://api.cmr.nasa.gov/cmr/dtd/CMRGranuleResults.dtd).
Collection metadata conforms to the Collection Results DTD Appendix F, Results DTDs (also located at: http://api.cmr.nasa.gov/cmr/dtd/CMRCollectionResults.dtd). 
Metadata attributes are made up of two values: the XML metadata attribute name and a primitive type name. The CMR currently ignores the type name. The allowable metadata attribute names are specified in the appropriate DTD (CMRGranuleResults.dtd for granules and CMRCollectionResults.dtd for collections). If you specify a metadata attribute name that has sub-attributes, all of the sub-attributes will be included as well. For example, if you specify Platform, the following elements will be included in the metadata:

  • Platform
  • PlatformShortName
  • Instrument
  • InstrumentShortName
  • Sensor
  • SensorShortName
  • SensorCharacteristics
  • SensorCharacteristicName
  • SensorCharacteristicValue

OperationMode
When you specify a sub-attribute, the CMR will return the ―parent‖ attribute in the hierarchy as well as the sub-attribute. This allows you to ensure that data are correctly scoped. For example, if you specify Sensor, the following elements will be included in the metadata:

  • Platform
  • PlatformShortName
  • Instrument
  • InstrumentShortName
  • GranuleUR
  • GranuleURMetaData

Detailed spatial attributes cannot be used as MetadataAttributes; only their containing element may be specified. For example, you cannot use BoundingBox as a MetadataAttribute, but you can use HorizontalSpatialDomainContainer. The following spatial elements cannot be specified as MetadataAttributes:

  • Point
  • Circle
  • BoundingRectangle
  • GPolygon
  • Polygon
  • PointLongitude
  • PointLatitude
  • CenterLongitude
  • CenterLatitude
  • Radius
  • WestBoundingCoordinate
  • NorthBoundingCoordinate
  • EastBoundingCoordinate
  • SouthBoundingCoordinate
  • Boundary
  • ExclusiveZone
  • SinglePolygon
  • MultiPolygon
  • OutRing
  • InnerRing

Specifying GranuleURMetaData as a MetadataAttribute is equivalent to not specifying any MetadataAttributes; the result set includes all the elements in the result DTD. 
The following code snippet shows how to execute a query for all of the metadata for matching items.

 

Code Listing 6: Executing a Simple Query
// Execute a query to get results
QueryResponse response = catalogService.executeQuery(userToken, queryString, ResultType.RESULTS,
    10, // Iterator
    0, // Cursor
    3000, // max results
    null); // no metadata attributes specified
 
MetadataAttribute[] attributes = new MetadataAttribute[] { new MetadataAttribute(
    "HorizontalSpatialDomainContainer", null) };
 
QueryResponse response = catalogService.executeQuery(userToken, queryString, ResultType.RESULTS,
    10, // Iterator
    0, // Cursor
    3000, // max results
    attributes);

Handling Large Result Sets

Given the CMR's large store of Earth Science data, it is possible for queries to return very large result sets. The CMR supports retrieving the results from a query in a number of ways. The simplest is to ask the CMR to return the results directly from the ExecuteQuery request by passing RESULTS as the ResultType. However, to prevent a single query from monopolizing CMR resources, the CMR limits the number of results available in response to a query. By default, this limit is 2,000 items. CMR Operations may change this limit depending on CMR usage patterns. 
For larger results, the CMR supports a paging mechanism. This allows you to page through the available data in page sizes that you select (up to the CMR Operations configurable limit). For all ResultTypes, the CMR will create and store a result set and return the corresponding GUID. You can page through the result set using the GetQueryResults operation. The arguments to GetQueryResults are similar to ExecuteQuery with the exception that you specify the result set GUID rather than a new query. 
Result sets may change after they are created. Providers are continually changing the data they have registered in the CMR. New records may appear or may be removed from a result set. Because of this, you should watch the fields Cursor and CursorAtEnd when paging through a large result set: 
Cursor specifies the index of the first record to be returned in the result set. For example, a value of 5 will return results starting from the fifth record. If none is specified, it defaults to 1. If you repeat the same query later, use the same Cursor value. 
Use CursorAtEnd to determine when you have reached the end of the result set. This Boolean field is TRUE if the returned results were the last available results in the result set. 
The following code illustrates paging through a result set and displaying it to the user.

 

Code Listing 8: Paging Through Query Results
final int ITERATOR_SIZE = 10;
try
    {
        CatalogServiceLocator catalogServiceLocator = new CatalogServiceLocator();
        CatalogServicePort catalogService = catalogServiceLocator.getCatalogServicePort();
        QueryResponse response = catalogService.executeQuery(userToken, userQuery,
        ResultType.RESULT_SET_GUID, 0, 0, 1000, null);
        String resultSetGuid = response.getResults().getResultSetGuid();
        // begin paging through results
        int cursor = 1;
        boolean atEnd = false;
        while (!atEnd)
        {
            //Get next ITERATOR_SIZE results
            QueryResults results = catalogService.getQueryResults(userToken,
                resultSetGuid, null, ITERATOR_SIZE, cursor);
            //Print out results
            System.out.println(results.getReturnData());
            //Set cursor to next index
            cursor = results.getCursor();
            //Check if at end of result set
            atEnd = results.isCursorAtEnd();
        }
        System.out.println("All results retrieved");
    }
    catch (CMRFault e)
    {
        e.printStackTrace();
    }
    catch (ServiceException e)
    {
        e.printStackTrace();
    }
    catch (RemoteException e)
    {
        e.printStackTrace();
    }

Like ExecuteQuery, GetQueryResults takes an array of MetadataAttributes. Internally, the CMR only stores in a result set the item IDs that match a given query. This means that you may pull different metadata from a single result set with each call by varying what you pass to the MetadataAttribute array without needing to re-query the CMR . It is highly recommended you use the MetadataAttribute array to restrict the information the CMR returns and thus improve performance.

Visibility of Results

When you execute a query, the query is applied to all the data in the CMR. However, when the results are retrieved, you may not see all of the items. What you can see depends on the rules defined by the Data Partners and the privileges granted to you.

Restricted Items

If a particular item in your result set is restricted for you (i.e., you are not allowed to see it), based on your privileges, it will not be returned.

Deleted Items

It is possible that between the time you execute a query and the time you view the results some of the matched items may have been deleted from the CMR or restricted due to a request from the Data Partner who owns the metadata. In that case, the item will not be returned in your result set. For more information about notification of deleted or restricted order items, refer to section 7.8.1, Restricted or Deleted Order Items.

Querying for Orderable Data

The CMR allows you to exclude from your query data that cannot be ordered. Refer to section 1.1.1.

Searching for Orbit Data

4.4.1 Backtrack Orbit Search Algorithm

Orbit searching is by far the most accurate way to search for level 0-2 orbital swath data. Unfortunately orbital mechanics is a quite difficult field, and the most well known orbit model, the NORAD Propagator, is quite complex. The NORAD Propagator is designed to work with a wide range of possible orbits, from circular to extremely elliptical, and consequently requires quite a bit of information about the orbit to model it well. 
To facilitate earth science, the orbits of satellites gathering earth science data are quite restricted compared to the variety of orbits the NORAD Propagator is designed to work with. Generally, the earth science community would like global coverage, with a constant field of view, at the same time every day. For this reason, most earth science satellites are in a sun-synchronous, near-polar orbit. Even missions that are not interested in global coverage, e.g., the Tropical Rainfall Measuring Mission (TRMM), are still interested in having a constant field of view so the coverage of the sensor is at a constant resolution. For this reason, ALL earth science satellites are in circular orbits. 
The Backtrack Orbit Search Algorithm, designed and developed by Ross Swick, exploits this fact to simplify the orbit model by modeling an orbit as a great circle under which the Earth rotates. This reduces the number of orbital elements required for the model from 22 to three. Moreover, the NORAD Propagator is designed to predict future orbits based on current status, and consequently must be reinitialized periodically to correct for cumulative error as the model spins forward. As the name implies Backtrack spins the orbit backwards, and in practice spins backwards at most one orbit, so there is no cumulative error.
For more information on Backtrack, please see  http://geospatialmethods.org/bosa/.

Figure 2. Typical Orbit Path Represented on a Globe and the same Path on a Map

Backtrack orbit model 

Three parameters to define an orbit: 

  1. Instrument swath width (in kilometers)
  2. Satellite declination or inclination (in degrees)
  3. Satellite period (in minutes)
Orbit data representation

Three parameters to represent orbit data:

  1. Equatorial crossing longitude
  2. Start circular latitude (or start latitude and start direction)
  3. End circular latitude (or end latitude and end direction)

How the CMR Searches for Orbit Data

  • The user specifies a regular spatial window

Figure 3. Spatial Window

 

<granuleCondition>
    <spatial>
        <IIMSPolygon>
            <IIMSLRing>
                <IIMSPoint lon="-90" lat="49" />
                <IIMSPoint lon="-90" lat="39" />
                <IIMSPoint lon="-70" lat="39" />
                <IIMSPoint lon="-70" lat="49" />
                <IIMSPoint lon="-90" lat="49" />
            </IIMSLRing>
        </IIMSPolygon>
        <SpatialType>
            <list>
                <value>ORBIT</value>
            </list>
        </SpatialType>
    </spatial>
</granuleCondition>

 

  • Backtrack then calculates from both ascending and descending a path for equatorial longitude crossings and start/end circular latitudes according to user's query window.

Figure 4. Search Ascending Path
Sample queries

The following are sample queries that you can execute against the CMR. Note that the provider and the datasets used in these samples are representative only; you should modify the query to suit your needs. 

 

Code Listing 9: Sample Collection Query (Discovery Search)
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE query PUBLIC "-//CMR CatalogService (v{*}10{*})//EN" "http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd">
<!- Search for collections from ORNL_DAAC that have parameter value that contains 'IMAGERY'--> 
<query>
    <for value="collections"/> 
        <dataCenterId>
            <list>
                <value>ORNL_DAAC</value>
            </list> 
        </dataCenterId>
    <where>
        <collectionCondition>
            <parameter>
                <textPattern>'%Imagery%'</textPattern>
            </parameter>
        </collectionCondition>
    </where>
</query>

 

 

Code Listing 10: Sample Collection Query from Two Providers (Discovery Search)
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE query PUBLIC "-//CMR CatalogService (v{*}10{*})//EN" "http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd">
<!-- Search for collections from GSFCECS and ORNL_DAAC that have processing level 1A or 2 --> 
<query>
    <for value="collections"/> 
    <dataCenterId>
        <list>
            <value>GSFCECS</value> 
            <value>ORNL_DAAC</value>
        </list>
    </dataCenterId>
    <where>
        <collectionCondition negated="y"> 
            <processingLevel>
                <list>
                    <value>'1A'</value>
                    <value>'2'</value>
                </list>
            </processingLevel>
        </collectionCondition>
    </where>
</query>

 

 

Code Listing 11: Sample Collection Query with Temporal and Spatial Constraints (Discovery)
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE query PUBLIC "-//CMR CatalogService (v{*}10{*})//EN" "http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd">
<!--
 Search for collections from ORNL_DAAC with: temporal range: periodic 
range between Jan 1, 1990 and Dec. 31 1998from the 1st to the 300th day 
of each year, AND spatial extent: bounding box 60S, 70W to 60N, 70E. 
-->
<query>
    <for value="collections"/> 
    <dataCenterId>
        <value>ORNL_DAAC</value> 
    </dataCenterId>
    <where>
        <collectionCondition>
            <temporal>
                <startDate>
                    <Date YYYY="1990" MM="01" DD="01"/> 
                </startDate>
                <stopDate>
                    <Date YYYY="1998" MM="12" DD="31"/> 
                </stopDate>
                <startDay value="1"/> 
                <endDay value="300"/>
            </temporal>
        </collectionCondition> 
        <collectionCondition negated="n">
        <spatial operator="RELATE"> 
            <IIMSPolygon>
                <IIMSLRing>
                    <IIMSPoint long='-10' lat='85'/> <IIMSPoint long='10' lat='85'/> <IIMSPoint long='10' lat='89'/> <IIMSPoint long='-10' lat='89'/> <IIMSPoint long='-10' lat='85'/>
                </IIMSLRing>
            </IIMSPolygon>
        </spatial>
    </collectionCondition>
    </where>
</query>

 

 


Code Listing 12: Sample Collection Query with Complex Temporal and Spatial Conditions (Discovery)
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE query PUBLIC "-//CMR CatalogService (v{*}10{*})//EN" "http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd">
<!-- Search for collections from ORNL_DAAC with
temporal range: periodic range between Jan 1, 1990 and Dec. 31 1998 from the 1st to the 300th day of each year, AND
some days of January. source name: L7 or AM-1 AND
spatially covering any 'temperate' region or USA -->
<query>
    <for value="collections"/> 
    <dataCenterId>
        <list>
            <value>ORNL/value>
        </list>
    </dataCenterId>
    <where>
        <collectionCondition>
            <temporal>
                <startDate>
                    <Date YYYY="1990" MM="01" DD="01"/> 
                </startDate>
                <stopDate>
                    <Date YYYY="1998" MM="12" DD="31"/> 
                </stopDate>
                <startDay value="1"/> 
                <endDay value="300"/>
            </temporal>
        </collectionCondition> 
        <collectionCondition negated='n'>
            <sourceName>
                <list>
                    <value>'L7'</value> 
                    <value>'AM-1'</value>
                </list>
            </sourceName>
        </collectionCondition>
        <collectionCondition>
            <spatialKeywords>
                <list>
                    <value>'temperate'</value>
                    <value>'USA'</value>
                </list>
            </spatialKeywords>
        </collectionCondition>
        <collectionCondition>
            <temporalKeywords>
                <textPattern>'%january%'</textPattern>
            </temporalKeywords>
        </collectionCondition>
    </where>
</query>

 

 

Code Listing 13: Sample Collection Query Using Provider Specific Attributes (Discovery)
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE query PUBLIC "-//CMR CatalogService (v10)//EN" 
"http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd"> 
<query>
    <for value="collections"/> 
    <dataCenterId>
        <value>ORNL_DAAC</value> 
    </dataCenterId>
    <where>
        <collectionCondition>
            <additionalAttributeNames>
                <list> 
                    <value>'Provider_Specific_Attribute_1'</value> 
                    <value>'Provider_Specific_Attribute_3'</value>
                </list>
            </ additionalAttributeNames > 
        </collectionCondition>
    </where>
</query>

Chapter 4: Retrieving Metadata

 

Chapter 5: Alternative Query Language

CMR Alternative Query Language (AQL) is a query language that defines the format for searches on collections and granules in the CMR system. AQL is an XML-based language. You can find the DTD for AQL at http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd or the schema at https://cmr.sit.earthdata.nasa.gov/search/site/IIMSAQLQueryLanguage.xsd.

 

 

Code Listing 14: General AQL Structure
<?xml version="1.0" encoding="UTF-8"?> 
<!DOCTYPE query PUBLIC "-//CMR CatalogService (v10)//EN" 
"http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd"> 
<query> 
    <for value= "collections"/> 
    //"granules" can be substituted for "collections" 
        <dataCenterId> 
            <all/>
        </dataCenterId> 
    <where> 
        <list of conditions ... > 
    </where> 
</query>

 

 

Note: The <for/> tag shown in this example does not require that you set a value attribute; if you do not explicitly set this attribute, it will default to "collections." Your query will validate correctly, but you may still experience an error if you use granule settings for other attributes within the same query. 

 

There is no discussion regarding the strong suggestion that clients include dataset IDs. Might want to have more high level information regarding the structure of an AQL query and general contructs (list, all, value, etc). 
You can specify the data centers that you wish to search by explicitly listing them or by using the empty element <all/> to indicate that you want all the data centers searched. This is the default condition, that is, if you do not specify a condition for the dataCenterId element, the CMR will search all data centers. Do not enclose dataCenterId values in single-quotes. Is this still needed? 
To search all data providers:

 

<dataCenterId><all/></dataCenterId>

 

To search for collections/granules in the ORNL data provider's metadata only:

 

<dataCenterId> 
    <value>ORNL_DAAC</value>
</dataCenterId>

 

To search for collections/granules in the LPDAAC_ECS data providers' metadata only:

 

<dataCenterId>
<list> <value>ORNL_DAAC</value> <value>LPDAAC_ECS</value>
</list>
</dataCenterId>

 

The list of conditions consists of collectionCondition elements for Discovery or granuleCondition elements for Inventory Search. The fields you can search on within collections occur as children of the collectionCondition element. The fields you can search on within granules occur as children of the granuleCondition element. The CMR will return only those granules or collections that satisfy all the conditions of the query, that is, the CMR applies the Boolean AND between all conditions of the query. 
The where tag can have any number of collectionCondition or granuleCondition elements. Each of these tags can contain any one of the search elements listed in the Inventory Search and Discovery Search tables below. For the query to be valid, each of the search elements may appear only once.

Code Listing 15 shows the general structure of a Discovery query with multiple constraints.

 

Code Listing 15: Structure of a Discovery Query with Multiple Collection Conditions
<?xml version="1.0" encoding="UTF-8"?> 
<!DOCTYPE query PUBLIC "-//CMR CatalogService (v10)//EN" "http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd"> 
<query> 
    <for value="collections"/> 
        <dataCenterId> 
            <value>ORNL_DAAC</value> 
        </dataCenterId> 
    <where> 
        <collectionCondition>… </collectionCondition> 
        <collectionCondition>… </collectionCondition> 
        …
    </where> 
</query>

Code Listing 16 shows the general structure of an Inventory query with multiple constraints. 

 

Code Listing 16: Structure of an Inventory Query with Multiple Granule Conditions
<?xml version="1.0" encoding="UTF-8"?> 
<!DOCTYPE query PUBLIC "-//CMR CatalogService (v10)//EN" "http://api.cmr.nasa.gov/cmr/dtd/IIMSAQLQueryLanguage.dtd"> 
<query> 
    <for value="granules"/> 
        <dataCenterId> 
            <value>ORNL_DAAC</value> 
        </dataCenterId> 
    <where> 
        <granuleCondition>… </granuleCondition> 
        <granuleCondition>… </granuleCondition> 
        …
    </where> 
</query>

If you specify a list or a single value, then the search looks for an exact match between the string specified and the data stored. For example, if the search value is 'Imagery' and the data stored has the value 'Satellite Imagery', the data will not be returned since there was no exact match. In this instance, it is more useful to use the textPattern element for the search.

Collection/Discovery Search

Searches for collections must enclose the search criteria in a collectionCondition element. If you want to search the negated criteria, you should set the negated attribute in the collectionCondition element to 'Y'.

Code Listing 17 shows how to create a search for all collections except the ones with processing level 1, 1A, 1B, or 2. 

 

Code Listing 17: Using the Negated Attribute on Collection Conditions
<collectionCondition negated='Y'> 
    <processingLevel> 
        <list> 
            <value>'1'</value>
            <value caseInsensitive="Y">'1A'</value>
            <value>'1B'</value> 
            <value>'2'</value>
        </list> 
    </processingLevel>
</collectionCondition>

You can set the negated attribute to 'Y' for a criterion. 
The following table lists the criteria you can use to search for collections. For details about each of these criteria, refer to Common Search Elements (same chapter)

Table 4: Collection Search Criteria (Discovery)

Search Criteria
XML Element
Campaign Short Name<CampaignShortName>...</CampaignShortName>
Dataset ID<dataSetId>...</dataSetId>
CMR Insert Date<CMRInsertDate>...</CMRInsertDate>
CMR Last Update<CMRLastUpdate>...</CMRLastUpdate>
Online Collections Only<onlineOnly/>
Parameter<parameter>...</parameter>
Processing Level<processingLevel>...</processingLevel>
Sensor Name<sensorName>...</sensorName>
Short Name<shortName>...</shortName>
Source Name<sourceName>...</sourceName>
Spatial<spatial>...</spatial>
Spatial Keywords<spatialKeywords>...</spatialKeywords>
Temporal<temporal>...</temporal>
Temporal Keywords<temporalKeywords>...</temporalKeywords>
Additional Attribute Names<additionalAttributeNames>...</additionalAttributeNames>
Orderable Items<orderable/>
Version ID<versionId>...</versionId>
Archive Center<archiveCenter>...</archiveCenter>
Additional Attributes<additionalAttributes>… </additionalAttributes>
Instrument Short Name<instrumentShortName>…</instrumentShortName>

Granule/Inventory Search

Searches for granules must enclose the search criteria in the granuleCondition element. As in the case of collectionCondition, if you want to search the negated criteria, set the negated attribute in the granuleCondition element to 'Y'. You can set the negated attribute to 'Y' for any criterion. 
The following table lists the criteria you can use to search for granules. 

Table 5: Granule Search Criteria (Inventory)

Search Criteria
XML Element
Only Granules with Browse Data<browseOnly/>
Campaign Short Name<CampaignShortName>...</CampaignShortName>
Percentage of cloud Cover<cloudCover>...</cloudCover>
Dataset ID<dataSetId>...</dataSetId>
CMR Insert Date<CMRInsertDate>...</CMRInsertDate>
CMR Last Update<CMRLastUpdate>...</CMRLastUpdate>
Either Day or Night Granules Only<dayNightFlag/>
Search on CMR granule IDs (formerly granuleId)<CMRGranuleID>...</CMRGranuleID>
Search on Granule UR (provider specific)<GranuleUR>...</GranuleUR>
Online Granules Only<onlineOnly/>
Two-D Coordinate System<TwoDCoordinateSystem>...</TwoDCoordinateSystem>
Producer Granule ID<ProducerGranuleID>...</ ProducerGranuleID>
Additional Attributes<additionalAttributes>...</additionalAttributes>
Sensor Name<sensorName>...</sensorName>
Source Name<sourceName>...</sourceName>
Spatial<spatial>...</spatial>
Temporal<temporal>...</temporal>
Orderable Items<orderable/>
Version ID<versionId>...</versionId>
PGE Name<PGEName>...</PGEName>
PGE Version<PGEVersion>...</PGEVersion>
Measured Parameters<measuredParameters>...</measuredParameters>
Provider Production Date<providerProductionDate>…</providerProductionDate>
Provider Insert Date<providerInsertDate>…</providerInsertDate>
Instrument Short Name<instrumentShortName>…</instrumentShortName>
Icon

Effective with E CHO Release 7.0, the LocalGranuleID search criterion was renamed ProducerGranuleID. 

Common Search Elements

You may use the following search criteria for both collection and granule searches. 
Case Insensitive Searching 
Any value or textPattern tag has an attribute caseInsensitive that you can set to Y or N to indicate whether the search value or textPattern is case insensitive. The default is N, that is, the default is case sensitive searching. Refer to the dataSet ID example on the next page for an example. 
CampaignShortName 
CampaignShortName is the name(s) of the campaign or project that gathered data associated with the collection or granule. In CampaignShortName, you can use a single-quoted string in a value, a list of single-quoted strings in a list, or a textPattern element as defined in the section 6.8, Miscellaneous Elements. 
This example shows a search for collections/granules where CampaignShortName = 'River'.

 

<CampaignShortName>
    <value>'River'</value>
</CampaignShortName>

This example shows a search for collections/granules where CampaignShortName = 'River' or 
CampaignShortName = 'Lake'.

 

<CampaignShortName>
    <list>
        <value>'River'</value>
        <value>'Lake'</value>
    </list>
</CampaignShortName>

This example shows a search for collections/granules where CampaignShortName contains the string 'AS_A%D', where the % represents zero or more numbers or letters between the A and the D in the text string 'AS_A%D'

 

<CampaignShortName>
    <textPattern operator="LIKE">'AS_A%D'</textPattern> 
</CampaignShortName>

DataSetID 
dataSetID specifies the universal name of a collection. You can use this element to restrict the search to a few collections, using the value element to specify a single collection, a list for a list of collections, or a textPattern. For the value and list elements, you must know the exact name of the collection. For example, you may have performed a Discovery search to locate the collection of interest and now want to perform an Inventory search for granules. 
This example shows a search for all collections whose dataSetId ends with '1A'.

 

<dataSetId>
    <textPattern caseInsensitive="Y">'%1A'</textPattern>
</dataSetId>

This example shows a search for all granules whose dataSetId is either 'LEAF CHEMISTRY, 1992-1993 (ACCP)' or 'ASTER DEM Product V002'.

 

<dataSetId>
    <list>
        <value>'LEAF CHEMISTRY, 1992-1993 (ACCP)'</value> 
        <value>'ASTER DEM Product V002'</value>
    </list>
</dataSetId>

OnlineOnly 
Use OnlineOnly to search for granules or collections that are available online.

 

<onlineOnly/>

CMR Insert Date 
Use CMRInsertDate to specify a search for collections or granules based on their insertion date into the CMR. You can only specify a dateRange.

 

<CMRInsertDate>
    <dateRange>
        <startDate>
            <Date YYYY="2001" MM="04" DD="05"/> // endDate is not required
        </startDate>
    </dateRange>
</CMRInsertDate>

CMR Last Update 

Use CMRLastUpdate to specify a search for collections or granules based on the date of their last update within the CMR. You can only specify a dateRange.

 

<CMRLastUpdate>
    <dateRange>
        <startDate>
            <Date YYYY="2001" MM="01" DD="01"/> 
        </startDate>
        <stopDate>
            <Date YYYY="2001" MM="06" DD="01"/> 
        </stopDate>
    </dateRange>
</CMRLastUpdate>

 

Sensor Name 
Use sensorName to search for collections or granules based on the name of the sensor. known value in a value, a list of known values in a list, or a text pattern in a textPattern. relationship between each value with the Boolean "AND" or "OR".
The "AND" relationship means that results must match every value specified.
You can search for a You can specify the 
The "OR" relationship means that results must match either or both of the values specified. "OR" is the default setting for searching with multiple values.

 

<granuleCondition> 
    <sensorName operator="AND">
        <list>
            <value>'human observer'</value> 
            <value>'rain gauge'</value>
        </list>
    </sensorName>
</granuleCondition>

 

Instrument Short Name 
Use instrumentShortName to search for collections or granules based on the name of the instrument. You can search for a known value in a value, a list of known values in a list, or a text pattern in a textPattern. You can specify the relationship between each value with the Boolean "AND" or "OR".
The "AND" relationship means that results must match every value specified.
The "OR" relationship means that results must match either or both of the values specified. "OR" is the default setting for searching with multiple values.

 

<granuleCondition> 
    <instrumentShortName operator="AND">
        <list>
            <value>'GPS'</value>
            <value>'Telescope'</value>
        </list>
    </instrumentShortName>
</granuleCondition>

 

Source Name 
Use sourceName to search for collections or granules based on the name of the source. You may specify the name using a value, a list of known values in a list, or a text pattern in a textPattern. You can specify the relationship between each value with the Boolean "AND" or "OR".

The "AND" relationship means that results must match every value specified.

The "OR" relationship means that results must match either or both of the values specified. "OR" is the default setting for searching with multiple values

 

<granuleCondition> 
    <sourceName operator="AND">
        <list> 
            <value>'CV-580'</value> 
            <value>'C-130'</value>
        </list>
    </sourceName>
</granuleCondition>

 

Spatial

Use spatial constraints to restrict the search within a spatial extent of the earth. Specify the spatial extent using the elements IIMSPolygonIIMSMultiPolygonIIMSBox or IIMSLine
You may also specify a TwoDCoordinate element along with a spatial element in a spatial constraint. This is useful when you wish to find spatially coincident granules between collections that support TwoDCoordinate searches and those that do not. When you use TwoDCoordinate together with a spatial element, the CMR will return all granules that satisfy either the TwoDCoordinate constraint or the spatial element, or both. You may only use TwoDCoordinate in a spatial constraint for granule (inventory) queries, not collection queries. 
You may also use TwoDCoordinate outside of a spatial constraint. In this case, list TwoDCoordinate inside a granuleCondition. The CMR will return results that satisfy the TwoDCoordinate constraint and all other conditions (including any spatial conditions if they are present). 
Polygon consists of one or more rings. Some rings may be embedded within others that define the interior of the polygon. The CMR system can handle polygons with only one external ring and multiple internal rings. 

Figure 6: Allowed Geometries for Spatial Data and Query Windows

                                                                                                                   

To define a ring, the last point in the ring must be the same as the first point in the ring. You must specify the points in each polygonal ring in counterclockwise order. 

Figure 7: Order of Points in a Polygonal Ring

                                                                                                                                            

 

To specify multiple outer rings, use the MultiPolygon or IIMSMultiPolygon element, with each outer ring in a separate Polygon. There should be no overlap between any of the outer rings defined. Refer to Code Listing 19: A Multi-Polygon Spatial Query for examples. 
Use IIMSBox to specify a bounding box on the Earth. A bounding box contains two points. The first point is the lower left corner of the box. The second point is the upper right corner of the box. The horizontal lines of the box will follow the latitude lines of the earth. This means the points of the box will not connect by lines following great circle arcs but will curve to stay at the same latitude. 
Use IIMSLine or LString to specify a line on the Earth. You must specify the points in the line from one endpoint to the other. You can also use IIMSPoint to specify a single point on the Earth. Note that although IIMSPoint is used to describe corner points in a polygon or endpoints in a line, it also describes a specific searchable geometry. Refer to Code Listing 20: A Single Point Spatial Query and for examples. 
The table below describes Spatial Operators available in the CMR System, regarding the relationship between two objects.

Table 6: Spatial Operators

Spatial Operator
Description
EQUALThey have the same boundary and interior.
TOUCHThe boundaries intersect but the interiors do not.
WITHINInterior and boundary of the first object are completely contained in the interior of the second.
CONTAINSThe interior and boundary of the second object are completely contained in the interior of the first.
RELATEThe objects are non-disjoint, that is, their bodies and/or boundaries intersect.

 

                                                                                                                                                               Figure 8: Examples of Spatial Regions

                                                                                 

 

Define the query so that the user-specified spatial extent is the second object in the query.
Use CONTAINS to retrieve granules/collections that contain the specified polygon.
Use WITHIN to retrieve granules/collections that are within the specified spatial extent.
Use RELATE to retrieve granules/collections that overlap in some way. RELATE is the default operator. 
The CMR supports multiple spatial representations (or SpatialTypes). Spatial data of different spatial representations are described differently and hence are stored and queried differently. 
The SpatialType tag is an optional field in AQL used to specify the spatial type. If the user opts not to specify this field, the CMR performs a default search. By default, the CMR performs the search on NORMAL (Cartesian and Geodetic), and ORBIT

 

Code Listing 18: Spatial Search Element
<spatial operator="RELATE">
    <IIMSPolygon> 
        <IIMSLRing> 
            <IIMSPoint lon="-120" lat="-30" /> 
            <IIMSPoint lon="-100" lat="-60" /> 
            <IIMSPoint lon="5" lat="-90" /> 
            <IIMSPoint lon="-120" lat="-60" /> 
            <IIMSPoint lon="160" lat="5" /> 
            <IIMSPoint lon="160" lat="60" /> 
            <IIMSPoint lon="120" lat="85" /> 
            <IIMSPoint lon="5" lat="85" /> 
            <IIMSPoint lon="-120" lat="30" /> 
            <IIMSPoint lon="-120" lat="-30" /> 
        </IIMSLRing>
        <IIMSLRing> 
            <IIMSPoint lon="80" lat="20" /> 
            <IIMSPoint lon="80" lat="60" /> 
            <IIMSPoint lon="20" lat="60" /> 
            <IIMSPoint lon="20" lat="20" /> 
            <IIMSPoint lon="80" lat="20" />
        </IIMSLRing> 
    </IIMSPolygon>
    <TwoDCoordinateSystem> 
        <TwoDCoordinateSystemName>
            <value>'WRS2'</value>
        </TwoDCoordinateSystemName> 
        <Coordinate1><range lower='15' upper='20'/></Coordinate1> <Coordinate2><range lower='33' upper='42'/></Coordinate2> 
    </ TwoDCoordinateSystem> 
    <SpatialType> 
        <list> 
            <value>NORMAL</value>
            <value>ORBIT</value> 
        </list>
    </SpatialType>
</spatial>

Spatial Type: NORMAL 
Conventional spatial data are described as shapes such as polygons, multi-polygons, boxes, or lines. NORMAL is the spatial type to use for searching this type of data. Spatial data are stored in the database as Oracle objects to record the shape, spatial locations, and coordinate system used; i.e., Cartesian or Geodetic. 
All spatial objects with the exception of Box in AQL, the CMR interprets as Geodetic. Geodetic spatial data has some restrictions. The total area of a single polygon cannot exceed half the earth. Lines, either alone or in a polygon, cannot be longer than half the circumference of the earth. 
The same points in Cartesian and Geodetic describe two different areas. Great circle arcs connect points in Geodetic. These follow the shortest paths on the earth between two points. Latitude lines connect points in Cartesian. The image below shows a geodetic area Google Earth below as a red outlined polygon overlayed over a white polygon representing the Cartesian area described with the same points.

Figure 9: Spatial differences between a Geodetic area and a Cartesian Area 

As noted previously, Geodetic and Cartesian represent polygons in different ways. There are other differences though besides how the lines define the shape. These differences stem from the fact that Geodetic and Cartesian are different representations of the earth. Geodetic is an ellipsoidal model of the earth whereas Cartesian describes the surface of the earth as a flat plane.

Figure 10: Ellipsoid

Figure 11: Flat Cartesian plane

These fundamental differences lead to some polygons being valid in Geodetic and not valid in Cartesian. Polygons and lines follow Geodesics so the CMR uses them as-is for searching geodetic data. Boxes, on the other hand, follow the latitude lines of the earth so the CMR uses them as-is for searching Cartesian data. The CMR converts polygons and lines from Geodetic for searching the Cartesian data and converts boxes from Cartesian when searching Geodetic data. The converted areas are valid approximations of the original areas. The CMR performs the conversion first by dividing the spatial area into pieces so that each new area fits within the bounds of sections of the earth. The CMR uses eight octants when converting from Geodetic to Cartesian. The CMR uses four quadrants when converting from Cartesian to Geodetic. The Cartesian graph below shows the octant bounds. The CMR divides the spatial area so that it does not cross the antimeridian, the prime meridian, or either of the poles.

Figure 12: Bounds of the eight octants 

After dividing the spatial area, the lines of the spatial area are "densified". Densification means that the CMR adds more points along the lines between the original points. This has the effect of making the lines in Cartesian space approximate the lines in geodetic space and vice versa. The CMR adds points every 10 km during densification. We found this number to be a good compromise between performance and accuracy. It is important that client developers understand that the CMR uses an approximation of the area so in a small number of cases there may be some missed or extra results. Images, shown below, illustrate the Geodetic to Cartesian Conversion process.

   

Figure 13: Example geodetic polygon          

                                             

                                                                                               














Figure 14: Geodetic polygon divided by octants  

       

Figure 15: Cartesian polygon before densification

                                                                   

                                                                                  









Figure 16: Cartesian polygon after densification 

Spatial Type: ORBIT 
Spatial data are orbit-based and generated from satellites running along orbits. Because an orbit travels at a fixed direction with a fixed declination angle and the covered area is a stripe of fixed width, the spatial coverage of a granule can be derived from the latitude where the granule starts and ends and the crossing longitude on the equator where its orbit originally starts. Thus, the orbit-based spatial data are not stored as shape objects but as the original data. The CMR uses an algorithm called ―Backtrack‖ to perform searches on orbit-based data. Spatial coverage can be as large as the entire earth surface including polar area. The CMR does simple validation against the spatial window to make sure it does not exceed the scope of the earth (the latitude is within the range –90 to 90 and the longitude is within the range –180 to 180). 
There is one limitation in the current implementation: the only supported spatial operator is RELATE. In the CMR, multi-orbit as well as single orbit searches are supported. 
This example shows a multi-polygon spatial query. 

 

<spatial operator='RELATE'
    <IIMSMultiPolygon>
        <IIMSPolygon>
            <IIMSLRing>
                <IIMSPoint lat="85" long="-50"/> 
                <IIMSPoint lat="70" long="-60"/> 
                <IIMSPoint lat="60" long="-50"/> 
                <IIMSPoint lat="70" long="-40"/> 
                <IIMSPoint lat="85" long="-50"/>
            </IIMSLRing>
        </IIMSPolygon>
        <IIMSPolygon>
            <IIMSLRing>
                <IIMSPoint lat="-85" long="-50"/> 
                <IIMSPoint lat="-70" long="-40"/> 
                <IIMSPoint lat="-60" long="-50"/> 
                <IIMSPoint lat="-70" long="-60"/> 
                <IIMSPoint lat="-85" long="-50"/>
            </IIMSLRing>
        </IIMSPolygon>
    </IIMSMultiPolygon>
</spatial>

 

 

Refer to Figure 6: Allowed Geometries for Spatial Data and Query Windows for an example of an image of a multi-polygon.

This example shows a single point spatial query. 

 

<spatial operator='RELATE'>
    <IIMSPoint long='-75.0' lat='35.0'/> 
</spatial>

 

Temporal

Use temporal constraints to specify the temporal extent of the data. The dates are stored as Gregorian dates, and the times are stored in GMT. This search allows only dates in YYYY-DD-MM format. You can specify a range of time represented by beginning and ending dates or a periodic temporal range specified by start date and end date. For a periodic temporal range, you can specify the start and end day of each year. You should use integers between 1 and 365 to specify the start and end dates, with start day =< end day. You can specify the end day as 366, but if the temporal range contains a non-leap year, then an error will be generated. 
This example shows a search for granule/collections whose temporal range overlaps with the time between May 12, 1990, and June 13, 1992.

 

Code Listing 21: Temporal Search Example
<temporal> 
    <startDate>
        <Date YYYY="1990" MM="5" DD="12"/>
    </startDate> 
    <stopDate>
        <Date YYYY="1992" MM="2" DD="13"/>
    </stopDate> 
</temporal> 

This example shows a search for granules/collections whose temporal range overlaps with all of these time ranges:

  • May 16 - July 14, 1990
  • January 5 - March 5, 1991
  • January 5 - February 10, 1992 

 

Code Listing 22: Temporal Example with Different Day Notations
<temporal> 
    <startDate>
        <Date YYYY="1990" MM="5" DD="12"/>
    </startDate> 
    <stopDate>
        <Date YYYY="1992" MM="2" DD="10"/>
    </stopDate> 
    <startDay value="5"/> 
    <endDay value="60"/> 
</temporal> 

Version ID 
Use versionId to search for collections or granules based on the version identifier of the data collection. You can search for a known value in a value, a list of known values in a list, or a text pattern in a textPattern. In the CMR, versionId is stored as a string so that it may look like 3,0033,0 …. 

 

Code Listing 23: Version ID in a Collection Condition
<versionId> 
    <value>'0'</value>
</versionId>

 

 

Code Listing 24: Version ID in a Granule Condition with a Pattern
 <versionId> 
    <textPattern operator="LIKE">'3.%'</textPattern> 
</versionId> 

Additional Attributes 
Use additionalAttributes to search for collections or granules based on one or many groups of the additional attributes by specifying the additional attribute name/value pair for an exact match. This function includes searching on multiple additional attributes. You may specify the relationship between groups with either "AND" or "OR".

The "AND" relationship means that result granules must match across every group.

The "OR" relationship means that result granules match with any of the specified groups. "OR" is the default setting. 
Within each additional attribute, you must specify an additionalAttributeName in single quotation marks (that is, apostrophes) for an exact match. You may specify the additionalAttributeValue using valuelisttextPatternrange, or dateRange
This example shows the "AND" relationship between two additional attribute groups. You can use this sample search with either granuleCondition or collectionCondition

 

Code Listing 25: Additional Attributes Example with an AND Relationship
<additionalAttributes operator='AND'> 
    <additionalAttribute> 
        <additionalAttributeName>
            'AveragedFocalPlane1Temperature'
        </additionalAttributeName> 
        <additionalAttributeValue> 
            <range lower='169' upper='170'/> 
        </additionalAttributeValue>
    </additionalAttribute>
    <additionalAttribute> 
        <additionalAttributeName>
            'AveragedFocalPlane2Temperature'
        </additionalAttributeName>
        <additionalAttributeValue> 
            <range lower='269' upper='270'/> 
        </additionalAttributeValue>
    </additionalAttribute> 
</additionalAttributes> 

This example shows the "OR" relationship between two additional attribute groups. 

 

Code Listing 26: Additional Attribute Example with an OR Relationship
<additionalAttributes operator='OR'> 
    <additionalAttribute>
        <additionalAttributeName> 
            'AveragedFocalPlane1Temperature'
        </additionalAttributeName>
        <additionalAttributeValue> 
            <range lower='169' upper='170'/> 
        </additionalAttributeValue> 
    </additionalAttribute>
    <additionalAttribute>
        <additionalAttributeName> 
            'AveragedFocalPlane2Temperature'
        </additionalAttributeName> 
        <additionalAttributeValue>
            <range lower='269' upper='270'/> 
        </additionalAttributeValue> 
    </additionalAttribute>
</additionalAttributes>

Search Elements for Collection/Discovery Queries 

This section describes elements used specifically for collection (Discovery) searches. 

Science Keywords

Use scienceKeywords to specify the science keywords associate with a collection. Science Keywords are based on the GCMD managed list of science keywords. They have the following hierarchical structure Category> Topic > Term > Variable Level 1 > Variable Level 2 > Variable Level 3 > Detailed Variable. A value at any of the specific hierarchy levels can be specified or anykeyword can be used to search all the levels. Science Keywords are all stored in upper case. Queries with science keywords will automatically be changed to upper case for searching. 
If you specify a list or a single value, then the search looks for an exact match between the string specified and the data stored. For example, if the search value is 'IMAGERY' and the data stored has the value 'SATELLITE IMAGERY', the data will not be returned since there was no exact match. In this instance, it is more useful to use the textPattern element for the search. 

 

Code Listing 27: Sample Science Keyword Search
<collectionCondition> 
    <scienceKeywords operator="AND">
        <scienceKeyword> 
            <categoryKeyword>
                <value>'category'</value>
            </categoryKeyword> 
            <topicKeyword>
                <value>'topic'</value> 
            </topicKeyword>
            <termKeyword>
                <value>'term'</value> 
            </termKeyword>
            <variableLevel1Keyword> 
                <value>'level1'</value>
            </variableLevel1Keyword>
            <variableLevel2Keyword> 
                <value>'level2'</value>
            </variableLevel2Keyword> 
            <variableLevel3Keyword>
                <value>'level3'</value> 
            </variableLevel3Keyword> 
            <detailedVariableKeyword>
                <value>'detailed'</value> 
            </detailedVariableKeyword>
        </scienceKeyword>
    </scienceKeywords> 
<collectionCondition>

 

 

Code Listing 28: Sample Science Keyword Search with anyKeyword
<collectionCondition> 
    <scienceKeywords operator="AND"> 
        <scienceKeyword>
            <anyKeyword>
                <textPattern>'%WATER%'</textPattern> 
            </anyKeyword>
        </scienceKeyword> 
    </scienceKeywords>
<collectionCondition>

Processing Level

Use processingLevel to specify the level at which the data (collection/granule) has been processed, for example, level 0, 1, 1A, 1B, 2, 3, 4. The search can specify one or more processing levels. The values must be single-quoted strings. 
This example shows a search for all collections at processing level 1, 1A, or 1B. 

 

Code Listing 29: Processing Level Search Element
 <collectionCondition> 
    <processingLevel>
        <list>
            <value>'1'</value> 
            <value>'1A'</value>
            <value>'1B'</value> 
        </list>
    </processingLevel>
</collectionCondition>

This example shows a search for all collections at processing level 1, 1A, or 1B—any value starting with the digit 1. This also returns results for collections with processing levels 13, 14, etc. 

 

Code Listing 30: Processing Level Element with Text Pattern
 <collectionCondition> 
    <processingLevel> 
        <textPattern operator="LIKE">'1_'</textPattern> 
    </processingLevel> 
</collectionCondition>

Additional Attribute Name 

Use additionalAttributeName to search for collections based on the names of the additional attributes in the collections. You can search for a known value in a value, a list of known values in a list, or a text pattern in a textPattern. 
This example shows a search for data with an additionalAttributeName value of 1 or 3. 

 

Code Listing 31: Additional Attribute Search Element
<collectionCondition>
    <additionalAttributeName> 
        <list>
            <value>'value_1'</value> 
            <value>'value_3'</value> 
        </list>
    </additionalAttributeName>
</collectionCondition> 

Short Name

Use shortName to specify a search for the short name of collections, which may or may not be unique for a particular provider. 
This example shows a search for all collections that have shortNames starting with ‗BOREAS.' 

Code Listing 32: Short Name Search Element
<collectionCondition>
    <shortName> 
        <textPattern>'BOREAS%'</textPattern>
    </shortName> 
</collectionCondition>

Spatial Keywords

Use spatialKeywords to specify a word or phrase that summarizes the spatial region covered by the collection. A collection may cover several regions. 
This example shows a search for all collections that cover Africa or Bermuda or the Indian Ocean. 

 

Code Listing 33: Spatial Keywords Search Element
<collectionCondition>
    <spatialKeywords> 
        <list>
            <value>'Africa'</value> 
            <value>'Bermuda'</value> 
            <value>'Indian Ocean'</value> 
        </list> 
    </spatialKeywords>
</collectionCondition>

This example shows a search for all collections that cover some region(s) of the Americas. 

 

Code Listing 34: Spatial Keywords Element with Text Pattern
<collectionCondition> 
    <spatialKeywords> 
        <textPattern operator="LIKE">'%america%'</textPattern> 
    </spatialKeywords>
</collectionCondition>

Temporal Keywords

Use temporalKeywords to specify a word or phrase that summarizes the temporal characteristics referenced in the collection. 
This example shows a search for all collections that have accumulated data during the spring or fall. 

 

Code Listing 35: Temporal Keywords Element
<collectionCondition> 
    <temporalKeywords>
        <list>
            <value>'spring'</value> 
            <value>'fall'</value>
        </list> 
    </temporalKeywords>
</collectionCondition>

Archive Center

Use archiveCenter to search for collections based on the center where a collection is archived. You may search for a known value in a value, a list of known values in a list, or a text pattern in a textPattern

Code Listing 36: Archive Center Search Element
<collectionCondition> 
    <archiveCenter> 
        <value>'ORNL_DAAC'</value> 
    </archiveCenter>
</collectionCondition>

Search Elements for Granule/Inventory Queries

This section describes elements used specifically for granule (Inventory) searches. 

Granules with Browse Data 

Use browseOnly to specify that you want only granules that have browse data available. 

 

Code Listing 37: Browse-only Search Element
<granuleCondition>
    <browseOnly/> 
</granuleCondition>

Percentage of Cloud Cover 

Use cloudCover to specify a range of percentage of scene cloud coverage in a granule. For the value in each attribute of the range, you should use a floating number between 0 and 100 that specifies the range in percentage of cloud coverage in a granule. You cannot use the negated condition for this search. 
This example shows a search for only those granules that have cloud cover between 10% and 20%. 

 

Code Listing 38: Cloud Cover Search Element
<granuleCondition>
    <cloudCover> 
        <range lower="10" upper="20"/> 
    </cloudCover>
</granuleCondition>

Day, Night, or Both Granules 

Use dayNightFlag to specify a search for granules that are gathered during daylight only, nighttime only, or both. If you do not specify this criterion, then your results will include granules gathered at all times of day. 
This example shows a search for granules gathered at least partly during daylight. 

 

Code Listing 39: Search Element for Granules Gathered During Daylight Only
<granuleCondition> 
    <dayNightFlag value="DAY"/>
</granuleCondition>

This example shows a search for granules gathered at least partly during the night. 

 

Code Listing 40: Search Element for Granules Gathered During the Nighttime Only
<granuleCondition> 
    <dayNightFlag value="NIGHT"/>
</granuleCondition>

This example shows a search for granules gathered partly at night and partly during the daylight. 

 

Code Listing 41: Search Element for Granules Gathered During Daylight, Nighttime, or Both
<granuleCondition> 
    <dayNightFlag value="Both"/> 
</granuleCondition>

CMR Granule IDs 

Use CMRGranuleID to specify an inventory search for specific granules. This search does not need to specify any spatial or temporal constraint. You can use this as a second stage query when you know the list of granules you are interested in from a previous query to hone results. The search is then limited to granules in the list. The list may contain granules from different data sets. The CMR performs the search on CMR_GRANULE_ID, a CMR-wide unique ID. You must enclose the names of the granules in single quotation marks (that is, apostrophes). 

 

Code Listing 42: Granule ID Search Element
<granuleCondition> 
    <CMRGranuleID> 
        <list> 
            <value>'G1984-ORNL_DAAC'</value>
        </list> 
    </CMRGranuleID>
</granuleCondition>

Granule UR 

Use GranuleUR to specify an inventory search for specific granules. Like granule ID, this criterion can be used as a secondary stage query. Instead of using the CMR-specific ID for the granules, this criterion allows you to use the provider-given name for the granules. Provider-given names are more meaningful and easier to remember. Since the granule names given by the provider are not guaranteed to be unique within the CMR, if a granule with the same name exists for more than one dataCenterId specified in the query, all will be returned in the results. You must enclose the names of the granules in single quotation marks (that is, apostrophes). Note that the GranuleUR can contain value, list, textPattern or patternList children. 

 

Code Listing 43: Granule UR Search Element
<granuleCondition> 
    <GranuleUR> 
        <list> 
            <value>'ACCP_CANOPYCHEM.df_can_calc.dat'</value> 
            <value>'ANGLIA_10YRCLIMATE.cfrs1120.zip'</value>
        </list> 
    </GranuleUR>
</granuleCondition>

Two-Dimensional Coordinate System 

Use TwoDCoordinateSystem as an alternative to specifying a spatial or temporal range. For some collections, granules can be located by searching a range of values on a two dimensional grid, (for example, Path/Row for Landsat, X/Y for MODIS Grid data). 
The TwoDCoordinateSystem criterion needs three elements:

  • the TwoDCoordinateSystemName
  • the Coordinate1 range
  • the Coordinate2 range 

Coordinate1 and Coordinate2 may also take a single value. The query returns all granules that match your specified TwoDCoordinateSystemName and overlap with the Coordinate1 and Coordinate2 ranges. Coordinate1 and Coordinate2 ranges must have both lower and upper attributes specified. 
This example shows a search for a range value. 

 

Code Listing 44: TwoDCoordinateSystem Search Element
<granuleCondition> 
    <TwoDCoordinateSystem>
        <TwoDCoordinateSystemName>
            <value>'WRS2'</value> 
        </TwoDCoordinateSystemName> 
        <Coordinate1>
            <range lower='15' upper='20'/>
        </Coordinate1> 
        <Coordinate2>
            <range lower='33' upper='42'/>
        </Coordinate2> 
    </TwoDCoordinateSystem> 
</granuleCondition>

This example shows a search for a single value. 

 

Code Listing 45: TwoDCoordinateSystem Search Element with a Single Value
<granuleCondition>
    <TwoDCoordinateSystem> 
        <TwoDCoordinateSystemName> 
            <value>'WRS2'</value>
        </TwoDCoordinateSystemName> 
        <coordinate1>
            <value>15<value/>
        </coordinate1> 
        <coordinate2>
            <range lower='33' upper='42'/>
        </coordinate2> 
    </TwoDCoordinateSystem > 
</granuleCondition>

ProducerGranuleID 

Use ProducerGranuleID to specify an inventory search for specific granules. ProducerGranuleID represents the ID assigned to data by the Science team that produced the data. Instead of using the CMR-specific IDs for the granules, this criterion allows the user to use the producer's given name for the granules. Granule names are available to the Science teams and are easier to search on. Since the granule names given by the producer are not guaranteed to be unique within the CMR, if a granule with the same name exists for more than one dataCenterId specified in the query, all will be returned in the results. You must enclose the names of the granules in single quotation marks (that is, apostrophes). Note that the ProducerGranuleID can contain value, list, textPattern or patternList children. 

 

Code Listing 46: ProducerGranuleID Search Element
<granuleCondition> 
    <ProducerGranuleID>
        <list>
            <value>'MOD03.A2000107.0930.003.2002056052717.hdf'</value> 
            <value>'MOD01.A2000107.0740.003.2002056051321.hdf'</value>
        </list> 
    </ProducerGranuleID>
</granuleCondition>

PGE Name 

Use PGEName to search for granules based on the name of the product generation executive (PGE). You can search for a known value in a value element, a list of known values in a list, or a text pattern in a textPattern

 

Code Listing 47: PGE Name Search Element
<granuleCondition> 
    <PGEName> 
        <value>'test_PGEName'</value> 
    </PGEName>
</granuleCondition>

PGE Version 

Use PGEVersion to search for granules based on the version of PGE that originally produced the granule. You can search for a known value in a value element, a list of known values in a list, or a text pattern in a textPattern. 

 

Code Listing 48: PGE Version Search Element
<granuleCondition> 
    <PGEVersion> 
        <value>'3.0.0'</value> 
    </PGEVersion>
</granuleCondition>

Collection Short Name 

Use collectionShortName to search for granules based on the short name of the collection they belong to. You can search for a known value in a value element, a list of known values in a list, or a text pattern in a textPattern

 

Code Listing 49: Collection Short Name Search Element
<granuleCondition> 
    <collectionShortName> 
        <value>'MOD03'</value> 
    </collectionShortName>
</granuleCondition>

Measured Parameters 

Use measuredParameters to search for granules based on one or more groups of measured science parameters. You can specify the relationship between groups of parameters with either AND or OR.

An AND relationship means that result granules must match across all groups.

An OR relationship means that result granules can match any of the groups. OR is the default setting. 
Within each measured parameter, you must specify a measuredParameterName in single quotation marks (that is, apostrophes) for exact matching. The detailed measured parameters can be valuelist, and textPattern type, or range and dateRange type. 
This example shows a search that specifies an "AND" relationship between two measured parameter groups. 

 

Code Listing 50: Measured Parameter Search Element
<granuleCondition>
    <measuredParameters operator="AND"> 
        <measuredParameter> 
            <measuredParameterName>'EV_1KM_RefSB'</measuredParameterName>
            <operationalQualityFlag> 
                <value>'Passed'</value>
            </operationalQualityFlag> 
            <QAPercentOutOfBoundsData> <range lower='30' upper='40'/>
            </QAPercentOutOfBoundsData> 
        </measuredParameter>
        <measuredParameter> 
            <measuredParameterName>'EV_1KM_RefSB'</measuredParameterName> 
                <operationalQualityFlag>
                    <value>'Passed'</value> 
                </operationalQualityFlag>
            <QAPercentOutOfBoundsData>
                <range lower='39' upper='50'/> 
            </QAPercentOutOfBoundsData> 
        </measuredParameter>
    </measuredParameters>
</granuleCondition>

This example shows a search that specifies an "OR" relationship between two measured parameter groups. 

 

Code Listing 51: Measured Parameter Search Element
<granuleCondition>
    <measuredParameters operator="OR"> 
        <measuredParameter> 
            <measuredParameterName>'EV_1KM_RefSB'</measuredParameterName>
            <operationalQualityFlag> 
                <value>'Passed'</value>
            </operationalQualityFlag> 
            <QAPercentOutOfBoundsData> <range lower='30' upper='40'/>
            </QAPercentOutOfBoundsData> 
        </measuredParameter>
        <measuredParameter> 
            <measuredParameterName>'EV_1KM_RefSB'</measuredParameterName> 
                <operationalQualityFlag>
                    <value>'Passed'</value> 
                </operationalQualityFlag>
            <QAPercentOutOfBoundsData>
                <range lower='39' upper='50'/> 
            </QAPercentOutOfBoundsData> 
        </measuredParameter>
    </measuredParameters>
</granuleCondition>

ProviderInsertDate

Use providerInsertDate to search for granules that are within a desired date-time range. StartDate is a required field, and StopDate is optional. The Date format is:

 

<Date YYYY="2001" MM="01" DD="01" HH="00" MI="00" SS="00"/>

 

Where YYYY is year; MM is month, DD is day of the month; HH is hour; MI is minute, and SS is second. 
This example shows a search using providerInsertDate

 

Code Listing 52: Provider Insert Date Search Element
<granuleCondition>
    <providerInsertDate> 
        <dateRange>
            <startDate> 
                <Date YYYY="2001" MM="01" DD="01" HH="00" MI="00" SS="00"/>
            </startDate> 
            <stopDate> 
                <Date YYYY="2003" MM="12" DD="01" HH="23" MI="59" SS="59"/> 
            </stopDate>
        </dateRange> 
    </providerInsertDate>
</granuleCondition>

ProviderProductionDate

Use ProviderProductionDate to search for granules that are within a desired date-time range. StartDate is a required field, and StopDate is optional. The Date format is:

 

<Date YYYY="2001" MM="01" DD="01" HH="00" MI="00" SS="00"/>

Where YYYY is year; MM is month, DD is day of the month; HH is hour; MI is minute, and SS is second. 

This example shows a search of the Data Provider's production date search element. 

 

Code Listing 53: Provider Production Date Search Element
<granuleCondition> 
    <providerProductionDate> 
        <dateRange>
            <startDate> 
                <Date YYYY="2001" MM="01" DD="01" HH="00" MI="00" SS="00"/> 
            </startDate>
            <stopDate> 
                <Date YYYY="2003" MM="12" DD="01" HH="23" MI="59" SS="59"/> 
            </stopDate>
        </dateRange> 
    </providerProductionDate>
</granuleCondition>

Miscellaneous Elements

Use the following elements to search for other elements using AQL. 
List
Use List to specify a list of values. The values may be numbers, dates, or strings. Each individual value must appear as text of the value element. If you specify a list of strings, you must use a single-quoted string with no wild-card characters. The CMR searches the parent element using Oracle's IN operator for an exact match with any value in the list. 
In general, interpretation of the list will be dependent on the parent element in which it appears. 

 

Code Listing 54: The List Element
<list> 
    <value>'Africa'</value> 
    <value>'Bermuda'</value> 
    <value>'Indian Ocean'</value>
</list>

Pattern List 

Use patternList to specify a list of text patterns and/or values. See section 6.8.4 for rules associated with values and section 6.8.3 for rules associated with text patterns. 
The values may be numbers, dates, or strings. Each individual value must appear as text of the value element. If you specify a list of string values, you must use a single-quoted string with no wild-card characters. 
The text patterns must conform to the rules specified in section 6.8.3. The CMR searches the parent element using Oracle's IN operator for an exact match with any value in the list. 
In general, interpretation of the list will be dependent on the parent element in which it appears.

 

<patternList>
    <value>'Africa'</value>
        <textPattern operator="LIKE" caseInsensitive="Y">'america%'</textPattern>
    <value>'Indian Ocean'</value> 
</patternList>

Text Pattern 

Use textPattern to search using the Oracle operator ―LIKE‖ to perform pattern matching. Because of pattern matching, exact specification of what is desired is not needed. The code listing below shows how to invoke the pattern matching. Note that you must specify the operator, and you must use a single-quoted string with or without wild-card characters. 
The wild-card characters supported are '%' and ''. The character ‗%' is a placeholder that represents any number of characters. The character ―‖ is a placeholder that represents a single character.

Table 7: AQL Wildcard Use

Wildcard pattern
Interpretation
‗%JOHN%'Strings containing ‗JOHN' as a sub-string
‗MARY%'Strings beginning with the characters ‗MARY'
‗%BOB'Strings ending in ‗BOB'
‗D_VE'Strings like ‗DAVE', ‗DOVE', etc.

You can include the actual characters "%" or "" in the pattern by using the ESCAPE character '\'. If the escape character appears in the pattern before the character "%" or "", then Oracle interprets this character literally in the pattern, rather than as a special pattern-matching character. 

Table 8: Wildcard Examples with ESCAPE Character

Wildcard pattern with ESCAPE character

Interpretation

‗JOHN%THAN'

String ‗JOHN%THAN'

‗JOHN%THAN'

Strings ‗JOHNNATHAN', ‗JOHNASDTHAN', etc.

‗JOHN\\MARY'

String ‗JOHN\MARY'

If the parent element (collectionCondition or granuleCondition) has the attribute negated with value 'Y', then the search will look for a string NOT like the pattern specified. 

 

Code Listing 55: Text Pattern Element
<spatialKeywords>
    <textPattern operator="LIKE" caseInsensitive="Y">'america%'</textPattern> 
</spatialKeywords>

Value

Use value to specify one value. Refer to each individual parent element for a detailed explanation. In general, if the element represents a string pattern, the search will be case sensitive unless the caseInsensitive attribute is set to Y.

 

<value>12.6</value>

or

 

<value caseInsensitive="Y">'Asx'</value>

Date 

Use Date to specify a date value.

 

<Date YYYY="2000" MM="05" DD="03"/>

or

 

<Date YYYY="2000" MM="10" DD="09" HH="21" MI="04" SS="32"/>

Date Range 

Use dateRange to specify a range of time. The range can include just a startDate, just a stopDate or both. If you only include a startDate, the CMR searches for data from that time forward with no stop date. If you only include a stopDate, the CMR searches for data from the current time up until the stop date. If you include both a startDate and a stopDate, then the CMR searches for data falling between those two dates, for example, between January 1, 2005 and January 31, 2005.

 

<dateRange>
    <startDate><Date YYYY="2005" MM="01" DD="01"/></startDate> 
    <stopDate><Date YYYY="2005" MM="01" DD="31"/></stopDate>
</dateRange>

or

 

<dateRange>
    <stopDate>
        <Date YYYY="2000" MM="10" DD="09" HH="04" MI="12" SS="34"/> 
    </stopDate>
</dateRange>

DifEntryId 

Use DifEntryId to find collections by their GCMD (Global Change Master Directory) DIF (Directory Interchange Format) Entry-ID. You can find the GCMD at the following URL:  http://gcmd.nasa.gov.

Accessing data

After identifying data of interest through catalog queries, you may place a request for that data if the data provider(s) allow it. To create an request, it is critical that you understand the parts that make up a request
A request is a collection of request items. Each request item consists of:

  • The GUID for the catalog item you wish to request
  • The quantity of the item you wish to request

Other options associated with the item:
Many catalog items belonging to many different providers may comprise a single order. Each provider may have its own validation scheme and rules for creating, validating, and submitting particular catalog items. To ensure appropriate validation – as well as to demarcate a larger request for sending to each provider – requests are split up into smaller provider requests, sometimes called sub-requests. Each provider request includes all the request items in a particular request that belong to a specific provider. Each provider request has its own provider request GUID, a unique identifier comprised of the GUID of the provider and the GUID of the request that contains the catalog items for this provider. The following figure shows the structure of a request.

                                                                                  

CMR Request Structure 

The CMR is designed to support a variety of providers and their various requesting needs, which brings some complexity to the process of request creation. The following figure illustrates the most commonly used and direct approach for creating and submitting a request.

 

                                                                                 

The most common way to obtain the catalog items needed for creating a request is through the ExecuteQuery operation found in the Catalog Service. You can also specify how the results should be displayed. When doing so, ensure that the CatalogItemID tag is returned. The string value under this tag is the actual catalog item GUID that you can use to request items from the CMR.

Access Options 

Access Options use CMR Forms (refer to  http://www.cmr.nasa.gov/data_partners/data_tools10.shtml for more information) for providers to indicate additional information required from users in order to fulfill their orders, such as the media to put the ordered data on and the specific subset of the data to order. Providers assign Order Option definitions to catalog items. When you add an order item to an order that has required order options, it must contain an option selection that corresponds to one of the assigned option definitions for the catalog item you are ordering. See section 1.3 of the CMR Forms Specification for more details. 
To list the possible options for each catalog item, use the GetCatalogItemOrderInfomation operation on the 
OrderManagementService. The operation returns one object called CatalogItemOrderInformation. This contains a list of catalog items and a list of option definitions. Not every option definition in the list applies to every catalog item returned. Each catalog item is associated with zero or more option definition GUIDs, which will be included in the list returned in CatalogItemOrderInformation. Use only one selection from one of the definitions when you add a catalog item to the order. Use your discretion when choosing the definition to use for the order and the selection from within that definition.

Icon

Sometimes a catalog item may not be orderable; GetCatalogItemOrderInformation will return an exception if any catalog item is not orderable. 

Icon

Not all catalog items have associated definitions, in which case they do not need option selections when ordering. However, if there are any options that are required by a catalog item, you are required to fill them out before you can validate and/or submit the order, or else an error will be returned. 

The code listing below shows an example of getting the order options for two granules. 

Code Listing 56: Getting Order Options for a Catalog Item

 

String token = "[token obtained from login]"
// The items we're interested in 
String[] catalogItemGuids = new String[] { "G12345-PROV1""G23456-PROV1" };
Get the service OrderManagementServiceLocator locator = new
 
 
OrderManagementServiceLocator(); OrderManagementServicePort orderService = 
locator.getOrderManagementServicePort();
Get the order information CatalogItemOrderInformation itemInfo =
 
 
orderService.getCatalogItemOrderInformation(token, catalogItemGuids);

 

Icon

In order to utilize the description and sort key option definition fields, clients must use the new complimentary operations. New operations are identified with the number 2, e.g. GetCatalogItemOrderInformation2 utilizes the new OptionDefinition2 type. See the CMR Web Services API documentation for all instances.

Creating an Order

Once you have collected the catalog item GUIDs and associated options, you can create an order by calling the CreateOrder operation on the OrderManagementService. Simply pass an array of item IDs to add to the order. 
Common Selection 
If you are adding one or more items to an order at the same time, and their selections are the same, you can choose a common selection for the CreateOrderAddOrderItemsCreateAndSubmitOrder, and UpdateOrderItems operations that will be applied to all order items passed in without an option selection. This reduces the amount of data transmitted to the CMR and is less resource-intensive because the CMR will need to validate the common selection only once. 
For each item in the array, you should set the following parameters:

Parameters on a Catalog Item

Parameter
Description
ItemGuidThe catalog item GUID from the item's metadata
QuantityOrderedThe quantity of the item you want to order
OptionSelection

Option selection that matches one of the option definitions associated with this item.

If no required option definitions are associated with the item, this will be an empty list.

The other attributes of Order Item should be empty when creating an order; the CMR will populate them automatically. The CMR will process the request and return a GUID identifying the newly created order. The code listing below shows an example of creating a simple order. 

Code Listing 57: Creating a Simple Order

 

OrderItem[] orderItems = new OrderItem[2]; orderItems[0] =
new OrderItem(nullnull"G12345-PROV1"null, (short2null); orderItems[1] =
new OrderItem(nullnull"G23456-PROV1"null, (short2null); String orderGuid = orderService.createOrder(token, orderItems,null);

 

Adding Order Items 
Once you have created an order, you may add other items to it using the AddOrderItems operation. This operation uses the GUID of the target order and a list of items to add. As with CreateOrder, the options other than those listed in the table above should be empty; the CMR will process the additions and return a list of GUIDs identifying these additions. The code listing below shows an example of adding items to an existing order. 

Code Listing 58: Adding Items to an Existing Order

 

orderItems[0] = new OrderItem(nullnull"G34567-PROV1"null, (short2null); 
orderItems[1] = new OrderItem(nullnull"G45678-PROV1"null, (short2null); 
orderService.addOrderItems(token, orderGuid, orderItems,null);

 

Icon

The CMR assigns each item in an order a GUID unique to that order. For example, adding the same granule to two different orders will produce two new GUIDs, one linking the granule to the first order and one linking the granule to the second order. 

Updating Order Items
Once you have added an item to an order, you can modify the options or quantity by calling UpdateOrderItems and passing in the modified items. In this case, you must fill in the GUID of each item since the CMR needs to findthe original item in your order to update it. To ensure you are updating the appropriate item with the correct information, follow these steps:

  • Call GetOrderItems to populate the data.
  • Modify the desired order items.
  • Pass the results to UpdateOrderItems.

The code listing below shows an example of updating an existing item.

 

Code Listing 59: Updating an Item in an Order

 

NameGuid[] itemNames = orderService.getOrderItemNamesByOrder(token, orderGuid); 
String[] orderItemGuids = new String[] { itemNames[0].getGuid() }; 
OrderItem[] items = orderService.getOrderItems(token, orderItemGuids); items[0].setQuantityOrdered((short22); 
orderService.updateOrderItems(token, items,null);

 

Removing Order Items 

You may remove items from an order using the RemoveOrderItems operation as shown below: 

Code Listing 60: Removing an Item from an Order

 

itemNames = orderService.getOrderItemNamesByOrder(token, orderGuid); 
orderItemGuids = new String[] { itemNames[0].getGuid() }; 
orderService.removeOrderItems(token, orderItemGuids);

 

Removing Orders and Canceling Orders

To delete a specific provider order(s), use the RemoveProviderOrders operation. You cannot delete orders that you have already submitted. For post-submittal orders, a registered user can use the CancelOrder operation on the OrderManagementService to cancel an open order. 
Guest orders cannot be deleted in the post-submittal phase; the CancelOrder operation will delete the entire order, including all associated orders.

Setting User Information for an Order

Every order must have its own user-specific information attached to it so that a data provider can process the order. Contact information, billing, and shipping addresses are all required as well as user domain and region. Each order is independent of other orders, so user information must be set up for each order created. 
You may set up user information at any point in the order process prior to submission. User information is not required until you validate, quote, or submit the order. If the required information is not present at that time, you will receive an error. You can set up user information for an order by calling the SetUserInformationForOrder operation on the OrderManagementService.

Validating an Order

To ensure that an order is complete, with all the required options and user-associated information for that order filled out, the CMR validates all orders when submitted or quoted. If you attempt to submit an invalid order, you will receive an error message. Currently, the message reflects only the first validation error encountered; you need to fix the reported error and revalidate until the order is error-free. At this point, if you change any aspect of the validated order before finally submitting it, then the order will again require validation. To validate orders, call SubmitOrder or QuoteOrder. You can manually validate orders with the ValidateOrder operation in the 
OrderManagementService.

Keep in mind that running the ValidateOrder operation may return a validation error related to one of the many provider-specific validation rules, which may change over time. You should find the error messages complete enough to help you determine what aspects of the order you need to correct. 

Note that validation errors can occur if the order specified does not exist, or does not belong to you.

Requesting a Quote for an Order

Requesting a quote is an optional step that not all providers support. A quoted order provides a binding price for the order as a whole. The binding quote response from each of the necessary providers may take a day or more to receive. You cannot make changes to an order while it is in the quoting process. If you make changes to the order subsequent to the receipt of a quote, the changes may invalidate the quoted price. You may request a quote from a provider by calling QuoteOrder.

Submitting a Data Request

Once you have fully validated a data request and have made no subsequent changes to the request, submit the request to the system using the SubmitOrder operation in the OrderManagementService
Specify your preference for receiving order status information using one of the NotificationLevel choices shown in Table 10: CMR Order Notification Levels

CMR Order Notification Levels

Notification Level
Description
VERBOSE

The system will e-mail you all provider order state changes and status message updates.

(See the next table for more on order state changes.)

DETAILThe system will e-mail you provider order state changes only.
INFOThe system will e-mail you when a provider order is closed or canceled.
CRITICALThe system will e-mail you if the order fails during submission or is rejected by the provider. 
NONEThe system will not send e-mail.

If you do not specify any notification level for a registered user's order, the CMR will use your preference value. If you, as a registered user, have not set a preference value, the default value is INFO. The notification for all guests' orders is also INFO
Once an order transmits to the system using the SubmitOrder operation, the CMR sends each provider order contained within your order to the appropriate provider. Each provider will decide whether to accept your provider order and how to process it. 

Tracking and Canceling Orders 
From this point on, you can track the status of an order using the GetOrders operation in the OrderManagementService. At any point before the order actually ships, you may request cancellation of the order through the CancelOrder operation. You can then use the GetOrders operation again to track whether you successfully cancelled the order. 
When the CMR sends a request to a provider to quote, submit or cancel a particular order, both the CMR and that provider must have the same ID for that order. The CMR identifies the specific provider order by the ProviderOrderGUID (which is the combination of the provider GUID and the order GUID). However, the provider may also have its own ID system for tracking a particular provider order. As a solution, the provider order in the CMR also contains a provider-tracking ID. When the provider receives the request from the CMR, the provider has the option to accept the order GUID that the CMR has assigned to that provider order, or to provide the CMR with its own unique ID. If the provider does pass back its own unique ID as the provider-tracking ID, then the CMR will use that ID in addition to the normal CMR ProviderOrderGUID to refer to this provider order. If the provider does not specify a tracking ID, then the provider order will be referenced using only the CMR ProviderOrderGUID.

Order States

An order consists of zero or many provider orders. An order state is calculated based on the state of the individual provider orders that make up the order. The table below shows the order states for orders made up of zero or one provider order or of multiple provider orders having the same provider order state. 

Order States with 0 or 1 Provider Order

# of Provider Orders 

Provider Order StateOrder State
0-NOT_VALIDATED

 

 

1

-Same as provider order state
QUOTE_FAILED

QUOTED_WITH_EXCEPTIONS

QUOTE_REJECTED
SUBMIT_FAILEDSUBMITTED_WITH_EXCEPTIONS
SUBMIT_REJECTED

Restricted or Deleted Order Items 
Some catalog items you may order only with permission. Each provider sets the restrictions and permissions on catalog item ordering. Providers may also delete catalog items at any time. Restrictions can apply to individual users, groups of users or all users. If a catalog item is not available to you, executing the 
CreateAndSubmitOrder, GetCatalogItemOrderInformationCreateOrder, or AddOrderItems operation with this catalog item will return an error message indicating that it is not an orderable item. If an item becomes unavailable after you have created an order, you will receive an error message when you invoke the 
UpdateOrderItemsValidateOrderQuoteOrder, or SubmitOrder operation on that order.

Metadata subscriptions

Appendices

 

Preface

This document provides a look and guide to the Common Metadata Repository (CMR) from the perspective of the Client Partner.

Conventions

  • All references to time are in Universal Time Coordinated (UTC).
  • Data Partners are also referred to as Data Providers.
  • Client Partners are also referred to as Client Developers.
  • Words in bold text are key words or concepts. 

  • Programming examples use a fixed width font, have upper/lower lines separating them 
    from the rest of the text, and are in this color font. 

     

  • Comments (denoted by // within examples) 

  • Best practices or warnings appear in italicized, boxed text.

 

  • No labels