Chapter 2: Getting Started
This chapter will discuss
- Creating and managing user accounts
- CMR session management - creating, using, and deleting tokens to provide authorization
If searching for and retrieving publicly available data is the only desired operation, this section can be skipped and the reader can go straight to Chapter 3.
User Accounts
User accounts are used to get access to restricted data, manage privileges, or to interact with other services and tools provided by the CMR or ECHO. User accounts for the CMR system are created and manage by the Earthdata Login (URS) system. If you need an account and don't already have one, please click on Earthdata Login to create one. Once created you can always go back to Earthdata Login to manage it. If you are part of a Data Provider group or other team the team administrator can set up permissions for you to access their restricted data. If you need special privileges you can always contact the CMR operational team at support@earthdata.nasa.gov and they can help you.
Creating and Managing CMR Sessions
The CMR uses tokens in request messages - the http call to CMR - to validate per request who the requester is and what privileges they have. For most searches, a token is not needed because the metadata records are open to everyone. When certain metadata records are restricted a token is needed so that privileged users can see and access those records. A Session is nothing more than a series of requests that use the same token meaning that you can use the same token for many requests before you delete it. All tokens expire at the end of a time period. At the time of this writing, the duration is 30 days. Because the token is used to track your session, it must be protected by client applications with the same level of security that you use for your login name and password.
To conduct a session the normal steps are:
- Create a token
- Do one or more of the following in any order:
- Search for records
- Retrieve records
- Delete the token
Create a Token
Now that you have created a token, you can search, retrieve records, and conduct other functionality through the CMR or ECHO APIs. This functionality is covered in later chapters of this document. Once finished interacting with the CMR the token can be deleted.
Delete the Token
Chapter 3: Searching for metadata
Client partners can search the CMR for metadata. Currently clients can search for collection and granule metadata. In the future clients will also be able to search for metadata describing services, visualizations, parameters (variables), and documents.
CMR Environment URLs
The CMR system, as described in the before you begin section, has three environments: The Systems Integration Test environment is where the CMR development team tests new functionality. This is the environment that first gets the newest upgrades, but it is the least stable. Once the CMR software has been tested it gets deployed to the User Acceptance Test environment. The environment here is quite stable and it is tested as a system for a couple of weeks before the software is deployed to the operational environment. Client Partners can test their software in either the SIT or UAT environment depending the level of integration and testing. The operational environment is the live system available to users around the world. All of the examples provided in the rest of the document are using the Systems Integration Test environment. To run the commands in the other environments just replace the SIT URL with either the UAT or OPS URL to use the API in the respective environments.
CMR Environment | Base API URL |
---|---|
Operations/Production (OPS) | |
User Acceptance Test (UAT) | |
Systems Integration Test (SIT) |
CMR Environments
Headers
Headers are a part of HTTP requests and for the CMR they provide information such as the content of the message (Content-Type), tokens to allow increased privileges (Echo-Token), the format of the data that gets returned (accept), etc. Content-Type is a standard HTTP header that specifies the content type of the body of the request for POST method messages. Search and retrieval requests support the following Content-Types. If the Content-Type is not specified, XML is assumed.
Body format | Content-Type |
---|---|
XML | application/xml |
JSON | application/json |
Content-Type headers
The Echo-Token allows the CMR to know who is making a request. The Token is in the format of XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX. A token must first be generated as described in the previous section. Once the requester has the token, the token can be placed into the http header for the necessary API calls.
If the caller wishes to control in what format and or specification the data gets returned they can use the Accept header. The following table lists the valid values. If this header or an alternative method is not used, the returned outcome will be a reference list of results in XML format.
Type Received | Accept HeaderValue | Comments |
---|---|---|
xml | application/xml | returns a reference list of results using the XML format |
json | application/json | returns a subset of metadata data list of results using the JSON format |
echo10 | application/echo10+xml | returns a full metadata record list of results in the echo 10 specification using the XML format |
iso | application/iso19115+xml | returns a full metadata record list of results in the ISO 19115-2 (MENDS) specification using the XML format |
iso19115 | application/iso19115+xml | returns a full metadata record list of results in the ISO 19115-2 (MENDS) specification using the XML format |
dif | application/dif+xml | supported for collections only and returns a full metadata record list of results in the DIF 9 specification using the XML format |
dif10 | application/dif10+xml | supported for collections only and returns a full metadata record list of results in the DIF 10 specification using the XML format |
csv | text/csv | supported for granules only and returns a subset of metadata list of results in a comma separated value format |
atom | application/atom+xml | returns a subset of metadata list of results in the ATOM specification using the XML format |
opendata | application/opendata+json | supported for collections only and returns a full metadata record list of results in the open data specification using the JSON format |
kml | application/vnd.google-earth.kml+xml | returns a subset of spatial metadata list of results using the KML specification in the XML format |
native | application/metadata+xml | returns a full metadata record list of results in their individual native specification using the XML format |
Accept Headers
For more information about the types please see the CMR API documentation.
Client-Id
Client-Id is another header that allows the client to specify their name. Client Partners are strongly encouraged to use this header for several reasons. The ID helps the CMR operations team monitor query performance per client and it can also make it easier for them to identify the requests if the client needs contact them for assistance. The ID helps NASA track how much traffic flows through a client provider and what kind of data their users are looking for.
Following are some examples for using the headers. The purple part of the example will be explained in this section, the rest will be described later:
The following curl command issues a search request with the search parameters contained in a file called searchterms in the current directory. The Content-Type - the specification and format of the searchterms file - is using the json format to specify the search parameters. The accept header states that we want the results as a reference list using the XML format. The Echo-Token header allows the CMR to know who is making the request for authorization purposes. The Client-Id header allows the operations team and NASA to monitor and track statistics.
curl -v -XPOST -H "Content-Type: application/json" -H "Echo-Token: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Accept: application/xml" -H "Client-Id: Client Partner Name" -d @searchterms
-i https://cmr.sit.earthdata.nasa.gov/search/collections
In the next example a user is issuing a search request using publicly available data and is returned a full metadata record list of results. Notice that only the Accept header is needed, but the Client-Id is encouraged.
curl -v -H "Accept: application/metadata+xml" -H "Client-Id: Client Partner Name" -i https://cmr.sit.earthdata.nasa.gov/search/collections
In the next example a user is issuing a search request using publicly available data and the default result reference list. Notice that no headers are needed, but again the Client-Id is encouraged.
curl -v -H "Client-Id: Client Partner Name" -i https://cmr.sit.earthdata.nasa.gov/search/collections
As an alternate to using the Accept header are extensions where the client can use the Type Received name in the query to get the same results. Instead of using "Accept: application/opendata+json", "opendata" can be used at the end of the main query before parameters are specified.
curl -v -H "Client-Id: Client Partner Name" -i "https://cmr.sit.earthdata.nasa.gov/search/collections.opendata"
Other examples use the DIF 10 specification and the ISO specification respectively.
curl -v -H "Client-Id: Client Partner Name" -i "https://cmr.sit.earthdata.nasa.gov/search/collections.dif10"
curl -v -H "Client-Id: Client Partner Name" -i "https://cmr.sit.earthdata.nasa.gov/search/collections.iso"
To change what is wanted in the request just replace the header as needed per the tables above.
Results
There are a several of types of results that can be returned. For each of these types different formats are supported.:
- A reference list of results
- XML
- A list of results with partial metadata records being provided
- XML
- JSON
- ATOM
- CSV - supported for granules only
- KLM
- A list of results with full metadata records being provided
- XML
- opendata
The following is an example of a reference list of results
<results>
<hits>2215</hits>
<took>16</took>
<references>
<reference>
<name>100m Digital Elevation Model Data V001</name>
<id>C1000000803-DEV08</id>
<location>https://cmr.sit.earthdata.nasa.gov:443/search/concepts/C1000000803-DEV08>
<revision-id>8</revision-id>
</reference>
<reference>
<name>100m Digital Elevation Model Data V001</name>
<id>C1000000719-EDF_OPS</id>
<location>https://cmr.sit.earthdata.nasa.gov:443/search/concepts/C1000000719-EDF_OPS>
<revision-id>8</revision-id>
</reference>
...
</references>
</results>
The results specify
- how many metadata records were found by the "hits" tag
- how long the query took in milliseconds
- a list of metadata record results specified by the "reference" tag
Within each reference tag a limited amount of information about the metadata is provided.
- The metadata name
- The CMR profile or concept ID - a CMR generated unique ID. The ID is encoded by a letter of the profile or concept (C for collection, G for granule, S for service), followed by a CMR generated number, followed by a "-" and then followed by the ID of the metadata provider.
- The exact CMR location to download the metadata
- The latest revision number of the metadata record.
The following is an example of a full metadata record list of results in the ECHO 10 specification using the XML format.
<results>
<hits>2215</hits>
<took>53</took>
<result concept-id="C1000000803-DEV08"
format="application/echo10+xml" revision-id="8">
<Collection>
<ShortName>DEM_100M</ShortName>
<VersionId>1</VersionId>
<InsertTime>2002-04-27T15:27:55.293Z</InsertTime>
<LastUpdate>2013-10-04T08:49:26.783Z</LastUpdate>
<LongName>100m Digital Elevation Model Data</LongName>
...
</Collection>
</result>
<result concept-id="C1000000719-EDF_OPS"
format="application/echo10+xml" revision-id="8">
<Collection>
<ShortName>DEM_100M</ShortName>
</result>
</results>
The results specify
- how many metadata records were found by the "hits" tag
- how long the query took in milliseconds
- a list of metadata record results specified by the "result" tag
Within each result tag three attributes are shown about the metadata record followed by the full metadata record. The attributes display the CMR concept id (profile ID), the specification and format of the metadata, and the revision number of the shown metadata record. For detailed information about the result specifications and formats available with examples please see the CMR API documentation.
Searching
There are several ways to search the CMR system all using the RESTful principles:
- Using the API calls and parameters with the GET method
- Using the API call and parameters with the POST method
- Using a JSON query language with a POST method
- Using the Alternative Query Language (AQL)
API calls and parameters GET method
The most popular and preferred way is to use the API calls and parameters with the GET or POST methods. For detailed documentation the API documentation is located at https://cmr.earthdata.nasa.gov/search/site/search_api_docs.html.
The CMR URL character limit is currently set to take roughly 500k characters. Clients using the Search API with query parameters should be careful not to exceed this limit or they will get an HTTP response of 413 FULL HEAD. If a client expects that the query url could be extra long so that it exceeds 500k characters, they should use the POST method for searching instead of the GET method. First we will describe search using the GET method.
The basic search command is shown in the following example
curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/collections"
The query returns the first 10 publicly available collection results in a reference list using the XML format. As described in the header section to see restricted data, if you have the privileges you will need to use a token.
There are search parameters that can be applied to provide more functionality. They are listed below
- page_size - The number of results per page - the default is 10. 0 and 2000 are the minimum and maximum values respectively. For example: page_size=100 shows 100 result records per page if that many results exist.
- page_num - The page number to return. For example: page_num=1 is the first page of results; page_num=2 is the second page of results; page_num=10 is the 10th page of results.
- sort_key - Indicates one or more elements to sort on. For example: sort_key[]=platform (the brackets "[" and "]" may need to be escaped by using the \ character)
- pretty - Returns formatted - readable - results if set to true. For example pretty=true. For all the returned examples in this document this flag is used.
- token - Specifies the client token. This is an alternative to using the Echo-Token header.
- echo-compatible - This is used by systems requiring ECHO results. To get the best use out of CMR Client Partners shouldn't use this parameter.
These search parameters are for collection requests only.
- include_has_granules - Includes a has-granules tag or attribute in the response so the client knows if the collection encompasses any granules. E.g. include_has_granules=true
- include_granule_counts - Includes a granule-counts tag or attribute in the response with the number of granules represented by the collection. E.g. include_granule_counts=true
- include_facets - Includes a list of facets and their counts at the end of the results. This is mainly used for collection search displays. E.g. include_facets=true
- include_facets with hierarchical_facets - Includes a list of facets preserving the hierarchical order. This is mainly used for collection search displays. E.g include_facets=true&hierarchical_facets=true
In next example box we will see a set of examples conducting a basic search with using the search parameters just described. The first example shows a client wanting to see 50 metadata references per page. The second example shows 50 metadata references per page using the formatted print. The third example shows page 2 of 20 results per page showing full records in the echo 10 specification using formatted print. The fourth example issues a request including token and client id headers and does a basic search sorting the results using the platform element. The request returns page 2 results of 20 results per page showing full records in the ISO 19115 specification using the XML format in a formatted fashion. As one can see the ? character separates the URL from the search parameters and the search parameters are separated by the & character. The parameters can be in any order.
curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/collections?page_size=50"
curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/collections?page_size=50&pretty=true"
curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/collections.echo10?page_num=2&page_size=20&pretty=true"
curl -v -i -H "Echo-Token: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team" "https://cmr.sit.earthdata.nasa.gov/search/collections.iso?sort_key\[\]=platform&page_num=2&page_size=20&pretty=true"
The previous examples all demonstrated searches for collections. The same search parameters apply to granules if it isn't stated that a parameter applies only to collections. To conduct a granule search, just replace collections with granules in the URL. Following are the same four search requests just listed above but for granules instead of collections.
curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/granules?page_size=50"
curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/granules?page_size=50&pretty=true"
curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/granules.echo10?page_num=2&page_size=20&pretty=true"
curl -v -i -H "Echo-token: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team" "https://cmr.earthdata.nasa.gov/search/granules.iso?sort_key\[\]=platform&page_num=2&page_size=20&pretty=true"
Now to refine our searches we can use another set of search parameters documented in the table below. These parameters support collection searches and most of these parameters have the brackets next to them and may need to be escaped (\[\]) depending on the language used or how the query is being sent. All of CMR time search parameters (temporal, updated_since, revision_date, and equator_crossing_date) formats are specified as yyyy-MM-ddTHH:mm:ss.SSSZ format. Where yyyy is year; MM is month; dd is day; T is the date time separator character; HH is the hour; mm is the minute; ss is the second; SSS is the milliseconds (the .SSS can be omitted); and Z specifies Zulu time. January 2 2000 at 5 seconds and 4 minutes past 3 O'clock in the morning Zulu time is represented as 2000-01-02T03:04:05Z.
To get the complete and most up to date set of parameters please see https://cmr.earthdata.nasa.gov/search/site/search_api_docs.html.
Search Parameters | Example | Notes | Supports Pattern Option | Supports Case Insensitivity Option | Supports AND Option (ALL values have to be present within an element) | Supports OR Option (ANY value has to be present within an element) |
---|---|---|---|---|---|---|
concept_id | concept_id\[\]=C123456-LPDAAC_ECS | NO | NO | NO | NO | |
echo_collection_id | echo_collection_id\[\]=C1000000001-CMR_PROV2 | uses concept_id | NO | NO | NO | NO |
entry_title | entry_title\[\]=this is a title | YES | YES | NO | NO | |
dataset_id | dataset_id\[\]=this is a title | uses entry_title | N/A | N/A | NO | NO |
entry_id | entry_id\[\]=SHORT_V5 | NO | NO | NO | NO | |
dif_entry_id | dif_entry_id\[\]=SHORT_V5 | matches either entry_id or associated difs | NO | NO | NO | NO |
archive_center | archive_center\[\]=SEDAC | YES | YES | NO | NO | |
temporal | temporal\[\]=2000-01-01T10:00:00Z,2010-03-10T12:00:00Z,30,60 or temporal\[\]=2000-01-01T10:00:00Z/P10Y2M10DT2H,30,60 | format is: begin datetime, end datetime, period, duration or: begin datetime/ISO 8601 time interval One can leave out the begin time or end time or both the period and duration ranges are inclusive unless otherwise specified | N/A | N/A | NO | NO |
project | project\[\]=ESI | YES | YES | YES | NO | |
campaign | campaign\[\]=ESI | uses project | YES | YES | YES | NO |
updated_since | updated_since=2000-01-01T01:00:00Z | The time is inclusive. | NO | N/A | NO | NO |
revision_date | revision_date\[\]=2000-01-01T01:00:00Z,2010-01-01T12:34:56Z revision_date\[\]=2000-01-01T01:00:00Z, | The beginning or ending date time can be left off, but comma must remain. Inclusive boundary search | NO | N/A | YES | NO |
processing_level_id | processing_level_id\[\]=1B | YES | YES | NO | NO | |
platform | platform\[\]=AQUA | platform short name | YES | YES | YES | NO |
instrument | instrument\[\]=CERES | instrument short name | YES | YES | YES | NO |
sensor | sensor\[\]=CCD | sensor short name | YES | YES | YES | NO |
spatial_keyword | spatial_keyword\[\]=VA | YES | YES | YES | NO | |
science_keywords | science_keywords\[0\]\[category\]=EARTH SCIENCE&science_keywords\[0\]\[topic\]=BIOLOGICAL CLASSIFICATION&science_keywords\[0\]\[term\]=ANIMALS/VERTEBRATES&science_keywords\[0\]\[variable-level-1\]=MAMMALS&science_keywords\[0\]\[variable-level-2\]=CARNIVORES&science_keywords\[0\]\[variable-level-3\]=BEARS | There is a hierarchy for science keywords. These can be ANDed together which is the default or ORed. | NO | NO | YES | YES |
two_d_coordinate_system_name | two_d_coordinate_system_name\[\]=Alpha | YES | NO | NO | NO | |
two_d_coordinate_system[name] | two_d_coordinate_system\[name\]=Alpha | alias of two_d_coordinate_system_name but does not support pattern | NO | NO | NO | NO |
collection_data_type | collection_data_type\[\]=NEAR_REAL_TIME | valid values for near real time: "NEAR_REAL_TIME": "near_real_time", "nrt", "NRT", "near real time", "near-real time", "near-real-time", "near real-time" ALSO uses OTHER, SCIENCE QUALITY | NO | YES | NO | NO |
provider | provider=ASF | YES | YES | YES | NO | |
short_name | short_name=MINIMAL | YES | YES | YES | NO | |
version | version=1 | used together with short_name | YES | YES | YES | NO |
polygon | polygon=10,10,30,10,30,20,10,20,10,10 | Polygon points are provided in counter-clockwise order. The last point should match the first point to close the polygon. The values are listed comma separated in longitude latitude order, i.e. lon1, lat1, lon2, lat2, lon3, lat3, and so on. | N/A | N/A | NO | NO |
bounding_box | bounding_box=-10,-5,10,5 | Bounding boxes define an area on the earth aligned with longitude and latitude. The Bounding box parameters must be 4 comma-separated numbers: lower left longitude, lower left latitude, upper right longitude, upper right latitude. | N/A | N/A | NO | NO |
point | point=100,20 | Search using a point involves using a pair of values representing the point coordinates as parameters. The first value is the longitude and second value is the latitude. | N/A | N/A | NO | NO |
line | line=-0.37,-14.07,4.75,1.25,25.13,-15.51 | Lines are provided as a list of comma separated values representing coordinates of points along the line. The coordinates are listed in the format lon1, lat1, lon2, lat2, lon3, lat3, and so on. | N/A | N/A | NO | NO |
keyword | keyword=alpha
| By default keyword searches are case insensitive and support wild cards ? and *. The following elements are searched by a keyword search:
| NO | NO | NO | NO |
online_only | online_only=true | valid values: true, false | NO | NO | NO | NO |
downloadable | downloadable=true | valid values: true, false | NO | NO | NO | NO |
browse_only | browse_only=false | valid values: true, false | NO | NO | NO | NO |
browsable | browsable=true | valid values: true, false | NO | NO | NO | NO |
Collection Search Parameters
Documented in the table below are granule supported search parameters.
Search Parameters | Example | Notes | Supports Pattern Option | Supports Case Insensitivity Option | Supports AND Option (ALL values have to be present within an element) | Supports OR Option (ANY value has to be present within an element) |
---|---|---|---|---|---|---|
granule_ur | granule_ur\[\]=SC:AST_L1B.003:2082836137 | NO | NO | NO | NO | |
producer_granule_id | producer_granule_id\[\]=AST_L1B_00304092000162008_20110111183559_9769.hdf | NO | NO | NO | NO | |
readable_granule_name | readable_granule_name\[\]=SC:AST_L1B.003:2082836137 | matches either granule ur or producer granule id | NO | NO | NO | NO |
online_only | online_only=true | valid values: true, false | NO | NO | NO | NO |
downloadable | downloadable=true | valid values: true, false | NO | NO | NO | NO |
attribute | attribute\[\]=UpperLeftQuadCloudCoverage attribute\[\]=float,UpperLeftQuadCloudCoverage,25.5,30 attribute\[\]=float,UpperLeftQuadCloudCoverage,25.5, attribute\[\]=float,UpperLeftQuadCloudCoverage,,30 attribute\[\]=int,UpperLeftQuadCloudCoverage,4 attribute\[\]=float,UpperLeftQuadCloudCoverage,25.5,30&options\[attribute\]\[or\]=true attribute\[\]=float,UpperLeftQuadCloudCoverage,25.5,30&options\[attribute\]\[exclude_boundry\]=true attribute\[\]=float,UpperLeftQuadCloudCoverage,25.5,30&options\[attribute\]\[exclude_collection\]=true | full syntax:name - attribute name only full syntax: value type, attribute name, min value, max value - range search, can leave off beginning or ending of range, but comma is still needed. Ranges are inclusive. If this is not desired set to true the exclude_boundry option. full syntax:value type, attribute name, value - single value attribute. These searches include the granule collection - if this is not desired set to true the option exclude_collection | NO | NO | YES | YES |
polygon | polygon=10,10,30,10,30,20,10,20,10,10 | Polygon points are provided in counter-clockwise order. The last point should match the first point to close the polygon. The values are listed comma separated in longitude latitude order, i.e. lon1, lat1, lon2, lat2, lon3, lat3, and so on. | N/A | N/A | NO | NO |
bounding_box | bounding_box=-10,-5,10,5 | Bounding boxes define an area on the earth aligned with longitude and latitude. The Bounding box parameters must be 4 comma-separated numbers: lower left longitude, lower left latitude, upper right longitude, upper right latitude. | N/A | N/A | NO | NO |
point | point=100,20 | Search using a point involves using a pair of values representing the point coordinates as parameters. The first value is the longitude and second value is the latitude. | N/A | N/A | NO | NO |
line | line=-0.37,-14.07,4.75,1.25,25.13,-15.51 | Lines are provided as a list of comma separated values representing coordinates of points along the line. The coordinates are listed in the format lon1, lat1, lon2, lat2, lon3, lat3, and so on. | N/A | N/A | NO | NO |
orbit_number | orbit_number=10 orbit_number=0.5,1.5 | value or range | NO | NO | NO | NO |
equator_crossing_longitude | equator_crossing_longitude=90 equator_crossing_longitude=170,-170 | value or range | NO | NO | NO | NO |
equator_crossing_date | equator_crossing_date=2000-01-01T10:00:00Z,2010-03-10T12:00:00Z | date range searches can be expressed using ISO 8601 | NO | NO | NO | NO |
updated_since | updated_since=2015-01-01T13:12:11Z | NO | N/A | NO | NO | |
revision_date | revision_date\[\]=2015-03-04T16:15:14Z,2015-04-04T17:18:19Z revision_date\[\]=2015-03-04T16:15:14Z, | The beginning or ending date time can be left off, but comma must remain. Inclusive boundary search | NO | N/A | YES | NO |
cloud_cover | cloud_cover=-70.0,120.0 | The beginning or ending range can be left off, but comma must remain. Inclusive boundary search | NO | N/A | NO | NO |
platform | platform\[\]=AQUA | platform short name | YES | YES | YES | NO |
instrument | instrument\[\]=CERES | instrument short name | YES | YES | YES | NO |
sensor | sensor\[\]=CCD | sensor short name | YES | YES | YES | NO |
project | project\[\]=ESI | YES | YES | YES | NO | |
campaign | campaign\[\]=ESI | uses project | YES | YES | YES | NO |
concept_id | concept_id\[\]=G123456-LPDAAC_ECS concept_id\[\]=C123456-LPDAAC_ECS | This finds either the granule or the collection parent record - the difference is in the ID (C vs G) | NO | NO | NO | NO |
echo_granule_id | echo_granule_id\[\]=G1000000001-CMR_PROV2 | uses concept_id | NO | NO | NO | NO |
collection_concept_id | collection_concept_id\[\]=C123456-LPDAAC_ECS | NO | NO | NO | NO | |
echo_collection_id | echo_collection_id\[\]=C123456-LPDAAC_ECS | NO | NO | NO | NO | |
day_night_flag | day_night_flag=day | valid values are day, night, unspecified | YES | YES | NO | NO |
day_night | day_night=unspecified | valid values are day, night, unspecified - uses the day-night-flag element. | YES | YES | NO | NO |
two_d_coordinate_system | two_d_coordinate_system\[\]=wrs-1:5,10:8-10,0-10:8,12 | see API docs for description | NO | NO | NO | NO |
grid | grid\[\]=wrs-1:5,10:8-10,0-10:8,12 | uses two_d_coordinate_system element | NO | NO | NO | NO |
provider | provider=ASF | YES | YES | YES | NO | |
short_name | short_name=MINIMAL | YES | YES | YES | NO | |
version | version=1 | used together with short_name | YES | YES | YES | NO |
entry_title | entry_title\[\]=this is a title | YES | YES | NO | NO | |
temporal | temporal\[\]=2000-01-01T10:00:00Z,2010-03-10T12:00:00Z,30,60 or temporal\[\]=2000-01-01T10:00:00Z/P10Y2M10DT2H,30,60 | format is: begin datetime, end datetime, period, duration or: begin datetime/ISO 8601 time interval One can leave out the begin time or end time or both the period and duration ranges are inclusive unless otherwise specified | N/A | N/A | NO | NO |
exclude | exclude\[echo_granule_id\]\[\]=G100000006-CMR_PROV exclude\[concept_id\]\[\]=G100000006-CMR_PROV exclude\[concept_id\]\[\]=C100000006-CMR_PROV | exclude metadata records by echo_granule_id, concept id, or parent concept id. | NO | NO | NO | NO |
Granule Search Parameters
In the following example we wish to find all collection metadata records that contain an AQUA platform and we would like the to see only a formatted reference list of results that contain 20 references.
curl -v -i -H "Echo-Token: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team" "https://cmr.sit.earthdata.nasa.gov/search/collections?platform\[\]=AQUA&page_size=20&pretty=true"
In the next example we wish to find all collection metadata records that contain an AQUA or an AURA platform and we would like the to see only a formatted reference list of results that contain 20 references.
curl -v -i -H "Echo-Token: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team" "https://cmr.sit.earthdata.nasa.gov/search/collections?platform\[\]=AQUA&platform\[\]=AURA&page_size=20&pretty=true"
There are a couple of extra options that certain search parameters have to aid the user. To use these options the syntax is: options[parameter name][option_key]=value. Parameter name is the name of the search parameter to be affected such as platform. Value is either set to true or false. Option_key is one of the following:
Option name | Description |
---|---|
ignore_case | If "ignore_case" is set to true the search will be case insensitive and if set to false the search will be case sensitive. The default value is true. E.g. ignore_case=true - the search will match on both AQUA and aqua |
pattern | This is the wildcard capability. If "pattern" is set to true the CMR will treat '*' as matches zero or more characters and '?' matches any single character. For example: platform[]=AQUA will match only on the value 'AQUA'. if platform[]=A?U*&options[platform][pattern]=true platforms containing A followed by any alphanumeric character followed by U followed by any number of alphanumeric characters will be found. So AQUA, ASUBB, ADUSD34H, AUU, etc. will all be found. The pattern option defaults to false. |
and | If "and" is set to true and if multiple values are listed for the parameter, the metadata records must contain ALL of these values in order to match. The default is false meaning metadata records that match ANY of the values will match. |
or | This option only applies to granule attributes or science-keyword searches. If "or" is set to true, the search will find records that match any of the attributes. The default for this option is false. |
Extra Options
The following is an example of using options with the platform search parameter. This example will find any platform that matches A followed by any number of alphanumeric characters and ends with A. This will find both platforms of AQUA and AURA.
curl -v -i -H "Echo-Token: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team" "https://cmr.sit.earthdata.nasa.gov/search/collections.echo10?platform\[\]=A*A&options\[platform\]\[pattern\]=true&pretty=true"
This next example demonstrates a user looking to find records that only contain an instrument that matches "HELLO" in uppercase letters.
curl -v -i -H "Echo-Token: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team" "https://cmr.sit.earthdata.nasa.gov/search/collections.echo10?instrument\[\]=HELLO&options\[instrument\]\[ignore-case\]=false&pretty=true"
Besides science_keywords, if any of the parameters that are searched are repeated, the metadata records that have ANY of the values will match. The following example demonstrates that the CMR system will match any metadata record containing either concept id.
curl -v -i -H "Echo-Token: 75E5CEBE-6BBB-2FB5-A613-0368A361D0B6" -H "Client-Id: Test_Team" "https://cmr.sit.earthdata.nasa.gov/search/collections.iso?concept_id\[\]=C123456-LPDAAC_ECS&concept_id\[\]=C123457-LPDAAC_ECS&pretty=true"
For a complete set of examples using all of the search parameters please see the API documentation: https://cmr.earthdata.nasa.gov/search/site/search_api_docs.html
API calls and parameters POST method
The API using the POST method is the same as with the GET method with the exception being the method used is POST instead of GET and the parameters are in the body of the message without a length constraint instead of existing in the URL string. Using the curl command the following example shows the query.xml file that contains the query we want to execute followed by the curl search request. In the query.xml file the parameters can be left as one long set or formatted to be more easily read - so long as syntax remains the same. Also notice in this example we did not escape the brackets ([ ])
query.xml:
pretty=true&
page_size=1&
page_num=3&
sort_key[]=platform&platform[]=AQUA&platform[]=AURA&revision_date[]=2015-07-01T01:00:00Z,2016-01-01T01:00:00Z&revision_date[]=2014-01-01T01:00:00Z,2014-06-01T01:00:00Z&temporal[]=2000-01-01T10:00:00Z/2010-03-10T12:00:00Z&include_has_granules=true&include_granule_counts=true&include_facets=true&hierarchical_facets=true
curl -v -XPOST -i -d @query.xml "https://cmr.sit.earthdata.nasa.gov/search/collections.echo10"
Notice in this specific instance that the Content-Type header is not used. Don't use it, it will cause an error.
JSON query language with a POST method
For those who understand the JSON format, the CMR provides a JSON RESTful interface. The elements that can be searched are the same as already described above but this interface is for collection searches only. This searching method does provide additional functionality of using conditions (AND, OR, NOT) against the elements to conduct a search. See the JSON schema https://cmr.sit.earthdata.nasa.gov/search/site/JSONQueryLanguage.json for more details. The example provided below demonstrates a query with conditions and uses several elements.
curl -XPOST -H "Content-Type: application/json" -H "Client-Id: GCMD" https://cmr.sit.earthdata.nasa.gov/search/collections
-d '{"condition": { "and": [{ "not": { "or": [{ "provider": "TEST" },
{ "and": [{ "project": "test-project",
"platform": "mars-satellite" }]}]}},
{ "bounding_box": [-45,15,0,25],
"science_keywords": { "category": "EARTH SCIENCE" }}]}}'
Alternative Query Language (AQL)
The CMR supports the ECHO Alternative Query Language (AQL) if a client wishes to use this capability. While the AQL is supported it is not being enhanced nor modified to take advantage of new CMR features. For a very detailed explanation of AQL with examples of how to use it, please see the ECHO AQL documentation.
Chapter 4: Retrieving Metadata
There are several ways of retrieving metadata:
Retrieve a result list consisting of full metadata records. While this has been demonstrated above, a example has been replicated here for convenience.
Example
curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/collections.native?pretty=true"
curl -v -i "https://cmr.sit.earthdata.nasa.gov/search/granules.native?pretty=true"
Use the concept id to retrieve a record: The syntax is https://cmr.sit.earthdata.nasa.gov/search/concepts/<concept id>.
The concept id/revision number if a specific revision of a metadata record is wanted: The syntax is https://cmr.sit.earthdata.nasa.gov/search/concepts/<concept id>/<revision number>.
- The CMR supports retrieving metadata records using different specifications and formats, which are listed in the table below:
Type Received | Accept HeaderValue | Supports Revision | Supports Granules | Comments |
---|---|---|---|---|
xml | application/xml | YES | YES | returns a reference list of results using the XML format |
json | application/json | NO | YES | returns a subset of metadata data list of results using the JSON format |
echo10 | application/echo10+xml | YES | YES | returns a full metadata record list of results in the echo 10 specification using the XML format |
iso | application/iso19115+xml | YES | YES | returns a full metadata record list of results in the ISO 19115-2 (MENDS) specification using the XML format |
iso19115 | application/iso19115+xml | YES | YES | returns a full metadata record list of results in the ISO 19115-2 (MENDS) specification using the XML format |
dif | application/dif+xml | YES | NO | supported for collections only and returns a full metadata record list of results in the DIF 9 specification using the XML format |
dif10 | application/dif10+xml | YES | NO | supported for collections only and returns a full metadata record list of results in the DIF 10 specification using the XML format |
atom | application/atom+xml | NO | YES | returns a subset of metadata list of results in the ATOM specification using the XML format |
native | application/metadata+xml | YES | YES | returns a full metadata record list of results in their individual native specification using the XML format |
Supported Standards
Below are several examples using the supported standards mime types:
- Ex 1: Retrieves a granule metadata record in the JSON format.
- Ex 2: Retrieves a granule metadata record with a revision of 8 in the ISO specification.
- Ex 3: Retrieves a collection metadata record with a revision of 7 in the DIF 10 specification. (*Note: If the record was sent and stored using the ECHO10 specification, that record will be translated to the DIF 10 specification and returned to the caller.)
Ex 4: Lists a granule record using the native option with the pretty print option turned on. The native option lists the metadata in the specification it was sent and stored within the CMR.
Example
curl -v "https://cmr.sit.earthdata.nasa.gov/search/concepts/G23447-ASF.json"
curl -v "https://cmr.sit.earthdata.nasa.gov/search/concepts/G23447-ASF/8.iso"
curl -v "https://cmr.sit.earthdata.nasa.gov/search/concepts/C1000000803-DEV08/7.dif10"
curl -v "https://cmr.sit.earthdata.nasa.gov/search/concepts/G23447-ASF.native?pretty=true"
Chapter 5: Accessing data
There are a variety of ways to access the data of interest from the response of a search query:
- Access the data provider's site via the landing page in the collection or granule and follow their instructions for data retrieval. Note: This information is not required and may not be present.
- Order the data through the ECHO system. A detailed explanation of this process is located in the ECHO Client Partners Users Guide Chapter 6: Ordering data through ECHO.
Acronyms
Acronyms used throughout this document are contained in the table below.
API | Application Programming Interface |
AQL | Alternative Query Language |
ASF | Alaska Satellite Facility DAAC |
COTS | Commercial Off The Shelf |
DAAC | Distributed Active Archive Center |
ECHO | EOS Clearinghouse |
ECS | EOSDIS Core System |
EOS | Earth Observing System |
EOSDIS | EOS Data and Information System |
ESDIS | Earth Science Data and Information System |
FTP | File Transfer Protocol |
GCMD | Global Change Master Directory |
GMT | Greenwich Mean Time |
NASA | National Aeronautics and Space Administration |
SSL | Secure Sockets Layer |
URL | Uniform Resource Locator |
UTC | Universal Time, Coordinated (also called GMT/UTC) |
WRS | Worldwide Reference System |
XML | eXtensible Markup Language |
Best Practices for Queries
Below are some tips and recommended practices to increase the efficiency of queries.
To Enhance the Speed of Queries:
- Limit the end user choices — This will promote efficiency by displaying only choices applicable to user needs.
- Search for collections first and limit the collection search spatially, temporally, and/or by data center — Limiting the collection will result in a narrower search and a smaller, more focused result set.
- Request only what the user would see in the first few pages. For example, if the client only supports displaying 10 pages of 10 items, using a page size of 100 items will allow the client to pre-fetch the next page of results while the user is examining the first page.
- Use the value element — As a general rule, it will be more efficient than range element.
To Increase Efficiency of Spatial Queries:
- If querying a single Data Partner — name the Data Partner in the query.
- If querying a single collection — include the name of the collection in the query.
- Note: Queries for smaller spatial regions return faster than queries for broader regions.
- Note: Queries for spatial regions with fewer points return faster results than queries with more points