ECHO has been replaced by the Common Metadata Repository (CMR), a high-performance, high-quality, continuously evolving metadata system that catalogs all data and service metadata records for the EOSDIS system and will be the authoritative management system for all EOSDIS metadata.
The information contained within this ECHO wiki is now archived for historical reference. Please navigate to the CMR wiki pages, or to the CMR Overview page on Earthdata.
Deprecated
Metadata Subscriptions
Using Metadata Subscriptions to Automate Queries
The subscription service is used to deliver metadata updates on one or more collections from specific providers from ECHO to the subscribers. Users can create or update subscriptions for collections, granules, both collections and granules, or all collections and the spatial condition that they are interested in. These conditions include the addition, update, or deletion of metadata by a provider. This capability can be extremely valuable for client developers, because it provides the opportunity to set up post-processing and data massaging that runs when new or changed metadata is delivered, to convert it into an application-specific form.
The three key questions in creating a subscription are:
- What dataset are you interested in? To determine what providers exist in the system, you can call the GetProviderNames operation on the Provider Service, passing null for the GUIDs. Additionally, the Catalog Service discovery mechanism retrieves the datasets currently stored in the metadata repository.
What type of metadata from the dataset are you interested in? ECHO allows you to subscribe to collection-level metadata, granule-level metadata, or both
. Collection metadata will rarely change whereas granule metadata will be frequently updated or created; most users choose to subscribe to granule metadata for this reason. Be aware that providers may also delete granule metadata from time to time.
- Where should ECHO send the metadata when it is updated? ECHO supports two methods of subscription delivery: FTP and e-mail. Due to security restrictions, ECHO requires all FTP deliveries to be transferred via passive mode. If delivery is not possible (because the FTP server is full or the mail server rejects the large file attachment), ECHO sends an e-mail notification to the subscriber. Consult your local administrator for more information on passive-mode transfers and file size limitations.
- In addition to the basic subscription information listed above, you may provide an AQL query to the Subscription Service that will be executed on your behalf when your subscription is triggered. In this case, the results of your query are sent rather than the raw updated metadata.
Note that users can specify an expiration date for the subscription, whether the result should be compressed (only GZIP compression is implemented), a limit on the size (in megabytes) of what is sent to the user, a format type (defaulted to ECHO format type), the options to specify which attributes are to be included in the metadata update, and a mechanism for sending the metadata update.
Creating a Metadata Subscriptions
To create a metadata subscription, call CreateSubscriptions in the Subscription Service, and pass the detailed MetadataSubscription information as described in the API documentation. Note the subscription GUID should be null since you are requesting a new subscription be created. CreateSubscriptions will return a list of GUIDs to identify the newly created subscriptions. The GUIDs will be listed in the same order in which you supplied the
MetadataSubscription information.
Code Listing 61: Creating a Basic Subscription
String token = "[token obtained from login]";
Get the subscription service SubscriptionServiceLocator locator = new SubscriptionServiceLocator();
SubscriptionServicePort subService = locator.getSubscriptionServicePort();
//The information we want to subscribe to
String providerGuid = "[provider guid obtained using ListAllProviders]";
String datasetName = "dataset name obtained through a query]";
String query = "[AQL query assembled to limit the data returned]";
// Filtering information
MetadataFilterInfo filterInfo = new MetadataFilterInfo(query, 5);
MetadataActionInfo actionInfo = new MetadataActionInfo(null, CompressionType.GZIP, SubscriptionUpdateType.COLLECTIONS_ONLY);
//Delivery information
DeliveryInfo deliveryInfo = new DeliveryInfo(DeliveryType.EMAIL, "example@example.org", null, null, 20);
Calendar stopTime = Calendar.getInstance(TimeZone.getTimeZone("GMT"));
stopTime.add(Calendar.YEAR, 1);
//Package it up in a MetadataSubscription element
MetadataSubscription[] metadataSubscriptions = new MetadataSubscription[1];
metadataSubscriptions[0] = new MetadataSubscription(null, null, "Example Simple Subscription", providerGuid, datasetName, filterInfo, actionInfo, deliveryInfo,
stopTime);
//Ask ECHO to create the subscription
String[] guids = subService.createSubscriptions(token, metadataSubscriptions);
If you provide a temporal condition in an AQL query for your subscription, you must leave NumberOfDays empty in MetadataFilterInfo. Likewise, if you use NumberOfDays, you may not have a temporal constraint in your AQL.
Each metadat subscription must have the following information:
Table 12: Metadata Subscription Information
| Subscription Parameter | Description |
|---|---|
| Subscription Name | A unique name to describe this subscription. In ECHO 10.10, the name must be unique. |
| Provider GUID | The unique ID of the provider you wish to subscribe to. If you want to subscribe to all providers, you may use an asterisk (*) with constraints. |
| Dataset Name | The name of the dataset you wish to subscribe to or asterisk (*) to indicate all datasets from the selected provider(s). |
| Metadata Filter Info | The spatial and/or temporal constraint to use for the subscription to narrow the data to a specific geographic location or period(s) of time. The AQL query filter is used to specify an IIMS AQL condition when validating metadata updates specified in your subscription preferences. The temporal filter element is an integer that represents a specified number of days prior to today‘s date--that is, a span of days--that allows you to focus on data whose acquisition dates fall within that span. Note that you can only set the temporal filter either in the AQL TemporalCondition or in the NumberOfDays. It is an ERROR to set it in both. |
| Metadata Action Info | The information related to processing of the subscription. Like Query, you can limit the metadata fields returned by a subscription by specifying a list of MetadataAttributes. |
| Compression Type | The compression type to use on the metadata file. Note the subscription files delivered from ECHO are XML-formatted and highly compressible. It is recommended that you use a CompressionType of GZIP to conserve space and bandwidth. |
| Subscription Update Type | The type of metadata to subscribe to:
|
| Delivery Info | The way in which metadata should be delivered from the subscription: EMAIL or FTPPUSH. If email is chosen, the DeliveryAddress should be of the format "username@domain". If FTP Push is chosen, the DeliveryAddress should also be of the format "username@ftp.domain". For FTP deliveries, the DeliveryFolder and Password fields are required. The Subscription Service has a governor that prevents delivery of metadata above a certain threshold. LimitSize is your specified maximum acceptable size (in megabytes) of the delivery file. The system calculates the size of the data file prior to delivery; if the size exceeds LimitSize, the file is not delivered. |
| StopTime | The time after which metadata should cease to be delivered. |
Deleting a Subscription
To remove a subscription, call the RemoveSubscriptions operations with the GUIDs of the subscriptions to remove, as shown below.
Code Listing 62: Removing a Subscription
String[] subGuids = new String[] { "[subscription guid]" };
subService.removeSubscriptions(token, subGuids);
Listing a Subscription
You can list the names and GUIDs of subscriptions using the GetSubscriptionNames and GetSubscriptionNamesByState operations.
Code Listing 63: Listing Subscriptions
subGuids = new String[] { "[subscription guid]" };
NameGuid[] allActiveNames = subService.getSubscriptionNames(token, null);
NameGuid[] specificActiveNames = subService.getSubscriptionNames(token, subGuids);
NameGuid[] allExpiredNames = subService.getSubscriptionNamesByState(token, SubscriptionState.EXPIRED);
Code Listing 64: Sample Granule Query Using Various Granule Conditions (Inventory)
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE query PUBLIC "-//ECHO CatalogService (v10)//EN" "http://api.echo.nasa.gov/echo/dtd/IIMSAQLQueryLanguage.dtd"> <!-- Search for granules from the L70R or L70RWRS or GLCF_GRANULE_METADATA datasets that have Browse data and were categorized as day granules. --> <query> <for value="granules"/> <dataCenterId> <list> <value>ORNL_DAAC</value> </list> </dataCenterId> <where> <granuleCondition> <browseOnly/> </granuleCondition> <granuleCondition> <cloudCover> <range lower="10" upper="20"/> </cloudCover> </granuleCondition> <granuleCondition> <dataSetId> <list> <value>'L70R'</value> <value>'L70RWRS'</value> <value>'GLCF_GRANULE_METADATA'</value> </list> </dataSetId> </granuleCondition> <granuleCondition> <dayNightFlag value="DAY"/> </granuleCondition> </where> </query>
Code Listing 65: Sample Granule Query Using Spatial and Temporal Bounds (Inventory)
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE query PUBLIC "-//ECHO CatalogService (v10)//EN" "http://api.echo.nasa.gov/echo/dtd/IIMSAQLQueryLanguage.dtd"> <!-- Search for granules the specified spatial region, and periodic temporal extent --> <query> <for value="granules"/> <dataCenterId> <list> <value>ORNL_DAAC</value> </list> </dataCenterId> <where> <granuleCondition> <spatial operator="RELATE"> <IIMSPolygon> <IIMSLRing> <IIMSPoint lon="-120" lat="-30" /> <IIMSPoint lon="-100" lat="-60" /> <IIMSPoint lon="-5" lat="-90" /> <IIMSPoint lon="5" lat="85" /> <IIMSPoint lon="-120" lat="30" /> <IIMSPoint lon="-120" lat="-30" /> </IIMSLRing> <IIMSLRing> <IIMSPoint lon="80" lat="20" /> <IIMSPoint lon="80" lat="60" /> <IIMSPoint lon="20" lat="60" /> <IIMSPoint lon="20" lat="20" /> <IIMSPoint lon="80" lat="20" /> </IIMSLRing> </IIMSPolygon> </spatial> </granuleCondition> <granuleCondition> <temporal> <startDate> <Date YYYY="1989" MM="01" DD="01"/> </startDate> <stopDate> <Date YYYY="1998" MM="12" DD="31"/> </stopDate> <startDay value="1"/> <endDay value="300"/> </temporal> </granuleCondition> </where> </query>
Code Listing 66: Sample Granule Query Using Sensor Name (Inventory)
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE query PUBLIC "-//ECHO CatalogService (v10)//EN" "http://api.echo.nasa.gov/echo/dtd/IIMSAQLQueryLanguage.dtd"> <!-- Search for granules from ORNL_DAAC with source name SWIR or TIR --> <query> <for value="granules"/> <dataCenterId> <list> <value>ORNL_DAAC</value> </list> </dataCenterId> <where> <granuleCondition> <sensorName> <list> <value>'SWIR'</value> <value>'TIR'</value> </list> </sensorName> </granuleCondition> </where> </query>
Code Listing 67: Sample Granule Query Using Temporal Constraints (Inventory)
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE query PUBLIC "-//ECHO CatalogService (v10)//EN" "http://api.echo.nasa.gov/echo/dtd/IIMSAQLQueryLanguage.dtd"> <!-- Search for granules with temporal range: periodic range between Jan 1, 1990 and Dec. 31 1998 from the 1st to the 300th day of each year. --> <query> <for value="granules"/> <dataCenterId> <list> <value>ORNL_DAAC</value> </list> </dataCenterId> <where> <granuleCondition> <temporal> <startDate> <Date YYYY="1990" MM="01" DD="01"/> </startDate> <stopDate> <Date YYYY="1998" MM="12" DD="31"/> </stopDate> <startDay value="1"/> <endDay value="300"/> </temporal> </granuleCondition> </where> </query>
Code Listing 68: Sample Granule Query Using Additional Provider Specific Attributes (Inventory)
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE query PUBLIC "-//ECHO CatalogService (v10)//EN" "http://api.echo.nasa.gov/echo/dtd/IIMSAQLQueryLanguage.dtd"> <query> <for value="granules"/> <dataCenterId> <list> <value>ORNL_DAAC</value> </list> </dataCenterId> <where> <granuleCondition> <additionalAttributes> <additionalAttribute> <additionalAttributeName>'COORDINATE_UNITS_NAME'</ additionalAttributeName> < additionalAttributeValue> <value>'METERS'</value> </ additionalAttributeValue> <additionalAttribute> </additionalAttributes> </granuleCondition> </where> </query>
Code Listing 69: Sample Granule Query Using Additional Provider Specific Attributes with Complex Values (Inventory)
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE query PUBLIC "-//ECHO CatalogService (v10)//EN" "http://api.echo.nasa.gov/echo/dtd/IIMSAQLQueryLanguage.dtd"> <query> <for value="granules"/> <dataCenterId> <list> <value>ORNL_DAAC</value> </list> </dataCenterId> <where> <granuleCondition> <additionalAttributes> <additionalAttribute> <additionalAttributeName>'SAMPLE_DATE' </ additionalAttributeName> < additionalAttributeValue> <dateRange> <startDate> <Date YYYY="2000" MM="05" DD="12"/> </startDate> <stopDate> <Date YYYY="2000" MM="10" DD="12"/> </stopDate> </dateRange> </ additionalAttributeValue> <additionalAttribute> </additionalAttributes> </granuleCondition> </where> </query>