Versions Compared

Old Version 6

changes.mady.by.user Erich Reiter

Saved on

compared with

New Version 7

changes.mady.by.user Jyna Maeng

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
{
  "Version": "2012-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      "Sid": "__owner_statement",
      "Effect": "Allow",
      "Principal": {
        "AWS": "<Your Account>"
      },
      "Action": [
        "SQS:*"
      ],
      "Resource": <Your Queue ARN>
    },
    {
      "Sid": "<Your Unique ID for this Access Policy",
      "Effect": "Allow",
      "Principal": {
        "Service": "sns.amazonaws.com"
      },
      "Action": "SQS:SendMessage",
      "Resource": <Your Queue ARN>
    }
  ]
}

Create a Subscription with the CMR

Once your queue is set up then you need to create a new subscription with the CMR

To create a new subscription you will need to use the CMR API and send a subscription request.  In the request you will need to send a subscription record that follows the UMM-Sub schema version 1.1.1

Following is an example UMM-Sub record 

Code Block
{"Name": "<The name of your subscription>",
 "Type": "granule",
 "SubscriberId": "<Your Earthdata Login Name>",
 "CollectionConceptId": "C1200463968-CMR_ONLY",
 "EndPoint": "arn:aws:sqs:<region>:<account>:<queue-name>",
 "Mode": ["New", "Update", "Delete"],
 "Method": "ingest",
 "MetadataSpecification": {
   "URL": "https://cdn.earthdata.nasa.gov/umm/subscription/v1.1.1",
   "Name": "UMM-Sub",
   "Version": "1.1.1"
 }
}

The above subscription record that gets sent as the payload with the subscription create/update request is described below.

The subscriber ID must be your Earthdata Login user name. 

The collection concept id is the CMR concept id of the collection for which granule ingest notifications are wanted.  Make sure that you have read permission to the collection, otherwise the subscription creation will fail. Permission can be checked with the following command: curl "https://cmr.sit.earthdata.nasa.gov/access-control/permissions?user_id=<Earthdata Login Name>&concept_id=<CMR Collection Concept ID>".

The EndPoint is the ARN for the SQS Queue that was created in step 1.

The Mode element describes of what kind of granule ingest notification is wanted.

  • Notifications will be sent if 
    • New - A granule is ingested into the CMR for the first time.
    • Update - A granule is ingested into the CMR as an update, meaning it already exists in the CMR, but another ingest of that granule occurs. 
    • Delete - A granule is deleted from the CMR.
  • One, two, or all three options can be used.

The Method element signifies if this is an search type of subscription or an ingest type of subscription. Valid values are either Ingest or Search. Use ingest, if the desire is to be notified when a granule is ingested, updated, or deleted.  Use Search otherwise.

Copy the rest of the elements as they are in the example.

Use the CMR API to create the subscription

...


Setup Collection to Subscribe To

Before you can create an ingest subscription, you will want to choose a pre-existing collection or create a new collection. This is the collection you will subscribe to so that if a granule is created, updated, or deleted you will see it in your queue.

Check You Have Correct Permissions to Your Chosen Collection

In order to subscribe to a collection, you need to make sure you can update and read it. Use the following command to make sure you have permissions.

Creating permissions for first time

Code Block
curl --location 'https://localhost:8080/access-control/acls' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
    "group_permissions": [
        {
            "permissions": [
                "read",
                "update",
                "delete",
                "order"
            ],
            "user_type": "guest"
        },
        {
            "permissions": [
                "read",
                "update",
                "delete",
                "order"
            ],
            "user_type": "registered"
        }
    ],
    "catalog_item_identity": {
        "name": "Guest and Registered Users Collections",
        "provider_id": "JM_PROV1",
        "granule_applicable": true,
        "collection_applicable": true,
        "collection_identifier": {
            "concept_ids": [
                "C3261894786-JM_PROV1", "C3261894785-JM_PROV1", "C3261906797-JM_PROV1"
            ],
            "entry_titles": [
                "Collection for CMR-10162_1 v1.18.1", "Collection for CMR-10162_2 v1.18.1", "Collection for CMR-10162_3 v1.18.1"
            ]
        }
    }
}'

Response

Code Block
{
    "revision_id": 1,
    "concept_id": "ACL3261894787-CMR"
}


Updating permissions for existing ACL

Code Block
curl --location --request PUT 'https://localhost:8080/access-control/acls/ACL3261894787-CMR' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
    "group_permissions": [
        {
        "user_type": "registered",
        "permissions": [
                "read",
                "update",
                "delete",
                "order"
            ]
        }, 
        {
        "user_type": "guest",
        "permissions": [
                "read",
                "update",
                "delete",
                "order"
            ]
        }],
    "catalog_item_identity": {
        "name": "All Granules",
        "provider_id": "JM_PROV1",
        "granule_applicable": true,
        "collection_applicable": true,
        "collection_identifier": {
            "concept_ids": ["C3261894786-JM_PROV1", "C3261894785-JM_PROV1", "C3261906797-JM_PROV1"],
            "entry_titles":["Collection for CMR-10162_1 v1.18.1", "Collection for CMR-10162_2 v1.18.1", "Collection for CMR-10162_3 v1.18.1"]
        }
    }
}
'

Response:

Code Block
{
    "revision_id": 5,
    "concept_id": "ACL3261894787-CMR"
}

Check you have correct permissions

Code Block
curl --location 'https://localhost:8080/access-control/permissions?user_id=jmaeng&concept_id=C3261894785-JM_PROV1'

Response:

Code Block
{"C3261894785-JM_PROV1":["read","order"]}


Create a Subscription with the CMR

Once your queue is set up then you need to create a new subscription with the CMR

To create a new subscription you will need to use the CMR API and send a subscription request.  In the request you will need to send a subscription record that follows the UMM-Sub schema version 1.1.1

Following is an example UMM-Sub record 

Code Block
{"Name": "<The name of your subscription>",
 "Type": "granule",
 "SubscriberId": "<Your Earthdata Login Name>",
 "CollectionConceptId": "C1200463968-CMR_ONLY",
 "EndPoint": "arn:aws:sqs:<region>:<account>:<queue-name>",
 "Mode": ["New", "Update", "Delete"],
 "Method": "ingest",
 "MetadataSpecification": {
   "URL": "https://cdn.earthdata.nasa.gov/umm/subscription/v1.1.1",
   "Name": "UMM-Sub",
   "Version": "1.1.1"
 }
}

The above subscription record that gets sent as the payload with the subscription create/update request is described below.

The subscriber ID must be your Earthdata Login user name. 

The collection concept id is the CMR concept id of the collection for which granule ingest notifications are wanted.  Make sure that you have read permission to the collection, otherwise the subscription creation will fail. Permission can be checked with the following command: curl "https://cmr.sit.earthdata.nasa.gov/access-control/permissions?user_id=<Earthdata Login Name>&concept_id=<CMR Collection Concept ID>".

The EndPoint is the ARN for the SQS Queue that was created in step 1.

The Mode element describes of what kind of granule ingest notification is wanted.

  • Notifications will be sent if 
    • New - A granule is ingested into the CMR for the first time.
    • Update - A granule is ingested into the CMR as an update, meaning it already exists in the CMR, but another ingest of that granule occurs. 
    • Delete - A granule is deleted from the CMR.
  • One, two, or all three options can be used.

The Method element signifies if this is an search type of subscription or an ingest type of subscription. Valid values are either Ingest or Search. Use ingest, if the desire is to be notified when a granule is ingested, updated, or deleted.  Use Search otherwise.

Copy the rest of the elements as they are in the example.

Use the CMR API to create the subscription

https://cmr.earthdata.nasa.gov/ingest/site/docs/ingest/api.html#subscription


Example CURL command:

Code Block
curl --location 'https://localhost:8080/ingest/subscriptions/jyna_ingest_subscription_3' \
--header 'Content-Type: application/vnd.nasa.cmr.umm+json' \
--header 'Authorization: ••••••' \
--data-raw '{
    "Name": "Ingest-Subscription-Test",
    "Type": "granule",
    "SubscriberId": "mysubscriberid",
    "EmailAddress": "myemail",
    "CollectionConceptId": "C3261906797-JM_PROV1",
    "EndPoint": "arn:aws:sqs:<region>:<account-id>:<my-queue-name>",
    "Mode": ["New","Update"],
    "Method":"ingest",
    "MetadataSpecification": {
        "URL": "https://cdn.earthdata.nasa.gov/umm/subscription/v1.1.1",
        "Name": "UMM-Sub", 
        "Version": "1.1.1"
    } 
}

Response:

Code Block
<?xml version="1.0" encoding="UTF-8"?>
<result>
    <concept-id>SUB3261906801-JM_PROV1</concept-id>
    <native-id>jyna_ingest_subscription_3</native-id>
    <revision-id>1</revision-id>
</result>


Confirm the Subscription

Once the subscription request has been successfully ingested into the CMR look in your queue either programmatically or through the AWS Console to find a subscription confirmation message.  If a subscription message is not received a CMR operator will have to look at the subscription logs to see what the errors are.  Almost all of the errors are because the queue access policy isn't sufficient from step 1.

...