Page History
Table of Contents | ||
---|---|---|
|
Element Description
The direct distribution information Direct Distribution Information main element allows data providers to provide users information on getting direct access to data products that are stored in the Amazon Web Service (AWS) S3 buckets when they are initially looking at a collection. The end users get information such as the S3 credentials end point, a credential documentation URL, as well as bucket prefix names, and an AWS region.
Best Practices
There are multiple sub-elements that comprise Archive and Direct Distribution Information for Collections:
FormatRegion: Defines a single format for an archival artifact (e.g., binary, jpeg, HDF5, HDF-EOS5, GeoTIFF). It is strongly recommended that the format be selected from the GCMD Granule Data Format vocabulary list in order to ensure consistent use of data formats across the CMR.
Format Type: Allows the provider to define if the archival artifact's format is in its native format or another supported format
Format Description: Allows the data provider to provide added information about the provided Format.
Average File Size: An approximate average of the size of the the archivable item, which gives users an idea of the magnitude of each archivable file
Average File Size Unit: Unit of measure for the average file size (e.g., KB, MB, TB)
Total Collection File Size: An approximate total size of the archivable items, which gives users an idea of the magnitude of all the archivable files combined
Total Collection File Size Unit: Unit of measure for the total collection file size (e.g., KB, MB, TB)
Total Collection File Size Begin Date: The date that data collection began for the collection, which should be in the following format: yyyy-MM-ddTHH:mm:ssZ
Description: Enables the provider to provide more information about the archivable item
Media: Defines how the distributable material can be obtained by an end user (e.g., CD-ROM, hard drives, online, etc.)
Fees: Provide the price an end user will need to pay in order to obtain the distributable material
Each of these elements are available under 'FileArchiveInformation' and 'FileDistributionInformation'. FileArchiveInformation describes the data in its archived format. FileDistributionInformation describes the data as distributed by a data provider; it can be the original format in which the data are archived, or can be used to specify when the data is distributed in a format other that its original format.
Examples:
Format: "HDF-EOS5"
Format Type: "Native"
Format Description: "This is an EOSDIS specific format and it is also available in ASCII."
Average File Size: "250"
Average File Size Unit: "KB"
Total Collection File Size: "25"
Total Collect File Size Unit: "GB"
Total Collection File Size Begin Date: "2018-04-01T00:00:00Z"
Element Specification
Cardinality of Archive and Distribution Information:
ArchiveAndDistributionInformation/ FileArchiveInformation/
FormatNative
Supported
ArchiveAndDistributionInformation/ FileArchiveInformation/ AverageFileSizeUnit
KB
MB
GB
TB
PB
NA
n/a
ArchiveAndDistributionInformation/ FileArchiveInformation/ TotalCollectionFileSize
ArchiveAndDistributionInformation/ FileArchiveInformation/ TotalCollectionFileSizeUnit
KB
MB
GB
TB
PB
NA
ArchiveAndDistributionInformation/ FileArchiveInformation/ TotalCollectionFileSizeBeginDate
ArchiveAndDistributionInformation/ FileArchiveInformation/ Description
1 - 1024 characters
ArchiveAndDistributionInformation/
FileDistributionInformation/Format
ArchiveAndDistributionInformation/
FileDistributionInformation/ FormatType
Native
Supported
ArchiveAndDistributionInformation/
FileDistributionInformation/ FormatDescription
ArchiveAndDistributionInformation/ FileDistributionInformation/
Media1 - 80 characters
ArchiveAndDistributionInformation/ FileDistributionInformation/ AverageFileSizeUnit
KB
MB
GB
TB
PB
NA
ArchiveAndDistributionInformation/ FileDistributionInformation/ TotalCollectionFileSize
ArchiveAndDistributionInformation/ FileDistributionInformation/ TotalCollectionFileSizeUnit
KB
MB
GB
TB
PB
NA
ArchiveAndDistributionInformation/ FileDistributionInformation/ TotalCollectionFileSizeBeginDate
ArchiveAndDistributionInformation/ FileDistributionInformation/ Description
1 - 1024 characters
Fees
1 - 255 characters
Metadata Validation and QA/QC
All metadata entering the CMR goes through the below process to ensure metadata quality requirements are met. All records undergo CMR validation before entering the system. The process of QA/QC is slightly different for NASA and non-NASA data providers. Non-NASA providers include interagency and international data providers and are referred to as the International Directory Network (IDN).
Please see the expandable sections below for flowchart details.
Expand | ||
---|---|---|
| ||
|
Expand | ||
---|---|---|
| ||
|
title | ARC Metadata QA/QC |
---|
This element is categorized as highest priority when:
- No data format is provided
- The data format provided is incorrect
This element is categorized as medium priority when:
- The data format provided is correct, but could be more specific
- E.g. Providing 'NetCDF' when the format should be 'NetCDF-4'
- E.g. Providing 'ASCII' when the data format should be 'CSV'
This element is categorized as low priority when:
- The data format provided is correct, however, it does not exactly match its entry in the GCMD Granule Data Format vocabulary list.
- E.g. providing 'netcdf4' versus 'NetCDF-4'
- E.g. providing 'geo-tiff' versus 'GeoTIFF'
ARC Automated Checks
- TBD
Dialect Mappings
Expand | ||
---|---|---|
| ||
DIF 9 (Note: DIF-9 is being phased out and will no longer be supported after 2018) |
the possible values for the Amazon Web Service US Regions where the data product resides. The current valid values are: us-east-1,us-east-2,us-west-1, and us-west-2.
S3BucketAndObjectPrefixNames: Defines the possible values for the Amazon Web Service US S3 bucket and/or object prefix names.
S3CredentialsAPIEndpoint: Defines the URL where the credentials are stored.
S3CredentialsAPIDocumentationURL: Defines the URL where the credential documentation is stored.
Each of these elements describes the information that is necessary to pull out data products that are stored in the AWS cloud using a S3 URL that is located inside each granule's (data product file's) metadata.
Examples:
Region: "us-east-1"
S3BucketAndObjectPrefixNames: "Bucket 1"
S3CredentialsAPIEndpoint: "https://some-data-product-credentials-api.com/api.html"
S3CredentialsAPIDocumentationURL: "https://some-data-product-credentials-api.com/docs.html"
Element Specification
Cardinality of Direct Distribution Information:
Model | Element | Type | Usable Valid Values | Constraints | Required? | Cardinality | Notes |
---|---|---|---|---|---|---|---|
UMM-C | DirectDistributionInformation/ Region | String | us-east-1 us-east-2 us-west-1 us-west-2 | n/a | Yes | 1 | |
UMM-C | DirectDistributionInformation/ S3BucketAndObjectPrefixNames | String | n/a | 1 - 1024 characters | No | 0..* | |
UMM-C | DirectDistributionInformation/ S3CredentialsAPIEndpoint | URI | n/a | 1 - 1024 characters | Yes | 1 | |
UMM-C | DirectDistributionInformation/ S3CredentialsAPIDocumentationURL | URI | n/a | 1 - 1024 characters | Yes | 1 |
Metadata Validation and QA/QC
All metadata entering the CMR goes through the below process to ensure metadata quality requirements are met. All records undergo CMR validation before entering the system. The process of QA/QC is slightly different for NASA and non-NASA data providers. Non-NASA providers include interagency and international data providers and are referred to as the International Directory Network (IDN).
Lucidchart | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Please see the expandable sections below for flowchart details.
Expand | ||
---|---|---|
| ||
|
Expand | ||
---|---|---|
| ||
|
Expand | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
ARC Automated Checks ARC uses the pyQuARC library for automated metadata checks. Please see the pyQuARC GitHub for more information. |
Dialect Mappings
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
DIF 10The definitions are the same between UMM-C and ECHO 10.
|
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ECHO 10The definitions are the same between UMM-C and ECHO 10.
|
Expand | |||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||
ISO 19115-2 MENDSArchive and Distribution Information is an optional metadata element in ISO 19115-2 MENDS.
| |||||||||||||||||||||||||||||||||
Expand | |||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||
DIF 10All of the sub-elements are optional in DIF 10. | |||||||||||||||||||||||||||||||||
UMM-C Element | Path | Type | Usable Valid Values | Constraints | Required in DIF 10? | Cardinality | Notes | ArchiveAndDistributionInformation/FileDistributionInformation/ Format | /DIF/Distribution/Distribution_Format | String | No | 0..1 | To ensure consistency, it is highly recommended that the format be selected from the GCMD Granule Data Format vocabulary list. | ArchiveAndDistributionInformation/FileDistributionInformation/FormatType | /DIF/Distribution/Distribution_Format_Type | String | Native Supported | No | 0..1 | This is a DIF 10.3 field. Currently in DIF 10.2 there is just one field called '/DIF/Distribution/Distribution_Format' | ArchiveAndDistributionInformation/FileDistributionInformation/Media | /DIF/Distribution/Distribution_Media | String | No | 0..1 | ArchiveAndDistributionInformation/FileDistributionInformation/AverageFileSize | /DIF/Distribution/Average_Granule_Size | This is a DIF 10.3 field. Currently in DIF 10.2 there is just one field called '/DIF/Distribution/Distribution_Size' | ArchiveAndDistributionInformation/FileDistributionInformation/AverageFileSizeUnit | /DIF/Distribution/Average_Granule_Size_Unit | Enumeration | KB MB GB TB PB | No | 0..1 | This is a DIF 10.3 field. Currently in DIF 10.2 there is just one field called '/DIF/Distribution/Distribution_Size' | ArchiveAndDistributionInformation/FileDistributionInformation/TotalCollectionFileSize | /DIF/Distribution/Total_Collection_Size | Decimal | No | 0..1 | This is a DIF 10.3 field. Currently in DIF 10.2 there is just one field called '/DIF/Distribution/Distribution_Size' | ArchiveAndDistributionInformation/FileDistributionInformation/TotalCollectionFileSizeUnit | /DIF/Distribution/Distribution_Collection_Size_Unit | Enumeration | KB MB GB TB PB | No | 0..1 | This is a DIF 10.3 field. Currently in DIF 10.2 there is just one field called '/DIF/Distribution/Distribution_Size' | ArchiveAndDistributionInformation/FileDistributionInformation/ Fees | /DIF/Distribution/Fees | String | n/a | No | 0..1 | Example Mapping
Section | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
title | ECHO 10 |
---|
ECHO 10
Only 2 UMM-C Archive and Distribution Information sub-elements map to ECHO 10, Data Format and Fees, and neither element is required.
N/A
N/A
Section | |||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Expand | |||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||
ISO 19115-2 SMAP
| |||||||||||||||||||||||||||||||||
Expand | |||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||
ISO 19115-2 MENDSArchive and Distribution Information is an optional metadata element in ISO 19115-2 MENDS. | |||||||||||||||||||||||||||||||||
UMM-C Element | Path | Type | Notes | ArchiveAndDistributionInformation/FileArchiveInformation/Format | [=> /gmd:name/gco:CharacterString | String | ArchiveAndDistributionInformation/FileArchiveInformation/FormatType | [=> /gmd:specification/gco:CharacterString = FormatType: <FormatType> | String | ArchiveAndDistributionInformation/FileArchiveInformation/AverageFileSize | [=> /gmd:specification/gco:CharacterString = AverageFileSize: <AverageFileSize> | Decimal | ArchiveAndDistributionInformation/FileArchiveInformation/AverageFileSizeUnit | [=> /gmd:specification/gco:CharacterString = AverageFileSizeUnit: <AverageFileSizeUnit> | String | ArchiveAndDistributionInformation/FileArchiveInformation/TotalCollectionFileSize | [=> /gmd:specification/gco:CharacterString = TotalCollectionFileSize: <TotalCollectionFileSize> | Decimal | ArchiveAndDistributionInformation/FileArchiveInformation/TotalCollectionFileSizeUnit | [=> /gmd:specification/gco:CharacterString = TotalCollectionFileSizeUnit: <TotalCollectionFileSizeUnit> | Enumeration | ArchiveAndDistributionInformation/FileArchiveInformation/TotalCollectionFileSizeBeginDate | [=> /gmd:specification/gco:CharacterString = TotalCollectionFileSizeBeginDate: <TotalCollectionFileSizeBeginDate> | Date | ArchiveAndDistributionInformation/FileArchiveInformation/Description | [=> /gmd:specification/gco:CharacterString = Description: <Description> | String | ArchiveAndDistributionInformation/FileDistributionInformation/Format | [=> /gmd:distributionFormat xlink:href="FileDistributionInformation_<Block Number>"/gmd:MD_Format/gmd:name/gco:CharacterString | String | ArchiveAndDistributionInformation/FileDistributionInformation/FormatType | [=> /gmd:distributionFormat xlink:href="FileDistributionInformation_<Block Number>"/gmd:MD_Format/gmd:specification/gmd:CharacterString = FormatType: <Native or Supported> | String | ArchiveAndDistributionInformation/FileDistributionInformation/Media | [=> /gmd:transferOptions xlink:href="FileDistributionInformation_Media_<Block Number>"gmd:MD_DigitalTransferOptions/gmd:offLine/gmd:MD_Medium/gmd:name/gmd:MD_MediumNameCode codeList="https://cdn.earthdata.nasa.gov/iso/resources/Codelist/gmxCodelists.xml#MD_MediumNameCode" codeListValue="<value>" = <value> | String | ArchiveAndDistributionInformation/FileDistributionInformation/AverageFileSize | [=> /gmd:transferOptions xlink:href="FileDistributionInformation_AverageFileSize_<Block Number>"/gmd:MD_DigitalTransferOptions/gmd:unitsOfDistribution/gco:CharacterString | Decimal | ArchiveAndDistributionInformation/FileDistributionInformation/AverageFileSizeUnit | [=> /gmd:transferOptions xlink:href="FileDistributionInformation_AverageFileSize_<Block Number>"/gmd:MD_DigitalTransferOptions/gmd:unitsOfDistribution/gco:CharacterString | Enumeration | ArchiveAndDistributionInformation/FileDistributionInformation/TotalCollectionFileSize | [=> /gmd:transferOptions xlink:href="FileDistributionInformation_TotalCollectionFileSize_<Block Number>"/gmd:MD_DigitalTransferOptions/gmd:transferSize/gco:Real | Decimal | ArchiveAndDistributionInformation/FileDistributionInformation/TotalCollectionFileSizeUnit | [=> /gmd:transferOptions xlink:href="FileDistributionInformation_TotalCollectionFileSize_<Block Number>"/gmd:MD_DigitalTransferOptions/gmd:unitsOfDistribution/gco:CharacterString | Enumeration | ArchiveAndDistributionInformation/FileDistributionInformation/TotalCollectionFileSizeBeginDate | [=> /gmd:distributionFormat xlink:href="FileDistributionInformation_<Block Number>"/gmd:MD_Format/gmd:specification/gmd:CharacterString = TotalCollectionFileSizeBeginDate: <Date use MMDDYYYYTHH:MM:SSZ format> | Date | ArchiveAndDistributionInformation/FileDistributionInformation/Description | [=> /gmd:distributor xlink:href="FileDistributionInformation_<Block Number>"/gmd:MD_Distributor/gmd:distributionOrderProcess/gmd:MD_StandardOrderProcess/gmd:orderingInstructions/gmd:CharacterString | String | ArchiveAndDistributionInformation/FileDistributionInformation/Fees | [=> /gmd:distributor xlink:href="FileDistributionInformation_<Block Number>"/gmd:MD_Distributor/gmd:distributionOrderProcess/gmd:MD_StandardOrderProcess/gmd:fees/gmd:CharacterString | String | Example Mapping
Section | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
title | ISO 19115-2 SMAP |
---|
ISO 19115-2 SMAP
Archive and Distribution Information does not currently map to ISO 19115-2 SMAP.
N/A
N/A
Section | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
UMM Migration
None
Excerpt | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Future Mappings
|
UMM Migration
None
Excerpt | |||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||
Future Mappings
|
History
UMM Versioning
Excerpt Include | ||||||
---|---|---|---|---|---|---|
|
Version | Date | What Changed |
---|---|---|
1.16.0 | 03/24/2021 |
History
UMM Versioning
Version | Date | What Changed |
---|---|---|
1.15.5 | 12/3/2020 | The description of the format elements now includes text about the controlled vocabulary for this field coming from KMS. |
1.15.4 | 9/18/2020 | No changes were made for Archive and Distribution Information during the transition from version 1.15.3 to 1.15.4 |
1.15.3 | 7/1/2020 | FormatDescription was added to FileArchive and FileDistribution Information types to version 1.15.3 |
1.15.2 | 5/20/2020 | No changes were made for Archive and Distribution Information during the transition from version 1.15.1 to 1.15.2 |
1.15.1 | 3/25/2020 | No changes were made for Archive and Distribution Information during the transition from version 1.15.0 to 1.15.1 |
1.15.0 | 2/26/2020 | No changes were made for Archive and Distribution Information during the transition from version 1.14.0 to 1.15.0 |
1.14.0 | 10/21/2019 | No changes were made for Archive and Distribution Information during the transition from version 1.13.0 to 1.14.0 |
1.13.0 | 04/11/2019 | This element was added to the UMM-C specification. |
ARC Documentation
Version | Date | What Changed | Author |
---|---|---|---|
1.0 | 056/1630/1821 | Recommendations/priority matrix transferred from internal ARC documentation to wiki space |