| Panel |
|---|
| borderColor | #9933FF |
|---|
| titleColor | White |
|---|
| titleBGColor | #9933FF |
|---|
| title | General Information |
|---|
|
| Expand |
|---|
| title | What is the Common Metadata Repository (CMR)? |
|---|
| The Common Metadata Repository (CMR) catalogs all data and service metadata records for the EOSDIS system and will be the authoritative management system for all EOSDIS metadata. These metadata records are registered, modified, discovered, and accessed through programmatic interfaces leveraging standard protocols and APIs. |
| Expand |
|---|
| title | What are the objectives of CMR? |
|---|
| CMR is designed to: - serve as a middleware replacement for the ECHO and GCMD’s backend. Note: Users of GCMD should see no impact - as the GCMD frontend will remain as is.
- handle metadata at the Concept level; including Collections, Granules, Visualizations, Parameters, Documentation, Services, and more.
- manage hundreds of millions of metadata records; making them available through high performance, standards compliant, temporal, spatial, and faceted search.
- incorporate both human and machine metadata assessment features that work to ensure the highest quality metadata possible.
- support multiple metadata standards using an evolvable Unified Metadata Model (UMM).
|
| Expand |
|---|
| The UMM is an extensible metadata model which provides a ‘Rosetta stone’ or cross-walk for mapping between CMR-supported metadata standards. Information detailing what the UMM is and how it works is available on Earthdata.gov |
| Expand |
|---|
|
|
| Expand |
|---|
|
The CMR system is composed of:
- CMR itself (formerly the [ECHO])
- GCMD
- International Data Network (IDN)
- Earth Science Data and Information System (ESDIS) Metrics System (EMS)
- All related tools (internal and external)
- All metadata — including UMM concepts, the GCMD Keywords Controlled Vocabulary, and other controlled vocabularies.
Image Removed
High Level Architecture Diagram
The CMR is designed to handle metadata at the Concept level. Collections and Granules are common metadata concepts, but this can be extended out to Visualizations, Variables, Documentation, Services, and more. The CMR provides a flexible ingest system with pluggable adapters that can handle multiple metadata record formats, multiple metadata record concepts, and relationships and validations between them. As new formats are introduced, new ingest adapters can be written for the CMR to provide ingest, validation and search support and response adapters provide format conversions for backward compatibility.
Image Removed
Metadata at the Concept Level
| Expand |
|---|
| title | What are the benefits of CMR? |
|---|
|
Modern Earth Science applications strive to provide end users with nearly immediate access and interactivity across massive stores of Earth Science data. That data is discovered, navigated, and often interrogated through science metadata. As the range of applications grow and more and more information moves from the underlying science data to metadata, the challenges of navigating even just the metadata increases. CMR is designed to handle hundreds of millions of metadata records; making them available through high-speed performance, standards compliant, temporal, spatial, and faceted search.
The CMR builds on the work done by ECHO and the GCMD to provide a unified, authoritative repository for NASA's Earth Science metadata.
Image Removed End-User Interface Diagram
High performance access to metadata is only part of the problem. To be useful to a broad range of Earth Science applications, the metadata must be of high quality, complete, and consistent. The CMR incorporates both human and machine metadata assessment features that work to ensure the highest quality metadata possible. During ingest, automated metadata scoring rubrics are applied giving data providers insight into how to make their data more discoverable or usable by end users. Science Coordinators and review teams can review metadata that fails verification or lacks required information to help providers make their metadata more consistent and complete.
Image Removed
Metadata Quality Assurance Diagram
The CMR's ingest adapter framework supports pluggable adapters which validate distinct metadata formats such as ECHO10, GCMD DIF, and ISO19115 against a common set of core metadata requirements, the Unified Metadata Model (UMM).The CMR will be able to take multiple metadata records associated with a common core concept, such as a Collection, and merge the disparate information into a robust and standards compliant ISO19115 representation for interested clients.As additional metadata concepts are introduced to the CMR, new ingest adapters will provide verification and search indexing capabilities across diverse metadata such as visualization and parameter information.
Image Removed| Expand |
|---|
| title | CMR and reconcilliation |
|---|
|
Reconciliation making sure your local provider metadata inventory matches CMR's metadata inventory
ECHO provided long and short form reconciliation functionality to providers. This was a vital piece of functionality for ECHO, prior to catalog-rest, for the following reasons,
- ECHO ingest was asynchronous. Once you FTP'd your files at some point in the future it was ingested and it may fail. How do I know what failed?
- ECHO ingest was a batch process. If a batch failed it was tricky trying to figure out what part of that batch failed.
- ECHO normalized the metadata in a database. It deconstructed what the provider put in and reconstructed it when it was searched for. That was an error-prone process
Consequently, providers needed to verify that the ECHO inventory matched what they had locally using separate processes. Short form verification allowed you to match inventory at a coarse level using id matching. Long form allowed you to determine discrepancies in content of metadata.
What is important to a provider is the following,
- The same collection and granule metadata is represented in both systems - short form reconciliation
- The content of the above is the same - long form reconciliation
How can we do that in CMR?
- When you ingest metadata into CMR it is a synchronous process. It is also searchable within CMR within a few milliseconds. After a successful ingest, CMR will return a concept id of that piece of metadata. You can search for that piece of metadata using that concept id and/or the native id. This way you can do long-form reconciliation on a metadata-by-metadata basis.
- You can get the number of collections or granules in a provider using the holdings endpoint. This, in conjunction, with step 1, gives you short-form reconciliation.
If step 1 fails for a piece of inventory you can trouble-shoot and repeat the exercise. If step 2 fails them additional triage is required form the CMR Operations group.
