Attendees:

Notes:

Hook:

Multi Mission Analutis and Algorithm Project (MAAP)

  • A lot revolves around CMR and utilization of Cumulus data ingest

Rama:

  • Global Change Science Index (GCSI) - all had to be traced back to origins
  • Hook was designing the technical aspect
  • Rama was looking at the science aspect.

PROV-ES Background

  • This was an extension of WC3 Prov
  • This is the one Hook used for GCSI
  • There was two camps
    • Assessing ISO or PROV for provenance lineage about 5 years ago
      • Many that did liket he WC3 model
      • Was so abstract did not have any ES pieces
      • Ted Haberer called them profiles - how can we extend the generic concepts to make ES flavored PROV
  • PROV-ES is not the only player in town
  • NOAA, NSF, Data One has one as well.
  • April of last year  - ESIP Provenance workshop
    • There are a lot of Provenance things going on
    • One of the key take aways
      • Instead of pick a winner
      • How do we make them operable or do translation between each other.
  • PROV-ES has not gone through the official process
  • ESDSWG Umbrella - hosted it all on the wiki
  • ISO 19165-2  
  • Step Functions in Cumulus
    • Just an implementation different from a specification
    • A way to describe the provenance from the implementations
    • Still need a data record that can codify the transformations steps
    • Provenance capability beyond that - transcend across different systems
    • PROV-ES last specs could support this probably
    • Link to Google Docs PROV-ES mapping to EC3 Provs
      • Not deviating from WC3 but adding too
    • Need persistent identifier that get connected to each other if you need reproduce what is done
    • WC3 PROV specification every concpet - prov entity or activity, or prov agent -t hey all need a a an indentifier globally. If a DOI is used it should be used
    • Unique identifier for he software that was used and operations that were used
    • Is ES Provenance pulling rom DAAC, running through Giovanni, and then in a paper.
    • Snippets of provenance
  • Kurt?
    • on WC3 prov working groups
    • WC3 has to be a way for independent organization to create their own traces
    • If Giovanni was spitting out traces for PROV
    • We ought to be a way interoperable way to connect up in a connect graph
  • 2012 a MEASURES project - Erik?
    • Had some demonstration that provided the cross traces as well and delivered to the DAAC
    • Provenance trace to be collated with the data product - if you embed them 
    • If every system outputs their own traces that may be good enough - like DOIs that re unvierally consistent that would be our primary key to do joins across these systems.
  • Tools
    • GCSI created a PROV-ES visualization tool - all PROV records could be ingested in stand along Elastic search tool with faceted search with proto3 visualization - with tree and graphs and faceted to drill down
    • Key was jsut ingested provenance record into the database
    • Was on github.com
  • Filenames?
    • lesson we learned to decouple the model for teh encoding
      • Not presupposing the encoding such as XML
      • similar to ISO
      • What is the schema independent to the encoding
        • The encoding we focuses on was JSON
    • Filename - Convention to have the granule .proves.json extension
  • if you ned to make globally unique IDs - do a little a more than granule ID
  • If you use the DOI and Granule ID
  • Has to be persistent and permanent - unique could be good now but how to do it later
  • or temp files - how do you account for that
  • DOI and landing page they lead to must be permanent

In CMR is there an entry for Provenance record? (Joe Lee)

  • At some point would want to put a link to a provenance record
  • Early discussions with Andy Mitchel - Umm-Prov never went anywhere
  • A model to formally represent provenance or artifacts in CMR - whole project in

Any future conversations?

  • Not aware of anything currently
  • ESDSWG from last year
    • Thought he saw some provenance working group being bootstrapped (Rama was trying to but it did not fly)

Always wanted good provenance for all of our products

We have not implemented them to the extent we want - if our project could demonstrate some of these concepts

Build from granules and grow over time and demonstrate the utility of other DAACs

Recommend going to the google spreadsheet - PROV-ES to WCS3

  • Intended to be as simple, simple, simple as possible - so simple why would you not use it.
  • Two other tabs in additona to mapping - show provenance encoding examples

NISAR Science team meeting during the week




  • No labels