Contents

Introduction

This page contains instructions about how to set up a new Request for Comment (RFC) review. It also contains child pages that are templates that, when copied, provide the wiki pages needed to track the RFC.The ESCO role in processing an RFC is described in the Process for Earth Science Data Systems Standards and References document. Also refer to the material in Instructions to RFC Authors, a description of the process as viewed from the document authors' point of view.

Scope

This page covers only RFC document reviews.

It does not cover RFCs that are evaluated as ASCII File Formats as described in ASCII File Format Guidelines for Earth Science Data.

It does not cover ESCO-PUB documents (they are not usually reviewed). 

Previous RFCs

It's always helpful to look at previous RFC review materials, particularly those that are in the same category as this new RFC. Look at the Standards and Practices page and the ESCO Document List to find relevant RFCs. Then look in the ESCO Review and Publication Archives on the wiki to see if there are helpful materials there.

RFC Requirements

Upon initial submission, the document might not meet all these requirements. ESCO staff should work with authors to help them understand the requirements and how to meet them. It is possible that after working with the authors it becomes clear that the document won't meet the requirements. In that case, ESCO should reject the submission (gently!).

S

CS

TNRequirement
  •  
  •  
  •  
Motivation must be clear and relevant to ESDS
  •  
  •  
  •  
Document must be well-written
  •  
  •  

Sufficiently well-documented to be technically implementable

  •  
  •  

At least two representative implementations

  •  
  •  

Evidence of significant operational experience (In the form of Points of Contact who are using the spec/convention operationally)


  •  

Format specifications should be published in open literature and freely available


  •  

No patent, copyright or other intellectual property encumbrances on the format or its specification




Besides the commercial entity that developed and/or maintains the format, there should be


  •  

at least two (2) independent implementations that read the format


  •  

at least one (1) independent implementation that writes the format

S: Specification or Convention
CS: Commercial Specification or Convention
TN: Tech Note

RFC Development

Generally carried out by non-ESCO individuals or teams with ESCO assistance. Authors should contact ESCO as early in this process as possible to increase chances of success. ESCO will help authors understand the process, how to format the document, other requirements such as completeness, evidence of implementation and operational experience. ESCO should also try to determine whether the proposed material is likely to be approved at the end of the process. Some material may not be suitable for a community review. In that case, ESCO could consider working having the authors submit an ESCO-PUB document instead (see the ESCO Publications section of the ESCO Document List for examples).

Sometimes ESCO will receive fully-formed RFC documents. This is rare if the authors have not had any prior submissions (but it has happened) and less rare for experienced authors. In that case, use this section as a reminder of things that should be checked and discussed with the submitters.

Sometimes ESCO writes the RFC, as was the case with the GeoTIFF File Format specification. In that case, ESCO has to adhere to the same process, and it's probably doubly important to make sure the document is well-written.

  •  

Potential submitters are encouraged to contact ESCO staff at: esco-staff@lists.nasa.gov

  •  

Authors are recommended to read this material before they start writing:

It's also helpful if they become familiar with the process so they know what to expect:

ESCO should make sure the authors know they should use one of our document templates.

(warning) There are two templates. Which one should be used depends on the kind of document they are writing. If in doubt, start with the ESDS RFC Template.

ESDS RFC Template: Generally, the ESDS RFC Template should be used (there are links to various forms of the template on the Instructions to RFC Authors page).

COMET Template: The COMET Template is only for material that will be registered with the COMET system. It's not hard to convert an ESDS RFC formatted document to a COMET formatted document but it's better not to have to do that. If at all possible, confer with the NASA ESCO lead about the proposed document to decide whether the document will need to be submitted into the COMET system. The COMET template is available on the Templates and Instructions page.

  •  

ESCO helps Determine the Document Type: Standard, Convention, Suggested Practice, Lessons Learned, Technical Note, ...

see ESCO Document List

  •  

ESCO helps Determine the Resource Type (Topics): Software, Specification, Process, Technical Note

see ESCO Document List

(If the document doesn't fit one of these, more can be added but there should be a good reason!)

  •  

ESCO helps set-up communication vehicle (emails, telecons, etc.) that includes the authors and designated ESCO staff members

  •  

Submitters determine who should be listed as Author(s) and who should be listed as Contributors (if any)

  • Authors are listed when the DOI is registered
  • Authors' contact info should be in the Authors section of the document
  • Contributors included in the DOI
  • Contributors can be listed individually, with or without contact information
  • Contributors can be included as a group, e.g. "Earth Science Data Systems Dataset Interoperability Working Group"
  •  

For standards and conventions, authors should focus on providing as many implementations of the standard, specification, or process as possible and to list these in a document known as Evidence of Use or Evidence of Implementation (some RFCs embed this in the RFC, e.g., GeoTIFF). The ESCO will use these references to target reviewers for their views on the operational readiness of the proposed material.

RFC Submission and Intake

When the authors are ready, the RFC is submitted to ESCO for processing.

Document Naming and Numbering

All versions of RFC documents prior to publication are 0.x

Some authors will use their own document naming and numbering convention during the editing phases.

Versions sent out for public review should be named "ESDS-RFC-nnnv0.x" or "423-ESCO-nnnv0.x"

The first published version of an approved document is version 1.0. The document name of this version does not contain the version number. COMET will use its own naming convention for versions.

Subsequent versions of RFC documents that are created to fix editorial problems are "v1.x". If an RFC has to get bumped to "2.x" because of non-editorial changes, then ESCO has to decide whether there has to be a new public review (the standards process document says it does).

Initial Screening

The initial screening process involves designating a member of ESCO to be the RFC Editor. The RFC Editor is the main point of contact between ESCO and the authors and also serves as the chair of the Technical Working Group (TWG).

Before setting up the review infrastructure, ESCO has to decide whether the document meets all the requirements and whether the material should be processed as an RFC and whether the material should be processed as a different document type than initially proposed by the authors. There are times when authors submit an RFC as a proposed standard or convention, but ESCO feels that it should be re-written as a Technical Note or Suggested Practice. This can happen if the material is not strong enough (not enough use within NASA being a primary reason) to be a standard or convention. (If the authors worked with ESCO to create the document, hopefully this is not the case.)

If the material is strong but the document itself is not well-formulated, then ESCO can work with the authors to correct that.

  •  

Assign an ESCO Staff member to be the RFC Editor to act as the liaison between ESCO, the RFC authors and the TWG.

  •  

Assure that the RFC meets requirements, see Instructions to RFC Authors: https://earthdata.nasa.gov/standards/instructions-to-rfc-authors

  •  

Decide whether to accept / reject / or ask for revisions

  •  
If RFC requires revisions
  •  

RFC Editor coordinates with RFC Author till it's ok

  •  

RFC Editor and ESCO Staff decide when to accept RFC or whether to reject (warning) reasons for rejection?

  •  

Assign RFC unique document number (i.e. ESDS-RFC-nnn) RFC numbers are assigned consecutively.

If the document is going to be posted to COMET, the document number usually becomes 423-ESCO-nnn where nnn is the next available RFC number. In some cases, the COMET number does not follow the 423-ESCO-nnn pattern. 

Create a Digital Object Identifier (DOI)

Updated 04/18/22 - this is VERY early in the process to create a DOI and, based on discussions within ESCO, it was decided that waiting to generate a DOI until the RFC is further through the community review process and near approval would be more efficient.

Follow the process in DOIs for ESCO Documents to create a DOI for the document. Most ESCO documents will be assigned DOIs.

Initial Review Set Up

Once ESCO knows there is a new RFC about to be submitted (this can happen in the RFC Development phase or in the RFC Submission and Intake phase), a set of wiki pages is needed to track the review process.

To set up the wiki tracking pages, copy this page, its attachments and its child pages. During the copy process you can rename the pages to include the RFC number.

Note: RFC numbers are assigned consecutively.

After the copy operation, the new pages will be in the Work in Progress area and should look like this:

Page titlePage nickname
ESDS-RFC-nnn Review Tracking PageTracking Page

ESDS-RFC-nnn Review - <TITLE OF RFC>

Public Review Page

ESDS-RFC-nnn TWG Work Area

TWG Page


  •  

Using the Confluence "Copy" function, bring up the page copying dialog box.

Select both "Include attached files and images" and "Include child pages", then click Next.

Make the Parent Page be "Work in Progress"

  •  

Fill in the fields to rename the pages. In this example, the pages are going to be used to track RFC-042

Replace the "RFC" in the title with, e.g., "RFC-042", then click Copy.

If it's going to be a COMET document, then you can replace "ESDS-RFC" with the COMET number, e.g. "423-ESCO-042". If you don't know yet which it will be, go with RFC, not COMET.

  •  
You can now begin using the new tracking pages in the Work in Progress area.

Finish Setting up Wiki pages

If there will be non-ESCO people on the TWG

Request creation of a Wiki group named esco-twg-nnn from confluence admins.

Move the Public Review Page to be a child of the ESCO Reviews page.

Make sure the permissions on the Public Review Page are set so that only members of the ESCO and esco_review groups can view/edit.

If there was a special wiki group for this TWG, add the esco-twg-nnn group to be able to view/edit Public Review Page and the TWG Page  (Confluence permissions may prevent non-esco members from being able to edit these pages. If it's necessary to allow this, adding them to the esco_review group will work but will also give full access to the entire ESCO Private space.)

Attach the submitted copy of the RFC to the TWG Page. You can just drag and drop the document into the Attachments area.

At this point, the Tracking Page remains in the Work in Progress area and the Public Review Page and the TWG Page are in the ESCO Reviews area.

Recruit Technical Working Group (TWG)

There is a description of what a TWG does in the ESCO Technical Working Group Guide.

If ESCO members have enough expertise about the topic of the RFC, then the TWG can consist of ESCO members.

The RFC Editor chairs the TWG (this is not spelled out in the process but has generally been the case in practice).

The TWG should use the TWG Page to keep lists of potential reviewers, formulate review questions, etc.

  •  
Recruit/appoint ate least 2 members into the TWG (in addition to the RFC Editor)
  •  
Ensure the TWG members have Confluence access

Set up the ESCO Reviews Pages on Earthdata and the Wiki

  •  

Add a notice saying that the review is upcoming on the ESCO Reviews page on Earthdata. Don't link to the review since none of the pages are publicly visible yet. This example shows how the page looks with a current review, an upcoming review and two in-progress views. (see Earthdata Page Editing, Creation, etc.)

  •  

Add a notice saying that the review is upcoming on the ESCO Reviews page on the wiki. Don't link to the review since none of the pages are publicly visible yet.

TWG Activity (pre-review phase)

  •  
Review the RFC for completeness (this may already have been done as part of the initial screening but if there are problems with the RFC that are discovered in this phase, they should be addressed in cooperation with the RFC authors).
  •  

Ensure RFC is ready to be reviewed more broadly

  •  

Ensure there is sufficient evidence of implementation

  •  
Identify reviewers
  •  

Target potential reviewers, including people listed in the Evidence of Implementation

  •  
Set up a list of potential reviewers on the TWG Page
  •  
Develop a review strategy - does the review need to include a technical review? Should the technical review be combined with the operational readiness review?
  •  

Develop review questions - try to ask questions that help to uncover strengths, weaknesses, applicability and limitations of the contents of the RFC (needed for the recommendation for adoption that is sent to ESDIS along with the RFC). Reviewers tend to be very good at commenting about the specifics of the document and it helps to get them to think about the overall context. A good question to end with is whether they think this document should be approved.

Optional: You can ask the authors for help with the questions, either by asking them what they would like to ask reviewers or by showing them the questions and asking for feedback.

Pre-Review Logistics

Review Period

The TWG and ESCO decide on when the review should start and how long it should run. Typical periods are on the order of 4 weeks. Reviews in August as well as in late November through early January don't tend to attract as many reviewers. It might be advisable to either avoid those times or set a longer review period. Perhaps also plan for more reminders.

Document Preparation

To set up a review in Jama and on the Wiki, you need to prepare the following documents

  • A clean copy of the RFC with no comments left over from the internal reviews. Create a PDF copy and a Word copy. This will get attached to the Public Review Page. This copy should be named "ESDS-RFC-nnnv0.x" where nnn is the RFC number and x is the latest revision number. For COMET, use "423-ESCO-nnnv0.x".
  • A copy of the review questions in a separate document. Create a PDF copy and a Word copy. This will get attached to the Public Review Page.
  • A copy of the RFC that has a section inserted above all other content, even above the title. This section is titled Review Introduction and Review Questions and contains a brief introduction to the review as well as the questions developed by the TWG. Create only a Word copy. This will get uploaded into Jama.

Add RFC into Jama

  •  

Send request to esco-staff@lists.nasa.gov asking to upload the review to Jama, ((warning) do we need a template for the email?) that includes:

  •  
  • The RFC with the review questions embedded in the top, as a Word document
  •  
  • Review end dates
  •  
  • List of people needing to be moderators (typically all of ESCO staff)

Finalize Public Review Pages

  •  
Let the authors know that the review is about to go live.
  •  
Attach the clean copies of the RFC and the questions in both Word and PDF to the Public Review Pages. You have to attach copies to that page so they will be accessible to the public. Non-ESCO or TWG members cannot see files that are links to attachments on the TWG page.
  •  
Fill in the red items on the Public Review Page
  •  

Do a final check of the Public Review Page - If you do these steps in an anonymous browser mode, you'll notice permission problems that might arise.

  • Double-check the contents of the Public Review Page to make sure it's complete.
  • Follow the Jama link to make sure it points to the public review
  • Download all the documents and open them to make sure they are the correct versions
  • Edit the restrictions so that the Top Level Review Page becomes visible to the public and that the remaining child pages are still private
  •  

Edit the ESCO Reviews page on Earthdata to move the review from the Upcoming section to the Current section.

Add the comment period and a link to the review's Top Level Review Page.

Check the link on this page once it's been published to make sure it points to the right page!

This example shows how the page looks with a current review, an upcoming review and two in-progress views.

(see Earthdata Page Editing, Creation, etc.)

  •  

Edit the ESCO Reviews page on the wiki to move the review from the Upcoming section to the Current section.

Add the comment period and a link to the review's Top Level Review Page.

Check the link on this page once it's been published to make sure it points to the right page!

This example shows how the page looks with a current review, an upcoming review and two in-progress views.

Community Review

Draft an Announcement

Create an announcement that includes

  • A short introduction to the review
  • Why the review is important
  • the deadline
  • A short note about ESCO's role
  • Link to the Public Review Page
  • Ask recipients of the email to spread the word to colleagues

Here is a sample review announcement. It's always easier to start from an email sent from a previous review!

Hello,

NASA's Earth Science Data and Information Systems (ESDIS) Standards Coordination Office (ESCO) is requesting comments on recommendations from the Earth Science Data Systems Working Group (ESDSWG) on Guidance for Data Product DevelopersYou are invited to review the Data Product Development Guide (DPDG) for Data Producers in the context of your experience with Earth science data, for its suitability for operational use.  Your insights will help improve this document, and determine whether it should be adopted as a Suggested Practice reference for Earth Science data producers.

This guide is primarily intended for developers of Earth Science data products derived from remote sensing data, and particularly for the development of Level 1B through Level 4 products.  However, developers of related data products including Level 0 and 1A satellite data, as well as airborne and in situ data products, will also find useful guidance.  The DPDG aims to compile the most applicable parts of earlier guides into an easy-to-follow document that logically outlines the typical development process for Earth Science data products. Emphasis has been given to standards and best practices formally endorsed by the ESCO, outputs from ESDSWGs, and recommendations from DAACs and data producers. Ultimately, the DPDG provides developers with guidelines for how to make data products that best serve end user communities—the primary beneficiaries of data product development.

The ESCO review will run through January 24, 2020. 

The Earth Science Data Systems Working Groups (ESDSWG) is an organization charged with the exploration and development of recommendations derived from pertinent community insights of NASA's heterogeneous and distributed Earth science data systems.  

The role of the ESDIS Standards Coordination Office (ESCO) is to conduct a review of this document by soliciting comments from a cross section of the Earth Science community. Reviewers are invited to read the document and answer the questions provided on the review web page. The ESCO recognizes that not all reviewers will be familiar with all of the content of the document. Reviewers are welcome to review those parts of the document that they have experience with.  You only need to answer questions applicable to you.  Additional comments are welcome.

If approved, these recommendations will serve as a Suggested Practice reference for data producers.

The document to review, the review questions, and the instructions for review are at ESO Review - Data Product Development Guide for Data Producers.

If you or your team member can review this document, please send names, email addresses, and Earthdata login usernames to esco-staff@lists.nasa.gov.  The ESCO Staff will add your names as reviewers in the Jama system.  You may also send any questions about the review to the ESCO staff at esco-staff@lists.nasa.gov.

Disseminate the Announcement

Email Lists - lists to consider

  • DAAC Managers - Confer with the NASA ESCO lead before sending this (and, in fact, have the NASA lead send the email to this list)
  • cmr-announce - This list is managed by ESCO.
  • eso-sig - This list is managed by ESCO (not functioning as of 5/4/22)
  • ESIP mailing lists (such as esip-all) - consider one or more ESIP lists, but confer with ESIP first

Targeted Reviewers

People respond best if they are asked personally. If you can develop a list of potential reviewers, then send a separate email to each one, if possible. If you send an email to a group of potential reviewers, be sure to list them in the BCC field of the email.

Keep track of targeted reviewers on the TWG Page.

Slack Channel

Paste the announcement into the #eosdis-standards channel on Slack.

Verbal mentions

Person-to-person recruiting works well. If you're at a conference or a meeting when the review starts (or is about to start), mention the review and offer to send the review information to people you meet. If you send announcements to anyone, add them to the list of reviewers on the TWG Page.

Add Reviewers to Jama

While the comment period is open, we will get requests from people to be given access to the Jama review.

Steps

  •  

Receive an email asking for Jama access

From: Paul Lemieux - NOAA Federal via esco-staff <esco-staff@lists.nasa.gov>
Subject: [esco-staff] [EXTERNAL] Jama Access
Date: August 26, 2021 at 9:39:08 AM EDT
To: esco-staff@lists.nasa.gov
Reply-To: Paul Lemieux - NOAA Federal <paul.lemieux@noaa.gov>

Hi,

I'm requesting access to Jama to review the latest GCMD keyword additions from NOAA TPIO. My EarthData login is my NOAA email address.

Thanks,

Paul Lemieux III (he/him), MSIS 
Physical Scientist
Data Stewardship Division - Archive Branch
NOAA National Centers for Environmental Information (NCEI)
...

  •   

Acknowledge the request so they know what to expect.

From: Allan Doyle <adoyle@intl-interfaces.com>
Subject: Re: [esco-staff] [EXTERNAL] Jama Access
Date: August 26, 2021 at 12:58:28 PM EDT
To: Paul Lemieux - NOAA Federal <paul.lemieux@noaa.gov>
Cc: ESCO Staff <esco-staff@lists.nasa.gov>

Hi Paul,

I've requested Jama access for you. I'll let you know when you can start with the review.

Thanks for being a reviewer.

Allan

  •  
Follow the process at How to add reviewers to Jama.
  •  

Let them know when they can start reviewing

From: Allan Doyle <adoyle@intl-interfaces.com>
Subject: Re: [esco-staff] [EXTERNAL] Jama Access
Date: August 26, 2021 at 1:37:22 PM EDT
To: Paul Lemieux - NOAA Federal <paul.lemieux@noaa.gov>
Cc: ESCO Staff <esco-staff@lists.nasa.gov>

Hi Paul,

You should be able to access the review now using your Earthdata login ID and password. The direct link is 
https://rms.earthdata.nasa.gov/review.req#/r:REV-567

Allan

  •  
For each person you add, also add their name and the date you added them into the Review Team page near the bottom under the heading 

Reviewers added based on requests sent to esco-staff@lists.nasa.gov

Collect Comments

Some people may send comments via email. If they do, copy the email text and save it on the TWG page for the review as a new comment. Thank the sender for the email and add your email to the comment as a reply. Also forward the comment to the authors. Use your discretion whether or not to remove the commenter's name and email address.

Comments that are received in Jama will be retained by Jama.

Reminder Announcements

At about the 1/2 or 3/4 point of the review period consider sending out a reminder announcement. Send reminder emails to targeted reviewers if they have not already submitted any comments.

Review Period Extension

Deciding whether to extend the review can be hard. Many people wait until the last week or last few days to start. However, if there are not enough commenters near the end of the review period, consider extending the review period. This provides a good reason for sending another announcement.

Review Period Conclusion

When the review period is over, the Public Review Page, the ESCO Reviews page on Earthdata and the ESCO Reviews page on the Wiki should be edited to indicate that the review period is over.

  •  

Edit the Public Review page to indicate that the comment period is closed.

  •  

Edit the ESCO Reviews page on Earthdata to move the review from the Current section to the Reviews in Progress section.

Change the comment period to indicate that it is closed.

This example shows how the page looks with a current review, an upcoming review and two in-progress views.

(see Earthdata Page Editing, Creation, etc.)

  •  

Edit the ESCO Reviews page on the wiki to 

move the review from the Current section to the Reviews in Progress section.

Change the comment period to indicate that it is closed.

This example shows how the page looks with a current review, an upcoming review and two in-progress views.

Assess Comments

TWG Assessment

When the comment period closes, the TWG should assess the comments to decide

  • Did enough people review the document? There is no specific number. Some topics can be very specialized and you may only find a small number of reviewers. Other documents attract as many as 20+ reviewers.
  • Do the comments warrant any revision of the document? The answer to this is usually "yes".
  • Do the comments support approval by ESDIS for use in EOSDIS? There should be more favorable comments than negative ones to support this.

If the TWG feels that there were not enough comments or that the comments were too unfavorable, ESO needs to decide how to proceed. There is no specific guidance for this and each document has to be handled on a case-by-case basis. In the past, there have been a few different scenarios:

  • A documented standard (RFC-005) was found to have little operational use but during the review it became clear that the prior version of that standard was used extensively. The submitters were encouraged to change the category of the current version to be a Technical Note and to submit the prior version as a Specification (RFC-006).
  • A documented convention (RFC-013) was submitted by an ESDIS project team. This convention was only in use within the system maintained by that team and data providers who submitted information to that system using the specified format. No other implementations existed or were expected to be developed. The TWG recommendation was to not recommend the spec for adoption. Subsequently, the status was noted as "submitted/in progress" on the ESO Document List. In 2016, ESO made the decision to mark it as Deprecated.
  • As part of an ESDIS desire to have the GeoTIFF file format be documented properly (GeoTIFF was a fairly mature format with no current format specification at the time), ESO invited someone to write a GeoTIFF RFC. We expected the RFC to contain an up-to-date format specification, but that turned out to be a far more complex task than originally expected. We had an RFC number dedicated to the document and the author submitted a draft that did not document the current format. The decision was made to stop processing the document even before sending it out for comment and the document number was reused for a different RFC.
  • An RFC covering the topic of Datacasting was submitted in 2012 but after over a year it was dropped due to insufficient evidence of implementation. See the comment at the bottom of the page for some background.

If the decision is made to stop processing this RFC or to change course, then the rest of the process will be different.

Authors' Assessment

The RFC Editor notifies the Authors that they can/should start addressing the comments and work towards a revised document. For recent reviews, particularly those with extensive comments, the authors have set up Google spreadsheets to keep track of the process. There is an Excel spreadsheet here that can be used as an example. The authors have shared the spreadsheets with the TWG but the spreadsheets are not meant to be preserved as public documents. The material in the spreadsheets can be valuable resource material for the final TWG report.

The authors will need to be upgraded to Moderator status on the Jama review so that they can work with the comments. 

Once the authors have finished addressing the comments, they should provide a new version of the document.

  •  

Contact the authors to work with them on addressing the comments

  •  

Give the authors copies of comments that were submitted via email

  •  

Add the authors as Moderators in the Jama review.

In the Actions menu, you will find Update Moderators. Add the authors by clicking their names in the left panel, either in Project Team or Outside Project Team.

  •  

Provide the authors with the sample spreadsheet and ask them to track the comments.

  •  
You may need to periodically check in with the authors to see how they are progressing.

Community Review - 2nd Round

The 2nd round of reviews is meant to give the 1st round commenters a chance to give feedback on how their comments were handled. Only those people who provided comments in the 1st round are invited to re-review the document.

Pre-review Logistics

Review Period

For the 2nd round, two weeks has been typical in the past. Adjust this as needed.

Document Preparation

For the 2nd round, you don't need new review questions. If all the 1st round commenters used Jama, then you also don't need to post updated documents on the Public Review Page.

You do have to ensure you have a clean copy of the updated document, named and numbered appropriately with a new version number to upload into Jama.

Add the new document into Jama

  •  

Send request to esco-staff@lists.nasa.gov asking to upload the review to Jama that includes:

  •  
  • The updated document (let her know that you'd like this in a new review, not a revision of the prior review)
  •  
  • Review end date
  •  
  • List of people needing to be moderators (typically all of ESCO staff)

Contact the 1st round commenters

Just before you send out the review announcement, add the 2nd round commenters to the Jama review as reviewers. They will probably get an automated email from the Jama system letting them know they've been added to the review.

This is a sample round 2 review announcement.

NOTE: This was adapted from the DQWG 2nd round review letter.

Thank you for your thoughtful reviews of ESDS-RFC-041: Data Product Development Guide for Data Producers. Your comments will help make these recommendations more generally useful and widely applicable.

Together with the authors, the ESDIS Standards Coordination Office has produced an updated version of the document to address the comments we received. The authors of this document also replied to each of your comments in the original review of this document. The original review copy and comments are still available here:  https://rms.earthdata.nasa.gov/review.req#/r:REV-514. Please note that since the review on the original document has closed, you can only view the authors' replies there, but can not comment anymore. You can add new comments to the updated version of this document (review link provided below).

Please take a look at this updated version and provide any additional feedback you may have by June 19th. Please note that since you’ve already replied to the reviewer questions (questions under the Guide for Reviewers section) in the original document, you don’t need to reply to them again but are you are welcome to comment further.

As before, you can provide comments into Jama directly at the URL

https://rms.earthdata.nasa.gov/review.req#/r:REV-527

Or you can respond via email.  The revised document is attached.

The due date for this review is June 19, 2020 at 11:00 PM EDT.

Thanks for your help, and best regards,

ESDIS Standards Coordination Office

Keep track of who you've sent the 2nd round review invitation to, their response, etc. on the TWG page.

Post Review

Document Clean-up

The TWG evaluates the comments received. The authors may need to make some additional revisions to the document.

Once the authors have finished the document, create a clean copy with all headings such as version number, title, etc filled in.

Send recommendation to ESDIS

The TWG is responsible for writing a recommendation to ESDIS to approve the document. The recommendation includes a summary of the RFC along with a short summary of the Strengths, Weaknesses, Applicability, and Limitations (SWAL) of the RFC. Most current RFC landing pages (see the ESCO Document List) have examples of the SWAL.

When the recommendation is ready, it is sent to the NASA ESCO lead along with the document. The NASA ESO lead is responsible for either approving the RFC, getting further feedback about whether to recommend from within ESDIS, or rejecting the RFC (the latter has not happened so far).

Publication of the RFC

COMET

Skip this part if the RFC is not going to be published in COMET.

At this point, if the RFC is meant to be published in COMET, then if it's not already in COMET format, it must be turned into a properly formatted COMET document.

Once there is a COMET formatted version, the document has to be submitted to COMET and be approved by ESDIS within the COMET system.

The final, COMET-approved, PDF version of the document is then published the same way a non-COMET RFC would be.

Earthdata

There are several steps to publishing an RFC (or COMET) document. First, the final, cleaned up and properly named PDF file should be uploaded to Earthdata. This is done via Conduit. Then a landing page has to be created for it (or updated if the document will share a landing page with another - see the Dataset Interoperability Recommendations for Earth Science landing page for an example with two documents or the Recommendations from the Data Quality Working Group landing page for an example with three documents).

The document should also be added to the ESCO Document List and probably (but not always) to the Standards and Practices page. There are a few Mission Requirements documents that are on the ESDIS Mission Requirements page and not on the Standards and Practices page.

Once the document is in Conduit on Earthdata and the landing page is public, let the authors know about it. They may have helpful comments about the landing page.

Additional Housekeeping

ESO Reviews pages

Once a document has been published on Earthdata, the review is removed from the ESCO Reviews page on Earthdata and from the ESCO Reviews page on the Wiki.

Wiki Pages

The wiki pages that were public as well as the private ones are gathered into a bundle with the Public Review page as the parent and the rest of the pages as child pages. These are moved to the ESCO Review and Publication Archives on the wiki.

Announcement

When the publication steps have been completed, it's a good idea to send an announcement email and to post the announcement on the #eosdis-standards channel on Slack.


Attachments

  File Modified
JPEG File update-moderators.jpg Sep 10, 2024 by Steve Olding
JPEG File review-in-progress.jpg Sep 10, 2024 by Steve Olding
Microsoft Excel Spreadsheet ESO Review Comment Disposal template.xlsx Sep 10, 2024 by Steve Olding
JPEG File ESO Reviews page on Wiki example.jpg Sep 10, 2024 by Steve Olding
JPEG File ESO Reviews page on Earthdata example.jpg Sep 10, 2024 by Steve Olding
JPEG File copy-page-2.jpg Sep 10, 2024 by Steve Olding
JPEG File copy-page-1.jpg Sep 10, 2024 by Steve Olding

  • No labels