Overview

The Common Metadata Repository (CMR) must ensure that the needs of existing systems (GCMD, ECHO and EMS) are met. In order to achieve this goal, the system will have to evolve its own requirements by collecting use cases and goals from users, customers and other stakeholders. These efforts will benefit from a common and consistent vocabulary, structure, and process for requirements gathering and tracking. This guide is intended to document these items. This guide is intended for use by the entire integrated CMR development team.


While it's almost certain that these practices will evolve over time, the items covered here will help keep the CMR team focused on developing the system with minimum distraction from confusing terminology and inconsistent workflows.

Tools in Use

The CMR will utilize Jama to record, review, approve, and manage system requirements and JIRA to record and track system issues and manage development tasks. Both tools utilize URS (https://urs.eosdis.nasa.gov/) for authentication. To gain authorization for either the Jama or JIRA tool, please request CMR Jama Access via support@earthdata.nasa.gov

Resource

The Jama User Guide can be a valuable resource for people using the tool: https://rms.earthdata.nasa.gov/help/index.html

Assumptions

It is assumed that all requirements will be developed and maintained in the Jama tool. If requirements have previously been maintained in Word or Excel, these requirements shall be imported into Jama.
CMR Systems Requirements will be an evolution of existing ECHO and GCMD Requirements. As different system components are developed and implemented, those requirements should be reconciled with existing system requirements to eliminate duplication and obsolete functionality.

CMR Requirements Hierarchy At Glance

Requirement Level

Tracking Location

Summary

Example

Theme

Jama via tags

Cross-cutting aspects that may span several components, these should be associated with Epics in Jama using tags on Epics or user stories.

Metrics; Monitoring; User Experience

Component

Jama

CMR Sub-system

Search; Ingest Adaptors; Bulk Change Tool

Epic

Jama

Large piece of functionality; a feature

Fast Keyword Searching; Controlled Vocabulary Adaptation

User Story

Jama/JIRA

Small, independent and testable piece of functionality that can be completed in 1-3 days. These are the lowest level of requirements tracking within Jama.

  • As a CMR user I want to be able to query datasets using keyword parameters so that I can find the data I want.
  • As a Science Coordinator, I want to be able to add a Science Keyword so that I can maintain current controlled vocabularies

Bootstrap Story

JIRA

Internal development activity, not necessarily demonstrable to end users, should be associated with one or more user stories, may be longer in duration than a user story

  • As a Developer, I want a default project generator so that I can easily setup new projects

Task

JIRA

In JIRA, user stories may be even further broken down into tasks. This is not required and is not tracked within Jama.

  • Formulate tests
  • Update documentation

Visual Hierarchy of CMR Components, Epics and User Stories

Vocabulary

Establishing a common vocabulary for requirements will help the team focused on the work at hand without creating confusion. Some of these terms can be become semantically confusing so it helps to document how they are defined within the context of CMR requirements. The following terms should be used as specified. 

Resource

Note: Some of these definitions borrow heavily from Agile 101, available here: http://agile101.wordpress.com/2009/08/10/the-difference-between-agile-themes-epics-and-user-stories/

Themes

A Theme is a top-level objective that may span projects and products but is does not assign functionality to a sub-component. Themes will describe how the CMR as a system will interact with external interfaces and systems, as well as exhibit the behaviors that span projects and products. Themes may be broken down into sub-themes, which are more likely to be product-specific. 
For CMR requirements, themes can be tracked via tagging in JIRA. The exact mechanism for recording and tracking themes is still to be worked out.

Components

The CMR will consist of several sub-system components that are descended from the system level requirements for the CMR, and are the result of assigning the CMR level functionality to specific implementation-level components. Requirements in Jama will be developed for each of these sub-systems and will have a corresponding component within the tool. These components can be created as needed and have a set of Epics and a set of User Stories underneath a component for organizational purposes.

Component Ownership

Each component within the CMR will have an owner responsible for tracking epics and stories within that component and conducting reviews on that component as need. This owner should also be involved in prioritizing work, planning sprint activities, and maintaining the component's work backlog. Owners are ideally are not members of the development team but should be able to understand the wider impact of the component and work with stakeholders.

Epics

Epics represent a high level piece of functionality that rolls up one or more related user stories. Epics are frequently bigger than a single sprint's worth of effort. These are often also referred to as features of a system. Epics should be traced back to one or more SOW Requirements. You would be unlikely to introduce an Epic into a sprint without first breaking it down into it's component User Stories so to reduce uncertainty. A component's owner will ensure that each Epic is integrated and properly regression tested.

User Stories

User stories represent a specific piece of functionality that is to be developed for the system. Stories are written from the perspective of the actor (e.g. "As an authenticated user, I want to perform a spatial search with a 4 point Cartesian bounding box."). User stories are typically 1-3 days in implementation length. Exceptions can be made to 4 days, but in general stories longer than 3 days should be split into multiple stories. Stories are allocated to sprints (usually several stories are planned in a single sprint) and are generally the unit of discussion for sprint planning. User stories should be traced back to one or more Epics.
Agile projects are also assumed to need a JIRA connector that will allow User Stories from the Jama project to be synced to and from their corresponding JIRA project for development purposes.

Good User Stories

The acronym "INVEST" can remind you that good user stories are:

  • I – Independent
  • N – Negotiable
  • V – Valuable
  • E – Estimable
  • S – Small
  • T – Testable

http://xp123.com/articles/invest-in-good-stories-and-smart-tasks/

Bootstrap Stories

In addition to the more traditional user stories, the CMR development effort will introduce the concept of Bootstrap Stories. These stories are longer in duration than User Stories (maybe as long as several weeks) and might not result in end-user demonstrable functionality. It is expected that initially there will be much "behind the scenes" work to set up databases and environments for development and deployment. Laying the groundwork for future development efforts may result in several weeks delay before the first user stories are ready. The Bootstrap Story is intended to track these activities and will be maintained in JIRA, but is not directly applicable to system requirements and therefore will not be in Jama.

Tasks

Tasks are subcomponents of User Stories and Bootstrap stories that are not tracked in Jama and are intended to aid the development team during sprints. Tasks should be short in duration and very specific in scope and should be directly related to a parent story.

Review and Approval Process

Frequency

When a Component and its initial descendant Epics and User Stories are gathered, a review of that component should be conducted using Jama's Review Center. Because reviews done using Jama can occur via the web, meetings are not required, allowing reviews to span several days among several reviewers.
It is expected that after initial component requirements are established, new requirements will continue to be introduced to the various sub-system components. These additional requirements will generally be reviewed by the CMR team on a bi-weekly or monthly basis.
If a large group of related requirements are added, the group may be subject to the same longer review process as the initial component requirements.

Participants

The CMR requirements review team should consist of at least one participant from ESDIS, ECHO, GCMD and EMS and will ideally include other stakeholders from DAACs or other interested Earthdata systems and projects (e.g. GIBS).

Process

As mentioned above component requirement reviews will occur within Jama and will span several days. The component owner will initiate this review and determine participants. Depending on the outcome of the web-based review, the component owner may choose to call a meeting (WebEx or in-person) to hash out details for particularly contentious stories.
Once the review is complete and outstanding issues have been addressed, the requirements/stories are approved and work may be prioritized in upcoming sprints.

Review Resources

For guidance on using the Review Center tool, please refer to the relevant section Jama User's Guide, found at the following link: https://rms.earthdata.nasa.gov/help/index.html?review_center.htm
The following video might also be of use http://www.jamasoftware.com/how-to-review-center/

4 Comments

  1. Do you have documentation on how to link JAMA components/stories with JIRA?

  2. Based on my understanding, when a story is marked "Approved" within Jama a ticket is created within JIRA,  this should happen automatically.

    1. I am not sure this is the case. I have several user stories in JAMA marked as "Draft" and they are added and synced in Jira. Who can we contact to verify this case and process?

      1. From what I was told, this should not be how it works. Problems with the sync should be raised as issues in the RMS JIRA project: https://bugs.earthdata.nasa.gov/browse/RMS