The ECHO metadata dialect includes two kinds of quality flags: QAStats and QAFlags. We examined sample granule metadata records for ~4100 collections to determine how these flags were currently used. This table shows the number of flags / record in metadata collections from sixteen DAACS. Cells that are shaded red indicate that no occurrences exist. These data indicate that four of the DAACs include a significant number of standard quality flags in their granule metadata.
| Path Elements | ASF | CDDIS | GES_DISC | GHRC | GSFCS4PA | LAADS | LANCEAMSR2 | LANCEMODIS | LARC | LARC_ASDC | LPDAAC_ECS | NSIDC_ECS | OB_DAAC | ORNL_DAAC | PODAAC | USGS_EROS |
| QAFlags/AutomaticQualityFlag | 3.45 | 0.10 | 0.32 | 2.31 | 0.81 | |||||||||||
| QAFlags/AutomaticQualityFlagExplanation | 3.45 | 0.86 | 0.32 | 2.31 | 0.81 | |||||||||||
| QAStats/QAPercentMissingData | 0.30 | 0.21 | 2.25 | 0.73 | 0.36 | |||||||||||
| QAFlags/OperationalQualityFlag | 0.30 | 0.10 | 0.82 | 0.62 | ||||||||||||
| QAFlags/OperationalQualityFlagExplanation | 0.30 | 0.10 | 0.82 | 0.62 | ||||||||||||
| QAFlags/ScienceQualityFlag | 0.30 | 0.61 | 2.28 | 0.73 | ||||||||||||
| QAFlags/ScienceQualityFlagExplanation | 0.14 | 0.06 | 2.28 | 0.73 | ||||||||||||
| QAStats/QAPercentOutOfBoundsData | 0.25 | 0.06 | 2.17 | 0.36 | ||||||||||||
| QAStats/QAPercentInterpolatedData | 0.10 | 0.01 | 2.17 | |||||||||||||
| QAStats/QAPercentCloudCover | 0.06 | 2.02 | 0.25 | 0.82 |
The most commonly occurring quality flag is the AutomaticQualityFlag and the most common values for this flag are shown below along with the % of the values they make up. There are a total of sixty-four different AutomaticQualityFlag values and the nine values that occur more than 1% of the time are included in this table.
Common values for AutomaticQualityFlag
| Value | % |
| Passed | 46.83% |
| parameter is produced correctly | 27.32% |
| Based on percentage of product that is good. Suspect used where true quality is not known. | 5.15% |
| No automatic quality assessment is performed in the PGE | 5.07% |
| DummyValue | 2.22% |
| Automatic quality determination software not yet implemented | 1.61% |
| Process Terminated Normally - Check Science Quality Flag for Science Team quality assessment | 1.43% |
| Passed indicates parameter passed for specific automatic test; Suspect, QA not run; Failed, parameter failed specific automatic test. | 1.10% |
| Validated | 1.01% |
There are several immediate observations from these flag values. First, we would expect flags to be simple and somewhat consistent indicators of quality that might be used to drive a granule search. Many of these flags are more like phrases than flags and they are inconsistent. For example, the two most common flags are “Passed” and “parameter is produced correctly”. As a general indicator of quality these two values seem equivalent, but they could not be easily searched for. Similarly, the value “Validated”, while relatively infrequent, means essentially the same as the other two values. The same situation exists at the other end of the spectrum. The values “No automatic quality assessment is performed in the PGE” and “Automatic quality determination software not yet implemented” provide essentially the same information.
A second aspect of these values is that some appear to be explanations rather than flag values. For example, “Based on percentage of product that is good. Suspect used where true quality is not known.” or “Passed indicates parameter passed for specific automatic test; Suspect, QA not run; Failed, parameter failed specific automatic test.”
Similar observations can be made with the values in the AutomaticQualityFlagExplanation values (shown below) where “parameter is produced correctly” is the overwhelming most common value. The role of this field is to explain how that is determined, not to reiterate the AutomaticScienceQualityFlag.
Common values for ScienceQualityFlag
parameter is produced correctly | 52.55% |
Based on percentage of product that is good. Suspect used where true quality is not known. | 9.90% |
No automatic quality assessment is performed in the PGE | 9.76% |
DummyValue | 4.27% |
Automatic quality determination software not yet implemented | 3.09% |
Process Terminated Normally - Check Science Quality Flag for Science Team quality assessment | 2.74% |
Passed indicates parameter passed for specific automatic test; Suspect, QA not run; Failed, parameter failed specific automatic test. | 2.12% |
Validated | 1.95% |
QA flag explanation | 1.36% |
All data passed during checkout | 1.33% |
No automatic quality assessment is performed in the PGE. | 1.00% |
Other standard flags yield similar observations, shown here for completeness.
Common values for OperationalQualityFlag
Passed | 42.38% |
Not Investigated | 23.93% |
Q/A process has not yet been run | 8.79% |
Passed,parameter passed the specified operational test. Inferred Pass,parameter terminated with warnings. Failed parameter terminated with fatal errors. | 7.32% |
Inferred Passed | 5.86% |
This granule passed operational tests that were administered by the OMI SIPS. QA metadata was extracted and the file was successfully read using standard HDF-EOS utilities. | 2.64% |
currently not used | 1.95% |
Data believed to be good, but there may be isolated exceptions | 1.56% |
Suspect | 1.46% |
no error detected | 1.17% |
Process Terminated Normally | 1.17% |
Common values for OperationalQualityFlagExplanation
Passed | 34.38% |
Not Investigated | 12.89% |
Q/A process has not yet been run | 17.58% |
Passed,parameter passed the specified operational test. Inferred Pass,parameter terminated with warnings. Failed parameter terminated with fatal errors. | 14.65% |
This granule passed operational tests that were administered by the OMI SIPS. QA metadata was extracted and the file was successfully read using standard HDF-EOS utilities. | 5.27% |
currently not used | 3.91% |
Data believed to be good, but there may be isolated exceptions | 3.13% |
no error detected | 2.34% |
Process Terminated Normally | 2.34% |
Process Terminated Normally - Check Science Quality Statement | 1.17% |
Common values for ScienceQualityFlag
Not Investigated | 34.70% |
Inferred Passed | 19.86% |
Passed,parameter passed the specified science test. Inferred Pass,parameter terminated with warnings for specified science test. Failed parameter terminated with fatal errors for specified science test. | 4.53% |
See http://landweb.nascom.nasa.gov/cgi-bin/QA_WWW/qaFlagPage.cgi?sat=aqua for the product Science Quality status. | 4.53% |
See http://landweb.nascom.nasa.gov/cgi-bin/QA_WWW/qaFlagPage.cgi?sat=terra for the product Science Quality status. | 3.86% |
Validated, see http://disc.gsfc.nasa.gov/Aura/MLS/ for quality document | 3.50% |
Passed | 2.96% |
See http://landweb.nascom.nasa.gov/cgi-bin/QA_WWW/qaFlagPage.cgi?sat=aqua&ver=C6 for the product Science Quality status. | 1.57% |
See http://landweb.nascom.nasa.gov/cgi-bin/QA_WWW/qaFlagPage.cgi?sat=terra&ver=C6 for the product Science Quality status. | 1.57% |
See http://landweb.nascom/nasa.gov/cgi-bin/QA_WWW/qaFlagPage.cgi?sat=aqua the product Science Quality status. | 1.45% |
An updated science quality flag and explanation is put in the product .met file when a granule has been evaluated. The flag value in this file, Not Investigated, is an automatic default that is put into every granule during production. | 1.33% |
Common values for ScienceQualityFlagExplanation
Note that the ScienceQualityFlags introduce URLs into the quality flag and explanation fields. This is an appropriate approach to providing more detailed explanations, but it is a strong commitment to persistence of these URLs.