A discussion of data quality flags during the ESDSWG meeting during April, 2016 prompted me to take a look at existing quality flags in the CMR. These all originated in ECHO and we know how to represent them in ISO. The data clearly indicate that the current quality flags suffer from considerable inconsistency which makes them hard to use if we should decide to support quality information as part of a search strategy.
In this blog I propose a small effort to reframe the automatic quality flags in terms of a small number of consistent values. Specifically, I propose the values Passed, Suspect, Failed, and Undetermined. The table below maps the current values to these proposed values. It also shows a number of current flag values that appear to be more appropriate as flag explanations than as flag values.
The table also shows the occurrence frequency of current flags. If the mapping is accepted as proposed, the distribution of values would be: Passed - 77%, Suspect - < 1%, Failed - < 1%, Undetermined - 13.5%. Roughly 8.5% of the current values would be migrated to explanations.
Current flag values | % |
Migrated to Explanation | |
| Based on percentage of product that is good. Suspect used where true quality is not known. | 5.15% |
| Passed indicates parameter passed for specific automatic test; Suspect, QA not run; Failed, parameter failed specific automatic test. | 1.10% |
| QA flag explanation | 0.70% |
| Passed if algorithm ran within bounds of execution constraints. Suspect if bounds of execution constraints violated. Failed if PGE failed. | 0.31% |
| All three channels WN, SW, and Total had values less than 50% bad | 0.23% |
| All MODIS data is classified suspect at this point Set QA flag to pass if geolocation can be performed on one footprint Exhau SS4.5-6: | 0.17% |
| Same as the flag for the SSF. | 0.11% |
| An updated automatic quality flag and explanation is put in the product .met file when a granule has been evaluated. The flag value in this file, Not Investigated, is an automatic default that is put into every granule during production. | 0.09% |
| Inspection estimate: Passed Failed | 0.08% |
| Number of good profiles is more than number of bad profiles. | 0.06% |
| Percentage of good pixels of 3 channels is more than 50. | 0.06% |
| Flag set to Passed if NrofGoodOutputSamples >= 75% of NrofGoodInputSamples, to Suspect if NrofGoodOutputSamples >= 50% but < 75% of NrofGoodInputSamples, and to Failed if NrofGoodOutputSamples < 50% of NrofGoodInputSamples. | 0.05% |
| Passed, indicates parameter passed for specific automatic test; Suspect, QA not run; Failed, parameter failed specific automatic test. | 0.05% |
| The value is based on a combination of the RadianceScienceQualityFlag, IrradianceScienceQualityFlag, QAPctRadianceError, QAPctFitError, QAPctCloudFractionMissing, QAPctCloudPressureMissing. Thresholds used: 50% for Failed and 20% for Suspect. | 0.05% |
| To be set as 'Passed' or 'Failed' to indicate failure of PGE test. | 0.05% |
| Flag set to Passed if QAPercentHighQualityData >= 90%, Flag set to Suspect if QAPercentHightQualityData >= 60% or or input L1B do not have its AutomaticQualityFlag set to Passed, Flag set to Failed if QAPercentHighQualityData < 60%. | 0.03% |
| The value is based on a combination of the RadianceScienceQualityFlag, IrradianceScienceQualityFlag, QAPctRadianceError, QAPctCloudDataError, QAPctSCDError, QAPctGhostColumnError, QAPctVCDError. Thresholds used: 50% for Failed and 20% for Suspect. | 0.03% |
| The value is based on a combination of the RadianceScienceQualityFlag, IrradianceScienceQualityFlag, QAPctRadianceError, QAPctMWError and QAPctNUVError. Thresholds used: 50% for Failed and 20% for Suspect. | 0.03% |
| Flag set to Passed if QAPercentHighQualityData >= 80%, Flag set to Suspent if QAPercentHighQualityData >= 20%, or L1B AutomaticQualityFlag not set to Passed, otherwise Flag set to Failed | 0.02% |
| Flag set to Passed if QAPercentHighQualityData >= 90%, Flag set to Suspect if QAPercentHightQualityData >= 60% or or input L1B do not have its AutomaticQualityFlag set to Passed, Flag set to Failed if QAPercentHighQualityData < 60%. | 0.02% |
| Set QA flag to pass if geolocation can be performed on one footprint All VIIRS data is classified as good Exhausted IES D SS4.5-6: no errorscessed pixel and IES data to SSF. | 0.02% |
| Set QA flag to pass if geolocation can be performed on one footprint All VIRS data is passed as is to convolution Exhaust SS4.5-6: no errorsully processed pixel and IES data to SSF. | 0.02% |
| Set to 'Failed' if processing error occurred,'Passed' otherwise | 0.02% |
| The value is based on a combination of the RadianceScienceQualityFlag, IrradianceScienceQualityFlag, QAPctRadianceError and the ProfileErrorFlag. Thresholds used: 30% for Failed and 50% for Suspect. | 0.02% |
Mapped to Passed | |
| Passed | 46.83% |
| parameter is produced correctly | 27.32% |
| Validated | 1.01% |
| All data passed during checkout | 0.69% |
| Data passed automatic quality checks. | 0.26% |
| Process terminated normally - Check HIRDLS Data Description and Quality document for more information on the quality | 0.25% |
| no error detected | 0.18% |
| Process Terminated Normally | 0.18% |
| This file passed automatic quality checks. | 0.15% |
| The data passed all automatic quality tests | 0.08% |
| This granule passed automatic tests that were performed by the PGE. The percentage of missing, out-of-bounds and contaminated data were computed to determine data quality. | 0.02% |
Mapped to Undetermined | |
| No automatic quality assessment is performed in the PGE | 5.07% |
| DummyValue | 2.22% |
| Automatic quality determination software not yet implemented | 1.61% |
| Process Terminated Normally - Check Science Quality Flag for Science Team quality assessment | 1.43% |
| No automatic quality assessment is performed in the PGE. | 0.52% |
| No automatic quality assessment done in the PGE. | 0.46% |
| No automatic quality assessment done in the PGE | 0.28% |
| Program Execution Completed | 0.28% |
| Refer to Log Files | 0.26% |
| Passed was set as a default value. More algorithm will be developed | 0.18% |
| Passed or Failed | 0.15% |
| Always Passed | 0.12% |
| Not implemented | 0.11% |
| Passed was set as a default value. May or may not require further study | 0.11% |
| Not Investigated | 0.09% |
| Process Terminated Normally - Check Science Quality Statement | 0.09% |
| Currently assuming all data received good, check percent_bad fields | 0.09% |
| TBD | 0.08% |
| Program execution completed. | 0.06% |
| set Passed as default, will decide later | 0.06% |
| Always 'Passed'. | 0.06% |
| Automatic quality determination software not implemented | 0.06% |
| not being investigated | 0.06% |
| Define VAL=Maximum(QAStatPctPixBad,QAStatPctPixProcessingError,QAStatPctGeolocationError,QAStatPctMeasError). Flag set to Passed if VAL is less or equal than 10%, to Suspect if VAL is between 10% and 20% and to Failed if VAL greater or equal than 20% | 0.05% |
| currently not used | 0.03% |
| An automatic quality investigation has not yet been devised. | 0.02% |
| df | 0.02% |
Suspect | |
| Suspect | 0.64% |
Failed | |
| Failed | 0.40% |
Overview
Content Tools