Use Case: I need to provide information about the quality of my data and how it was measured.
Overview
A principle goal of metadata is to ensure that the data they describe can be independently understood and used effectively. Data quality measures and reports play a critical role in achieving this goal. Connecting these to the metadata record is clearly important.
The approach to including quality information in ISO 19115 metadata records is similar to the approach used in ECHO and improves on it considerably. It includes the capability to describe quality measures that are used and the techniques used to apply them. Understanding how to take advantage of this flexibility and implementing systems that maximize the value that this capability provides will certainly be a challenge for the environmental data community.
The Data Quality Section of the ISO standard supports flexibility at several levels. The DQ_Data_Quality object (see Figure) includes two sections: scope, and element. A metadata record can have any number of associated DQ_DataQuality objects.
Presentations available that describe the ISO data quality standard and compare it with the DIF and ECHO approaches.
Recommendations
An important initial goal is to connect users to existing literature that includes quality information. This should be done at the collection level using standaloneQualityReports. A complete report includes a description of the scope of the quality information with spatial and temporal extent information, an abstract, and a complete citation.
Conceptual Model
This Figure shows an overview of the data quality information included in the ISO 19157 standard. The elements are:
Scope: quality information can be provided at many different level of detail from an entire series down to specific attributes in particular regions or time periods. The scope attribute lets users know what the current information pertains to.
StandaloneQualityReport: quality information exists in papers and reports that are already part of the scientific literature. Users can find this information using citations included in the metadata record.
Report: quality reports hold the bulk of the structured quality information in the metadata record. They describe the quality measure, how it was applied, and the result of the application.
An overview of the complete data quality standard is also available.
Scope:
The DQ_Scope object enables a complete description of the scope of a data quality report whether it pertains to a complete data series or to a particular temporal and spatial extent for a particular parameter in a dataset.
level:
The level element of the scope provides a general description of the scope using a codelist of standard terms. The level can be used in an initial search of the quality information to answer questions like "what kinds of quality information are available for this collection or granule".
attribute | dimensionGroup | model | aggregate |
attributeType | feature | tile | product |
collectionHardware | featureType | metadata | collection |
collectionSession | propertyType | initiative | coverage |
dataset | fieldSession | sample | application |
series | software | document |
|
nonGeographicDataset | service | repository |
levelDescription:
In some cases, the general description provided by the codelist is not enough. In those cases the levelDescription can be used to provide more detail. For example, the scope code might be "attribute", indicating that the report concerns an attribute of the dataset, and the levelDescription would provide the name(s) of the specific attributes covered, i.e. precipitation rate.
extent:
In some cases, the quality information may apply to a particular spatial and temporal extent of the dataset. This extent can be described at several different levels of detail using the EX_Extent object. The extent should be described quantitatively, if possible, or with a text description if quantitative information is not possible.
StandAlone Report
ISO 19157 recognizes that important data quality information can exist outside of the conceptual framework of the model and that it may be helpful to provide that information as a supplement to the metadata. The standaloneQualityReport was added to enable connections between the metadata and these reports. It includes an abstract (CharacterString) that provides a brief description of the quality report in the metadata record and a the citation (CI_Citation) that provides the reference that a user needs for the complete report.
In many cases quality information is provided on the Web pages. Those can also be referenced as StandAloneReports.
As an example, consider this data quality description from a GCMD metadata record:
Abstract: The fire training-set may also have been biased against savanna and savanna woodland fires since their detection is more difficult than in humid, forest environments with cool background temperatures [Malingreau, 1990]. There may, therefore, be an under-sampling of fires in these warmer background environments.
Citation: Malingreau J.P, 1990, The contribution of remote sensing to the global monitoring of fires in tropical and subtropical ecosystems. In: Fire in Tropical Biota, (J.G. Goldammer , editor), Springer Verlag , Berlin: 337-370.
Report
Quality reports are encapsulated in the 19115 DQ_Element class that includes three kinds of information, the measure, the evaluationMethod and the result. It is expected that an organization uses a standard set of quality measures that are managed centrally and have unique names, descriptions, and identifiers. The nameOfMeasure and measureDescription are included in the metadata record and should help users understand the nature of the measure. The measureIdentification is an identifier that can be used to find a more detailed description of the measure.
The evaluation method provides information on how the quality measure was applied for this report. These methods can also be standardized across an organization. Citations are available to describe the specific proceedure and more complete references.
Finally, the result reports the results of the application of the measure in this specific case. Several types of reports are included in the standard (see below).
DQ_MeasureReference [0..1]
+ measureIdentification: MD_Identifier [0..1]
+ nameOfMeasure: CharacterString [0..*]
+ measureDescription: CharacterString [0..1]
DQ_EvaluationMethod [0..1]
+ dateTime: DateTime [0..*]
+ evaluationMethodDescription: CharacterString [0..1]
+ evaluationProceedure: CI_Citation [0..1]
+ referenceDoc: CI_Citation [0..*]
+ evaluationMethodType: DQ_EvaluationMethodTypeCode [0..1]
DQ_Result [1..*]
+ dateTime: DateTime [0..*]
+ resultScope: DQ_ScopeCode [0..1]
Result Types
The ISO Standard include four types of data quality results in the DQ_Element object.
The first, the DQ_ConformanceResult, describes how the dataset was tested for conformance to a published standard and whether the dataset passed the test.
The second, the DQ_QuantitativeResult, provides a mechanism for describing the results of a quantitative quality evaluation.
Finally, the QE_CoverageResult (from ISO 19115-2), allows a quality result to be expressed as a spatial object. For example, if a gridded dataset has an associated grid of quality flags, that quality grid could be described here. Note also that the coverage results have an associated file and format.
The quality of many datasets varies spatially within the dataset. For example, gridded satellite datasets include grids that provide quality flag values for every pixel. Radiosondes in the atmosphere and profiles in the ocean also can have quality that varies along their paths. In situations like these, the quality information can be described using a spatial feature like a grid or a line. In ISO quality information in spatial features is described using a coverageResult. The coverageResult includes MD_SpatialRepresentation and MD_CoverageDescriptions that describe the quality coverage. In this case, the MD_CoverageDescription can serve a role that is similar to the role of the measure and evaluationMethod elements in the report. The MD_CoverageDescription includes a contentType that can currently be image, thematicRepresentation, or physicalMeasurement. The revision of 19115 may add qualityInformation to this codelist and allow multiple contentType values.
Finally, the DQ_Descriptive result was introduced in ISO 19157 in order to include simple textual descriptions of data quality in the standard. It is a simple CharacterString.
DQ_ConformanceResult
+ specification : CI_Citation
+ explanation : CharacterString
+ pass : Boolean
Or
DQ_QuantitativeResult
+ valueType [0..1] : RecordType
+ valueUnit : UnitOfMeasure
+ errorStatistic [0..1] : CharacterString
+ value [1..*] : Record
Or
QE_CoverageResult (added in ISO 19115-2, see UML below)
+ spatialRepresentationType: MD_SpatialRepresentationTypeCode, one of vector, grid, textTable, tin, stereoModel, video
+ resultFile: MX_DataFile
+ resultFormat: MD_Format
+ resultSpatialRepresentation: MD_SpatialRepresentation
+ resultContentDescription: MD_CoverageDescription
Or
DQ_DescriptiveResult
+ statement: CharacterString
Implementation (XML)
The general structure of a data quality report in XML is:
<gmd:DQ_DataQuality xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.isotc211.org/2005/gmd http://www.isotc211.org/2005/gmd/gmd.xsd http://www.isotc211.org/2005/gmi http://www.isotc211.org/2005/gmi/gmi.xsd"
xmlns:gmi="http://www.isotc211.org/2005/gmi" xmlns:gco="http://www.isotc211.org/2005/gco" xmlns:gmd="http://www.isotc211.org/2005/gmd"
xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:srv="http://www.isotc211.org/2005/srv" xmlns:gml="http://www.opengis.net/gml/3.2">
<gmd:scope>
<!--
The DQ_Scope object can include an id attribute that allows the same quality scope to be referenced from several different sections of the metadata record.
For example, if all of the quality reports in the record pertain to the entire granule, they could all reference the same scope object with a value of "dataset".
In this case, the id is set to datasetScope.
-->
<gmd:DQ_Scope id="datasetScope">
<gmd:level>
<!--
The MD_ScopeCode is a codelist that includes two standard attributes: a URL that gives the location of the codeList,
and a value from the codelist. The location can be set to the standard location if the standard codelist is sufficient
for the data provider needs or, if necessary, it can be set to a codelist that has been extended to include dataset
specific values.
-->
<gmd:MD_ScopeCode codeList="http:CodeListLocation//#MD_ScopeCode" codeListValue="dataset">dataset</gmd:MD_ScopeCode>
</gmd:level>
<!--
The extent of the quality report can include specific information, if appropriate, or it can reference an extent defined
in another part of the metadata record. In this case, the extent of the quality report is the same as that of the dataset,
so the identifier boundingExtent is used. If necessary, the qualityReport can contain more than one extent. See extent examples.
-->
<gmd:extent xlink:href="#boundingExtent"/>
<gmd:levelDescription>
<!--
The levelDescription is used to provide more detail about the scope of the quality report. It can include name(s) of
attributes or features or a text description of the scope. The properties also include dataset and other which are
both character strings. If necessary, the quality report can contain more than one levelDescription.
-->
<gmd:MD_ScopeDescription>
<gmd:dataset>
<gco:CharacterString>levelDescription</gco:CharacterString>
</gmd:dataset>
</gmd:MD_ScopeDescription>
</gmd:levelDescription>
</gmd:DQ_Scope>
</gmd:scope>
<gmd:report>
<!--
The report holds the meat of the DQ_DataQuality section og the metadata. It includes four major sections: the measure,
the method, the date, and the result. There are a number of different types of reports that reflect general areas of
data quality, e.g. DQ_Completeness, DQ_LogicalConsistency, DQ_PositionalAccuracy, DQ_ThematicAccuracy, DQ_TemporalAccuracy,
DQ_QuantitativeAttributeAccuracy, and DQ_Usability. These are very general descriptors that should be used if clearly
appropriate. At the same time, some of these names were created with collections of geographic features in mind and so
may not make sense in the context of other data types. In those cases, the very general type DQ_QuantitativeAttributeAccuracy
serves as a generally applicable catch-all.
-->
<gmd:DQ_QuantitativeAttributeAccuracy>
<!--
Information about the quality measure: To facilitate data set comparisons, it is helpful if the results in the data quality
reports are expressed in a comparable way and that there is a common understanding of the data quality measures that have
been used. ISO 19157 provides a set of standard measures to facilitate comparisons, but these are very general, e.g.
"number of missing items" or "relative horizontal error". The standard also describes how user defined quality
measures can be cataloged for sharing across a community.
ISO 19115 quality reports include three elements that are used to describe the quality measure. The nameOfMeasure and
measureDescription are used to provide enough information for a user to understand the measure being used if they are
familiar with the measures used in a community. The measureIdentifier is used to support a search for more information.
-->
<gmd:nameOfMeasure>
<gco:CharacterString>nameOfMeasure</gco:CharacterString>
</gmd:nameOfMeasure>
<gmd:measureIdentification>
<gmd:MD_Identifier>
<gmd:code>
<gco:CharacterString>measureIdentifier</gco:CharacterString>
</gmd:code>
</gmd:MD_Identifier>
</gmd:measureIdentification>
<gmd:measureDescription>
<gco:CharacterString>measureDescription</gco:CharacterString>
</gmd:measureDescription>
<!--
The evaluation method provides a description of how the measure was used in the context of a particular dataset
in order to generate the quality report. ISO 19115 includes three elements that are relevant to the evaluationMethod:
the type is a codelist with values of directInternal, directExternal, or indirect; the evaluationMethodDescription
is a text description that is available to a user along with the metadata record; the evaluationProceedure provides
a citation to more detailed information.
-->
<gmd:evaluationMethodType>
<gmd:DQ_EvaluationMethodTypeCode codeList="URI to codelist location" codeListValue="direct_external">direct_external</gmd:DQ_EvaluationMethodTypeCode>
</gmd:evaluationMethodType>
<gmd:evaluationMethodDescription>
<gco:CharacterString>evaluationMethodDescription</gco:CharacterString>
</gmd:evaluationMethodDescription>
<gmd:evaluationProcedure>
<gmd:CI_Citation>
<gmd:title>
<gco:CharacterString>title of reference material</gco:CharacterString>
</gmd:title>
<gmd:date gco:nilReason="missing"/>
</gmd:CI_Citation>
</gmd:evaluationProcedure>
<!--
The dateTime gives the date and time of the quality test in ISO1601 compliant text format
-->
<gmd:dateTime>
<gco:DateTime>2012-10-02T15:44:00</gco:DateTime>
</gmd:dateTime>
<!--
The result is the actual result of the quality test. There are four types of results: DQ_QuantitativeResult,
DQ_ConformanceResult, QE_CoverageResult (from ISO 19115-2), and DQ_DescriptiveResult (from ISO 19157).
-->
<gmd:result>
<gmd:DQ_QuantitativeResult>
<gmd:valueUnit xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="AnyValidURI"/>
<gmd:value>
<gco:Record></gco:Record>
</gmd:value>
</gmd:DQ_QuantitativeResult>
</gmd:result>
<gmd:result>
<gmi:QE_CoverageResult>
<gmi:spatialRepresentationType/>
<gmi:resultSpatialRepresentation/>
<gmi:resultContentDescription/>
<gmi:resultFormat/>
<gmi:resultFile/>
</gmi:QE_CoverageResult>
</gmd:result>
<gmd:result>
<gmd:DQ_ConformanceResult>
<gmd:specification/>
<gmd:explanation/>
<gmd:pass/>
</gmd:DQ_ConformanceResult>
</gmd:result>
</gmd:DQ_QuantitativeAttributeAccuracy>
</gmd:report>
</gmd:DQ_DataQuality>
Details and examples of how ECHO data quality content can be represented in ISO.
Notes
Tracking Compliance With Standards
The importance of standards in data integration is well know and broadly acknowledged. It is also well known that compliance tests will be required so that developers can systematically measure compliance with standards that evolve with time. Of course, these tests can only useful to users if the results are available and understandable.
There are two ISO Standard objects that apply to compliance reports. The first is the DQ_DataQuality object. This object associates a DQ_Element with a service, a dataset, or some subset of a dataset described by the DQ_Scope. The DQ_Element references a test and an evaluation procedure that produces a DQ_Result at a particular time. There are three types of DQ_Results, but the DQ_ConformanceResult is the type that is relevant here. It references the specification being tested, explains the meaning of conformance, and gives a Boolean result.
How might these new types of reports be used? The environmental data community is using more and more standard formats for data files. The conformance test might indicate how well the dataset conforms to those file formats or associated conventions like the Climate Forecast conventions for netCDF.
Tracking Problems Identified by Users
The second ISO object that could be related to conformance testing is the MD_Usage object. This provides a mechanism for incorporating user problems into the metadata for a dataset or service. The object includes a description of the specific usage, the problem encountered (the limitation), the date and the user contact information. In the case of standards compliance, the usage would be attempting to access the dataset or service in a way that is consistent with an advertised standard and the limitation would be finding the dataset to be non-compliant with the advertisement. These problem reports would be available from the archive and, hopefully, would motivate data providers to be careful about advertising services that they could not support.
MD_Usage
+ specificUsage : CharacterString
+ usageDateTime [0..1] : DateTime
+ userDeterminedLimitations [0..1] : CharacterString
+ userContactInfo [1..*] : CI_ResponsibleParty
