Sampling Designs for Assessing Map Accuracy

Stephen V. Stehman
State University of New York (SUNY)
College of Environmental Science and Forestry (ESF)
320 Bray Hall, Syracuse, NY 13210 USA

Abstract. Assessing map accuracy requires comparing the categories or quantities mapped to the reality of what is on the ground. Practical necessity dictates that the ground condition can only be determined for a sample of locations. Thus sampling design becomes a critical component of accuracy assessment. Historically, the basic sampling designs implemented for map accuracy assessment were simple random, systematic, stratified random, and cluster sampling. These designs remain the fundamental building blocks of effective sampling design for accuracy assessment. The demands placed upon accuracy assessment have increased as the richness of spatial data and applications have expanded. Desirable assessment objectives now extend beyond the analysis based on an error matrix to include accuracy of gross and net change, composition of the classes mapped (at one or more levels of support), and landscape features (e.g. patch size and shape distributions). Quantitative map products, for example, maps of percent impervious surface or percent forest canopy cover, pose new sampling design challenges. Perhaps the biggest challenge of an expanded set of objectives is the requirement to collect reference data for assessment units of different sizes (e.g. 30 m by 30 m pixel, 3x3 pixel block, or 5 km by 5 km block). Multi-stage cluster sampling becomes a prominent design option when attempting to meet multiple objectives targeting multiple sizes of assessment units. Sampling design choices become more difficult as the number of accuracy objectives increases. Different sampling designs are suited to achieve some objectives better than others, and trade-offs among desirable design criteria must be recognized and factored into the decision-making process. As mapping science continues to advance, to keep pace, accuracy assessment sampling designs need to be developed and evaluated to address these emerging new objectives. Sampling designs can no longer just target the traditional descriptive accuracy objectives encapsulated by the error matrix analyses, but must simultaneously permit assessment of additional objectives such as accuracy of land-cover composition and landscape pattern, and accuracy of quantitative map products.

Keywords: multi-stage cluster sampling, landscape pattern, land-cover composition, quantitative outputs

In: Wan, Y. et al. (eds) Proceeding of the 8th international symposium on spatial accuracy assessment in natural resources and environmental sciences, World Academic Union (Press).

Stehman2008accuracy.pdf202.07 KB