Methods for Estimating the Accuracy of Per-Pixel, Per-Parcel and Expert Visual Classification of High Resolution Optical Satellite Imagery

Neil Stuart +, Tom Jaas, Ioannis Zisopoulos and Karin Viergever
Institute of Geography, University of Edinburgh, Drummond Street Edinburgh EH8 9XP. 

Abstract. We describe methods for collecting appropriate quantities and  types of reference data for validating classifications of high resolution satellite data. We use the example of collecting reference data to test classifications of 1m spatial resolution IKONOS data for an open woodland savanna in Central America. Reference data was collected in the field by GPS survey to ensure the purity and representativeness of the ground areas and a precise matching between the ground data and the corresponding image pixels. The image is then classified by three methods: by automatic per-pixel maximum-likelihood (ML), by automatic per- parcel nearest neighbour and by a visual classification by experienced image interpreters. We find that the per-parcel classifier achieves higher accuracy than the per-pixel ML classifier for all the required land cover classes. The overall accuracy for the per-parcel classifier is 82% (producer accuracy range: 47-95%, k=0.73) compared to 57% (range: 36-70%, k=0.5) for ML.  The classification by expert visual interpretation yields an overall accuracy of 96% (range: 89-100%, k=0.95). The per-parcel classification exceeded the minimum accuracy requirement of 70% for two of five required land cover classes and approached the target of 85% suggested for the overall accuracy required in natural resource mapping.   We conclude that a per-parcel classifier can achieve an acceptable standard of accuracy for some of the savanna land cover classes, but that further work is needed to improve the classification of smaller groups trees and sparse woodlands.   Since visual classification is still commonly used in developing countries for classifying imagery and in some cases is the desired output that an automated classifier seeks to reproduce, we developed a means to measure the stability or reliability of a visual classification. We estimate the average accuracy of a series of manual, visual classifications of the same image by different interpreters, by comparing each to the agreed ‘master’ classification by an expert interpreter. The result shows which map features are frequently classified correctly (or not) by different interpreters and according to their level of expertise.  This information allows further training to focus on these classes.  

Keywords: accuracy comparison, per-parcel, visual interpretation, reference data, IKONOS

In: Wan, Y. et al. (eds) Proceeding of the 8th international symposium on spatial accuracy assessment in natural resources and environmental sciences, World Academic Union (Press).

Stuart2008accuracy.pdf344.99 KB