Deriving thematic uncertainty measures in remote sensing using classification outputs

Kyle M. Brown 1, Giles M. Foody 2 and Peter M. Atkinson 2
1 Environment Agency, Science Group - Technology, 
Lower Bristol Road, Bath, UK BA2 9ES, UK,
2 School of Geography, University of Southampton, 
Highfield, Southampton SO17 1BJ, UK, ;

The process of estimating thematic errors in the classification of remotely sensed imagery generally involves the use of the confusion matrix. Though measures of overall and per-class accuracy may be derived from the confusion matrix there is no information on where thematic error occurs. However, spatial variation in  thematic error can be a key variable in determining errors when overlay operations such as change detection are carried out. One method of indicating thematic error on a per-pixel basis is to use the outputs of a classifier to estimate the uncertainty associated with the allocation of a particular class to the pixel. This probabilistic approach has been  used previously, but studies have generally used a single classifier and so comparisons of the relative accuracy of classifiers for deriving thematic uncertainty measures have not been made. Also, the effects of classifier training on estimation of thematic uncertainty have not yet been examined. This paper compared three classification methods for estimating thematic uncertainty for a sand dune test site at Ainsdale, Southport, UK using data acquired by the Compact Airborne Spectrographic Imager. The classifiers used were the maximum likelihood (ML), the multi-layer perceptron (MLP) neural network and probabilistic neural network (PNN). The paper also examined the effect of varying the training of neural network classifiers on estimating thematic uncertainty by altering the number of iterations and the architecture of the MLP and the smoothing function of the PNN. The MLP and PNN with the largest proportion of correctly allocated cases, Po, had larger overall accuracies than the ML (ML Po = 0.774; MLP Po = 0.827; PNN Po = 0.827). A significant, at 99.999% confidence, one-to-one relationship between predicted and actual thematic uncertainty was found for all classifiers. This indicates that all the classifiers tested were able to estimate thematic  uncertainty. The MLP and PNN estimated thematic uncertainty with similar accuracy, but the ML was less accurate than both classifiers. However, the MLP and PNN that had the largest overall accuracy were  not the classifiers that estimated thematic uncertainty most accurately. The number of iterations used in training for the MLP was significantly correlated with accuracy of estimation of thematic uncertainty (adjusted- r 2  = 0.324,  p = 0.046).  The  smoothing function of the PNN clearly influenced the overall accuracy and the accuracy of thematic uncertainty estimation, though a significant correlation was not found when linear, log-linear and polynomial regressions were applied. The results of the study are discussed in terms of selecting the most suitable classifier for mapping land cover or predicting thematic error, as the most appropriate classifier for each task may be different.

Keywords: thematic uncertainty, classification, maximum likelihood classifier, multi-layer perceptron, probabilistic neural network

In: Caetano, M. and Painho, M. (eds). Proceedings of the 7th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, 5 – 7 July 2006, Lisboa, Instituto Geográfico Português

Brown2006accuracy.pdf1.3 MB