Accuracy assessment of High Resolution Satellite Imagery by Leave-one-out method

Maria Antonia Brovelli1, Mattia Crespi2, Francesca Fratarcangeli 2, Francesca Giannone2 and Eugenio Realini1
1 DIIAR, Politecnico di Milano, Polo Regionale di Como
via Valleggio 11 - 22100 Como, Italy
Tel.: ++39 0313327517; Fax: ++39 0313327519;
2 DITS - Area di Geodesia e Geomatica, Università di Roma “La Sapienza”
via Eudossiana, 18 - 00184 Rome, Italy
Tel.: +39 0644585068; Fax: +39 0644585515;

Interest in high-resolution satellite imagery (HRSI) is spreading in several application fields, at both scientific and commercial levels. A fundamental and critical goal for the geometric use of this kind of imagery is their orientation and orthorectification, the process able to correct the geometric deformations they undergo during acquisition. One of the main objectives of the studies about orthorectification  is the definition of an effective methodology to assess the spatial accuracy achievable from orthorectified imagery. Currently, the most used method (hold-out validation - HOV) to compute this accuracy just consists in partitioning the known ground points in two sets, the first used into the orientation-orthorectification model (GCPs – Ground Control Points) and the second to validate the model itself (CPs – Check Points); in this respect, the accuracy is just the RMSE of residuals between imagery derived coordinates with respect to CPs coordinates. However this method has some drawbacks: it is generally not reliable and it is not applicable when a low number of ground points is available. First of all, once the two sets are selected, accuracy estimate is not reliable since it is strictly dependent on the points used as CPs; if outliers or poor quality points are included in the CPs set, accuracy estimate is biased. In addition, when a low number of ground points is available, almost all of them are used as GCPs and very few CPs remain, so that RMSE may be computed on a poor (not significant) sample. In these cases accuracy assessment with the usual procedure is essentially lost. In the present work we propose an alternative to the  previously described method to perform a spatial accuracy assessment,  that is the use of the Leave-one-out cross- validation (LOOCV) method for the orientation and orthorectification of HRSI. The method consists in the iterative application of the orthorectification model using all the known ground points (or a subset of them) as GCPs except one, different in each iteration, used as CP. In every iteration the residual between imagery  derived coordinates with respect to CP coordinates (prediction error of the model on  CP coordinates) is calculated; the overall spatial accuracy achievable from the orthorectified image may be estimated by calculating the usual RMSE or, better, a robust accuracy index like the mAD (median Absolute Deviation) of the prediction errors on all the iterations. In this way we solve both mentioned drawbacks of the classical procedure: it is a reliable and robust method, not dependent on a particular set of CPs and on outliers, and it allows us to use each known ground point both as a GCP and as a CP, capitalising all the available ground information. To test this method we modified the software SISAR, developed by the Geodesy and Geomatics Team at the University of Rome “La Sapienza” to perform rigorous orientation of HRSI, integrating it with a module suited to carry out iteratively the core algorithm with point configurations  required  to  apply  the  Leave-one-out  method.  The  software  was  tested  on EROS-A1 and Quickbird imagery, confirming the good features of the Leave-one-out method. Moreover SISAR was compared to the world recognized commercial software OrthoEngine v. 10 (PCI Geomatica), which required manual iterations to realize the Leave-one-out procedure; this comparison showed the quite good performances of the SISAR rigorous model. 

Keywords: HRSI, accuracy, Leave-one-out

In: Caetano, M. and Painho, M. (eds). Proceedings of the 7th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, 5 – 7 July 2006, Lisboa, Instituto Geográfico Português

Brovelli2006accuracy.pdf665.52 KB