LCDB v3.0 Accuracy Assessment

The accuracy of LCDB mapping prior to LCDB v3.0 is not well known. No accuracy assessment of any second edition LCDB mapping has ever been performed. An independent accuracy assessment of the first edition of LCDB for the 1996/97 date was undertaken and any further mapping was assumed to be of a higher standard. The current standard of the LCDB, while often assumed to be at least as accurate as the 93% reported for the first edition, is not known and no investigation of change accuracy exists. This assessment is intended to provide useful statistics for reporting of map accuracy and change for LCDB v3.0 and also inform further mapping about classes that require additional targeted mapping effort.

Map Assessment

An initial accuracy assessment comprised a 9750 point random survey of overall map accuracy at LCDB3 date (2007/8). The sampling was stratified by mainland island and mapped class with 150 points per class for the North Island and 150 points per class for the South Island. The assessment was rapidly performed on screen (~ 1000 sample points per day) using all available satellite imagery. This includes imagery from the nominal dates of 1990 (Landsat 4), 1997 (SPOT 3), 2002 (Landsat 7), 2008 (SPOT5) and 2009 (SPOTMaps). Manual interpretation by an experienced operator, who was not involved with mapping process, was performed at each random location to assess accuracy. If a point is found to be incorrect the most appropriate class at that location was chosen by the operator. The results of this survey were summarised into a confusion matrix for the North Island (see attached file niacc.xlsx) and the South Island (see attached file siacc.xlsx).

Attached are North & South Island spreadsheets with the detailed results of the initial accuracy assessment. The results are very similar between the North and South islands

Mean Users Class Accuracy

North Island 89.8%

South Island 90.2 %

This average class accuracy result of approximately 90% does not account for the differing areas of each of the classes. Where the area contribution to accuracy is taken into account we obtain the overall map accuracy.

Overall Map Accuracy

North Island 96.4%

South Island 96.6%

The result is approximately 96%, a better result than that originally reported for LCDB1 (93%) even though there are twice as many classes in LCDB3 than there were in the original LCDB1 map. The reason for the difference between the mean class and overall map accuracy is the extensive classes of pasture & indigenous forest are mapped very well and they compensate for the larger proportional error in some of the smaller classes.

Change Assessment

A two-stage process similar to that outlined above is currently being performed to investigate change accuracy. Change is typically is only a few percent of the area of the overall map and as such is better investigated in a separate targeted process. The assessment comprises 11000 random points in each of the North and South Islands (10000 in areas on "no change" and 1000 and areas of "change"). The map changes are being investigated between 2001/2 and 2007/8 dates and change accuracy matrix generated.

Download  niacc.xlsx  for the North Island accuracy table

Download  siacc.xlsx  for the South Island accuracy table