7 Accuracy assessment
Once we have produced a land cover (or other) classification from a remote sensing image, an obvious questions is “how accurate is that map?” It is important to answer this question because we want users of the map to have an appropriate amount of confidence in it. If the map is perfect, we want people to know this so they can get the maximum amount of use out of it. And if the map is no more accurate that a random assignment of classes to pixels would have been, we also want people to know that, so they don’t use it for anything (except maybe hanging it on the wall or showing it to students as an example of what not to do…).
The subject of accuracy assessment also goes beyond classifications to maps of continuous variables, such as the Earth’s surface temperature, near-surface CO2 concentration, vegetation health, or other variables come in the form of continuous rather than discrete variables. Regardless of what your map shows, you’ll want people to know how good it is, how much they can trust it. While there are similarities between assessing maps of categorical and continuous variables, the specific measures used to quantify accuracy are different between the two, so in this chapter we will treat each in turn.
Accuracy assessment for classifications
The basic principle for all accuracy assessment is to compare estimates with reality, and to quantify the difference between the two. In the context of remote sensing-based land cover classifications, the ‘estimates’ are the classes mapped for each pixel, and ‘reality’ is the actual land cover in the areas corresponding to each pixel. Given that the classification algorithm has already provided us with the ‘estimates’, the first challenge in accuracy assessment is to find data on ‘reality’. Such data are often called ‘ground-truth’ data, and typically consist of georeferenced field observations of land cover. A technique often used is to physically go into the study area with a GPS and a camera, and take georeferenced photos that in turn allow the land cover to be determined visually from each photo. Because people can visually distinguish between different kinds of land cover with great accuracy, such data can reasonably be considered to represent ‘reality’. In many cases, though, the term ‘ground truth’ oversells the accuracy of this kind of information. People may be good at distinguishing between ‘desert’ and ‘forest’ in a photo, but they are clearly less good at distinguishing between ‘high-density forest’ and ‘medium-density forest’. Especially if the difference between two classes is based on percentage cover (e.g. the difference between medium-density and high-density forest may be whether trees cover more or less than 50% of the surface area) field observations may not always lead to a perfect description of reality. Many remote sensing scientists therefore prefer the term ‘validation data’, suggesting that these data are appropriate as the basis for comparison with