How is land-cover classification accuracy measured?
Coloring every satellite pixel as forest, cropland, settlement or water is classification; its accuracy must be measured separately. An error matrix compares predictions with independent reference samples that were not used in model training. High overall agreement can still conceal frequent errors in a small but important class.
Independent reference data and sampling
1. Why must classes be defined first? The boundaries between forest, shrubland and agriculture can vary by institution, season and minimum mapping unit. A rule for a pixel containing both trees and fields should be written before validation. Vague class definitions make disagreement reflect interpretation as well as sensor error. Identify the class dictionary, acquisition date, pixel size and reference data date. An aerial photograph from several years ago cannot always serve as unquestioned truth for a newly developed site. A difference can represent real change between dates rather than failure of the classifier. 2. How should reference locations be selected? Reusing the same pixels employed to train a classification produces an overly favorable test. Choose independent locations randomly or through a stratified design that includes adequate samples from small classes. Convenient paved roads and obvious lakes alone are insufficient. Compare the mapped category at each site with reliable aerial imagery, field observation or another documented reference, and flag uncertainty in that reference itself. USGS LCMAP land-cover products have been validated against independently collected reference data. Record sampling probabilities so that regional estimates can be calculated correctly. 3. How is an error matrix read? Rows may list mapped predictions and columns reference classes, or the reverse; state the orientation explicitly. Diagonal cells show agreement and off-diagonal cells show confusion. A pixel mapped as farmland but observed as shrubland affects the two classes in different ways. The total number correct divided by the number sampled gives overall accuracy, while user's and producer's perspectives answer distinct class-specific questions. How reliable are pixels labeled forest, and how many actual forest pixels were captured? Those cannot be answered by one overall percentage.
Area and change accuracy
4. Why is overall accuracy insufficient? If most of an area is an easily classified open landscape, overall accuracy above 90 percent can coexist with poor detection of rare wetlands. Report sample counts, confused categories and uncertainty by class. A stratified sample needs appropriate weights; a raw fraction from deliberately oversampled rare classes cannot simply represent a whole country's area. Pixel counts in the finished map and area estimates adjusted through independent sampling can differ. State which one supports a policy statement. 5. How should a change map be tested? Two annual maps may each be fairly accurate while rare transitions, such as forest converted to cropland, remain uncertain. Match dates of before-and-after reference observations and avoid labeling seasonal harvest as permanent land-use conversion. USGS LCMAP provides separate validation tables for land-cover change agreement. Sample transitions as well as stable areas in sufficient numbers. Cloud-covered and otherwise unobserved pixels should not be forced into an unchanged class. A transition map adds two sets of classification uncertainty plus the challenge of interpreting real time change. 6. How should the findings be used? For claims about forest loss, farming plans or urban growth, the class dictionary, sample design and error matrix belong beside the map. A regional product may lack the spatial and temporal detail for one parcel decision. Publish data version, sample size, independence, class-specific errors and uncertainty intervals. A colorful classification does not mean 'the satellite proved it'; it supports conclusions only within the measured accuracy and the definitions used by the producer. For consequential local decisions, seek appropriate authoritative data and field confirmation.