Day 3 Question 3: Problems with image classification and optimism for the future

Question 3:

Image classification is probably the most important task for remote sensing. Yet it can be quite challenging for many applications. Over the past three decades, many classification strategies have been developed. Write an essay to discuss the status and problems of image classification. Are you optimistic about the future of automated classifiers applied to the physical and human environments, and why?

---

Introduction

One of the most useful and often employed tools for extracting land cover information from remotely sensed data is image classification (Jensen, 1996). Classification of multispectral imagery is generally performed by one of several algorithms including supervised, unsupervised, fuzzy classification, band math, or hybrid techniques. Hybrid techniques include any combination of the other techniques, each of which will be described in more detail below.

Supervised classification methods

Supervised classification techniques require the operator to identify pre-determined land-cover classes by selecting image pixels that are representative of each of those classes (Mausel et al., 1990). These representative pixel sets are referred to as training sets. Mean spectral reflectance and standard deviations of each land-cover classes is derived from the training sets. From these training set statistics, various supervised classification algorithms are able to select the most appropriate class for each pixel in the image (Richards and Jia, 1986).

One of the simplest supervised classification methods is the parallelepiped technique. A parallelepiped is an n-dimensional shape with planar surfaces. The parallelepiped algorithm defines each training class as an n-dimensional parallelepiped, where n is the number of spectral bands in the image. The dimensions of the parallelepiped are defined using a threshold specified by the operator and the standard deviations from the mean of each class. Pixels that fall within a single class parallelepiped are assigned to that class. Pixels that do not fall within a class parallelepiped or in regions where two parallelepipeds overlap are not classified. Although simple and efficient, this method has several shortcomings. When too small a threshold is specified, the class parallelepipeds will be small and many pixels will go unclassified. If too great a threshold is chosen, class parallelepipeds will overlap and pixels that fall in the overlapping regions are also not assigned to a class. This is especially problematic for highly-correlated data sets (Richards and Jia, 1986).

The spectral angle mapper (SAM) algorithm also treats each pixel as a vector in n-dimensional space. The spectral signature of each pixel is described by the angle its corresponding vector makes with the axes that define the n-dimensional space of the image. A mean vector for each class is calculated and all pixel vectors are classified based on the angle they make with each class vector (Richards and Jia, 1986). A maximum angle may be defined whereby pixels that are not within that angle of any class are not classified. This algorithm is less sensitive to illumination and albedo effects when used with calibrated reflectances.

Most supervised classification algorithms assume an equal probability of each pixel belonging to each class (Hord, 1982). One of the most popular supervised classification algorithms, the maximum likelihood classifier, is based on Bayesian probability theory (Eastman, 2000), and therefore does not assume equal probabilities for each class. Instead, this algorithm utilizes the mean measurement vector and the covariance matrix from each signature to determine a probability that each pixel in an image belongs to each class. For this reason, the maximum likelihood procedure is considered the most powerful hard classification system in many remote sensing applications, yet it is recommended only when the training sites are well-defined with large sample sizes and are relatively homogeneous (Eastman, 2000).

One special case of the maximum likelihood method is the minimum distance to means (or minimum distance) classifier. This algorithm assumes identical and symmetric class distributions, so it is able to classify pixels using the mean measurement vector of each class without calculating a covariance matrix for each (Richards and Jia, 1986).

Unsupervised classification methods

The greatest weaknesses of supervised classification methods are their sensitivity to training site definitions and their assumption of a normally distributed probability distribution function for each class (Richards and Jia, 1986). Unsupervised classification techniques, on the other hand, use clustering techniques to generate classifications by grouping pixels with similar spectral characteristics. The operator then combines and labels the spectral clusters into meaningful land cover classes (Jensen, 1996). This is not always straightforward as clusters sometimes represent mixed classes of land cover as discussed above.

The most common clustering method is the iterative isodata algorithm in which initial candidate clusters are selected and their means are allowed to migrate in the spectral domain, optimizing the classification with each iteration (Ball and Hall, 1965). In each iteration of the algorithm, every pixel is compared to each new cluster mean and assigned to the nearest cluster. The means are then recalculated and the process repeats until the pixel-to-cluster mean distances are minimized or a pre-defined number of iterations is reached. Generally, the time required for class merging and labeling after an unsupervised clustering algorithm is less than the time required to define training sets for supervised classifications (Albert, 2002).

Fuzzy classification methods

Unsupervised and supervised classification methods are considered hard classifiers in that all pixels are forced into discrete groups (Jensen, 1996). In reality, the instantaneous field of view or pixel size of a sensor records the radiation emitted or reflected from a mixture of land covers, which do not typically exhibit sharp geometric boundaries like image pixels. Accordingly, a single pixel is often a mixture of multiple surface types as land covers grade into one another (Lam, 1993; Wang, 1990a,b). Fuzzy classification methods, however, assign a set of probabilities to each pixel based on the likelihood that it belongs to each land-cover class. This information can then be used to determine more precise land-cover classes, including mixed pixel classes (Jensen, 1996). There are several types of fuzzy classification techniques, including linear spectral unmixing, mixed tuned matched filtering (MTMF), and spectral feature fitting (SFF).

Linear spectral unmixing is based on the assumption that the spectral reflectance of a pixel is a linear combination of the unique reflectance spectrum of each material present in the pixel in the proportion in which they cover the pixel area (Menke, 1984). Although this technique is often reserved for hyperspectral imagery, it has been used with multispectral imagery with limited success (Richards and Jia, 1986). Other classification schemes are usually better suited for multispectral imagery (Albert, 2002).

Generally to perform an unmixing, the reflectance spectra of each land-cover type is needed. These can be obtained in situ using a field spectrometer, or training sets can be established using the endmembers, or samples of pure cover type (Richards and Jia, 1986) found from the results of running a pixel purity index (PPI) algorithm. The PPI is performed on a minimum noise fraction (MNF) transformation of the image data (Green et al., 1988). The MNF transform is a two step transformation. First, a principal component analysis (PCA) is performed on the data to decorrelate and rescale the noise. Any band-to-band correlation in the noise is removed and the resulting noise has unit variance. Next, a second PCA transformation is done on the noise-whitened data. The results are the MNF transformed image. The PPI procedure then continually re-projects the MNF transform result onto random unit vectors for a user-specified number of iterations (typically 1×104 to 1×108 times). Pixels at the ends of these vectors are tagged with each rotation. The number of times each pixel is tagged is recorded. Purer pixels tend to be tagged more often (Boardman, 1993; Boardman et al., 1995) and, thus, the highest scoring pixels are taken as potential endmembers. The user than rotates the purest pixels in n-dimensional space to identify endmember clusters. This method is extremely time consuming, both in terms of operator time and processing time (Albert, 2002), but is extremely useful in hyperspectral applications where spectral libraries are not available for all land-cover types.

The MTMF technique is a partial unmixing method that does not require all image endmembers to be defined. The algorithm returns a percent cover image for each defined endmember as well as an infeasibility score for each to help reduce falsely classified pixels. Endmember mixtures can then be compared to their infeasibility scores and pixels that have a high mixture score and a low infeasibility score are confidently classified as that endmember (Albert, 2002).

One additional fuzzy classification technique is the SFF algorithm. This algorithm returns a scale image which is a measure of how well the spectral signature of a pixel matches each training set spectrum. The algorithm also returns a root-mean-square (RMS) error image for each training set. The RMS image can then be plotted against the individual endmember scale images and pixels with low RMS error and high scale scores for a given class can then be assigned to that class (Albert, 2002).

Band math methods

Since the reflectivity of some surfaces depend on solar insolation angles and viewing angles, the pixel vectors created using multiple bands are often better at classifying a pixel than any single spectral band. These techniques included band ratios, and normalized differences, PCA and MNF transforms, among others. A tremendous benefit of these methods is that they typically require very little operator time and are often some of the most accurate methods (e.g. Albert, 2002, Kääb et al., 2002b; Paul, 2002a, b).

Problems with classifications

One of the difficulties in performing an accurate image classification is identifying all of the environmental factors which affect the spectral reflectance of each land cover you are attempting to extract (Jensen, 1996), and how each alters the reflectance patterns. These may include soil moisture and soil type, water turbidity, plant species variations, scattered patterns of atmospheric haze or water vapor, variations in snow grain size or melt water content, plant stress level, canopy structure and biochemistry, variations in outcrop mineral content, and a variety of other factors. When one includes the variety of materials in an urban landscape, the list of factors increases in magnitude.

Selecting homogeneous and representative training sites for supervised and fuzzy classification algorithms is difficult in many cases, and the performance of the techniques relies on an accurate set of endmembers. The alternatives include spectral libraries, which are becoming more common, and endmembers created from PPI analyses. The latter can be quite costly in terms of human-input and computer processing time.

Another difficulty of image classification is that it is often difficult to gauge the accuracy of the classification. This leads to difficulty in selecting the most accurate classification algorithm, especially as some of the more complex algorithms do not necessarily produce the best results (Albert, 2002).

Optimism for the future

New digital image analysis methods for extracting land cover classifications from remotely sensed data are constantly being introduced. Bateson et al. (2000) have encorporated endmember variability in their training sets to improve spectral mixture analyses. Doucette et al. (2001) utilized a neural network to create a self-organizing road map algorithm which extracts elongated road features from multispectral imagery, while others have used edge-detection techniques. Hubert-Moy et al. (2000) present an original parametric classification method for determining land-cover type. Steele (2000) uses a combination of multiple classification techniques to derive a map of land cover. These new technologies, that continue to augment the science of classification, lend hope to the promise of more accurate automated classification systems for the future.

---

Works Cited:
 
Albert, T. H. (2002), Evaluation of remote sensing techniques for ice-area classification applied to the tropical Quelccaya Ice Cap, Peru, Polar Geography, 26, 210-226.

Ball, G. H., and D. J. Hall (1965), A Novel Method of Data Analysis and Pattern Classification, Stanford Research Institute, Menlo Park, CA.

Bateson, C. A., et al. (2000), Endmember bundles: a new approach to incorporating endmember variability into spectral mixture analysis, IEEE Transactions on Geoscience and Remote Sensing, 38, 1083-1094.

Boardman, J. W. (1993), Automated spectral unmixing of AVIRIS data using concept geometry concepts, paper presented at Proceedings from the Airborne Geosciences Workshop.

Boardman, J. W., et al. (1995), Mapping target signatures via partial unmixing of AVIRIS data, paper presented at Proceedings from the Airborne Geosciences Workshop.

Doucette, P., et al. (2001), Self-organized clustering for road extraction in classified imagery, ISPRS Journal of Photogrammetry and Remote Sensing, 22, 347-358.

Eastman, J. R. (2000), Idrisi32 Help Files, edited, Clark Labs, The Idrisi Project, Worcester, MA.

Green, A. A., et al. (1988), A transformation for ordering multispectral data in terms of image quality with implications for noise removal, IEEE Transactions on Geoscience and Remote Sensing, 26, 65-74.

Hord, R. M. (1982), Digital Image Processing of Remotely Sensed Data, , 256 pp., Academic Press, New York.

Hubert-Moy, L., et al. (2000), A comparison of parametric classification procedures of remotely sensed data applied on different landscape units, Remote Sensing of Environment, 75, 174-187.

Jensen, J. R. (1996), Introductory Digital Image Processing: A Remote Sensing Perspective, 2nd edition ed., 316 pp., Prentice Hall, New Jersey.

Kääb, A., et al. (2002), Glacier monitoring from ASTER imagery: accuracy and applications, paper presented at EARSeL Proceedings. LIS-SIG Workshop, Berne, March 11-13, 2002.

Lam, S. (1993), Fuzzy sets advance spatial decision analysis, GIS World, 6, 58-59.

Mausel, P. W., et al. (1990), Optimum band selection for supervised classification of multispectral data, Photogrammetric Engineering and Remote Sensing, 56, 55-60.

Menke, W. (1984), Geophysical Data Analysis: Discrete Inverse Theory, Harcourt Brace Jovanovich, San Diego, CA.

Paul, F. (2002), Changes in glacier area in Tyrol, Austria, between 1969 and 1992 derived from Landsat 5 Thematic Mapper and Austrian Glacier Inventory data, International Journal of Remote Sensing, 23, 787-799.

Paul, F. (2002), Combined technologies allow rapid analysis of glacier changes, EOS, Transactions, 83, 253, 260-261.

Richards, J. A., and X. Jia (19996), Remote Sensing Digital Image Analysis: An Introduction, 3rd ed., 363 pp., Springer, Berlin.

Steele, B. M. (2000), Combining multiple classifiers: an application using spatial and remotely sensed information for land cover type mapping, Remote Sensing of Environment, 74, 545-556.

Wang, F. (1990a), Fuzzy supervised classification of remote sensing images, IEEE Transactions on Geoscience and Remote Sensing, 28, 194-201.

Wang, F. (1990b), Improving remote sensing image analysis through fuzzy information representation, Photogrammetric Engineering and Remote Sensing, 56, 1163-1169.