Day 3 Question 2: Challenges of image fusion for data extraction

Question 2:

Image fusion has recently emerged as a major research area in remote sensing and image processing. Write an essay to examine the significance of image fusion for information extraction, to discuss the major problems and challenges of image fusion, and to suggest some possible solutions to the problems you identified.

Often times, a single remotely-sensed data source is inadequate for a given study, giving rise to the need for analysis of a mixed set of spatial data (Richards and Jai, 1999). Drawing inferences about a single pixel from multiple data sources is referred to as data fusion. Data fusion is most easily addressed when the data sources are of the same type, and can therefore be handled by the same algorithms and systems. The issue becomes increasingly complex, however, as various data types are added to the analysis.

Some solutions to data fusion issues with various data types include numerically-based quantitative methods and methods based on evidential theory and expert systems. Expert systems even allow for non-numerical data sources. These results of any of these methods, however, is greatly influenced by the accuracy of the registration of the data (Richards and Jai, 1999).

One numerical method for handling classification of mixed data sets is to stack the data and extend the pixel vectors through the spectral and non-spectral data. This is known as the stacked vector approach. One benefit of this methods is that it allows for the use of traditional classification techniques, except for those which rely on training class statistics, such as the maximum likelihood algorithm (discussed further in question 3). Alternatives to the maximum likelihood classifier which work for mixed data sets include the parallelepiped classification (Richards and Jai, 1999).

Numerical solutions like the stacked vector approach lack the ability to handle non-numerical data, yet many of the data in a typical GIS are implicitly non-numerical. Expert systems and evidentially theory methods allow for non-numerical data types, and allow these data to effect the classification or numerical labeling of pixels without being treated in explicitly numerical ways. The links between the numerical and non-numerical data are still handled quantitatively.

Richards and Jai (1999) provide an example of data fusion using different data types. The example involves a multispectral (MS) satellite image with synthetic aperture radar (SAR) data. The MS image alone is not able to distinguish built-up urban areas from areas cleared for construction. The SAR data provides structural information, but no land-cover types. A knowledge-based expert system is then constructed which uses the MS imagery to decompose the pixels into likely classes and then uses the SAR structural information to either further segment those classes, like separating woody vegetation from grassland, or deciding whether a pixel belongs in one class or another, like determining between construction areas and urban areas. These decisions are made on a pixel-by-pixel basis.

Works Cited: 
Richards, J. A., and X. Jia (19996), Remote Sensing Digital Image Analysis: An Introduction, 3rd ed., 363 pp., Springer, Berlin.