Issue 8.2 | November 2004
Information within this issue may be outdated. Click here to link to the most recent issue

Fusing Aerial Multispectral Imagery and High-Resolution Photography

by M. Bajic, Faculty, Geodesy University of Zagreb and Scientific Council, CROMAC
Center for Testing Development Training

Introduction

Aerial remote sensing is useful for spotting indicators in the survey of mine-suspected areas and minefields.1, 2, 3, 4, 5 Because the detection of landmines in the soil and under the vegetation cover is not yet practical, the minefield indicators and their electromagnetic signatures provide means for detection of the mine-suspected areas and minefields. The minefield indicators can be natural (e.g., vegetation cover) over a large spatial extent, or else artificial or man-made (e.g., trenches, bunkers, demolished objects) over a small spatial extent.1 The dimensions of artificial minefield indicators require use of multi-spectral sensors and aerial images with spatial resolution below one meter. Due to a mixture of natural and artificial minefield indicators, aerial survey should provide wide-area coverage, with very high spatial and spectral resolution. Two different approaches were applied to resolve this conflict. One approach used very high spatial resolution sensors (in the range of five to 10 cm), four wavelengths (green [G], red [R], near infrared [NIR] and thermal infrared), and image acquisition at low heights above terrain (starting with 30 m).4 Due to a large number of images, a serious problem was mosaicking and fusing the images acquired by used sensors (spatial accuracy and matching the radiometry). Another approach used three different sensors: multispectral line scanner (12 channels), synthetic aperture radar (Experimental SAR, four wavelengths) and photogrammetric camera RMK (color infrared photography [CIR], [G], [R] and [NIR]).6 Both cases used digital sensors, while aerial photography was used in one of them, but only as an auxiliary source of ground truth and contextual information.

A current paper reports preliminary results of the fusion of aerial multispectral imagery and very high-resolution photography by principal components transformation, aimed at detecting the minefield indicators that are otherwise invisible. Two parameters were used as a measure of benefit: Image Quality (IQ) and estimated National Image Interpretation Rating Scale (NIIRS) in accordance with those established by Nill and Bouzas.7 Both a multispectral line scanner (spatial resolution one m, wide area coverage three by four km, 12 channels) and CIR photography (spatial resolution three cm, area coverage rectangle with sides 450 to 500 m, 3 channels) analyzed images for the project. Analysis shows the possibility of combining these different images in an efficient manner and provides data and information about important artificial minefield indicators that are otherwise invisible. This was not foreseen in a previous study6 but will be applied in the last phase of the Space and Airborne Mined Area Reduction Tools (SMART) project as a new tool for anomaly detection.

Fusing the Aerial Multispectral Images and Very High Spatial Resolution Photography Through Principal Components Transformation

The fusion of the multispectral images and very high spatial resolution photography was made by principal components transformation and results are reported in this paper, although the intensity, hue and saturation (IHS) method was analysed as well. An overview of methods and architectures of data fusion8 and a novel approach given by Thomas and Wald9 served as general guidance in analysis. Because the goal was to improve the possibility and benefits of data fusion to detect artificial minefield indicators that are otherwise invisible, the fine analysis of spatial and spectral quality was left for the next phase of the research.

Typically, one set of the multispectral images of the surveyed area covers an area three by four km. Although the project entailed some 90 very high spatial resolution photographs available for the area, processing time and memory resources decreased the number of photographs that were actually developed. The a priori data and information about existing minefield indicators serve to guide the selection of the relevant photographs.

Figure 1: Aerial very high spatial resolution CIR photography. The photography covers a rectangular area having sides approximately 450–500 m in G, R and NIR wavelengths with spatial resolution three cm.6 The number of pixels in a line or column varies from photograph to photograph in the range of 14827 to 15090. Part of this image was used in analysis.

The considered photographs have three layers: G, R and NIR. The first processing step is to calculate principal components of the G, R and NIR layers, while only the first principal component (PC1CIR) will be utilized in the next process. Generally, only a relevant part of the PC1CIR should be registered on the geocoded multispectral image. Parts of the multispectral layers (channels) common to PC1CIR were extracted and resampled to the pixel size of the PC1CIR. The resampled layers (channels) have small pixels (3.95 cm) while the ground resolution is one m. The next phase of processing (often named sharpening) calculates the first principal component of the resampled multispectral layers (channels) and PC1CIR. The new images (fused channels) have the spatial resolution of the PC1CIR, which is 3.95 cm.

The images obtained by the above process (fused channels or layers) are designed to be a new multispectral set for further interpretation. The processing was done by the software TNTmips.10

Figure 2: This multispectral image covers an area approximately four by three km, in 12 channels, from visible to thermal infrared wavelengths, having spatial resolution one by one m and containing 3600x4800 pixels. This figure shows visible channels 5, 3 and 1 combined as a red, green and blue image.6 The inserted gray-scale image is part of PC1CIR. The enlarged part of the multispectral images and PC1CIR is shown in Figure 3.

Very High Spatial Resolution Aerial Photography

The aerial color infrared CIR photography (Figure 1, above) was acquired at the same time, at the same height above the terrain and on the same airplane as multispectral line scanner images, but the mapping modes were different. The first principal component PC1CIR was calculated using G, R and NIR layers of the CIR image and PC1CIR was used in data fusion. PC1CIR is a gray-scale image.

Figure 3:The inserted gray-scale image is part of the first principal component, PC1CIR; the background is part of the multispectral image, channels 8, 5 and 3. The gray area of PC1CIR defines the borders of the multispectral sub-images that were extracted and resampled for each channel. Resampled sub-images (they have same pixel dimensions as PC1CIR) were processed with PC1CIR by means of principal component transformation and new derived images (indeed only the first principal components) are fused images of the respective channels.

Multispectral Geocoded Images

The multispectral images were collected by line scanner in 12 channels in visible, NIR and thermal infrared wavelengths (Figure 2). They were processed by the use of the digital elevation model and thus they are ortho images.2 Combined PC1CIR and multispectral images are shown on Figure 3 as well. The current article presents results for seven selected wavelengths in Table 1.

Channel Wavelength range Description
Ch01 424–450 nm Visible blue
Ch03 522–600 nm Visible green-yellow
Ch05 626–691 nm Visible orange
Ch06 692–759 nm Red-near infrared
Ch08 897–1022 nm Middle infrared
Ch10 3,300–5,000 nm Short-wave thermal infrared
Ch12 8,000–13,000 nm Long-wave thermal infrared
Table 1: Analysed multispectral channels. Registering the aerial photography to geocoded multispectral images.

Comparison of the Original and Fused Images

Comparison of the original multispectral images (one-meter resolution) and fused multispectral images of all channels (pixels 3.95 cm) shows that data fusion provided improvement in interpretability of minefield indicators (Figure 4 and Figure 5). Two examples were selected to be presented in the current paper. The first scene consists of the asphalt road, field road, ploughed land, land covered by low vegetation and land covered by trees (Figure 4).

The landcover on Figure 4 is as follows: an asphalt road crosses the image from the upper left to the lower right corner. The land below this road is covered with several trees and bushes. On the upper right side above the asphalt road is ploughed land, while in the left upper side is land covered with low vegetation. A thin field road divides the upper part of the scene. The same classification was applied on original and on fused channels; results are presented in Figure 4b and Figure 4d. The difference is evident in favor of fused channels.

Figure 4: A first example of the comparison of the original (a, b) and fused (c, d) images. These images cover common although not the same area. Figure 5: A second example of the comparison of the original (a, b) and fused (c, d) images.
Legend:
  1. original image, one meter resolution
  2. unsupervised classification in small region of interest on original multispectral channels
  3. fused image, 3.95 cm resolution
  4. unsupervised classification in small region of interest on fused multispectral channels
Classification was Fuzzy C Means, 10 classes11

The second example contains the remnants of the demolished house, an important example of minefield indicators (Figure 5). The scene of the second example is a simple target: the remnants of the demolished house, surrounded by several trees and grassy land (Figure 5). The original images (one-meter resolution) show the target could be detected (Figure 5a) but not classified as demolished (Figure 5b). Fused images enable detection (Figure 5c) and classification (Figure 5d) of the target as a demolished house. This example demonstrates the benefit that is enabled due to proposed fusion by principal component transformation.

Improvement of the IQ and NIIRS

The fusion of multispectral images (one-meter resolution) and the derivative of the aerial photography of very high spatial resolution (resolution of the original is three cm; derivative has pixels of 3.95 cm) indicated the benefits regarding the detection and classification of artificial minefield indicators. While further development and operation of the reported method are under way in project SMART, the preliminary quantitative analysis was performed by method7 and results are shown in Figure 6 and Figure 7. The greatest increase of IQ shows fused channels 6 and 7. The increase of NIIRS is significant and is more visible for channels 1 to 8. The increase of NIIRS is lowest for channels 10 and 12 in thermal infrared wavelengths.

 
Figure 6: IQ increase obtained
by data fusion.
Figure 7: NIIRS increase obtained by data fusion.

Conclusion

The preliminary results validated the benefits of fusing the aerial multispectral imagery and very high-resolution photography using principal components transformation to detect the minefield indicators that are otherwise invisible. IQ and estimated NIIRS were used as a measure of benefit. Both a multispectral line scanner (spatial resolution one meter, wide-area coverage three by four km, 12 channels) and CIR photography (spatial resolution three cm, area coverage rectangle with sides 450 to 500 m, three channels) analyzed images for the project. Analysis shows the possibility of combining these different images in an efficient manner and providing data and information about artificial minefield indicators that are otherwise invisible.

*All figures courtesy of the author.

Acknowledgements

Research was conducted in a frame of the project EC IST SMART, 2000, SMART, EC IST-2000-25044. Discussion with Prof. L. Wald in May 2004 about different approaches to data fusion initiated the reported analysis.

Endnotes

  1. Tables of Potential Indicators and Suspect Locations, in SMART, Space and Airborne Mined Area Reduction Tools, http://www.smart.rma.ac.be/Indicators.htm, 12 March 2002.
  2. Y. Yvinec, D. Borgis, M. Acheroy, H. Suess, M. Keller, M. Bajic, E. Wolf, S. Vanhuysse, I. Bloch, Y. Yu, O. Damanet, SMART: Space and Airborne Mined Area Reduction Tools—Presentation. Proceedings EUDEM2-SCOT—2003, International conference on requirements and technologies for the detection, removal, and neutralization of landmines and UXO, 15–18 September 2003, VUB Brussel, Brussels, Belgium, pp. 595–602.
  3. R. Sapina, An Overview of Airborne-Collected Indicators of Minefield-Suspected Areas. Proceedings EUDEM2-SCOT—2003, International conference on requirements and technologies for the detection, removal, and neutralization of landmines and UXO, 15–18 September 2003, VUB Brussel, Brussels, Belgium, pp. 587–591.
  4. M.M. Eisl, M. Khalili, ARC—Airborne Minefield Area Reduction. Proceedings EUDEM2-SCOT—2003, International conference on requirements and technologies for the detection, removal, and neutralization of landmines and UXO, 15–18 September 2003, VUB Brussel, Brussels, Belgium, pp. 603–609.
  5. R. Pernar, R. Sapina, A. Marinov, C. Matic, D. Vuletic, M. Bajic, The Relevance, Strength and Likelihood of Occurrence of the Minefield Indicators and Signatures Used in the Airborne and Space-Borne Remote Sensing of Mine-Contaminated Areas. 24th EARSeL Symposium, New Strategies for European Remote Sensing, 25–27 May 2004, Dubrovnik, Croatia (Proceedings in print).
  6. Space and Airborne Mined Area Reduction Tools—SMART, EC IST—2000-25044, http://www.smart.rma.ac.be/#work.
  7. N. B. Nill, B.H. Bouzas, "Objective Image Quality Measure Derived From Digital Image Power Spectra." Optical Engineering, April 1992, Vol. 31 No. 4, pp. 813–825.
  8. FAS, 1968, National Image Interpretability Rating Scales, http://www.fas.org/irp/imint/niirs.htm, 16 January 1998.
  9. L. Wald, Data Fusion Definitions and Architectures, Fusion of Images of Different Spatial Resolutions, Les Presses de l'Ecole des Mines, Paris, France, 2002.
  10. C. Thomas, L. Wald, Assessment of the Quality of Fused Products. 24th EARSeL Symposium, New Strategies for European Remote Sensing, 25–27 May 2004, Dubrovnik, Croatia (Proceedings in print).
  11. MicroImages. TNTmips, 6.9. Issue date: 16 January 2004, MicroImages, Inc.

Contact Information

Prof. D. Sc., Milan Bajic
Faculty of Geodesy University of Zagreb
Scientific Council of CROMAC
Center for Testing Development Training
Ulica grada Vukovara 226 c
10000 Zagreb
Croatia
Tel: +385 98 460 917
E-mail: milan.bajic@zg.htnet.hr