Real-time multi-spectral image fusion

• Published in: Concurrency and computation 2001-10, Vol.13 (12), p.1063-1081
• Main Authors: Achalakul, Tiranee, Taylor, Stephen
• Format: Article
• Language: English
• Subjects: multi-spectral camera; performance prediction; principal component transform; real-time image fusion
• ISSN: 1532-0626; 1532-0634
• DOI: 10.1002/cpe.591

This paper describes a novel real‐time multi‐spectral imaging capability for surveillance applications. The capability combines a new high‐performance multi‐spectral camera system with a distributed algorithm that computes a spectral‐screening principal component transform (PCT). The camera system uses a novel filter wheel design together with a high‐bandwidth CCD camera to allow image cubes to be delivered at 110 frames $^{-1}$s with a spectral coverage between 400 and 1000 nm. The filters used in a particular application are selected to highlight a particular object based on its spectral signature. The distributed algorithm allows image streams from a dispersed collection of cameras to be disseminated, viewed, and interpreted by a distributed group of analysts in real‐time. It operates on networks of commercial‐off‐the‐shelf multiprocessors connected with high‐performance (e.g. gigabit) networking, taking advantage of multi‐threading where appropriate. The algorithm uses a concurrent formulation of the PCT to de‐correlate and compress a multi‐spectral image cube. Spectral screening is used to give features that occur infrequently (e.g. mechanized vehicles in a forest) equal importance to those that occur frequently (e.g. trees in the forest). A human‐centered color‐mapping scheme is used to maximize the impact of spectral contrast on the human visual system. To demonstrate the efficacy of the multi‐spectral system, plant‐life scenes with both real and artificial foliage are used. These scenes demonstrate the systems ability to distinguish elements of a scene that cannot be distinguished with the naked eye. The capability is evaluated in terms of visual performance, scalability, and real‐time throughput. Our previous work on predictive analytical modeling is extended to answer practical design questions such as ‘For a specified cost, what system can be constructed and what performance will it attain?’ Copyright © 2001 John Wiley & Sons, Ltd.