Dual-branch spectral–spatial feature extraction network for multispectral image compression

• Published in: Multimedia systems 2023-12, Vol.29 (6), p.3579-3597
• Main Authors: Kong, Fanqiang, Tang, Jiahui, Li, Yunsong, Li, Dan, Hu, Kedi
• Format: Article
• Language: English
• Subjects: Algorithms; Coders; Coding; Computer Communication Networks; Computer Graphics; Computer Science; Cryptology; Data Storage Representation; Feature extraction; Floating point arithmetic; Image compression; Image quality; Image restoration; Integers; Machine learning; Multimedia Information Systems; Operating Systems; Performance indices; Regular Paper
• ISSN: 0942-4962; 1432-1882
• DOI: 10.1007/s00530-023-01179-7

The advent of the information age and the continuous development of spectrum imaging technologies have both triggered an explosion of multispectral image information data. Hence, image data need to be compressed first no matter transmitting or storing information. This paper designs an end-to-end compression algorithm based on spectral–spatial domain feature leaning (SSDFL). SSDFL consists of dual branches: one to obtain spectral features by Bi-ConvLSTM, and the other for spatial feature extraction via residual multiscale depth-wise convolutions (RMDCs). The attention mechanism is conductive to ulteriorly fuse features. The encoder with SSDFL serves to extract important features from multispectral images and decrease their size to 1/8 of its original size. After decreasing dimensions of images, the quantizer turns the latent floating-point characteristics into integer data. The following phase utilizes lossless entropy coding to execute coding on the integer input and estimate the bitstream. The rate-distortion is added to the training process in order to jointly optimize the image bitstream and loss. The decoder is nearly symmetrical to the encoder to restore high-quality images. Two datasets from the WorldView-3 and Setinel-2 satellites are adopted in experiment with PSNR, MS-SSIM, SS, and MSA as evaluation indexes. Corresponding results prove that the performance of the proposed approach is superior to that of traditional compression algorithms 3D-SPIHT and JPEG2000. In addition, compared with deep learning compression methods CNN-based, ConvGRU-based and Partitioned, the suggested method still outperforms at various bitrates.