Deep Feature Fusion for VHR Remote Sensing Scene Classification

• Published in: IEEE transactions on geoscience and remote sensing 2017-08, Vol.55 (8), p.4775-4784
• Main Authors: Chaib, Souleyman, Huan Liu, Yanfeng Gu, Hongxun Yao
• Format: Article
• Language: English
• Subjects: Categories; Classification; Correlation; Correlation analysis; Datasets; Discriminant correlation analysis (DCA); Feature extraction; features fusion; Frameworks; High resolution; Image classification; Image resolution; Imagery; Machine learning; Methods; Principal component analysis; Remote sensing; Resolution; Scene analysis; scene classification; Technology; unsupervised features learning; Visual system; Visualization
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2017.2700322

The rapid development of remote sensing technology allows us to get images with high and very high resolution (VHR). VHR imagery scene classification has become an important and challenging problem. In this paper, we introduce a framework for VHR scene understanding. First, the pretrained visual geometry group network (VGG-Net) model is proposed as deep feature extractors to extract informative features from the original VHR images. Second, we select the fully connected layers constructed by VGG-Net in which each layer is regarded as separated feature descriptors. And then we combine between them to construct final representation of the VHR image scenes. Third, discriminant correlation analysis (DCA) is adopted as feature fusion strategy to further refine the original features extracting from VGG-Net, which allows a more efficient fusion approach with small cost than the traditional feature fusion strategies. We apply our approach to three challenging data sets: 1) UC MERCED data set that contains 21 different areal scene categories with submeter resolution; 2) WHU-RS data set that contains 19 challenging scene categories with various resolutions; and 3) the Aerial Image data set that has a number of 10 000 images within 30 challenging scene categories with various resolutions. The experimental results demonstrate that our proposed method outperforms the state-of-the-art approaches. Using feature fusion technique achieves a higher accuracy than solely using the raw deep features. Moreover, the proposed method based on DCA fusion produces good informative features to describe the images scene with much lower dimension.