MSA-Net: Establishing Reliable Correspondences by Multiscale Attention Network

• Published in: IEEE transactions on image processing 2022-01, Vol.31, p.4598-4608
• Main Authors: Zheng, Linxin, Xiao, Guobao, Shi, Ziwei, Wang, Shiping, Ma, Jiayi
• Format: Article
• Language: English
• Subjects: Context; Context modeling; Data mining; Datasets; Deep learning; Feature extraction; Feature maps; Inliers (landforms); Matching; Outlier removal; Outliers (statistics); Parameters; Pose estimation; Robustness; Task analysis; wide-baseline stereo
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2022.3186535

In this paper, we propose a novel multi-scale attention based network (called MSA-Net) for feature matching problems. Current deep networks based feature matching methods suffer from limited effectiveness and robustness when applied to different scenarios, due to random distributions of outliers and insufficient information learning. To address this issue, we propose a multi-scale attention block to enhance the robustness to outliers, for improving the representational ability of the feature map. In addition, we also design a novel context channel refine block and a context spatial refine block to mine the information context with less parameters along channel and spatial dimensions, respectively. The proposed MSA-Net is able to effectively infer the probability of correspondences being inliers with less parameters. Extensive experiments on outlier removal and relative pose estimation have shown the performance improvements of our network over current state-of-the-art methods with less parameters on both outdoor and indoor datasets. Notably, our proposed network achieves an 11.7% improvement at error threshold 5° without RANSAC than the state-of-the-art method on relative pose estimation task when trained on YFCC100M dataset.