Kernel-based feature aggregation framework in point cloud networks

• Published in: Pattern recognition 2023-07, Vol.139, p.109439, Article 109439
• Main Authors: Zhang, Jianjia, Zhang, Zhenxi, Wang, Lei, Zhou, Luping, Zhang, Xiaocai, Liu, Mengting, Wu, Weiwen
• Format: Article
• Language: English
• Subjects: Deep learning; Feature aggregation; Kernel; Point cloud; Pooling
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2023.109439

•We are among the first ones to investigate the drawbacks of maxpooling in deep point cloud networks. And based on our insightful analysis, a kernel-based feature aggregation framework is innovatively proposed for orderless point analysis via deep networks. The framework enables modeling nonlinear feature relationships which are complementary to max-pooling in a flexible manner.•The order-invariance property of max-pooling can be naturally kept by simply choosing an order-invariant kernel function, e.g., the commonly used RBF kernel or Polynomial kernel. At the same time, the high efficiency of max-pooling is also maintained and only very minor additional computational cost is introduced, which will be demonstrated later through theoretical analysis and experimental study.•The superiority and generability of the proposed method have been extensively verified in both supervised and unsupervised point cloud analysis tasks with various representative backbone networks. Specifically, the supervised tasks include 3D object classification, part segmentation, scene segmentation while the unsupervised task refers to place retrieval. The backbone networks involved in the evaluation include PointNet, DGCNN and PCT. Various effective deep networks have been developed for analysis of 3D point clouds. One key step in these networks is to aggregate the features of orderless points into a compact representation for the cloud. As a typical order-invariant aggregation method, max-pooling has been widely applied. However, while enjoying simplicity and high efficiency, max-pooling does not fully exploit the feature information since it not only ignores the non-maximum elements in each feature dimension but also neglects the interactions between different dimensions. These drawbacks of max-pooling motivate us to explore advanced feature aggregation methods for 3D point cloud analysis. The desired advanced method should be capable of modeling richer information between the point features than max-pooling, and, at the same time, it can readily replace max-pooling without the need to modify other parts of the existing network architecture. To this end, this paper proposes a novel kernel-based feature aggregation framework for 3D point cloud analysis for the first time. The proposed method effectively considers all the elements in each dimension and models the nonlinear interactions between feature dimensions as complementary information to max-pooling. In addition, it is a plug-