3D-TRANS: 3D Hierarchical Transformer for Shape Correspondence Learning

• Main Authors: Huang, Hao, Yuan, Shuaihang, Wen, Congcong, Hao, Yu, Fang, Yi
• Format: Conference Proceeding
• Language: English
• Subjects: Grasping; hierarchical shape analysis; Image color analysis; K-d tree; Representation learning; Shape; Shape registration; Solid modeling; Surgery; Three-dimensional displays; Transformer
• ISSN: 2767-7745
• DOI: 10.1109/ICARA60736.2024.10552931

Motivated by the intuition that one can find correspondences between two 3D shapes in a coarse-to-fine manner, we propose 3D-TRANS, a novel learning-based model to match deformable 3D shapes hierarchically. Specifically, we first propose a new subspace feature learning approach that takes advantage of local and global geometric structures of 3D shapes. Then, we design a new hierarchical 3D transformer that adaptively learns the spatial relationships of different space partitions, which correspond to different regions of shapes at multiple scales. Inside the transformer, low-rank self-attention is utilized to extract discriminative region-aware shape descriptors, capturing both short- and long-range geometric dependencies of different shape regions at an economical computational cost. Finally, we develop a new multiple templates fusion scheme to fuse multiple deformed templates using region-aware shape descriptors to predict correspondences between input shapes and templates. We demonstrate that our approach improves over the state-of-the-art results on the difficult FAUST-inter and -intra shapes. Our model also performs well on real partial shapes from the SCAPE dataset and non-human shapes from the TOSCA dataset.