skip to main content
research-article

Relation Constrained Capsule Graph Neural Networks for Non-Rigid Shape Correspondence

Published: 19 November 2024 Publication History

Abstract

Non-rigid 3D shape correspondence aims to establish dense correspondences between two non-rigidly deformed 3D shapes. However, the variability and symmetry of non-rigid shapes usually lead to mismatches due to shape deformation, topological changes, or data with severe noise. To finding an accurate correspondence between 3D dynamic shapes for the local deformation complexity, this article proposes a Relation Constrained Capsule Graph Network (RC-CGNet), which combines global and local features by encouraging the relation constraints between the embedding feature space and the input shape space based on the functional maps framework. Specifically, we design a Diffusion Graph Attention Network (DGANet) to segment the surface into parts with correct edge boundary between two regions. The Minimum Spanning Tree (MST) of geodesic curves among the singularities obtained from the segmented parts is added as relation constraints, which can compute isometric correspondences in both direct and symmetric directions. Besides that, the relation-and-attention constrained neural networks are designed to learn the shape correspondence via attention-aware CapsNet and functional maps under relation constraints. To improve the convergence speed and matching accuracy, we propose an optimized residual network structure based on the Nesterov Accelerated Gradient (NAG) to extract local features, and use graph convolution structure to extract global features. Moreover, a lightweight Gated Attention Module (GAM) is designed to fuse global and local features to obtain a richer feature representation. Since the capsule network has better spatial reasoning ability than the traditional convolutional neural network, our novel network architecture is a dual-route capsule network based on Routing Attention Fusion Block (RAFB), filtering low-discriminative capsules from a holistic view by exploiting geometric hierarchical relationships of semantic parts. Experiments on open datasets show that our method has excellent accuracy and wide adaptability.

References

[1]
Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. Scape: Shape completion and animation of people. In Proceedings of the ACM SIGGRAPH 2005 Papers, 408–416.
[2]
Mathieu Aubry, Ulrich Schlickewei, and Daniel Cremers. 2011. The wave kernel signature: A quantum mechanical approach to shape analysis. In Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV Workshops). IEEE, 1626–1633.
[3]
Florian Bernard, Zeeshan Khan Suri, and Christian Theobalt. 2020. Mina: Convex mixed-integer programming for non-rigid shape alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13826–13835.
[4]
Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3794–3801.
[5]
Davide Boscaini, Jonathan Masci, Emanuele Rodolà, and Michael Bronstein. 2016. Learning shape correspondence with anisotropic convolutional neural networks. Advances in Neural Information Processing Systems 29 (2016), 3189–3197.
[6]
Alexander M. Bronstein, Michael M. Bronstein, and Ron Kimmel. 2008. Numerical Geometry of Non-Rigid Shapes. Springer Science & Business Media. Retrieved from https://papers.nips.cc/paper_files/paper/2016/hash/228499b55310264a8ea0e27b6e7c6ab6-Abstract.html
[7]
Etienne Corman, Maks Ovsjanikov, and Antonin Chambolle. 2014. Supervised descriptor learning for non-rigid shape matching. In Proceedings of the European Conference on Computer Vision. Springer, 283–298.
[8]
Nicolas Donati, Abhishek Sharma, and Maks Ovsjanikov. 2020. Deep geometric functional maps: Robust feature learning for shape correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8592–8601.
[9]
Yangbo Feng, Junyu Gao, and Changsheng Xu. 2022. Learning dual-routing capsule graph neural network for few-shot video classification. IEEE Transactions on Multimedia 25 (2022), 3204–3216.
[10]
Vignesh Ganapathi-Subramanian, Boris Thibert, Maks Ovsjanikov, and Leonidas Guibas. 2016. Stable region correspondences between non-isometric shapes. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 121–133.
[11]
Dvir Ginzburg and Dan Raviv. 2020. Cyclic functional mapping: Self-supervised correspondence between non-isometric deformable shapes. In Proceedings of the European Conference on Computer Vision. Springer, 36–52.
[12]
Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, and Mathieu Aubry. 2018. 3D-CODED: 3D correspondences by deep deformation. In Proceedings of the European Conference on Computer Vision (ECCV), 230–246.
[13]
Yulan Guo, Ferdous Sohel, Mohammed Bennamoun, Min Lu, and Jianwei Wan. 2013. Rotational projection statistics for 3D local surface description and object recognition. International Journal of Computer Vision 105, 1 (2013), 63–86.
[14]
Oshri Halimi, Or Litany, Emanuele Rodola, Alex M. Bronstein, and Ron Kimmel. 2019. Unsupervised learning of dense shape correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4370–4379.
[15]
Ling Hu, Qinsong Li, Shengjun Liu, and Xinru Liu. 2021. Efficient deformable shape correspondence via multiscale spectral manifold wavelets preservation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14536–14545.
[16]
Ling Hu, Qinsong Li, Shengjun Liu, Dong-Ming Yan, Haojun Xu, and Xinru Liu. 2023. RFMNet: Robust deep functional maps for unsupervised non-rigid shape correspondence. Graphical Models 129 (2023), Article 101189.
[17]
Haibin Huang, Evangelos Kalogerakis, Siddhartha Chaudhuri, Duygu Ceylan, Vladimir G. Kim, and Ersin Yumer. 2017. Learning local shape descriptors from part correspondences with multiview convolutional networks. ACM Transactions on Graphics 37, 1 (2017), 1–14.
[18]
Varun Jain, Hao Zhang, and Oliver Van Kaick. 2007. Non-rigid spectral correspondence of triangle meshes. International Journal of Shape Modeling 13, 1 (2007), 101–124.
[19]
Huan Li, Yibo Yang, Dongmin Chen, and Zhouchen Lin. 2018. Optimization algorithm inspired deep neural network structure design. In Proceedings of the Asian Conference on Machine Learning. PMLR, 614–629.
[20]
Junyi Li, Siqing Li, Wayne Xin Zhao, Gaole He, Zhicheng Wei, Nicholas Jing Yuan, and Ji-Rong Wen. 2020a. Knowledge-enhanced personalized review generation with capsule graph neural network. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 735–744.
[21]
Qinsong Li, Shengjun Liu, Ling Hu, and Xinru Liu. 2020b. Shape correspondence using anisotropic Chebyshev spectral CNNs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14658–14667.
[22]
Xiang Li, Congcong Wen, Lingjing Wang, and Yi Fang. 2020c. Topology constrained shape correspondence. IEEE Transactions on Visualization and Computer Graphics 27, 10 (2020), 3926–3937.
[23]
Yang Li, Wei Zhao, Erik Cambria, Suhang Wang, and Steffen Eger. 2021. Graph routing between capsules. Neural Networks 143 (2021), 345–354.
[24]
Yuanfeng Lian and Mengqi Chen. 2023. CA-CGNet: Component-aware capsule graph neural network for non-rigid shape correspondence. Applied Sciences 13, 5 (2023), 3261.
[25]
Yaron Lipman and Thomas Funkhouser. 2009. Möbius voting for surface correspondence. ACM Transactions on Graphics 28, 3 (2009), 1–12.
[26]
Or Litany, Tal Remez, Emanuele Rodola, Alex Bronstein, and Michael Bronstein. 2017. Deep functional maps: Structured prediction for dense shape correspondence. In Proceedings of the IEEE International Conference on Computer Vision, 5659–5667.
[27]
Yi Liu, Dingwen Zhang, Qiang Zhang, and Jungong Han. 2021. Part-object relational visual saliency. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 7 (2021), 3688–3704.
[28]
Robin Magnet, Jing Ren, Olga Sorkine-Hornung, and Maks Ovsjanikov. 2022. Smooth non-rigid shape matching via effective Dirichlet energy optimization. In Proceedings of the International Conference on 3D Vision (3DV). IEEE, 495–504.
[29]
Sazan Mahbub and Md Shamsuzzoha Bayzid. 2022. EGRET: Edge aggregated graph attention networks and transfer learning improve protein–protein interaction site prediction. Briefings in Bioinformatics 23, 2 (2022), Article bbab578.
[30]
Riccardo Marin, Simone Melzi, Emanuele Rodola, and Umberto Castellani. 2020. Farm: Functional automatic registration method for 3d human bodies. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 160–173.
[31]
Haggai Maron, Meirav Galun, Noam Aigerman, Miri Trope, Nadav Dym, Ersin Yumer, Vladimir G. Kim, and Yaron Lipman. 2017. Convolutional neural networks on surfaces via seamless toric covers. ACM Transactions on Graphics 36, 4 (2017), 71–1.
[32]
Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. 2015. Geodesic convolutional neural networks on riemannian manifolds. In Proceedings of the IEEE International Conference on Computer Vision Workshops, 37–45.
[33]
Simone Melzi, Jing Ren, Emanuele Rodola, Abhishek Sharma, Peter Wonka, and Maks Ovsjanikov. 2019. Zoomout: Spectral upsampling for efficient shape correspondence. ACM Transactions on Graphics 38, 6 (2019), 1–14. DOI:
[34]
Dorian Nogneng, Simone Melzi, Emanuele Rodola, Umberto Castellani, Michael Bronstein, and Maks Ovsjanikov. 2018. Improved functional mappings via product preservation. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 179–190.
[35]
Maks Ovsjanikov, Mirela Ben-Chen, Justin Solomon, Adrian Butscher, and Leonidas Guibas. 2012. Functional maps: A flexible representation of maps between shapes. ACM Transactions on Graphics 31, 4 (2012), 1–11.
[36]
Adrien Poulenard and Maks Ovsjanikov. 2018. Multi-directional geodesic neural networks via equivariant convolution. ACM Transactions on Graphics 37, 6 (2018), 1–14.
[37]
Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 652–660.
[38]
Jing Ren, Mikhail Panine, Peter Wonka, and Maks Ovsjanikov. 2019. Structured regularization of functional map computations. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 39–53.
[39]
Jing Ren, Adrien Poulenard, Peter Wonka, and Maks Ovsjanikov. 2018. Continuous and orientation-preserving correspondences via functional maps. ACM Transactions on Graphics 37, 6 (2018), 1–16.
[40]
Emanuele Rodolà, Luca Cosmo, Michael M Bronstein, Andrea Torsello, and Daniel Cremers. 2017. Partial functional correspondence. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 222–236.
[41]
Emanuele Rodolà, Zorah Lähner, Alexander M. Bronstein, Michael M. Bronstein, and Justin Solomon. 2019. Functional maps representation on product manifolds. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 678–689.
[42]
Emanuele Rodola, Samuel Rota Bulo, Thomas Windheuser, Matthias Vestner, and Daniel Cremers. 2014. Dense non-rigid shape correspondence using random forests. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4177–4184.
[43]
Jean-Michel Roufosse, Abhishek Sharma, and Maks Ovsjanikov. 2019. Unsupervised deep learning for structured shape matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1617–1627.
[44]
Yusuf Sahillioğlu. 2018. A genetic isometric shape correspondence algorithm with adaptive sampling. ACM Transactions on Graphics 37, 5 (2018), 1–14.
[45]
Yusuf Sahillioğlu. 2020. Recent advances in shape correspondence. The Visual Computer 36, 8 (2020), 1705–1721.
[46]
Yusuf Sahillioğlu and Yücel Yemez. 2011. Coarse-to-fine combinatorial matching for dense isometric shape correspondence. Computer Graphics Forum 30, 5 (2011), 1461–1470.
[47]
Y. Sahillioǧlu and Y. Yemez. 2011. Coarse-to-fine combinatorial matching for dense isometric shape correspondence. Computer Graphics Forum 30, 5 (2011), 1461–1470. DOI:
[48]
Konstantinos Sfikas, Theoharis Theoharis, and Ioannis Pratikakis. 2012. Non-rigid 3D object retrieval using topological information guided by conformal factors. The Visual Computer 28, 9 (2012), 943–955.
[49]
Nicholas Sharp, Souhaib Attaiki, Keenan Crane, and Maks Ovsjanikov. 2022. Diffusionnet: Discretization agnostic learning on surfaces. ACM Transactions on Graphics 41, 3 (2022), 1–16.
[50]
Yi Shi, Mengchen Xu, Shuaihang Yuan, and Yi Fang. 2020. Unsupervised deep shape descriptor with point distribution learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9353–9362.
[51]
Jian Sun, Maks Ovsjanikov, and Leonidas Guibas. 2009. A concise and provably informative multi-scale signature based on heat diffusion. In Computer Graphics Forum, Vol. 28. Wiley Online Library, 1383–1392.
[52]
Federico Tombari, Samuele Salti, and Luigi Di Stefano. 2010. Unique signatures of histograms for local surface description. In Proceedings of the European Conference on Computer Vision. Springer, 356–369.
[53]
Matthias Vestner, Zorah Lähner, Amit Boyarski, Or Litany, Ron Slossberg, Tal Remez, Emanuele Rodola, Alex Bronstein, Michael Bronstein, Ron Kimmel, and Daniel Cremers. 2017. Efficient deformable shape correspondence via kernel matching. In Proceedings of the International Conference on 3D Vision (3DV). IEEE, 517–526.
[54]
Kangkan Wang, Guofeng Zhang, Huayu Zheng, and Jian Yang. 2021. Learning dense correspondences for non-rigid point clouds with two-stage regression. IEEE Transactions on Image Processing 30 (2021), 8468–8482.
[55]
Sen Wang, Yang Wang, Miao Jin, Xianfeng David Gu, and Dimitris Samaras. 2007. Conformal geometry and its applications on 3D shape matching, recognition, and stitching. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 7 (2007), 1209–1220.
[56]
Yiqun Wang, Jianwei Guo, Dong-Ming Yan, Kai Wang, and Xiaopeng Zhang. 2019. A robust local spectral descriptor for matching non-rigid shapes with incompatible shape structures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6231–6240.
[57]
Yiqun Wang, Jing Ren, Dong-Ming Yan, Jianwei Guo, Xiaopeng Zhang, and Peter Wonka. 2020. MGCN: Descriptor learning using multiscale gcns. ACM Transactions on Graphics 39, 4 (2020), 122–1.
[58]
Lingyu Wei, Qixing Huang, Duygu Ceylan, Etienne Vouga, and Hao Li. 2016. Dense human body correspondences using convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1544–1553.
[59]
Rui Xiang, Rongjie Lai, and Hongkai Zhao. 2021. A dual iterative refinement method for non-rigid shape matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15930–15939.
[60]
Jin Xie, Guoxian Dai, Fan Zhu, Edward K. Wong, and Yi Fang. 2016. Deepshape: Deep-learned shape descriptor for 3d shape retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 7 (2016), 1335–1345.
[61]
Zhang Xinyi and Lihui Chen. 2018. Capsule graph neural network. In Proceedings of the International Conference on Learning Representations.
[62]
Jinyu Yang, Peilin Zhao, Yu Rong, Chaochao Yan, Chunyuan Li, Hehuan Ma, and Junzhou Huang. 2020. Hierarchical graph capsule network. In Proceedings of the AAAI Conference on Artificial Intelligence 35, 12 (2020), 10603–1061. DOI:
[63]
Rui Yang, Wenrui Dai, Chenglin Li, Junni Zou, and Hongkai Xiong. 2020. NCGNN: Node-level capsule graph neural network. IEEE Transactions on Neural Networks and Learning Systems 35, 1 (2022), 1025–1039.
[64]
Zhangsihao Yang, Or Litany, Tolga Birdal, Srinath Sridhar, and Leonidas Guibas. 2021. Continuous geodesic convolutions for learning on 3d shapes. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 134–144.
[65]
Silvia Zuffi and Michael J. Black. 2015. The stitched puppet: A graphical model of 3d human shape and pose. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3537–3546.
[66]
Silvia Zuffi, Angjoo Kanazawa, David W. Jacobs, and Michael J. Black. 2017. 3D menagerie: Modeling the 3D shape and pose of animals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6365–6373.

Index Terms

  1. Relation Constrained Capsule Graph Neural Networks for Non-Rigid Shape Correspondence

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Intelligent Systems and Technology
      ACM Transactions on Intelligent Systems and Technology  Volume 15, Issue 6
      December 2024
      727 pages
      EISSN:2157-6912
      DOI:10.1145/3613712
      • Editor:
      • Huan Liu
      Issue’s Table of Contents

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 November 2024
      Online AM: 16 August 2024
      Accepted: 26 July 2024
      Revised: 22 June 2024
      Received: 18 December 2023
      Published in TIST Volume 15, Issue 6

      Check for updates

      Author Tags

      1. Shape correspondence
      2. attention mechanism
      3. capsule network
      4. graph convolution network

      Qualifiers

      • Research-article

      Funding Sources

      • NSFC
      • NSF
      • OAC

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 251
        Total Downloads
      • Downloads (Last 12 months)251
      • Downloads (Last 6 weeks)17
      Reflects downloads up to 12 Feb 2025

      Other Metrics

      Citations

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      Full Text

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media