skip to main content
research-article

Quintuple-based Representation Learning for Bipartite Heterogeneous Networks

Published: 17 May 2024 Publication History

Abstract

Recent years have seen rapid progress in network representation learning, which removes the need for burdensome feature engineering and facilitates downstream network-based tasks. In reality, networks often exhibit heterogeneity, which means there may exist multiple types of nodes and interactions. Heterogeneous networks raise new challenges to representation learning, as the awareness of node and edge types is required. In this article, we study a basic building block of general heterogeneous networks, the heterogeneous networks with two types of nodes. Many problems can be solved by decomposing general heterogeneous networks into multiple bipartite ones. Recently, to overcome the demerits of non-metric measures used in the embedding space, metric learning-based approaches have been leveraged to tackle heterogeneous network representation learning. These approaches first generate triplets of samples, in which an anchor node, a positive counterpart, and a negative one co-exist, and then try to pull closer positive samples and push away negative ones. However, when dealing with heterogeneous networks, even the simplest two-typed ones, triplets cannot simultaneously involve both positive and negative samples from different parts of networks. To address this incompatibility of triplet-based metric learning, in this article, we propose a novel quintuple-based method for learning node representations in bipartite heterogeneous networks. Specifically, we generate quintuples that contain positive and negative samples from two different parts of networks. And we formulate two learning objectives that accommodate quintuple-based learning samples, a proximity-based loss that models the relations in quintuples by sigmoid probabilities and an angular loss that more robustly maintains similarity structures. In addition, we also parameterize feature learning by using one-dimensional convolution operators around nodes’ neighborhoods. Compared with eight methods, extensive experiments on two downstream tasks manifest the effectiveness of our approach.

References

[1]
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems, Vol. 26. Curran Associates, Inc., 1–9.
[2]
Ivan Brugere, Brian Gallagher, and Tanya Y. Berger-Wolf. 2018. Network structure inference, a survey: Motivations, methods, and applications. ACM Computing Surveys (CSUR) 51, 2 (2018), 1–39.
[3]
Hongyun Cai, Vincent W. Zheng, and Kevin Chen-Chuan Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering 30, 9 (2018), 1616–1637.
[4]
Hui Chen, Cangqi Zhou, Jing Zhang, and Qianmu Li. 2021. Heterogeneous graph embedding based on edge-aware neighborhood convolution. In 2021 International Joint Conference on Neural Networks (IJCNN’21). IEEE, 1–8.
[5]
Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. 2019. Multi-label image recognition with graph convolutional networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19). IEEE, 5177–5186.
[6]
Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu. 2018. A survey on network embedding. IEEE Transactions on Knowledge and Data Engineering 31, 5 (2018), 833–852.
[7]
Yuxiao Dong, Nitesh V. Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 135–144.
[8]
Yuxiao Dong, Ziniu Hu, Kuansan Wang, Yizhou Sun, and Jie Tang. 2020. Heterogeneous network representation learning. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI’20). 4861–4867.
[9]
Tao Yang Fu, Wang Chien Lee, and Zhen Lei. 2017. HIN2Vec: Explore meta-paths in heterogeneous information networks for representation learning. In 26th ACM International Conference on Information and Knowledge Management (CIKM’17). Association for Computing Machinery, 1797–1806.
[10]
Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. MAGNN: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of the Web Conference 2020 (WWW’20). 2331–2341.
[11]
Ming Gao, Leihui Chen, Xiangnan He, and Aoying Zhou. 2018. BiNE: Bipartite network embedding. In The 41st International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’18). 715–724.
[12]
Dario Garcia-Gasulla, Ulises Cortés, Eduard Ayguadé, and Jesús Labarta. 2015. Evaluating link prediction on large graphs. In Artificial Intelligence Research and Development. IOS Press, 90–99.
[13]
Palash Goyal and Emilio Ferrara. 2018. Graph embedding techniques, applications, and performance: A survey. Knowledge-based Systems 151 (2018), 78–94.
[14]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855–864.
[15]
Pili Hu and Wing Cheong Lau. 2013. A survey and taxonomy of graph sampling. arXiv preprint arXiv:1308.5865 (2013).
[16]
Le Hui, Xiang Li, Chen Gong, Meng Fang, Joey Tianyi Zhou, and Jian Yang. 2019. Inter-class angular loss for convolutional neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 3894–3901.
[17]
Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14). Association for Computational Linguistics.
[18]
Thomas N. Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint:1609.02907 (2016).
[19]
Rongjie Li, Songyang Zhang, Bo Wan, and Xuming He. 2021. Bipartite graph network with adaptive message passing for unbiased scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’21). 11109–11119.
[20]
Xiaocui Li, Hongzhi Yin, Ke Zhou, and Xiaofang Zhou. 2020. Semi-supervised clustering with deep metric learning and graph embedding. World Wide Web 23, 2 (2020), 781–798.
[21]
Fanzhen Liu, Shan Xue, Jia Wu, Chuan Zhou, Wenbin Hu, Cecile Paris, Surya Nepal, Jian Yang, and Philip S. Yu. 2020. Deep learning for community detection: Progress, challenges and opportunities. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI’20). 4981–4987.
[22]
Jiwen Lu, Junlin Hu, and Jie Zhou. 2017. Deep metric learning for visual understanding: An overview of recent advances. IEEE Signal Processing Magazine 34, 6 (2017), 76–84.
[23]
Xiaoxiao Ma, Jia Wu, Shan Xue, Jian Yang, Chuan Zhou, Quan Z. Sheng, Hui Xiong, and Leman Akoglu. 2023. A comprehensive survey on graph anomaly detection with deep learning. IEEE Transactions on Knowledge and Data Engineering 35, 12 (2023), 12012–12038.
[24]
Rada F. Mihalcea and Dragomir R. Radev. 2011. Graph-based Natural Language Processing and Information Retrieval (1st ed.). Cambridge University Press, USA.
[25]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NIPS’13). 3111–3119.
[26]
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 701–710.
[27]
Phu Pham and Phuc Do. 2019. W-MetaPath2Vec: The topic-driven meta-path-based model for large-scaled content-based heterogeneous information network representation learning. Expert Systems with Applications 123 (2019), 328–344.
[28]
Takaya Saito and Marc Rehmsmeier. 2015. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One 10, 3 (2015), e0118432.
[29]
Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European Semantic Web Conference. Springer, 593–607.
[30]
Chuan Shi, Binbin Hu, Wayne Xin Zhao, and Philip S. Yu. 2018. Heterogeneous information network embedding for recommendation. IEEE Transactions on Knowledge and Data Engineering 31, 2 (2018), 357–370.
[31]
Xing Su, Shan Xue, Fanzhen Liu, Jia Wu, Jian Yang, Chuan Zhou, Wenbin Hu, Cecile Paris, Surya Nepal, Di Jin, Quan Z. Sheng, and Philip S. Yu. 2021. A comprehensive survey on community detection with deep learning. arXiv preprint arXiv:2105.12584 (2021).
[32]
Zhu Sun, Jie Yang, Jie Zhang, Alessandro Bozzon, Long-Kai Huang, and Chi Xu. 2018. Recurrent knowledge graph embedding for effective recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems. 297–305.
[33]
Jian Tang, Meng Qu, and Qiaozhu Mei. 2015. PTE: Predictive text embedding through large-scale heterogeneous text networks. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1165–1174.
[34]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. 1067–1077.
[35]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
[36]
Jian Wang, Feng Zhou, Shilei Wen, Xiao Liu, and Yuanqing Lin. 2017. Deep metric learning with angular loss. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). 2593–2601.
[37]
Xiao Wang, Deyu Bo, Chuan Shi, Shaohua Fan, Yanfang Ye, and Philip S. Yu. 2020. A survey on heterogeneous graph embedding: Methods, techniques, applications and sources. arXiv preprint arXiv:2011.14867 (2020).
[38]
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous graph attention network. In The World Wide Web Conference (WWW’19). 2022–2032.
[39]
Xiao Wang, Yiding Zhang, and Chuan Shi. 2019. Hyperbolic heterogeneous information network embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 5337–5344.
[40]
Kilian Q. Weinberger and Lawrence K. Saul. 2009. Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research 10, 9 (2009), 207–244.
[41]
Carl Yang, Yuxin Xiao, Yu Zhang, Yizhou Sun, and Jiawei Han. 2022. Heterogeneous network representation learning: A unified framework with survey and benchmark. IEEE Transactions on Knowledge and Data Engineering 34, 10 (2022), 4854–4873.
[42]
Yao Yang, Haoran Chen, and Junming Shao. 2019. Triplet enhanced AutoEncoder: Model-free discriminative network embedding. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19). 5363–5369.
[43]
Xiang Yue, Zhen Wang, Jingong Huang, Srinivasan Parthasarathy, Soheil Moosavinasab, Yungui Huang, Simon M. Lin, Wen Zhang, Ping Zhang, and Huan Sun. 2020. Graph embedding on biomedical networks: Methods, applications and evaluations. Bioinformatics 36, 4 (2020), 1241–1251.
[44]
Chuxu Zhang, Ananthram Swami, and Nitesh V. Chawla. 2019. SHNE: Representation learning for semantic-associated heterogeneous networks. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining (WSDM’19). 690–698.
[45]
Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. 2018. Network representation learning: A survey. IEEE Transactions on Big Data 6, 1 (2018), 3–28.
[46]
Cangqi Zhou, Hui Chen, Jing Zhang, Qianmu Li, and Dianming Hu. 2022. AngHNE: Representation learning for bipartite heterogeneous networks with angular loss. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM’22). 1470–1478.
[47]
Cangqi Zhou, Jing Zhang, Kaisheng Gao, Qianmu Li, Dianming Hu, and Victor S. Sheng. 2022. Bipartite network embedding with symmetric neighborhood convolution. Expert Systems with Applications 198 (2022), 116757.

Cited By

View all
  • (2024)Deep Pre-Training Transformers for Scientific Paper RepresentationElectronics10.3390/electronics1311212313:11(2123)Online publication date: 29-May-2024

Index Terms

  1. Quintuple-based Representation Learning for Bipartite Heterogeneous Networks

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Intelligent Systems and Technology
    ACM Transactions on Intelligent Systems and Technology  Volume 15, Issue 3
    June 2024
    646 pages
    EISSN:2157-6912
    DOI:10.1145/3613609
    • Editor:
    • Huan Liu
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 17 May 2024
    Online AM: 26 March 2024
    Accepted: 16 December 2022
    Revised: 08 May 2022
    Received: 08 May 2022
    Published in TIST Volume 15, Issue 3

    Check for updates

    Author Tags

    1. Network representation learning
    2. heterogeneous networks
    3. quintuple sampling
    4. link prediction
    5. recommendation

    Qualifiers

    • Research-article

    Funding Sources

    • National Key Research and Development Program of China
    • National Natural Science Foundation of China

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)132
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 12 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Deep Pre-Training Transformers for Scientific Paper RepresentationElectronics10.3390/electronics1311212313:11(2123)Online publication date: 29-May-2024

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media