skip to main content
research-article

Decentralized Federated Recommendation with Privacy-aware Structured Client-level Graph

Published: 29 July 2024 Publication History

Abstract

Recommendation models are deployed in a variety of commercial applications to provide personalized services for users. However, most of them rely on the users’ original rating records that are often collected by a centralized server for model training, which may cause privacy issues. Recently, some centralized federated recommendation models are proposed for the protection of users’ privacy, which however requires a server for coordination in the whole process of model training. As a response, we propose a novel privacy-aware decentralized federated recommendation (DFedRec) model, which is lossless compared with the traditional model in recommendation performance and is thus more accurate than other models in this line. Specifically, we design a privacy-aware structured client-level graph for the sharing of the model parameters in the process of model training, which is a one-stone-two-bird strategy, i.e., it protects users’ privacy via some randomly sampled fake entries and reduces the communication cost by sharing the model parameters only with the related neighboring users. With the help of the privacy-aware structured client-level graph, we propose two novel collaborative training mechanisms in the setting without a server, including a batch algorithm DFedRec(b) and a stochastic one DFedRec(s), where the former requires the anonymity mechanism while the latter does not. They are both equivalent to probabilistic matrix factorization trained in a centralized server and are thus lossless. We then provide formal analysis of privacy guarantee of our methods and conduct extensive empirical studies on three public datasets with explicit feedback, which show the effectiveness of our DFedRec, i.e., it is privacy aware, communication efficient, and lossless.

References

[1]
Muhammad Ammad-ud-din, Elena Ivannikova, Suleiman A. Khan, Were Oyomno, Qiang Fu, Kuan Eeik Tan, and Adrian Flanagan. 2019. Federated collaborative filtering for privacy-preserving personalized recommendation system. Retrieved from https://arxiv.org/abs/1901.09888
[2]
Vito Walter Anelli, Yashar Deldjoo, Tommaso Di Noia, Antonio Ferrara, and Fedelucio Narducci. 2021. Federank: User controlled feedback with federated recommender systems. In Proceedings of the European Conference on Information Retrieval. 32–47.
[3]
Rodolfo Stoffel Antunes, Cristiano André da Costa, Arne Küderle, Imrana Abdullahi Yari, and Björn Eskofier. 2022. Federated learning for healthcare: Systematic review and architecture proposal. ACM Trans. Intell. Syst. Technol. 13, 4 (2022), 1–23.
[4]
Di Chai, Leye Wang, Kai Chen, and Qiang Yang. 2020. Secure federated matrix factorization. IEEE Intell. Syst. 36, 5 (2020), 11–20.
[5]
Chaochao Chen, Ziqi Liu, Peilin Zhao, Jun Zhou, and Xiaolong Li. 2018. Privacy preserving point-of-interest recommendation using decentralized matrix factorization. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18). 257–264.
[6]
Erika Duriakova, Elias Z. Tragos, Barry Smyth, Neil Hurley, Francisco J. Peña, Panagiotis Symeonidis, James Geraci, and Aonghus Lawlor. 2019. PDMFRec: A decentralised matrix factorisation with tunable user-centric privacy. In Proceedings of the 13th ACM Conference on Recommender Systems (RecSys’19). 457–461.
[7]
Yeting Guo, Fang Liu, Zhiping Cai, Hui Zeng, Li Chen, Tongqing Zhou, and Nong Xiao. 2021. PREFER: Point-of-interest recommendation with efficiency and privacy-preservation via federated edge learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1, 1–25.
[8]
Jialiang Han, Yun Ma, Qiaozhu Mei, and Xuanzhe Liu. 2021. DeepRec: On-device deep learning for privacy-preserving sequential recommendation in mobile commerce. In Proceedings of the 30th International Conference on World Wide Web (WWW’21). 900–911.
[9]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web (WWW’17). 173–182.
[10]
István Hegedüs, Gábor Danner, and Márk Jelasity. 2019. Decentralized recommendation based on matrix factorization: A comparison of gossip and federated learning. In Proceedings of the Workshop on Decentralized Machine Learning at the Edge with ECML PKDD. 317–332.
[11]
Pengqing Hu, Zhaohao Lin, Weike Pan, Qiang Yang, Xiaogang Peng, and Zhong Ming. 2023. Privacy-preserving graph convolution network for federated item recommendation. Artific. Intell. 324 (2023), 103996.
[12]
Mubashir Imran, Hongzhi Yin, Tong Chen, Quoc Viet Hung Nguyen, Alexander Zhou, and Kai Zheng. 2023. ReFRS: Resource-efficient federated recommender system for dynamic and diversified user preferences. ACM Trans. Info. Syst. 41, 3 (2023), 1–30.
[13]
Amir Jalalirad, Marco Scavuzzo, Catalin Capota, and Michael Sprague. 2019. A simple and efficient federated recommender system. In Proceedings of the 6th IEEE/ACM International Conference on Big Data Computing, Applications and Technologies. 53–58.
[14]
Farwa K. Khan, Adrian Flanagan, Kuan Eeik Tan, Zareen Alamgir, and Muhammad Ammad-Ud-Din. 2021. A payload optimization method for federated recommender systems. In Proceedings of the 15th ACM Conference on Recommender Systems (RecSys’21). 432–442.
[15]
Eugene Kharitonov. 2019. Federated online learning to rank with evolution strategies. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 249–257.
[16]
Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. Retrieved from https://arxiv.org/abs/1610.05492
[17]
Yehuda Koren. 2008. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’08). 426–434.
[18]
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. 2019. On the convergence of FedAvg on non-IID data. Retrieved from https://arxiv.org/abs/1907.02189
[19]
Feng Liang, Weike Pan, and Zhong Ming. 2021. FedRec++: Lossless federated recommendation with explicit feedback. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI’21). 4224–4231.
[20]
Feng Liang, Enyue Yang, Weike Pan, Qiang Yang, and Zhong Ming. 2022. A survey of recommender systems based on federated learning (in Chinese). Scientia Sinica Informationis (2022), 52(5):1–29.
[21]
Guanyu Lin, Feng Liang, Weike Pan, and Zhong Ming. 2021. FedRec: Federated recommendation with explicit feedback. IEEE Intell. Syst. (2021), 36(5):21–30.
[22]
Zhaohao Lin, Weike Pan, Qiang Yang, and Zhong Ming. 2022. A generic federated recommendation framework via fake marks and secret sharing. ACM Trans. Info. Syst. 41, 2 (2022), 1–37.
[23]
Yehida Lindell. 2005. Secure multiparty computation for privacy preserving data mining. In Encyclopedia of Data Warehousing and Mining. IGI Global, 1005–1009.
[24]
Shuchang Liu, Shuyuan Xu, Wenhui Yu, Zuohui Fu, Yongfeng Zhang, and Amelie Marian. 2021. FedCT: Federated collaborative transfer for recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’21). 716–725.
[25]
Xiaofeng Lu, Yuying Liao, Pietro Liò, and Pan Hui. 2020. Privacy-preserving asynchronous federated learning mechanism for edge network computing. IEEE Access 8 (2020), 48970–48981.
[26]
Zhifeng Luo, Shuhong Chen, and Yutian Li. 2013. A distributed anonymization scheme for privacy-preserving recommendation systems. In Proceedings of the IEEE 4th International Conference on Software Engineering and Service Science (ICSESS’13). 491–494.
[27]
Lorenzo Minto, Moritz Haller, Benjamin Livshits, and Hamed Haddadi. 2021. Stronger privacy for federated collaborative filtering with implicit feedback. In Proceedings of the 15th ACM Conference on Recommender Systems (RecSys’21). 342–350.
[28]
Andriy Mnih and Ruslan R. Salakhutdinov. 2007. Probabilistic matrix factorization. In Proceedings of the 21st Conference on Neural Information Processing Systems (NeurIPS’07). 1257–1264.
[29]
Khalil Muhammad, Qinqin Wang, Diarmuid O’Reilly-Morgan, Elias Tragos, Barry Smyth, Neil Hurley, James Geraci, and Aonghus Lawlor. 2020. FedFast: Going beyond average for faster training of federated recommender systems. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’20). 1234–1242.
[30]
Liang Qu, Ningzhi Tang, Ruiqi Zheng, Quoc Viet Hung Nguyen, Zi Huang, Yuhui Shi, and Hongzhi Yin. 2023. Semi-decentralized federated ego graph learning for recommendation. Retrieved from https://arXiv:2302.10900
[31]
Juan E. Rubio, Cristina Alcaraz, and Javier Lopez. 2017. Recommender system for privacy-preserving solutions in smart metering. Pervas. Mobile Comput. 41 (2017), 205–218.
[32]
Yichen Wan, Youyang Qu, Longxiang Gao, and Yong Xiang. 2022. Privacy-preserving blockchain-enabled federated learning for B5G-driven edge computing. Comput. Netw. 204 (2022), 108671.
[33]
Chuhan Wu, Fangzhao Wu, Yang Cao, Yongfeng Huang, and Xing Xie. 2021. FedGNN: Federated graph neural network for privacy-preserving recommendation. Retrieved from https://arxiv.org/abs/2102.04925
[34]
Chuhan Wu, Fangzhao Wu, Lingjuan Lyu, Yongfeng Huang, and Xing Xie. 2022. Communication-efficient federated learning via knowledge distillation. Nature Commun. 13, 1 (2022), 1–8.
[35]
Cong Xie, Sanmi Koyejo, and Indranil Gupta. 2019. Asynchronous federated optimization. Retrieved from https://arxiv.org/abs/1903.03934
[36]
Liu Yang, Ben Tan, Vincent W. Zheng, Kai Chen, and Qiang Yang. 2020. Federated recommendation systems. In Federated Learning. Springer, 225–239.
[37]
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol. 10, 2 (2019), 12:1–12:19.
[38]
Wei Yuan, Quoc Viet Hung Nguyen, Tieke He, Liang Chen, and Hongzhi Yin. 2023. Manipulating federated recommender systems: Poisoning with synthetic users and its countermeasures. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’23). 1690–1699. DOI:
[39]
Wei Yuan, Chaoqun Yang, Quoc Viet Hung Nguyen, Lizhen Cui, Tieke He, and Hongzhi Yin. 2023. Interaction-level membership inference attack against federated recommender systems. In Proceedings of the Conference on the World Wide Web(WWW ’23). New York, NY, 1053–1062. DOI:
[40]
Wei Yuan, Hongzhi Yin, Fangzhao Wu, Shijie Zhang, Tieke He, and Hao Wang. 2023. Federated unlearning for on-device recommendation. In Proceedings of the 16th ACM International Conference on Web Search and Data Mining (WSDM’23). 393–401.
[41]
Chunxu Zhang, Guodong Long, Tianyi Zhou, Peng Yan, Zijian Zhang, Chengqi Zhang, and Bo Yang. 2023. Dual personalization on federated recommendation. Retrieved from https://arXiv:2301.08143
[42]
Honglei Zhang, Fangyuan Luo, Jun Wu, Xiangnan He, and Yidong Li. 2023. LightFR: Lightweight federated recommendation with privacy-preserving matrix factorization. ACM Trans. Info. Syst. 41, 4 (2023), 1–28.
[43]
Shijie Zhang, Wei Yuan, and Hongzhi Yin. 2023. Comprehensive privacy analysis on federated recommender system against attribute inference attacks. IEEE Trans. Knowl. Data Eng. 36, 3 (2023), 987–999. https://www.computer.org/csdl/journal/tk/5555/01/10184063/1OM08WAv6k8
[44]
Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. Adv. Neural Info. Process. Syst. 32 (2019).

Cited By

View all
  • (2024)Distributed Recommendation Systems: Survey and Research DirectionsACM Transactions on Information Systems10.1145/369478343:1(1-38)Online publication date: 6-Sep-2024
  • (2024)Horizontal Federated Recommender System: A SurveyACM Computing Surveys10.1145/365616556:9(1-42)Online publication date: 8-May-2024
  • (2024)Subgraph-level federated graph neural network for privacy-preserving recommendation with meta-learningNeural Networks10.1016/j.neunet.2024.106574179(106574)Online publication date: Nov-2024

Index Terms

  1. Decentralized Federated Recommendation with Privacy-aware Structured Client-level Graph

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Intelligent Systems and Technology
    ACM Transactions on Intelligent Systems and Technology  Volume 15, Issue 4
    August 2024
    563 pages
    EISSN:2157-6912
    DOI:10.1145/3613644
    • Editor:
    • Huan Liu
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 29 July 2024
    Online AM: 22 January 2024
    Accepted: 03 January 2024
    Revised: 16 November 2023
    Received: 30 June 2023
    Published in TIST Volume 15, Issue 4

    Check for updates

    Author Tags

    1. Federated recommendation
    2. Matrix factorization
    3. Graph machine learning
    4. Explicit feedback

    Qualifiers

    • Research-article

    Funding Sources

    • National Natural Science Foundation of China
    • National Key Research and Development Program of China

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)416
    • Downloads (Last 6 weeks)20
    Reflects downloads up to 12 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Distributed Recommendation Systems: Survey and Research DirectionsACM Transactions on Information Systems10.1145/369478343:1(1-38)Online publication date: 6-Sep-2024
    • (2024)Horizontal Federated Recommender System: A SurveyACM Computing Surveys10.1145/365616556:9(1-42)Online publication date: 8-May-2024
    • (2024)Subgraph-level federated graph neural network for privacy-preserving recommendation with meta-learningNeural Networks10.1016/j.neunet.2024.106574179(106574)Online publication date: Nov-2024

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media