skip to main content
research-article

DNSRF: Deep Network-based Semi-NMF Representation Framework

Published: 07 November 2024 Publication History

Abstract

Representation learning is an important topic in machine learning, pattern recognition, and data mining research. Among many representation learning approaches, semi-nonnegative matrix factorization (SNMF) is a frequently-used one. However, a typical problem of SNMF is that usually there is no learning rate guidance during the optimization process, which often leads to a poor representation ability. To overcome this limitation, we propose a very general representation learning framework (DNSRF) that is based on a deep neural net. Essentially, the parameters of the deep net used to construct the DNSRF algorithms are obtained by matrix element update. In combination with different activation functions, DNSRF can be implemented in various ways. In our experiments, we tested nine instances of our DNSRF framework on six benchmark datasets. In comparison with other state-of-the-art methods, the results demonstrate the superior performance of our framework, which is thus shown to have a great representation ability.

References

[1]
Hervé Abdi and Lynne J. Williams. 2010. Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics 2, 4 (2010), 433–459.
[2]
Kais Allab, Lazhar Labiod, and Mohamed Nadif. 2016. A semi-NMF-PCA unified framework for data clustering. IEEE Transactions on Knowledge and Data Engineering 29, 1 (2016), 2–16.
[3]
Ossama S. Alshabrawy, Mohamed Elsayed Ghoneim, Wael Awad, and Aboul Ella Hassanien. 2012. Underdetermined blind source separation based on fuzzy c-means and semi-nonnegative matrix factorization. In Proceedings of the Federated Conference on Computer Science and Information Systems (FedCSIS ’12). IEEE, 695–700.
[4]
Kirk Baker. 2005. Singular Value Decomposition Tutorial, Vol. 24. The Ohio State University.
[5]
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. 2018. Relational inductive biases, deep learning, and graph networks. arXiv:1806.01261. Retrieved from
[6]
Mikhail Belkin and Partha Niyogi. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15, 6 (2003), 1373–1396.
[7]
Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 8 (2013), 1798–1828.
[8]
Deng Cai, Xiaofei He, Jiawei Han, and Thomas S. Huang. 2010. Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 8 (2010), 1548–1560.
[9]
Wei Chen, Hongjun Wang, Zhiguo Long, and Tianrui Li. 2022. Fast flexible bipartite graph model for co-clustering. IEEE Transactions on Knowledge and Data Engineering 35, 7 (2022), 6930–6940.
[10]
Yong Chen, Hui Zhang, Xiaopeng Zhang, and Rui Liu. 2017. Regularized semi-non-negative matrix factorization for hashing. IEEE Transactions on Multimedia 20, 7 (2017), 1823–1836.
[11]
Meriem Chouh and Kamel Boukhetala. 2016. Semi-nonnegative matrix factorization algorithm based on genetic algorithm initialization. International Journal of Machine Learning and Computing 6, 4 (2016), 231–234.
[12]
Ping Deng, Tianrui Li, Dexian Wang, Hongjun Wang, Hong Peng, and Shi-Jinn Horng. 2023. Multi-view clustering guided by unconstrained non-negative matrix factorization. Knowledge-Based Systems 266 (2023), 110425.
[13]
Ping Deng, Tianrui Li, Hongjun Wang, Dexian Wang, Shi-Jinn Horng, and Rui Liu. 2022. Graph regularized sparse non-negative matrix factorization for clustering. IEEE Transactions on Computational Social Systems 10, 3 (2022), 910–921.
[14]
Ping Deng, Hongjun Wang, Tianrui Li, Shi-Jinn Horng, and Xinwen Zhu. 2019. Linear discriminant analysis guided by unsupervised ensemble learning. Information Sciences 480 (2019), 211–221.
[15]
Chris Ding, Xiaofeng He, and Horst D. Simon. 2005. On the equivalence of nonnegative matrix factorization and spectral clustering. In Proceedings of the SIAM International Conference on Data Mining (SDM ’05). 606–610.
[16]
Chris H. Q. Ding, Tao Li, and Michael I. Jordan. 2008. Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 1 (2008), 45–55.
[17]
Zhiqiang Fu, Yao Zhao, Dongxia Chang, and Yiming Wang. 2021. A hierarchical weighted low-rank representation for image clustering and classification. Pattern Recognition 112 (2021), 107736.
[18]
Ziyu Guan, Lijun Zhang, Jinye Peng, and Jianping Fan. 2015. Multi-view concept learning for data representation. IEEE Transactions on Knowledge and Data Engineering 27, 11 (2015), 3016–3028.
[19]
Shudong Huang, Zenglin Xu, Zhao Kang, and Yazhou Ren. 2020. Regularized nonnegative matrix factorization with adaptive local structure learning. Neurocomputing 382 (2020), 196–209.
[20]
Yanyong Huang, Zongxin Shen, Yuxin Cai, Xiuwen Yi, Dongjie Wang, Fengmao Lv, and Tianrui Li. 2023. C2IMUFS: Complementary and consensus learning-based incomplete multi-view unsupervised feature selection. IEEE Transactions on Knowledge and Data Engineering 35, 10 (2023), 10681–10694.
[21]
Aapo Hyvärinen and Erkki Oja. 2000. Independent component analysis: Algorithms and applications. Neural Networks 13, 4 (2000), 411–430.
[22]
William Kahan. 2013. A Tutorial Overview of Vector and Matrix Norms. University of California, Berkeley, CA, Lecture Notes, 19.
[23]
Serhat Kiliçarslan and Mete Celik. 2021. RSigELU: A nonlinear activation function for deep neural networks. Expert Systems with Applications 174 (2021), 114805.
[24]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436–444.
[25]
Daniel D. Lee and H. Sebastian Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788–791.
[26]
Zhen Li, Jian Chen, Ying Fu, Guyu Hu, Zhisong Pan, and Liangliang Zhang. 2018. Community detection based on regularized semi-nonnegative matrix tri-factorization in signed networks. Mobile Networks and Applications 23, 1 (2018), 71–79.
[27]
Zechao Li, Jing Liu, Jinhui Tang, and Hanqing Lu. 2015. robust structured subspace learning for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 10 (2015), 2085–2098.
[28]
Hao Ma, Haixuan Yang, Irwin King, and Michael R Lyu. 2009. Semi-nonnegative matrix factorization with global statistical consistency for collaborative filtering. In Proceedings of the 18th ACM Conference on Information and Knowledge Management. 767–776.
[29]
Yang Meng, Ronghua Shang, Licheng Jiao, Wenya Zhang, and Shuyuan Yang. 2018. Dual-graph regularized non-negative matrix factorization with sparse and orthogonal constraints. Engineering Applications of Artificial Intelligence 69 (2018), 24–35.
[30]
Quanyi Mo and Bruce A. Draper. 2012. Semi-nonnegative matrix factorization for motion segmentation with missing data. In Proceedings of the European Conference on Computer Vision. Springer, 402–415.
[31]
Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. 2014. In search of the real inductive bias: On the role of implicit regularization in deep learning. arXiv:1412.6614. Retrieved from
[32]
Rong Pang, Yan Yang, Aiguo Huang, Yan Liu, Peng Zhang, and Guangwu Tang. 2023. Multi-scale feature fusion model for bridge appearance defects detection. Big Data Mining and Analytics 7, 1 (2023), 1–11.
[33]
Chong Peng, Zhilu Zhang, Chenglizhao Chen, Zhao Kang, and Qiang Cheng. 2022b. Two-dimensional semi-nonnegative matrix factorization for clustering. Information Sciences 590 (2022), 106–141.
[34]
Siyuan Peng, Wee Ser, Badong Chen, and Zhiping Lin. 2021. Robust semi-supervised nonnegative matrix factorization for image clustering. Pattern Recognition 111 (2021), 107683.
[35]
Siyuan Peng, Zhijing Yang, Feiping Nie, Badong Chen, and Zhiping Lin. 2022a. Correntropy based semi-supervised concept factorization with adaptive neighbors for clustering. Neural Networks 154 (2022), 203–217.
[36]
Anyong Qin, Zhuolin Tan, Xingli Tan, Yongji Wu, Cheng Jing, and Yuan Yan Tang. 2023. Distribution preserving-based deep semi-NMF for data representation. Neurocomputing 524 (2023), 69–83.
[37]
Akhand Rai and Sanjay H. Upadhyay. 2019. The application of semi-nonnegative matrix factorization for detection of incipient faults in bearings. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 233, 13 (2019), 4543–4555.
[38]
Florian Rousset, Françoise Peyrin, and Nicolas Ducros. 2018. A semi nonnegative matrix factorization technique for pattern generalization in single-pixel imaging. IEEE Transactions on Computational Imaging 4, 2 (2018), 284–294.
[39]
David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. 1986. Learning representations by back-propagating errors. Nature 323, 6088 (1986), 533–536.
[40]
Aghiles Salah, Melissa Ailem, and Mohamed Nadif. 2017. A way to boost semi-NMF for document clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 2275–2278.
[41]
Zhenqiu Shu, Yanwu Sun, Jiali Tang, and Congzhe You. 2022. Adaptive graph regularized deep semi-nonnegative matrix factorization for data representation. Neural Processing Letters 54, 6 (2022), 5721–5739.
[42]
Zhen-qiu Shu, Xiao-jun Wu, Cong Hu, Cong-zhe You, and Hong-hui Fan. 2021. Deep semi-nonnegative matrix factorization with elastic preserving for data representation. Multimedia Tools and Applications 80, 2 (2021), 1707–1724.
[43]
Meiping Song, Tingting Yang, Hongju Cao, Fang Li, Bai Xue, Sen Li, and Chein-I. Chang. 2022. Bi-endmember semi-NMF based on low-rank and sparse matrix decomposition. IEEE Transactions on Geoscience and Remote Sensing 60 (2022), 1–16.
[44]
Alaa Tharwat, Tarek Gaber, Abdelhameed Ibrahim, and Aboul Ella Hassanien. 2017. Linear discriminant analysis: A detailed tutorial. AI Communications 30, 2 (2017), 169–190.
[45]
George Trigeorgis, Konstantinos Bousmalis, Stefanos Zafeiriou, and Bjoern Schuller. 2014. A deep semi-NMF model for learning hidden representations. In Proceedings of the International Conference on Machine Learning. PMLR, 1692–1700.
[46]
Dexian Wang, Tianrui Li, Ping Deng, Jia Liu, Wei Huang, and Fan Zhang. 2022a. A generalized deep learning algorithm based on NMF for multi-view clustering. IEEE Transactions on Big Data 9, 1 (2022), 328–340.
[47]
Dexian Wang, Tianrui Li, Ping Deng, Hongjun Wang, and Pengfei Zhang. 2022b. Dual graph-regularized sparse concept factorization for clustering. Information Sciences 607 (2022), 1074–1088.
[48]
Dexian Wang, Tianrui Li, Ping Deng, Fan Zhang, Wei Huang, Pengfei Zhang, and Jia Liu. 2023a. A generalized deep learning clustering algorithm based on non-negative matrix factorization. ACM Transactions on Knowledge Discovery from Data 17, 7 (2023), 1–20.
[49]
Dexian Wang, Tianrui Li, Wei Huang, Zhipeng Luo, Ping Deng, Pengfei Zhang, and Minbo Ma. 2023b. A multi-view clustering algorithm based on deep semi-NMF. Information Fusion 99 (2023), 101884.
[50]
Jing Wang, Feng Tian, Hongchuan Yu, Chang Hong Liu, Kun Zhan, and Xiao Wang. 2018. Diverse non-negative matrix factorization for multiview data representation. IEEE Transactions on Cybernetics 48, 9 (2018), 2620–2632.
[51]
Wei Wang, Feiyu Chen, Yongxin Ge, Sheng Huang, Xiaohong Zhang, and Dan Yang. 2021. Discriminative deep semi-nonnegative matrix factorization network with similarity maximization for unsupervised feature learning. Pattern Recognition Letters 149 (2021), 157–163.
[52]
Zhongliang Wang, Mi He, Ling Wang, Ke Xu, Jingjing Xiao, and Yongjian Nian. 2020. Semi-NMF-based reconstruction for hyperspectral compressed sensing. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 13 (2020), 4352–4368.
[53]
Wei Xu and Yihong Gong. 2004. Document clustering by concept factorization. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 202–209.
[54]
Yesong Xu, Shuo Chen, Jun Li, Zongyan Han, and Jian Yang. 2020. Autoencoder-based latent block-diagonal representation for subspace clustering. IEEE Transactions on Cybernetics 52, 6 (2020), 5408–5418.
[55]
Yesong Xu, Shuo Chen, Jun Li, Chunyan Xu, and Jian Yang. 2023. Fast subspace clustering by learning projective block diagonal representation. Pattern Recognition 135 (2023), 109152.
[56]
Fanghua Ye, Chuan Chen, and Zibin Zheng. 2018. Deep autoencoder-like nonnegative matrix factorization for community detection. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 1393–1402.
[57]
Qiaolin Ye, Zechao Li, Liyong Fu, Zhao Zhang, Wankou Yang, and Guowei Yang. 2019. Nonpeaked discriminant analysis for data representation. IEEE Transactions on Neural Networks and Learning Systems 30, 12 (2019), 3818–3832.
[58]
Jingxing Yin, Siyuan Peng, Zhijing Yang, Badong Chen, and Zhiping Lin. 2023. Hypergraph based semi-supervised symmetric nonnegative matrix factorization for image clustering. Pattern Recognition 137 (2023), 109274.
[59]
Naoto Yokoya, Jocelyn Chanussot, and Akira Iwasaki. 2013. Nonlinear unmixing of hyperspectral data using semi-nonnegative matrix factorization. IEEE Transactions on Geoscience and Remote Sensing 52, 2 (2013), 1430–1437.
[60]
Zhong Yuan, Hongmei Chen, Pengfei Zhang, Jihong Wan, and Tianrui Li. 2021. A novel unsupervised approach to heterogeneous feature selection based on fuzzy mutual information. IEEE Transactions on Fuzzy Systems 30, 9 (2021), 3395–3409.
[61]
Xianhua Zeng, Shengwei Qu, and Zhilong Wu. 2016. Graph regularized deep semi-nonnegative matrix factorization for clustering. In Proceedings of the 8th International Conference on Digital Image Processing (ICDIP ’16), Vol. 10033. 1126–1130.
[62]
Ji Zhang, Jingkuan Song, Lianli Gao, Ye Liu, and Heng Tao Shen. 2022. Progressive meta-learning with curriculum. IEEE Transactions on Circuits and Systems for Video Technology 32, 9 (2022), 5916–5930.
[63]
Pengfei Zhang, Tianrui Li, Guoqiang Wang, Chuan Luo, Hongmei Chen, Junbo Zhang, Dexian Wang, and Zeng Yu. 2021a. Multi-source information fusion based on rough set theory: A review. Information Fusion 68 (2021), 85–117.
[64]
Pengfei Zhang, Tianrui Li, Guoqiang Wang, Dexian Wang, Pei Lai, and Fan Zhang. 2023. A multi-source information fusion model for outlier detection. Information Fusion 93 (2023), 192–208.
[65]
Yan Zhang, Zhao Zhang, Yang Wang, Zheng Zhang, Li Zhang, Shuicheng Yan, and Meng Wang. 2021b. Dual-constrained deep semi-supervised coupled factorization network with enriched prior. International Journal of Computer Vision 129 (2021), 3233–3254.
[66]
Zhao Zhang, Yan Zhang, Guangcan Liu, Jinhui Tang, Shuicheng Yan, and Meng Wang. 2019. Joint label prediction based semi-supervised adaptive concept factorization for robust data representation. IEEE Transactions on Knowledge and Data Engineering 32, 5 (2019), 952–970.
[67]
Zhao Zhang, Yan Zhang, Mingliang Xu, Li Zhang, Yi Yang, and Shuicheng Yan. 2021c. A survey on concept factorization: From shallow to deep representation learning. Information Processing & Management 58, 3 (2021), 102534.
[68]
Guoqiang Zhong, Li-Na Wang, Xiao Ling, and Junyu Dong. 2016. An overview on data representation learning: From traditional feature learning to recent deep learning. The Journal of Finance and Data Science 2, 4 (2016), 265–278.

Cited By

View all
  • (2025)A Contemporary Survey on Multisource Information Fusion for Smart Sustainable Cities: Emerging Trends and Persistent ChallengesInformation Fusion10.1016/j.inffus.2024.102667114(102667)Online publication date: Feb-2025
  • (2025)TsCANet: Three-stream contrastive adaptive network for cross-domain few-shot learningThe Journal of Supercomputing10.1007/s11227-024-06482-281:1Online publication date: 1-Jan-2025
  • (2024)An autoencoder-like deep NMF representation learning algorithm for clusteringKnowledge-Based Systems10.1016/j.knosys.2024.112597305(112597)Online publication date: Dec-2024

Index Terms

  1. DNSRF: Deep Network-based Semi-NMF Representation Framework

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Intelligent Systems and Technology
    ACM Transactions on Intelligent Systems and Technology  Volume 15, Issue 5
    October 2024
    719 pages
    EISSN:2157-6912
    DOI:10.1145/3613688
    • Editor:
    • Huan Liu
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 November 2024
    Online AM: 03 June 2024
    Accepted: 23 April 2024
    Revised: 11 November 2023
    Received: 23 November 2022
    Published in TIST Volume 15, Issue 5

    Check for updates

    Author Tags

    1. Representation learning
    2. semi-nonnegative matrix factorization
    3. deep network
    4. clustering

    Qualifiers

    • Research-article

    Funding Sources

    • Postdoctoral Fellowship Program of CPSF
    • China Postdoctoral Science Foundation
    • Natural Science Foundation of Sichuan Province
    • School Talent Introduction Program

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)223
    • Downloads (Last 6 weeks)19
    Reflects downloads up to 12 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)A Contemporary Survey on Multisource Information Fusion for Smart Sustainable Cities: Emerging Trends and Persistent ChallengesInformation Fusion10.1016/j.inffus.2024.102667114(102667)Online publication date: Feb-2025
    • (2025)TsCANet: Three-stream contrastive adaptive network for cross-domain few-shot learningThe Journal of Supercomputing10.1007/s11227-024-06482-281:1Online publication date: 1-Jan-2025
    • (2024)An autoencoder-like deep NMF representation learning algorithm for clusteringKnowledge-Based Systems10.1016/j.knosys.2024.112597305(112597)Online publication date: Dec-2024

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media