{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,29]],"date-time":"2026-03-29T06:24:18Z","timestamp":1774765458463,"version":"3.50.1"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2025,5,9]],"date-time":"2025-05-09T00:00:00Z","timestamp":1746748800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2025,7,31]]},"abstract":"<jats:p>Programmatic Weak Supervision (PWS) is a recent data labeling paradigm, which employs several Labeling Functions (LFs) to provide weak labels and involves a Label Model (LM) for label aggregation. Despite the significant progress, there still remain some inherent challenges in PWS. From the view of labeling, LFs may wrongly label some data points. From the view of data, some data points themselves may be low-quality (e.g., ambiguous texts or blurred images). These largely stem from the lack of an explicit evaluation mechanism for LFs or data points. To this end, inspired by confident learning focusing on label quality, we propose a Confident PWS (CPWS) approach for high-quality data labeling. Specifically, several LFs are firstly utilized to provide weak labels for unlabeled data. Then, we develop an explicit Dual Evaluation Mechanism (DEM) to evaluate the quality of both LFs and data points, which not only employs data to evaluate trained models but also leverages trained models to evaluate data. Along this line, we further design a Distribution-Guided Pruning Strategy (DPS) to prune low-quality data and aggregate weak labels under the guidance of label class distribution. Extensive experiments on various benchmark datasets demonstrate the effectiveness and generalization ability of our proposed approach.<\/jats:p>","DOI":"10.1145\/3725730","type":"journal-article","created":{"date-parts":[[2025,3,25]],"date-time":"2025-03-25T19:28:14Z","timestamp":1742930894000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["CPWS: Confident Programmatic Weak Supervision for High-Quality Data Labeling"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7466-8909","authenticated-orcid":false,"given":"Shulan","family":"Ruan","sequence":"first","affiliation":[{"name":"Shenzhen International Graduate School, Tsinghua University, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6090-9895","authenticated-orcid":false,"given":"Huijie","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Data Science, University of Science and Technology\u00a0of\u00a0China, Hefei, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0425-5855","authenticated-orcid":false,"given":"Zhao","family":"Chen","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0880-6678","authenticated-orcid":false,"given":"Bin","family":"Feng","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0743-9003","authenticated-orcid":false,"given":"Kun","family":"Zhang","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8050-2486","authenticated-orcid":false,"given":"Caleb Chen","family":"Cao","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4835-4102","authenticated-orcid":false,"given":"Enhong","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, University of Science and Technology\u00a0of\u00a0China, Hefei, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8257-5806","authenticated-orcid":false,"given":"Lei","family":"Chen","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology\u2014Guangzhou Campus, Guangzhou, China"}]}],"member":"320","published-online":{"date-parts":[[2025,5,9]]},"reference":[{"issue":"2","key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3439861","article-title":"Unbiased learning to rank: Online or offline","volume":"39","author":"Ai Qingyao","year":"2021","unstructured":"Qingyao Ai, Tao Yang, Huazheng Wang, and Jiaxin Mao. 2021. Unbiased learning to rank: Online or offline? ACM Transactions on Information Systems 39, 2 (2021), 1\u201329.","journal-title":"ACM Transactions on Information Systems"},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","first-page":"722","DOI":"10.1007\/978-3-540-76298-0_52","volume-title":"In International Semantic Web Conference","author":"Auer S\u00f6ren","year":"2007","unstructured":"S\u00f6ren Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In International Semantic Web Conference. Springer, 722\u2013735."},{"key":"e_1_3_2_4_2","volume-title":"International Conference on Learning Representations","author":"Awasthi Abhijeet","year":"2019","unstructured":"Abhijeet Awasthi, Sabyasachi Ghosh, Rasna Goyal, and Sunita Sarawagi. 2019. Learning from rules generalizing labeled exemplars. In International Conference on Learning Representations."},{"key":"e_1_3_2_5_2","first-page":"1298","volume-title":"International Conference on Machine Learning","author":"Baevski Alexei","year":"2022","unstructured":"Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. 2022. Data2vec: A general framework for self-supervised learning in speech, vision and language. In International Conference on Machine Learning. PMLR, 1298\u20131312."},{"key":"e_1_3_2_6_2","first-page":"26671","article-title":"Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods","author":"Balestriero Randall","year":"2022","unstructured":"Randall Balestriero and Yann LeCun. 2022. Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods. In Advances in Neural Information Processing Systems, 26671\u201326685.","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"3","key":"e_1_3_2_7_2","first-page":"1","article-title":"Meng Wang, and Xiangnan He. 2023. Bias and debias in recommender system: A survey and future directions","volume":"41","author":"Chen Jiawei","year":"2023","unstructured":"Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2023. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems 41, 3 (2023), 1\u201339.","journal-title":"ACM Transactions on Information Systems"},{"key":"e_1_3_2_8_2","first-page":"1062","volume-title":"International Conference on Machine Learning","author":"Chen Pengfei","year":"2019","unstructured":"Pengfei Chen, Ben Ben Liao, Guangyong Chen, and Shengyu Zhang. 2019. Understanding and utilizing deep neural networks trained with noisy labels. In International Conference on Machine Learning. PMLR, 1062\u20131070."},{"key":"e_1_3_2_9_2","first-page":"2580","volume-title":"IEEE\/CVF International Conference on Computer Vision","author":"Chen Vincent S.","year":"2019","unstructured":"Vincent S. Chen, Paroma Varma, Ranjay Krishna, Michael Bernstein, Christopher Re, and Li Fei-Fei. 2019. Scene graph prediction with limited labels. In IEEE\/CVF International Conference on Computer Vision, 2580\u20132590."},{"key":"e_1_3_2_10_2","first-page":"3915","volume-title":"International Conference on Machine Learning","author":"Chiu Chung-Cheng","year":"2022","unstructured":"Chung-Cheng Chiu, James Qin, Yu Zhang, Jiahui Yu, and Yonghui Wu. 2022. Self-supervised learning with random-projection quantizer for speech recognition. In International Conference on Machine Learning. PMLR, 3915\u20133924."},{"key":"e_1_3_2_11_2","first-page":"16670","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Chuang Ching-Yao","year":"2022","unstructured":"Ching-Yao Chuang, R. Devon Hjelm, Xin Wang, Vibhav Vineet, Neel Joshi, Antonio Torralba, Stefanie Jegelka, and Yale Song. 2022. Robust contrastive learning against noisy views. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 16670\u201316681."},{"issue":"1","key":"e_1_3_2_12_2","first-page":"20","article-title":"Maximum likelihood estimation of observer error-rates using the EM algorithm","volume":"28","author":"Philip Dawid Alexander","year":"1979","unstructured":"Alexander Philip Dawid and Allan M. Skene. 1979. Maximum likelihood estimation of observer error-rates using the EM algorithm. Journal of the Royal Statistical Society: Series C (Applied Statistics) 28, 1 (1979), 20\u201328.","journal-title":"Journal of the Royal Statistical Society: Series C (Applied Statistics)"},{"key":"e_1_3_2_13_2","first-page":"973","volume-title":"International Joint Conference on Artificial Intelligence","volume":"17","author":"Charles Elkan","year":"2001","unstructured":"Charles Elkan. 2001. The foundations of cost-sensitive learning. In International Joint Conference on Artificial Intelligence, Vol. 17. Lawrence Erlbaum Associates Ltd, 973\u2013978."},{"key":"e_1_3_2_14_2","first-page":"564","volume-title":"16th European Conference on Machine Learning (ECML \u201905)","author":"George Forman","year":"2005","unstructured":"George Forman. 2005. Counting positives accurately despite inaccurate classification. In 16th European Conference on Machine Learning (ECML \u201905). Springer, 564\u2013575."},{"key":"e_1_3_2_15_2","first-page":"3280","volume-title":"International Conference on Machine Learning. PMLR","author":"Fu Daniel","year":"2020","unstructured":"Daniel Fu, Mayee Chen, Frederic Sala, Sarah Hooper, Kayvon Fatahalian, and Christopher R\u00e9. 2020. Fast and three-rious: Speeding up weak supervision with triplet methods. In International Conference on Machine Learning. PMLR, 3280\u20133291."},{"key":"e_1_3_2_16_2","first-page":"5138","volume-title":"IEEE\/CVF International Conference on Computer Vision","author":"Han Jiangfan","year":"2019","unstructured":"Jiangfan Han, Ping Luo, and Xiaogang Wang. 2019. Deep self-learning from noisy labels. In IEEE\/CVF International Conference on Computer Vision, 5138\u20135147."},{"issue":"3","key":"e_1_3_2_17_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3494522","article-title":"2022. A survey on task assignment in crowdsourcing","volume":"55","author":"Hettiachchi Danula","year":"2022","unstructured":"Danula Hettiachchi, Vassilis Kostakos, and Jorge Goncalves. 2022. A survey on task assignment in crowdsourcing. ACM Computing Surveys 55, 3 (2022), 1\u201335.","journal-title":"ACM Computing Surveys"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002541"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2023.3310002"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2766634"},{"key":"e_1_3_2_21_2","first-page":"3326","volume-title":"IEEE\/CVF International Conference on Computer Vision","author":"Huang Jinchi","year":"2019","unstructured":"Jinchi Huang, Lie Qu, Rongfei Jia, and Binqiang Zhao. 2019. O2u-net: A simple noisy label detection approach for deep neural networks. In IEEE\/CVF International Conference on Computer Vision, 3326\u20133334."},{"key":"e_1_3_2_22_2","first-page":"4171","volume-title":"North American Chapter of the Association for Computational Linguistics\u2014Human Language Technologies Conference (NAACL-HLT)","author":"Kenton Jacob Devlin Ming-Wei Chang","year":"2019","unstructured":"Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics\u2014Human Language Technologies Conference (NAACL-HLT), 4171\u20134186."},{"key":"e_1_3_2_23_2","first-page":"619","volume-title":"Artificial Intelligence and Statistics","author":"Kim Hyun-Chul","year":"2012","unstructured":"Hyun-Chul Kim and Zoubin Ghahramani. 2012. Bayesian classifier combination. In Artificial Intelligence and Statistics. PMLR, 619\u2013627."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41551-022-00914-1"},{"key":"e_1_3_2_25_2","first-page":"16023","article-title":"Training subset selection for weak supervision","author":"Lang Hunter","year":"2022","unstructured":"Hunter Lang, Aravindan Vijayaraghavan, and David Sontag. 2022. Training subset selection for weak supervision. In Advances in Neural Information Processing Systems, 16023\u201316036.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_26_2","first-page":"1","volume-title":"CHI Conference on Human Factors in Computing Systems","author":"Li Chu","year":"2024","unstructured":"Chu Li, Zhihan Zhang, Michael Saugstad, Esteban Safranchik, Chaitanyashareef Kulkarni, Xiaoyu Huang, Shwetak Patel, Vikram Iyer, Tim Althoff, and Jon E. Froehlich. 2024. LabelAId: Just-in-time AI interventions for improving human labeling quality and domain knowledge in crowdsourcing systems. In CHI Conference on Human Factors in Computing Systems, 1\u201321."},{"key":"e_1_3_2_27_2","article-title":"Characterizing the impacts of semi-supervised learning for weak supervision","volume":"36","author":"Li Jeffrey","year":"2024","unstructured":"Jeffrey Li, Jieyu Zhang, Ludwig Schmidt, and Alexander J. Ratner. 2024. Characterizing the impacts of semi-supervised learning for weak supervision. In Advances in Neural Information Processing Systems, Vol. 36.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_28_2","first-page":"1","article-title":"Transferring annotator-and instance-dependent transition matrix for learning from crowds","volume":"1","author":"Li Shikun","year":"2024","unstructured":"Shikun Li, Xiaobo Xia, Jiankang Deng, Shiming Gey, and Tongliang Liu. 2024. Transferring annotator-and instance-dependent transition matrix for learning from crowds. IEEE Transactions on Pattern Analysis & Machine Intelligence 1 (2024), 1\u201315.","journal-title":"IEEE Transactions on Pattern Analysis & Machine Intelligence"},{"key":"e_1_3_2_29_2","first-page":"1","volume-title":"ACM Transactions on Information Systems","author":"Li Shujie","year":"2024","unstructured":"Shujie Li, Guanghu Yuan, Min Yang, Ying Shen, Chengming Li, Ruifeng Xu, and Xiaoyan Zhao. 2024. Improving semi-supervised text classification with dual meta-learning. ACM Transactions on Information Systems 42, 4 (2024), 1\u201328."},{"key":"e_1_3_2_30_2","first-page":"3886","volume-title":"International Conference on Machine Learning. PMLR","author":"Li Yuan","year":"2019","unstructured":"Yuan Li, Benjamin Rubinstein, and Trevor Cohn. 2019. Exploiting worker correlation for label aggregation in crowdsourcing. In International Conference on Machine Learning. PMLR, 3886\u20133895."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403149"},{"key":"e_1_3_2_32_2","first-page":"3122","volume-title":"International Conference on Machine Learning","author":"Lipton Zachary","year":"2018","unstructured":"Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. 2018. Detecting and correcting for label shift with black box predictors. In International Conference on Machine Learning. PMLR, 3122\u20133130."},{"issue":"6","key":"e_1_3_2_33_2","first-page":"5879","article-title":"Graph self-supervised learning: A survey","volume":"35","author":"Liu Yixin","year":"2022","unstructured":"Yixin Liu, Ming Jin, Shirui Pan, Chuan Zhou, Yu Zheng, Feng Xia, and S. Yu Philip. 2022. Graph self-supervised learning: A survey. IEEE Transactions on Knowledge and Data Engineering 35, 6 (2022), 5879\u20135900.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271737"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.5555\/944249.944255"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.12125"},{"issue":"3","key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3385186","article-title":"Outdoorsent: Sentiment analysis of urban outdoor images by using semantic and deep features","volume":"38","author":"Bonasoli de Oliveira Wyverson","year":"2020","unstructured":"Wyverson Bonasoli de Oliveira, Leyza Baldo Dorini, Rodrigo Minetto, and Thiago H. Silva. 2020. Outdoorsent: Sentiment analysis of urban outdoor images by using semantic and deep features. ACM Transactions on Information Systems 38, 3 (2020), 1\u201328.","journal-title":"ACM Transactions on Information Systems"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3569453"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503161.3548018"},{"key":"e_1_3_2_40_2","first-page":"269","volume-title":"VLDB Endowment, International Conference on Very Large Data Bases","volume":"11","author":"Ratner Alexander","year":"2017","unstructured":"Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher R\u00e9. 2017. Snorkel: Rapid training data creation with weak supervision. In VLDB Endowment, International Conference on Very Large Data Bases, Vol. 11. NIH Public Access, 269."},{"key":"e_1_3_2_41_2","first-page":"4763","volume-title":"AAAI Conference on Artificial Intelligence","volume":"33","author":"Ratner Alexander","year":"2019","unstructured":"Alexander Ratner, Braden Hancock, Jared Dunnmon, Frederic Sala, Shreyash Pandey, and Christopher R\u00e9. 2019. Training complex models with multi-task weak supervision. In AAAI Conference on Artificial Intelligence, Vol. 33, 4763\u20134771."},{"key":"e_1_3_2_42_2","article-title":"Data programming: Creating large training sets, quickly. In","volume":"29","author":"Ratner Alexander J.","year":"2016","unstructured":"Alexander J. Ratner, Christopher M. De Sa, Sen Wu, Daniel Selsam, and Christopher R\u00e9. 2016. Data programming: Creating large training sets, quickly. In Advances in Neural Information Processing Systems, Vol. 29.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_43_2","volume-title":"Findings of Conference on Empirical Methods in Natural Language Processing","author":"Ren Wendi","year":"2021","unstructured":"Wendi Ren, Yinghao Li, Hanting Su, David Kartchner, Cassie Mitchell, and Chao Zhang. 2021. Denoising multi-source weak supervision for neural text classification. In Findings of Conference on Empirical Methods in Natural Language Processing."},{"key":"e_1_3_2_44_2","first-page":"1","volume-title":"2020 IEEE International Conference on Multimedia and Expo (ICME)","author":"Ruan Shulan","year":"2020","unstructured":"Shulan Ruan, Kun Zhang, Yijun Wang, Hanqing Tao, Weidong He, Guangyi Lv, and Enhong Chen. 2020. Context-aware generation-based net for multi-label visual emotion recognition. In 2020 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1\u20136."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3118208"},{"key":"e_1_3_2_46_2","first-page":"13960","volume-title":"IEEE\/CVF International Conference on Computer Vision","author":"Ruan Shulan","year":"2021","unstructured":"Shulan Ruan, Yong Zhang, Kun Zhang, Yanbo Fan, Fan Tang, Qi Liu, and Enhong Chen. 2021. DAE-GAN: Dynamic aspect-aware GAN for text-to-image synthesis. In IEEE\/CVF International Conference on Computer Vision, 13960\u201313969."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2024\/540"},{"key":"e_1_3_2_48_2","article-title":"Mitigating source bias for fairer weak supervision","volume":"36","author":"Shin Changho","year":"2024","unstructured":"Changho Shin, Sonia Cromp, Dyah Adila, and Frederic Sala. 2024. Mitigating source bias for fairer weak supervision. In Advances in Neural Information Processing Systems, Vol. 36.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_49_2","first-page":"15020","volume-title":"AAAI Conference on Artificial Intelligence","volume":"38","author":"Shubham Kumar","year":"2024","unstructured":"Kumar Shubham, Pranav Sastry, and A. P. Prathosh. 2024. Fusing conditional submodular GAN and programmatic weak supervision. In AAAI Conference on Artificial Intelligence, Vol. 38, 15020\u201315028."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3152527"},{"key":"e_1_3_2_51_2","article-title":"Inferring generative model structure with static analysis. In","author":"Varma Paroma","year":"2017","unstructured":"Paroma Varma, Bryan D. He, Payal Bajaj, Nishith Khandwala, Imon Banerjee, Daniel Rubin, and Christopher R\u00e9. 2017. Inferring generative model structure with static analysis. In Advances in Neural Information Processing Systems, Vol. 30.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_52_2","first-page":"22919","volume-title":"International Conference on Machine Learning","author":"Wang Jianfeng","year":"2022","unstructured":"Jianfeng Wang, Thomas Lukasiewicz, Daniela Massiceti, Xiaolin Hu, Vladimir Pavlovic, and Alexandros Neophytou. 2022. Np-match: When neural processes meet semi-supervised learning. In International Conference on Machine Learning. PMLR, 22919\u201322934."},{"key":"e_1_3_2_53_2","first-page":"3938","article-title":"Usb: A unified semi-supervised learning benchmark for classification","author":"Wang Yidong","year":"2022","unstructured":"Yidong Wang, Hao Chen, Yue Fan, Wang Sun, Ran Tao, Wenxin Hou, Renjie Wang, Linyi Yang, Zhi Zhou, Lan-Zhe Guo, et\u00a0al. 2022. Usb: A unified semi-supervised learning benchmark for classification. In Advances in Neural Information Processing Systems, 3938\u20133961.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_54_2","volume-title":"the 11th International Conference on Learning Representations","author":"Wu Renzhi","year":"2023","unstructured":"Renzhi Wu, Shen-En Chen, Jieyu Zhang, and Xu Chu. 2023. Learning hyper label model for programmatic weak supervision. In the 11th International Conference on Learning Representations."},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1145\/3503161.3548366","volume-title":"30th ACM International Conference on Multimedia","author":"Xiao Fangxiong","year":"2022","unstructured":"Fangxiong Xiao, Lixi Deng, Jingjing Chen, Houye Ji, Xiaorui Yang, Zhuoye Ding, and Bo Long. 2022. From abstract to details: A generative multimodal fusion framework for recommendation. In 30th ACM International Conference on Multimedia, 258\u2013267."},{"issue":"4","key":"e_1_3_2_56_2","first-page":"1","article-title":"Learning text-image joint embedding for efficient cross-modal retrieval with deep feature engineering","volume":"40","author":"Xie Zhongwei","year":"2021","unstructured":"Zhongwei Xie, Ling Liu, Yanzhao Wu, Luo Zhong, and Lin Li. 2021. Learning text-image joint embedding for efficient cross-modal retrieval with deep feature engineering. ACM Transactions on Information Systems 40, 4 (2021), 1\u201327.","journal-title":"ACM Transactions on Information Systems"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3580594"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3220219"},{"key":"e_1_3_2_59_2","first-page":"13284","volume-title":"IEEE Transactions on Neural Networks and Learning Systems","author":"Yuan Di","year":"2023","unstructured":"Di Yuan, Xiaojun Chang, Qiao Liu, Yi Yang, Dehua Wang, Minglei Shu, Zhenyu He, and Guangming Shi. 2023. Active learning for deep visual tracking. IEEE Transactions on Neural Networks and Learning Systems 35, 10 (2023), 13284\u201313296."},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2020.2978844"},{"key":"e_1_3_2_61_2","unstructured":"Jieyu Zhang Cheng-Yu Hsieh Yue Yu Chao Zhang and Alexander Ratner. 2022. A survey on programmatic weak supervision. arXiv:2202.05433. Retrieved from https:\/\/arxiv.org\/abs\/2202.05433"},{"key":"e_1_3_2_62_2","first-page":"157","volume-title":"International Conference on Artificial Intelligence and Statistics. PMLR","author":"Zhang Jieyu","year":"2023","unstructured":"Jieyu Zhang, Linxin Song, and Alex Ratner. 2023. Leveraging instance features for label aggregation in programmatic weak supervision. In International Conference on Artificial Intelligence and Statistics. PMLR, 157\u2013171."},{"key":"e_1_3_2_63_2","first-page":"2862","article-title":"Understanding programmatic weak supervision via source-aware influence function","author":"Zhang Jieyu","year":"2022","unstructured":"Jieyu Zhang, Haonan Wang, Cheng-Yu Hsieh, and Alexander J. Ratner. 2022. Understanding programmatic weak supervision via source-aware influence function. In Advances in Neural Information Processing Systems, 2862\u20132875.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","unstructured":"Jieyu Zhang Yue Yu Yinghao Li Yujing Wang Yaming Yang Mao Yang and Alexander Ratner. 2021. Wrench: A comprehensive benchmark for weak supervision. arXiv:2109.11377. Retrieved from https:\/\/arxiv.org\/abs\/2109.11377","DOI":"10.5465\/AMBPP.2021.11377abstract"},{"key":"e_1_3_2_65_2","first-page":"14411","volume-title":"AAAI Conference on Artificial Intelligence","volume":"35","author":"Zhang Kun","year":"2021","unstructured":"Kun Zhang, Le Wu, Guangyi Lv, Meng Wang, Enhong Chen, and Shulan Ruan. 2021. Making the relation matters: Relation of relation learning network for sentence semantic matching. In AAAI Conference on Artificial Intelligence, Vol. 35, 14411\u201314419."},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3637870"},{"key":"e_1_3_2_67_2","first-page":"14471","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zheng Mingkai","year":"2022","unstructured":"Mingkai Zheng, Shan You, Lang Huang, Fei Wang, Chen Qian, and Chang Xu. 2022. Simmatch: Semi-supervised learning with similarity matching. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 14471\u201314481."},{"key":"e_1_3_2_68_2","volume-title":"Learning From Labeled and Unlabeled Data with Label Propagation","author":"Zhu Xiaojin","year":"2002","unstructured":"Xiaojin Zhu and Zoubin Ghahramani. 2002. Learning From Labeled and Unlabeled Data with Label Propagation. Technical Report. CMU-CALD-02-107. Carnegie Mellon University."},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3224228"},{"key":"e_1_3_2_70_2","first-page":"14502","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Ziegler Adrian","year":"2022","unstructured":"Adrian Ziegler and Yuki M. Asano. 2022. Self-supervised learning of object parts for semantic segmentation. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 14502\u201314511."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3725730","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3725730","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:57:04Z","timestamp":1750298224000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3725730"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,9]]},"references-count":69,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,7,31]]}},"alternative-id":["10.1145\/3725730"],"URL":"https:\/\/doi.org\/10.1145\/3725730","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,9]]},"assertion":[{"value":"2024-06-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-16","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}