{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,13]],"date-time":"2025-05-13T22:00:14Z","timestamp":1747173614495,"version":"3.40.5"},"reference-count":91,"publisher":"Cambridge University Press (CUP)","issue":"1","license":[{"start":{"date-parts":[[2021,8,9]],"date-time":"2021-08-09T00:00:00Z","timestamp":1628467200000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2023,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We give an in-depth account of compositional matrix-space models (CMSMs), a type of generic models for natural language, wherein compositionality is realized via matrix multiplication. We argue for the structural plausibility of this model and show that it is able to cover and combine various common compositional natural language processing approaches. Then, we consider efficient task-specific learning methods for training CMSMs and evaluate their performance in compositionality prediction and sentiment analysis.<\/jats:p>","DOI":"10.1017\/s1351324921000206","type":"journal-article","created":{"date-parts":[[2021,8,9]],"date-time":"2021-08-09T06:40:18Z","timestamp":1628491218000},"page":"32-80","update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":0,"title":["Compositional matrix-space models of language: Definitions, properties, and learning methods"],"prefix":"10.1017","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3905-1107","authenticated-orcid":false,"given":"Shima","family":"Asaadi","sequence":"first","affiliation":[]},{"given":"Eugenie","family":"Giesbrecht","sequence":"additional","affiliation":[]},{"given":"Sebastian","family":"Rudolph","sequence":"additional","affiliation":[]}],"member":"56","published-online":{"date-parts":[[2021,8,9]]},"reference":[{"key":"S1351324921000206_ref29","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-03079-6_14"},{"key":"S1351324921000206_ref58","doi-asserted-by":"publisher","DOI":"10.3115\/1219840.1219856"},{"key":"S1351324921000206_ref60","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-009-2213-6"},{"key":"S1351324921000206_ref50","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K15-1035"},{"volume-title":"Introduction to Modern Information Retrieval","year":"1986","author":"Salton","key":"S1351324921000206_ref72"},{"key":"S1351324921000206_ref43","unstructured":"Kordoni, V. and Simova, I. (2014). Multiword expressions in machine translation. In Calzolari N., Choukri K., Declerck T., Loftsson H., Maegaard B., Mariani J., Moreno A., Odijk J. and Piperidis S. (eds), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014). European Languages Resources Association, pp. 1208\u20131211."},{"key":"S1351324921000206_ref53","unstructured":"Mikolov, T. , Chen, K. , Corrado, G. and Dean, J. (2013a). Efficient estimation of word representations in vector space. In International Conference on Learning Representations (ICLR 2013)."},{"key":"S1351324921000206_ref1","unstructured":"Antonellis, I. and Gallopoulos, E. (2006). Exploring term-document matrices from matrix models in text mining. In Berry M.W. and Castellanos M. (eds), Proceedings of the Fourth Workshop on Text Mining (TM 2006) in Conjunction with the Sixth SIAM International Conference on Data Mining (SDM 2006). Society for Industrial and Applied Mathematics."},{"key":"S1351324921000206_ref61","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1098\/rsta.1894.0003","article-title":"Contributions to the mathematical theory of evolution","volume":"185","author":"Pearson","year":"1894","journal-title":"Philosophical Transactions of the Royal Society of London. A"},{"volume-title":"Introduction to Linear Algebra","year":"1993","author":"Strang","key":"S1351324921000206_ref80"},{"key":"S1351324921000206_ref6","first-page":"1","volume-title":"International Conference on Algebraic Informatics (CAI 2015)","author":"Balle","year":"2015"},{"key":"S1351324921000206_ref7","doi-asserted-by":"crossref","unstructured":"Baroni, M. , Bernardi, R. and Zamparelli, R. (2014). Frege in space: A program of compositional distributional semantics. In Linguistic Issues in Language Technology, Volume 9, 2014 - Perspectives on Semantic Representations for Textual Inference, vol. 9. CSLI Publications.","DOI":"10.33011\/lilt.v9i.1321"},{"key":"S1351324921000206_ref67","unstructured":"Reddy, S. , McCarthy, D. and Manandhar, S. (2011). An empirical study on compositionality in compound nouns. In Wang H. and Yarowsky D. (eds.), Proceedings of 5th International Joint Conference on Natural Language Processing (IJCNLP 2011). Asian Federation of Natural Language Processing, pp. 210\u2013218."},{"key":"S1351324921000206_ref82","doi-asserted-by":"publisher","DOI":"10.1613\/jair.3640"},{"first-page":"81","year":"2014","author":"Weller","key":"S1351324921000206_ref86"},{"key":"S1351324921000206_ref76","first-page":"895","volume-title":"Advances in Neural Information Processing Systems","volume":"5","author":"Sch\u00fctze","year":"1993"},{"key":"S1351324921000206_ref77","unstructured":"ShafieiBavani, E. , Ebrahimi, M. , Wong, R. and Chen, F. (2018). Summarization evaluation in the absence of human model summaries using the compositionality of word embeddings. In Bender E.M., Derczynski L. and Isabelle P. (eds.), Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, New Mexico, USA. Association for Computational Linguistics, pp. 905\u2013914."},{"key":"S1351324921000206_ref81","doi-asserted-by":"publisher","DOI":"10.1145\/1150402.1150445"},{"key":"S1351324921000206_ref89","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1201"},{"key":"S1351324921000206_ref75","doi-asserted-by":"publisher","DOI":"10.1162\/nol_a_00003"},{"key":"S1351324921000206_ref54","unstructured":"Mikolov, T. , Sutskever, I. , Chen, K. , Corrado, G.S. and Dean, J. (2013b). Distributed representations of words and phrases and their compositionality. In Burges C.J., Bottou L., Welling M., Ghahramani Z. and Weinberger K.Q. (eds), Advances in neural information processing systems (NIPS 2013). Curran Associates, Inc., pp. 3111\u20133119."},{"key":"S1351324921000206_ref3","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-2621"},{"key":"S1351324921000206_ref40","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog2502_1"},{"key":"S1351324921000206_ref33","doi-asserted-by":"publisher","DOI":"10.1177\/1745691619861372"},{"key":"S1351324921000206_ref45","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.104.2.211"},{"key":"S1351324921000206_ref38","unstructured":"Irsoy, O. and Cardie, C. (2015). Modeling compositionality with multiplicative recurrent neural networks. In Bengio Y. and LeCun Y. (eds), 3rd International Conference on Learning Representation (ICLR 2015), Sand Diego, CA, USA, pp. 1\u201310."},{"key":"S1351324921000206_ref46","unstructured":"Le, Q. and Mikolov, T. (2014). Distributed representations of sentences and documents. In Xing E.P. and Jebara T. (eds), Proceedings of the 31st International Conference on Machine Learning (ICML-14). JMLR.org, pp. 1188\u20131196."},{"key":"S1351324921000206_ref69","unstructured":"Sahlgren, M. , Holst, A. and Kanerva, P. (2008). Permutations as a means to encode order in word space. In Love B.C., McRae K. and Sloutsky V.M. (eds), Proceedings of 30th Annual Meeting of the Cognitive Science Society (CogSci\u201908). Cognitive Science Society, pp. 1300\u20131305."},{"key":"S1351324921000206_ref71","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1099"},{"key":"S1351324921000206_ref28","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/2076.001.0001"},{"key":"S1351324921000206_ref24","unstructured":"Finlayson, M.A. and Kulkarni, N. (2011). Detecting multi-word expressions improves word sense disambiguation. In Kordoni V., Ramisch C. and Villavicencio A. (eds), Proceedings of the Workshop on Multiword Expressions: from Parsing and Generation to the Real World (MWE 2011). Association for Computational Linguistics, pp. 20\u201324."},{"key":"S1351324921000206_ref17","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00341"},{"key":"S1351324921000206_ref12","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148287"},{"key":"S1351324921000206_ref8","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00016"},{"key":"S1351324921000206_ref34","doi-asserted-by":"publisher","DOI":"10.1007\/BF00126510"},{"key":"S1351324921000206_ref87","unstructured":"Widdows, D. (2008). Semantic vector products: Some initial investigations. In Bruza P.D., Lawless W., Rijsbergen K.V., Sofge D.A. and Coecke B. (eds), Proceedings of the Second AAAI Symposium on Quantum Interaction (QI-2008). College Publications."},{"key":"S1351324921000206_ref14","doi-asserted-by":"publisher","DOI":"10.1145\/1281192.1281211"},{"key":"S1351324921000206_ref10","unstructured":"Biemann, C. and Giesbrecht, E. (2011). Distributional semantics and compositionality 2011: Shared task description and results. In Biemann C. and Giesbrecht E. (eds), Proceedings of the Workshop on Distributional Semantics and Compositionality (DiSCO 2011). Association for Computational Linguistics, pp. 21\u201328."},{"key":"S1351324921000206_ref15","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K18-1049"},{"key":"S1351324921000206_ref62","doi-asserted-by":"crossref","unstructured":"Peters, M. , Neumann, M. , Iyyer, M. , Gardner, M. , Clark, C. , Lee, K. and Zettlemoyer, L. (2018). Deep contextualized word representations. In Walker M., Ji H. and Stent A. (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, Louisiana. Association for Computational Linguistics.","DOI":"10.18653\/v1\/N18-1202"},{"key":"S1351324921000206_ref35","doi-asserted-by":"publisher","DOI":"10.1080\/00437956.1954.11659520"},{"key":"S1351324921000206_ref36","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"S1351324921000206_ref44","doi-asserted-by":"publisher","DOI":"10.1080\/00029890.1958.11989160"},{"key":"S1351324921000206_ref37","unstructured":"Hong, J. and Fang, M. (2015). Sentiment analysis with deeply learned distributed representations of variable length texts. Technical report, Stanford University."},{"key":"S1351324921000206_ref25","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04930-9_14"},{"key":"S1351324921000206_ref16","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1187"},{"key":"S1351324921000206_ref52","doi-asserted-by":"publisher","DOI":"10.3115\/1119282.1119292"},{"key":"S1351324921000206_ref42","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-1128"},{"key":"S1351324921000206_ref66","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-2026"},{"key":"S1351324921000206_ref85","unstructured":"Wang, X. , Jiang, W. and Luo, Z. (2016). Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In Matsumoto Y. and Prasad R. (eds), Proceedings of the 26th International Conference on Computational Linguistics (COLING 2016). The COLING 2016 Organizing Committee, pp. 2428\u20132437."},{"key":"S1351324921000206_ref26","unstructured":"Frege, G. (1884). Die Grundlagen der Arithmetik: eine logisch-mathematische Untersuchung \u00fcber den Begriff der Zahl. Breslau, Germany: W. Koebner."},{"key":"S1351324921000206_ref19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum likelihood from incomplete data via the EM algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"Journal of the Royal Statistical Society. Series B"},{"key":"S1351324921000206_ref57","doi-asserted-by":"publisher","DOI":"10.1162\/coli.2007.33.2.161"},{"key":"S1351324921000206_ref13","first-page":"40","article-title":"On the theory of groups as depending on the symbolic equation","volume":"7","author":"Cayley","year":"1854","journal-title":"Philosophical Magazine"},{"key":"S1351324921000206_ref47","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-63558-3_43"},{"key":"S1351324921000206_ref48","unstructured":"Liu, N. , Zhang, B. , Yan, J. , Chen, Z. , Liu, W. , Bai, F. and Chien, L. (2005). Text representation: From vector to tensor. In Proceedings of the Fifth IEEE International Conference on Data Mining, Washington, DC, USA. IEEE Computer Society, pp. 725\u2013728."},{"key":"S1351324921000206_ref55","unstructured":"Mitchell, J. and Lapata, M. (2008). Vector-based models of semantic composition. In Moore J.D., Teufel S., Allan J. and Furui S. (eds), Proceedings of 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-08:HLT). Association for Computational Linguistics, pp. 236\u2013244."},{"key":"S1351324921000206_ref83","doi-asserted-by":"publisher","DOI":"10.1613\/jair.2934"},{"key":"S1351324921000206_ref11","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"S1351324921000206_ref73","doi-asserted-by":"publisher","DOI":"10.1145\/361219.361220"},{"key":"S1351324921000206_ref78","unstructured":"Socher, R. , Huval, B. , Manning, C.D. and Ng, A.Y. (2012). Semantic compositionality through recursive matrix-vector spaces. In Tsujii J., Henderson J. and Pasca M. (eds.), Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL 2012). Association for Computational Linguistics, pp. 1201\u20131211."},{"key":"S1351324921000206_ref84","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324910000148"},{"key":"S1351324921000206_ref18","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9"},{"key":"S1351324921000206_ref49","doi-asserted-by":"publisher","DOI":"10.3758\/BF03204766"},{"key":"S1351324921000206_ref2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-2408"},{"volume-title":"The Matrix Cookbook","year":"2012","author":"Petersen","key":"S1351324921000206_ref63"},{"key":"S1351324921000206_ref32","unstructured":"Guevara, E. (2010). A regression model of adjective-noun compositionality in distributional semantics. In Basili R. and Pennacchiotti M. (eds), Proceedings of the 2010 Workshop on GEometrical Models of Natural Language Semantics (GEMS\u201910). Association for Computational Linguistics, pp. 33\u201337."},{"key":"S1351324921000206_ref41","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-0410"},{"key":"S1351324921000206_ref51","doi-asserted-by":"publisher","DOI":"10.1016\/j.jml.2016.04.001"},{"key":"S1351324921000206_ref70","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-01492-5_4"},{"key":"S1351324921000206_ref20","unstructured":"Devlin, J. , Chang, M.-W. , Lee, K. and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Burstein J., Doran C. and Solorio T. (eds), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota. Association for Computational Linguistics, pp. 4171\u20134186."},{"key":"S1351324921000206_ref27","unstructured":"Gao, K. , Wang, Y. and Wang, Z. (2004). An efficient relevant evaluation model in information retrieval and its application. In Wei D., Wang H., Peng Z., Kara A. and He Y. (eds), Proceedings of the The Fourth International Conference on Computer and Information Technology (CIT\u201904). IEEE Computer Society, pp. 845\u2013850."},{"volume-title":"The Lexicon: An Introduction","year":"2016","author":"Je\u017eek","key":"S1351324921000206_ref39"},{"key":"S1351324921000206_ref88","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2016.2647384"},{"key":"S1351324921000206_ref21","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008395026374"},{"key":"S1351324921000206_ref90","unstructured":"Yessenalina, A. and Cardie, C. (2011). Compositional matrix-space models for sentiment analysis. In Barzilay R. and Johnson M. (eds), Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP 2011). Association for Computational Linguistics, pp. 172\u2013182."},{"key":"S1351324921000206_ref23","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W15-0904"},{"key":"S1351324921000206_ref79","unstructured":"Socher, R. , Perelygin, A. , Wu, J.Y. , Chuang, J. , Manning, C.D. , Ng, A.Y. and Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Yarowsky D., Baldwin T., Korhonen A., Livescu K. and Bethard S. (eds.), Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2013). Association for Computational Linguistics, pp. 1631\u20131642."},{"key":"S1351324921000206_ref91","doi-asserted-by":"publisher","DOI":"10.1162\/neco_a_01199"},{"key":"S1351324921000206_ref5","unstructured":"Balle, B. , Hamilton, W. and Pineau, J. (2014). Methods of moments for learning stochastic languages: Unified presentation and empirical comparison. In Xing E.P. and Jebara T. (eds), Proceedings of the 31st International Conference on Machine Learning (ICML 2014). JMLR.org, pp. 1386\u20131394."},{"key":"S1351324921000206_ref65","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9904-1946-08555-9"},{"key":"S1351324921000206_ref74","unstructured":"Sanh, V. , Debut, L. , Chaumond, J. and Wolf, T. (2019). Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter. In EMC2."},{"key":"S1351324921000206_ref64","doi-asserted-by":"publisher","DOI":"10.1109\/72.377968"},{"key":"S1351324921000206_ref30","unstructured":"Giesbrecht, E. (2014). Distributional Tensor Space Model of Natural Language Semantics. PhD Thesis, Karlsruhe Institute of Technology."},{"key":"S1351324921000206_ref56","doi-asserted-by":"publisher","DOI":"10.1111\/j.1551-6709.2010.01106.x"},{"key":"S1351324921000206_ref68","unstructured":"Rudolph, S. and Giesbrecht, E. (2010). Compositional matrix-space models of language. In Hajic J., Carberry S. and Clark S. (eds.), Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010). Association for Computational Linguistics, pp. 907\u2013916."},{"key":"S1351324921000206_ref4","first-page":"267","article-title":"Multiword expressions","volume":"2","author":"Baldwin","year":"2010","journal-title":"Handbook of Natural Language Processing"},{"key":"S1351324921000206_ref59","doi-asserted-by":"publisher","DOI":"10.1002\/9780470751305"},{"key":"S1351324921000206_ref31","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4615-2710-7"},{"key":"S1351324921000206_ref9","unstructured":"Baroni, M. and Zamparelli, R. (2010). Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Li H. and M\u00e0rquez L. (eds), Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP 2010). Association for Computational Linguistics, pp. 1183\u20131193."},{"key":"S1351324921000206_ref22","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10223-8_14"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324921000206","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,6]],"date-time":"2024-09-06T02:40:02Z","timestamp":1725590402000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324921000206\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,9]]},"references-count":91,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1]]}},"alternative-id":["S1351324921000206"],"URL":"https:\/\/doi.org\/10.1017\/s1351324921000206","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"type":"print","value":"1351-3249"},{"type":"electronic","value":"1469-8110"}],"subject":[],"published":{"date-parts":[[2021,8,9]]},"assertion":[{"value":"\u00a9 The Author(s), 2021. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http:\/\/creativecommons.org\/licenses\/by\/4.0\/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.","name":"license","label":"License","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This content has been made available to all.","name":"free","label":"Free to read"}]}}