{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T06:15:50Z","timestamp":1775283350187,"version":"3.50.1"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2014,8,1]],"date-time":"2014-08-01T00:00:00Z","timestamp":1406851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001457","name":"Media Development Authority - Singapore","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001457","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001381","name":"National Research Foundation-Prime Minister's office, Republic of Singapore","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001381","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2014,8]]},"abstract":"<jats:p>\n            Current music recommender systems typically act in a greedy manner by recommending songs with the highest user ratings. Greedy recommendation, however, is suboptimal over the long term: it does not actively gather information on user preferences and fails to recommend\n            <jats:italic>novel<\/jats:italic>\n            songs that are potentially interesting. A successful recommender system must balance the needs to\n            <jats:italic>explore<\/jats:italic>\n            user preferences and to\n            <jats:italic>exploit<\/jats:italic>\n            this information for recommendation. This article presents a new approach to music recommendation by formulating this exploration-exploitation trade-off as a reinforcement learning task. To learn user preferences, it uses a Bayesian model that accounts for both audio content and the novelty of recommendations. A piecewise-linear approximation to the model and a variational inference algorithm help to speed up Bayesian inference. One additional benefit of our approach is a single unified model for both music recommendation and playlist generation. We demonstrate the strong potential of the proposed approach with simulation results and a user study.\n          <\/jats:p>","DOI":"10.1145\/2623372","type":"journal-article","created":{"date-parts":[[2014,8,29]],"date-time":"2014-08-29T13:03:31Z","timestamp":1409317411000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":65,"title":["Exploration in Interactive Personalized Music Recommendation"],"prefix":"10.1145","volume":"11","author":[{"given":"Xinxi","family":"Wang","sequence":"first","affiliation":[{"name":"National University of Singapore, SG"}]},{"given":"Yi","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of High Performance Computing, A*STAR, Singapore"}]},{"given":"David","family":"Hsu","sequence":"additional","affiliation":[{"name":"National University of Singapore, SG"}]},{"given":"Ye","family":"Wang","sequence":"additional","affiliation":[{"name":"National University of Singapore, SG"}]}],"member":"320","published-online":{"date-parts":[[2014,9,4]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Proceedings of the 25th Annual Conference on Learning Theory (COLT'12)","author":"Agrawal S.","unstructured":"S. Agrawal and N. Goyal . 2012. Analysis of thompson sampling for the multi-armed bandit problem . In Proceedings of the 25th Annual Conference on Learning Theory (COLT'12) . S. Agrawal and N. Goyal. 2012. Analysis of thompson sampling for the multi-armed bandit problem. In Proceedings of the 25th Annual Conference on Learning Theory (COLT'12)."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187836.2187838"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.5555\/944919.944941"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13735-012-0032-2"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1137\/0916069"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1101149.1101181"},{"key":"e_1_2_2_7_1","volume-title":"Proceedings of the International Workshop on Diversity in Document Retrieval (DDR'11)","author":"Castells P.","unstructured":"P. Castells , S. Vargas , and J. Wang . 2011. Novelty and diversity metrics for recommender systems: Choice, discovery and relevance . In Proceedings of the International Workshop on Diversity in Document Retrieval (DDR'11) at the 33rd European Conference on Information Retrieval (ECIR'11). 29--36. P. Castells, S. Vargas, and J. Wang. 2011. Novelty and diversity metrics for recommender systems: Choice, discovery and relevance. In Proceedings of the International Workshop on Diversity in Document Retrieval (DDR'11) at the 33rd European Conference on Information Retrieval (ECIR'11). 29--36."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/502585.502625"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339643"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2433396.2433420"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2578726.2578751"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAAI.2010.21"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1037\/10011-000"},{"key":"e_1_2_2_14_1","volume-title":"Proceedings of the Neural Information Processing Systems Conference (NIPS'07)","volume":"20","author":"Eck D.","unstructured":"D. Eck , P. Lamere , T. Bertin-Mahieux , and S. Green . 2007. Automatic generation of social tags for music recommendation . In Proceedings of the Neural Information Processing Systems Conference (NIPS'07) . Vol. 20 . D. Eck, P. Lamere, T. Bertin-Mahieux, and S. Green. 2007. Automatic generation of social tags for music recommendation. In Proceedings of the Neural Information Processing Systems Conference (NIPS'07). Vol. 20."},{"key":"e_1_2_2_15_1","unstructured":"N. Friedman and D. Koller. 2009. Probabilistic Graphical Models: Principles and Techniques 1st Ed. The MIT Press.   N. Friedman and D. Koller. 2009. Probabilistic Graphical Models: Principles and Techniques 1 st Ed. The MIT Press."},{"key":"e_1_2_2_16_1","volume-title":"Proceedings of the International Conference on Information Technology: Coding and Computing.","volume":"1","author":"Golovin N.","unstructured":"N. Golovin and E. Rahm . 2004. Reinforcement learning architecture for web recommendations . In Proceedings of the International Conference on Information Technology: Coding and Computing. Vol. 1 , IEEE, 398--402. N. Golovin and E. Rahm. 2004. Reinforcement learning architecture for web recommendations. In Proceedings of the International Conference on Information Technology: Coding and Computing. Vol. 1, IEEE, 398--402."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/1577069.1755883"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2365952.2365979"},{"key":"e_1_2_2_19_1","doi-asserted-by":"crossref","unstructured":"T. Hastie R. Tibshirani and J. Friedman. 2009. The Elements of Statistical Learning. Springer.  T. Hastie R. Tibshirani and J. Friedman. 2009. The Elements of Statistical Learning. Springer.","DOI":"10.1007\/978-0-387-84858-7"},{"key":"e_1_2_2_20_1","volume-title":"Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR'11)","author":"Hu Y.","unstructured":"Y. Hu and M. Ogihara . 2011. Nextone player: A music recommendation system based on user behavior . In Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR'11) . Y. Hu and M. Ogihara. 2011. Nextone player: A music recommendation system based on user behavior. In Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR'11)."},{"key":"e_1_2_2_21_1","volume-title":"Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI'97)","author":"Joachims T.","unstructured":"T. Joachims , D. Freitag , and T. Mitchell . 1997. WebWatcher: A tour guide for the world wide web . In Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI'97) . 770--777. T. Joachims, D. Freitag, and T. Mitchell. 1997. WebWatcher: A tour guide for the world wide web. In Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI'97). 770--777."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2507157.2507180"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI.2011.182"},{"key":"e_1_2_2_24_1","first-page":"592","article-title":"On bayesian upper confidence bounds for bandit problems","volume":"22","author":"Kaufmann E.","year":"2012","unstructured":"E. Kaufmann , O. Capp\u00e9 , and A. Garivier . 2012 . On bayesian upper confidence bounds for bandit problems . J. Mach. Learn. Res. Proc. Track 22 , 592 -- 600 . E. Kaufmann, O. Capp\u00e9, and A. Garivier. 2012. On bayesian upper confidence bounds for bandit problems. J. Mach. Learn. Res. Proc. Track 22, 592--600.","journal-title":"J. Mach. Learn. Res. Proc. Track"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2542205.2542206"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2009.263"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835486"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772758"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935878"},{"key":"e_1_2_2_30_1","volume-title":"Dj-mc: A reinforcement-learning agent for music playlist recommendation","author":"Liebman E.","year":"2014","unstructured":"E. Liebman and P. Stone . 2014 . Dj-mc: A reinforcement-learning agent for music playlist recommendation . http:\/\/arxiv.org\/pdf\/1401.1880.pdf E. Liebman and P. Stone. 2014. Dj-mc: A reinforcement-learning agent for music playlist recommendation. http:\/\/arxiv.org\/pdf\/1401.1880.pdf"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCTD.2009.246"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2502081.2502126"},{"key":"e_1_2_2_33_1","volume-title":"Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR'02)","author":"Logan B.","year":"2002","unstructured":"B. Logan . 2002 . Content-based playlist generation: Exploratory experiments . In Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR'02) . 295--296. B. Logan. 2002. Content-based playlist generation: Exploratory experiments. In Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR'02). 295--296."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03193007"},{"key":"e_1_2_2_35_1","first-page":"2069","article-title":"Optimistic bayesian sampling in contextual-bandit problems","volume":"13","author":"May B. C.","year":"2012","unstructured":"B. C. May , N. Korda , A. Lee , and D. S. Leslie . 2012 . Optimistic bayesian sampling in contextual-bandit problems . J. Mach. Learn. Res. 13 , 1, 2069 -- 2106 . B. C. May, N. Korda, A. Lee, and D. S. Leslie. 2012. Optimistic bayesian sampling in contextual-bandit problems. J. Mach. Learn. Res. 13, 1, 2069--2106.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1080\/00107510500052444"},{"key":"e_1_2_2_37_1","volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS'13)","author":"Oord A. V. D.","unstructured":"A. V. D. Oord , S. Dieleman , and B. Schrauwen . 2013. Deep content-based music recommendation . In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS'13) . 2643--2651. A. V. D. Oord, S. Dieleman, and B. Schrauwen. 2013. Deep content-based music recommendation. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS'13). 2643--2651."},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390267"},{"key":"e_1_2_2_39_1","volume-title":"Proceedings of the 20th International Conference on MultiMedia Modeling (MMM'14)","author":"Schedl M.","unstructured":"M. Schedl and D. Schnitzer . 2014. Location-aware music artist recommendation . In Proceedings of the 20th International Conference on MultiMedia Modeling (MMM'14) . M. Schedl and D. Schnitzer. 2014. Location-aware music artist recommendation. In Proceedings of the 20th International Conference on MultiMedia Modeling (MMM'14)."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/1046920.1088715"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2502081.2502233"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339584"},{"key":"e_1_2_2_43_1","volume-title":"Proceedings of the 9th International Symposium on Computer Music Modelling and Retrieval (CMMR'12)","author":"Song Y.","unstructured":"Y. Song , S. Dixon , and M. Pearce . 2012. A survey of music recommendation systems and future perspectives . In Proceedings of the 9th International Symposium on Computer Music Modelling and Retrieval (CMMR'12) . Y. Song, S. Dixon, and M. Pearce. 2012. A survey of music recommendation systems and future perspectives. In Proceedings of the 9th International Symposium on Computer Music Modelling and Retrieval (CMMR'12)."},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/1089551.1089606"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.5555\/551283"},{"key":"e_1_2_2_46_1","series-title":"Synthesis Lectures on Artificial Intelligence and Machine Learning Series","volume-title":"Algorithms for Reinforcement Learning","author":"Szepesv\u00e1ri C.","unstructured":"C. Szepesv\u00e1ri . 2010. Algorithms for Reinforcement Learning . Synthesis Lectures on Artificial Intelligence and Machine Learning Series . Vol. 4 , Morgan and Claypool, San Rafael , CA. C. Szepesv\u00e1ri. 2010. Algorithms for Reinforcement Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning Series. Vol. 4, Morgan and Claypool, San Rafael, CA."},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1363686.1363954"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2393347.2393368"},{"key":"e_1_2_2_49_1","volume-title":"Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR'06)","author":"Yoshii K.","unstructured":"K. Yoshii , M. Goto , K. Komatani , T. Ogata , and H. G. Okuno . 2006. Hybrid collaborative and content-based music recommendation using probabilistic model with latent user preferences . In Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR'06) . 296--301. K. Yoshii, M. Goto, K. Komatani, T. Ogata, and H. G. Okuno. 2006. Hybrid collaborative and content-based music recommendation using probabilistic model with latent user preferences. In Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR'06). 296--301."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1080\/088395101750363993"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2124295.2124300"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772794"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2623372","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2623372","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:19:42Z","timestamp":1750231182000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2623372"}},"subtitle":["A Reinforcement Learning Approach"],"short-title":[],"issued":{"date-parts":[[2014,8]]},"references-count":52,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2014,8]]}},"alternative-id":["10.1145\/2623372"],"URL":"https:\/\/doi.org\/10.1145\/2623372","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,8]]},"assertion":[{"value":"2013-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-09-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}