{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T17:44:36Z","timestamp":1776447876870,"version":"3.51.2"},"reference-count":46,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,4,13]],"date-time":"2022-04-13T00:00:00Z","timestamp":1649808000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>At present, a common drawback of crowd simulation models is that they are mainly simulated in (abstract) 2D environments, which limits the simulation of crowd behaviors observed in real 3D environments. Therefore, we propose a deep reinforcement learning-based model with human-like perceptron and policy for crowd evacuation in 3D environments (HDRLM3D). In HDRLM3D, we propose a vision-like ray perceptron (VLRP) and combine it with a redesigned global (or local) perceptron (GOLP) to form a human-like perception model. We propose a double-branch feature extraction and decision network (DBFED-Net) as the policy, which can extract features and make behavioral decisions. Moreover, we validate our method\u2019s ability to reproduce typical phenomena and behaviors through experiments in two different scenarios. In scenario I, we reproduce the bottleneck effect of crowds and verify the effectiveness and advantages of HDRLM3D by comparing it with real crowd experiments and classical methods in terms of density maps, fundamental diagrams, and evacuation times. In scenario II, we reproduce agents\u2019 navigation and obstacle avoidance behaviors and demonstrate the advantages of HDRLM3D for crowd simulation in unknown 3D environments by comparing it with other deep reinforcement learning-based models in terms of trajectories and numbers of collisions.<\/jats:p>","DOI":"10.3390\/ijgi11040255","type":"journal-article","created":{"date-parts":[[2022,4,13]],"date-time":"2022-04-13T21:33:42Z","timestamp":1649885622000},"page":"255","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["HDRLM3D: A Deep Reinforcement Learning-Based Model with Human-like Perceptron and Policy for Crowd Evacuation in 3D Environments"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2510-9814","authenticated-orcid":false,"given":"Dong","family":"Zhang","sequence":"first","affiliation":[{"name":"National Engineering Research Center for Geoinformatics, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"given":"Wenhang","family":"Li","sequence":"additional","affiliation":[{"name":"National Engineering Research Center for Geoinformatics, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}]},{"given":"Jianhua","family":"Gong","sequence":"additional","affiliation":[{"name":"National Engineering Research Center for Geoinformatics, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Zhejiang-CAS Application Center for Geoinformatics, Jiaxing 314199, China"}]},{"given":"Lin","family":"Huang","sequence":"additional","affiliation":[{"name":"National Engineering Research Center for Geoinformatics, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}]},{"given":"Guoyong","family":"Zhang","sequence":"additional","affiliation":[{"name":"National Engineering Research Center for Geoinformatics, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}]},{"given":"Shen","family":"Shen","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Resources and Environmental Information System, Institute of Geographic Sciences and Natural Resources Research, CAS, Beijing 100101, China"}]},{"given":"Jiantao","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Surveying and Geo-Informatics, Shandong Jianzhu University, Jinan 250101, China"}]},{"given":"Haonan","family":"Ma","sequence":"additional","affiliation":[{"name":"National Engineering Research Center for Geoinformatics, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/s10694-007-0040-6","article-title":"A Post-fire Survey on the Pre-evacuation Human Behavior","volume":"45","author":"Zhao","year":"2008","journal-title":"Fire Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1002\/(SICI)1099-1018(199911\/12)23:6<297::AID-FAM702>3.0.CO;2-2","article-title":"Occupants\u2019 behaviour in response to the high-rise apartments fire in Hiroshima City","volume":"23","author":"Sekizawa","year":"2015","journal-title":"Fire Mater."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"046109","DOI":"10.1103\/PhysRevE.75.046109","article-title":"The Dynamics of Crowd Disasters: An Empirical Study","volume":"75","author":"Helbing","year":"2007","journal-title":"Phys. Rev. E"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"522","DOI":"10.1109\/THMS.2013.2283945","article-title":"Person Tracking in Large Public Spaces Using 3-D Range Sensors","volume":"43","author":"Brscic","year":"2013","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"11947","DOI":"10.1073\/pnas.2031912100","article-title":"Self-organized queuing and scale-free behavior in real escape panic","volume":"100","author":"Saloma","year":"2003","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"022808","DOI":"10.1103\/PhysRevE.91.022808","article-title":"Flow and clogging of a sheep herd passing through a bottleneck","volume":"91","author":"Pastor","year":"2015","journal-title":"Phys. Rev. E"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"032302","DOI":"10.1103\/PhysRevE.94.032302","article-title":"Effect of obstacle position in the flow of sheep through a narrow door","volume":"94","author":"Zuriguel","year":"2016","journal-title":"Phys. Rev. E"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.physa.2017.02.004","article-title":"Empirical study on social groups in pedestrian evacuation dynamics","volume":"475","author":"Schadschneider","year":"2017","journal-title":"Phys. A Stat. Mech. Its Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"033404","DOI":"10.1088\/1742-5468\/aa620d","article-title":"Fundamental diagrams for multidirectional pedestrian flows","volume":"2017","author":"Cao","year":"2017","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.ssci.2018.03.015","article-title":"Exit choice in an emergency evacuation scenario is influenced by exit familiarity and neighbor behavior","volume":"106","author":"Kinateder","year":"2018","journal-title":"Saf. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Huang, L., Gong, J., and Li, W. (2021). A Perception Model for Optimizing and Evaluating Evacuation Guidance Systems. ISPRS Int. J. Geo-Inf., 10.","DOI":"10.3390\/ijgi10020054"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1016\/j.buildenv.2008.04.002","article-title":"Modeling crowd evacuation of a building based on seven methodological approaches","volume":"44","author":"Zheng","year":"2009","journal-title":"Build. Environ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1038\/229381a0","article-title":"The Statistics of Crowd Fluids","volume":"229","author":"Henderson","year":"1971","journal-title":"Nature"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"37","DOI":"10.3141\/1646-05","article-title":"Effect of pedestrians on capacity of signalized intersections","volume":"1646","author":"Milazzo","year":"1998","journal-title":"Transp. Res. Rec."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1007\/s11067-005-2629-y","article-title":"Pedestrian Travel Behavior Modeling","volume":"5","author":"Hoogendoorn","year":"2005","journal-title":"Netw. Spat. Econ."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1016\/0191-2615(94)90013-2","article-title":"Modeling and simulation of pedestrian traffic flow","volume":"28","year":"1994","journal-title":"Transp. Res. Part B Methodol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1016\/j.physa.2007.04.006","article-title":"Cellular automaton model for evacuation process with obstacles","volume":"382","author":"Varas","year":"2007","journal-title":"Phys. A Stat. Mech. Its Appl."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1016\/S0378-4371(00)00630-0","article-title":"Scaling behavior of crowd flow outside a hall","volume":"292","author":"Tajima","year":"2001","journal-title":"Phys. A Stat. Mech. Its Appl."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4282","DOI":"10.1103\/PhysRevE.51.4282","article-title":"Social Force Model for Pedestrian Dynamics","volume":"51","author":"Helbing","year":"1995","journal-title":"Phys. Rev. E"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1016\/j.tics.2005.07.009","article-title":"Computational models of collective behavior","volume":"9","author":"Goldstone","year":"2005","journal-title":"Trends Cogn. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Torrey, L. (2010, January 11\u201313). Crowd Simulation Via Multi-Agent Reinforcement Learning. Proceedings of the Sixth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA.","DOI":"10.1609\/aiide.v6i1.12390"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/j.simpat.2014.06.005","article-title":"MARL-Ped: A multi-agent reinforcement learning based framework to simulate pedestrian groups","volume":"47","author":"Lozano","year":"2014","journal-title":"Simul. Model. Pract. Theory"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1016\/j.simpat.2017.03.003","article-title":"Emergent behaviors and scalability for multi-agent reinforcement learning-based pedestrian models","volume":"74","author":"Lozano","year":"2017","journal-title":"Simul. Model. Pract. Theory"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lee, J., and Won, J. (2018, January 8\u201310). Crowd simulation by deep reinforcement learning. Proceedings of the MIG \u201818: Motion, Interaction and Games, Limassol, Cyprus.","DOI":"10.1145\/3274247.3274510"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"109544","DOI":"10.1109\/ACCESS.2019.2933492","article-title":"Crowd Navigation in an Unknown and Dynamic Environment Based on Deep Reinforcement Learning","volume":"7","author":"Sun","year":"2019","journal-title":"IEEE Access"},{"key":"ref_26","unstructured":"Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., and Mordatch, I. (2019). Emergent tool use from multi-agent autocurricula. arXiv."},{"key":"ref_27","unstructured":"Juliani, A., Berges, V.-P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., and Mattar, M. (2018). Unity: A general platform for intelligent agents. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"809","DOI":"10.3934\/krm.2013.6.809","article-title":"Vision-based macroscopic pedestrian models","volume":"6","author":"Degond","year":"2013","journal-title":"Kinet. Relat. Models"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1273","DOI":"10.3934\/krm.2019049","article-title":"A kinetic theory approach to model pedestrian dynamics in bounded domains with obstacles","volume":"12","author":"Kim","year":"2019","journal-title":"Kinet. Relat. Models"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1893","DOI":"10.1142\/S0218202520400126","article-title":"Coupling kinetic theory approaches for pedestrian dynamics and disease contagion in a confined environment","volume":"30","author":"Kim","year":"2020","journal-title":"Math. Models Methods Appl. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1142\/S0218202520500013","article-title":"A unified multiscale vision of behavioral crowds","volume":"30","author":"Aylaj","year":"2020","journal-title":"Math. Models Methods Appl. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1142\/S0218202522500087","article-title":"Towards a mathematical theory of behavioral human crowds","volume":"32","author":"Bellomo","year":"2022","journal-title":"Math. Models Methods Appl. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"107621","DOI":"10.1016\/j.cie.2021.107621","article-title":"Deep deterministic policy gradient algorithm for crowd-evacuation path planning","volume":"161","author":"Li","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1016\/j.neucom.2019.08.021","article-title":"Data-driven crowd evacuation: A reinforcement learning method","volume":"366","author":"Yao","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Huang, L., Gong, J., Li, W., Xu, T., Shen, S., Liang, J., Feng, Q., Zhang, D., and Sun, J. (2018). Social Force Model-Based Group Behavior Simulation in Virtual Geographic Environments. ISPRS Int. J. Geo-Inf., 7.","DOI":"10.3390\/ijgi7020079"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"73841","DOI":"10.1109\/ACCESS.2019.2920913","article-title":"Improved multi-agent reinforcement learning for path planning-based crowd simulation","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_37","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_39","unstructured":"Adrian, J., Boltes, M., Holl, S., Sieben, A., and Seyfried, A. (2018, January 21\u201324). Crowding and Queuing in Entrance Scenarios: Influence of Corridor Width in Front of Bottlenecks. Proceedings of the 9th International Conference on Pedestrian and Evacuation Dynamics (PED2018), Lund, Sweden."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Berg, J.v.d., Guy, S.J., Lin, M., and Manocha, D. (2011). Reciprocal n-body collision avoidance. Robotics Research, Springer.","DOI":"10.1007\/978-3-642-19457-3_1"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"P10002","DOI":"10.1088\/1742-5468\/2005\/10\/P10002","article-title":"The fundamental diagram of pedestrian movement revisited","volume":"2005","author":"Seyfried","year":"2005","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Seyfried, A., Boltes, M., K\u00e4hler, J., Klingsch, W., Portz, A., Rupprecht, T., Schadschneider, A., Steffen, B., and Winkens, A. (2010). Enhanced empirical data for the fundamental diagram and the flow through bottlenecks. Pedestr. Evacuation Dyn. 2008, 145\u2013156.","DOI":"10.1007\/978-3-642-04504-2_11"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.simpat.2018.02.007","article-title":"A social force evacuation model driven by video data","volume":"84","author":"Liu","year":"2018","journal-title":"Simul. Model. Pract. Theory"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"103631","DOI":"10.1016\/j.robot.2020.103631","article-title":"C-Nav: Distributed coordination in crowded multi-agent navigation","volume":"133","author":"Godoy","year":"2020","journal-title":"Robot. Auton. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"104994","DOI":"10.1016\/j.envsoft.2021.104994","article-title":"A rapid 3D reproduction system of dam-break floods constrained by post-disaster information","volume":"139","author":"Li","year":"2021","journal-title":"Environ. Model. Softw."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1521","DOI":"10.1080\/13658816.2020.1833016","article-title":"An augmented representation method of debris flow scenes to improve public perception","volume":"35","author":"Li","year":"2021","journal-title":"Int. J. Geogr. Inf. Sci."}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/11\/4\/255\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:53:38Z","timestamp":1760136818000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/11\/4\/255"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,13]]},"references-count":46,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["ijgi11040255"],"URL":"https:\/\/doi.org\/10.3390\/ijgi11040255","relation":{},"ISSN":["2220-9964"],"issn-type":[{"value":"2220-9964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,13]]}}}