{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T22:51:43Z","timestamp":1776811903494,"version":"3.51.2"},"reference-count":30,"publisher":"European Society of Computational Methods in Sciences and Engineering","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"published-print":{"date-parts":[[2024,6,17]]},"abstract":"<jats:p>This paper examines the use of an Uninterruptible Power Supply (UPS) to enhance the operational efficiency of data centers. It focuses on developing an optimal energy scheduling strategy for a data center equipped with UPS, using the Markov decision process (MDP) framework. The MDP framework simulates the decision-making process involved in minimizing energy costs. Each unit\u2019s available power output in the data center is treated as a Markov state, taking into account the uncertainty associated with renewable distributed generation. This uncertainty drives the system to transition to other Markov states in subsequent decision times. A recursive optimization model is established for each Markov state at each decision time to guide state-based operations, which includes determining the unit output while considering both current and future costs. The challenge of dealing with high dimensionality, arising from a substantial number of states and actions in the model, is effectively addressed by adopting an approximate dynamic programming (ADP) method. This approach incorporates decision-state and forwards dynamic algorithms to tackle the complexity of the MDP-based model. By employing ADP, the computational burden is reduced, enabling efficient and practical solutions to be obtained.<\/jats:p>","DOI":"10.3233\/jcm-247149","type":"journal-article","created":{"date-parts":[[2024,6,18]],"date-time":"2024-06-18T12:12:11Z","timestamp":1718712731000},"page":"1317-1329","source":"Crossref","is-referenced-by-count":1,"title":["Markov decision process framework of optimal energy dispatch in a smart data center with uninterruptible power supplies"],"prefix":"10.66113","volume":"24","author":[{"given":"Haifeng","family":"Cen","sequence":"first","affiliation":[{"name":"Guangzhou Power Supply, GuangDong Power Grid Co., Ltd., Guangzhou, China"}]},{"given":"Yuan","family":"Xu","sequence":"additional","affiliation":[{"name":"Guangzhou Power Supply, GuangDong Power Grid Co., Ltd., Guangzhou, China"}]},{"given":"Kaiyuan","family":"Sun","sequence":"additional","affiliation":[{"name":"Guangzhou Power Supply, GuangDong Power Grid Co., Ltd., Guangzhou, China"}]},{"given":"Hao","family":"Tian","sequence":"additional","affiliation":[{"name":"State Key Lab of Control and Simulation of Power Systems and Generation Equipments, Department of Electrical Engineering, Tsinghua University, Haidian District Beijing, China"},{"name":"WuXi University, Xishan District Wuxi, China"},{"name":"La Consolacion University Philippines, Malolos, Philippines"}]},{"given":"Kun","family":"Chen","sequence":"additional","affiliation":[{"name":"Guangzhou Power Supply, GuangDong Power Grid Co., Ltd., Guangzhou, China"}]},{"given":"Lin","family":"Lin","sequence":"additional","affiliation":[{"name":"Guangzhou Power Supply, GuangDong Power Grid Co., Ltd., Guangzhou, China"}]}],"member":"55691","reference":[{"issue":"22","key":"10.3233\/JCM-247149_ref1","first-page":"12","article-title":"Towards a European renewable-based energy system enabled by smart grid: status and prospects","volume":"35","author":"Ding","year":"2011","journal-title":"Dianli Xitong Zidonghua\/Automation of Electric Power Systems."},{"key":"10.3233\/JCM-247149_ref2","doi-asserted-by":"crossref","first-page":"75568","DOI":"10.1109\/ACCESS.2021.3081430","article-title":"Future Greener Seaports: A Review of New Infrastructure, Challenges, and Energy Efficiency Measures","volume":"9","author":"Sadiq","year":"2021","journal-title":"IEEE Access."},{"issue":"3","key":"10.3233\/JCM-247149_ref3","doi-asserted-by":"crossref","first-page":"2198","DOI":"10.1109\/TIA.2018.2890789","article-title":"Integrated Stochastic Energy Management for Data Center Microgrid Considering Waste Heat Recovery","volume":"55","author":"Ding","year":"2019","journal-title":"IEEE Transactions on Industry Applications."},{"issue":"3","key":"10.3233\/JCM-247149_ref4","doi-asserted-by":"crossref","first-page":"2739","DOI":"10.1109\/JSYST.2016.2642218","article-title":"A hierarchical modeling for reactive power optimization with joint transmission and distribution networks by curve fitting","volume":"12","author":"Ding","year":"2018","journal-title":"IEEE Systems Journal."},{"issue":"2","key":"10.3233\/JCM-247149_ref5","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1109\/TSUSC.2017.2711925","article-title":"Carbon-aware electricity cost minimization for sustainable data centers","volume":"2","author":"Dou","year":"2017","journal-title":"IEEE Transactions on Sustainable Computing."},{"key":"10.3233\/JCM-247149_ref6","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/CCGrid.2016.101","article-title":"DiBA: Distributed Power Budget Allocation for Large-Scale Computing Clusters","author":"Badiei","year":"2016","journal-title":"16th IEEE\/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2016"},{"issue":"2","key":"10.3233\/JCM-247149_ref7","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1109\/TSG.2014.2363583","article-title":"Modeling demand response capability by internet data centers processing batch computing jobs","volume":"6","author":"Li","year":"2015","journal-title":"IEEE Transactions on Smart Grid."},{"key":"10.3233\/JCM-247149_ref8","first-page":"369","article-title":"Utilizing landfill gas (LFG) to electrify digital data centers in China for accelerating energy transition in Industry 40 era","author":"Liang","year":"2022","journal-title":"Journal of Cleaner Production."},{"issue":"2","key":"10.3233\/JCM-247149_ref9","doi-asserted-by":"crossref","first-page":"1017","DOI":"10.1109\/TSG.2013.2237929","article-title":"Energy and performance management of green data centers: A profit maximization approach","volume":"4","author":"Ghamkhari","year":"2013","journal-title":"IEEE Transactions on Smart Grid."},{"key":"10.3233\/JCM-247149_ref10","doi-asserted-by":"crossref","unstructured":"Ghamkhari M, Mohsenian-Rad H. Optimal integration of renewable energy resources in data centers with behind-the-meter renewable generator. IEEE International Conference on Communications. 2012; 3340-4.","DOI":"10.1109\/ICC.2012.6364475"},{"key":"10.3233\/JCM-247149_ref11","doi-asserted-by":"crossref","unstructured":"Yu L, Jiang T, Cao Y, Wu J. Risk-constrained operation for internet data centers under smart grid environment. In: 2013 International Conference on Wireless Communications and Signal Processing. IEEE; 2013. pp. 1-6.","DOI":"10.1109\/WCSP.2013.6677138"},{"key":"10.3233\/JCM-247149_ref12","doi-asserted-by":"crossref","unstructured":"Dou H, Qi Y, Wei W, Song H. Minimizing Electricity Bills for Geographically Distributed Data Centers with Renewable and Cooling Aware Load Balancing. Proceedings \u2013 2015 International Conference on Identification, Information, and Knowledge in the Internet of Things, IIKI 2015. 2016; 210-4.","DOI":"10.1109\/IIKI.2015.52"},{"key":"10.3233\/JCM-247149_ref13","doi-asserted-by":"crossref","unstructured":"Chen T, Wang X, Giannakis GB. Cooling-Aware Energy and Workload Management in Data Centers via Stochastic Optimization. IEEE Journal on Selected Topics in Signal Processing. 2016; 10(2): 402-15.","DOI":"10.1109\/JSTSP.2015.2500189"},{"issue":"8","key":"10.3233\/JCM-247149_ref14","doi-asserted-by":"crossref","first-page":"2030","DOI":"10.1109\/TPDS.2013.278","article-title":"Energy and network aware workload management for sustainable data centers with thermal storage","volume":"25","author":"Guo","year":"2014","journal-title":"IEEE Transactions on Parallel and Distributed Systems."},{"issue":"3","key":"10.3233\/JCM-247149_ref15","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1109\/TPDS.2013.69","article-title":"Temporal load balancing with service delay guarantees for data center energy cost optimization","volume":"25","author":"Lo","year":"2014","journal-title":"IEEE Transactions on Parallel and Distributed Systems."},{"key":"10.3233\/JCM-247149_ref16","doi-asserted-by":"crossref","first-page":"116120","DOI":"10.1016\/j.energy.2019.116120","article-title":"Predictive energy-saving optimization based on nonlinear model predictive control for cooperative connected vehicles platoon with V2V communication","volume":"189","author":"Ma","year":"2019","journal-title":"Energy."},{"issue":"4","key":"10.3233\/JCM-247149_ref17","doi-asserted-by":"crossref","first-page":"1969","DOI":"10.1109\/TSG.2014.2309961","article-title":"Rapid-charging navigation of electric vehicles based on real-time power systems and traffic data","volume":"5","author":"Guo","year":"2014","journal-title":"IEEE Transactions on Smart Grid."},{"issue":"2","key":"10.3233\/JCM-247149_ref18","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1109\/TIV.2020.3012947","article-title":"Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control","volume":"6","author":"Lin","year":"2021","journal-title":"IEEE Transactions on Intelligent Vehicles."},{"issue":"5","key":"10.3233\/JCM-247149_ref19","doi-asserted-by":"crossref","first-page":"1306","DOI":"10.1109\/TPDS.2013.2297095","article-title":"Risk-constrained operation for internet data centers in deregulated electricity markets","volume":"25","author":"Yu","year":"2014","journal-title":"IEEE Transactions on Parallel and Distributed Systems."},{"issue":"3","key":"10.3233\/JCM-247149_ref20","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1049\/pel2.12050","article-title":"Optimal switching sequence model predictive control for three-level NPC grid-connected inverters","volume":"14","author":"Lyu","year":"2021","journal-title":"IET Power Electronics."},{"issue":"10","key":"10.3233\/JCM-247149_ref21","doi-asserted-by":"crossref","first-page":"10799","DOI":"10.1109\/TPEL.2020.2981914","article-title":"Robust Predictive Torque Control of N3-Phase PMSM for High-Power Traction Application","volume":"35","author":"Wu","year":"2020","journal-title":"IEEE Transactions on Power Electronics."},{"key":"10.3233\/JCM-247149_ref22","doi-asserted-by":"crossref","first-page":"118460","DOI":"10.1016\/j.apenergy.2021.118460","article-title":"Energy efficient speed planning of electric vehicles for car-following scenario using model-based reinforcement learning","volume":"313","author":"Lee","year":"2022","journal-title":"Applied Energy."},{"issue":"3","key":"10.3233\/JCM-247149_ref23","doi-asserted-by":"crossref","first-page":"1896","DOI":"10.3390\/en8031896","article-title":"Hierarchical load tracking control of a grid-connected solid oxide fuel cell for maximum electrical efficiency operation","volume":"8","author":"Li","year":"2015","journal-title":"Energies."},{"issue":"13","key":"10.3233\/JCM-247149_ref24","doi-asserted-by":"crossref","first-page":"1486","DOI":"10.1049\/iet-rpg.2017.0871","article-title":"Bi-level decentralised active power control for large-scale wind farm cluster","volume":"12","author":"Huang","year":"2018","journal-title":"IET Renewable Power Generation."},{"issue":"7","key":"10.3233\/JCM-247149_ref25","doi-asserted-by":"crossref","first-page":"3076","DOI":"10.1109\/TII.2017.2771487","article-title":"A Robust Load Frequency Control Scheme for Power Systems Based on Second-Order Sliding Mode and Extended Disturbance Observer","volume":"14","author":"Liao","year":"2018","journal-title":"IEEE Transactions on Industrial Informatics."},{"key":"10.3233\/JCM-247149_ref26","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.ijepes.2017.05.025","article-title":"Predictive control of wind turbine for load reduction during ramping events","volume":"93","author":"Liu","year":"2017","journal-title":"International Journal of Electrical Power and Energy Systems."},{"key":"10.3233\/JCM-247149_ref27","doi-asserted-by":"crossref","first-page":"124732","DOI":"10.1016\/j.energy.2022.124732","article-title":"Energy-oriented car-following control for a front- and rear-independent-drive electric vehicle platoon","volume":"257","author":"Guo","year":"2022","journal-title":"Energy."},{"issue":"3","key":"10.3233\/JCM-247149_ref28","first-page":"92","article-title":"Multi-time-scale Slack Optimal Control in Distribution Network Based on Voltage Optimization for Point of Common Coupling of PV","volume":"43","author":"Huang","year":"2019","journal-title":"Dianli Xitong Zidonghua\/Automation of Electric Power Systems."},{"key":"10.3233\/JCM-247149_ref29","doi-asserted-by":"crossref","first-page":"119874","DOI":"10.1016\/j.apenergy.2022.119874","article-title":"Stochastic co-optimization of speed planning and powertrain control with dynamic probabilistic constraints for safe and ecological driving","volume":"325","author":"Sun","year":"2022","journal-title":"Applied Energy."},{"issue":"4","key":"10.3233\/JCM-247149_ref30","doi-asserted-by":"crossref","first-page":"2152","DOI":"10.1109\/TSTE.2019.2952122","article-title":"Distributed Optimal Voltage Control for VSC-HVDC Connected Large-Scale Wind Farm Cluster Based on Analytical Target Cascading Method","volume":"11","author":"Huang","year":"2020","journal-title":"IEEE Transactions on Sustainable Energy."}],"container-title":["Journal of Computational Methods in Sciences and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/JCM-247149","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T22:07:28Z","timestamp":1776809248000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/JCM-247149"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,17]]},"references-count":30,"journal-issue":{"issue":"3"},"URL":"https:\/\/doi.org\/10.3233\/jcm-247149","relation":{},"ISSN":["1472-7978","1875-8983"],"issn-type":[{"value":"1472-7978","type":"print"},{"value":"1875-8983","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,17]]}}}