Oliver Liu

Due to the popularity and importance of Parallel File Systems (PFSs) in modern High Performance Computing (HPC) centers, PFS designs and I/O optimizations are active research topics. However, the research process is often time-consuming and faces cost and complexity challenges in deploying experiments in real HPC systems. This paper describes PFSsim, a trace-driven simulator of distributed storage systems that allows the evaluation of PFS designs, I/O schedulers, network structures, and… Show more

Due to the popularity and importance of Parallel File Systems (PFSs) in modern High Performance Computing (HPC) centers, PFS designs and I/O optimizations are active research topics. However, the research process is often time-consuming and faces cost and complexity challenges in deploying experiments in real HPC systems. This paper describes PFSsim, a trace-driven simulator of distributed storage systems that allows the evaluation of PFS designs, I/O schedulers, network structures, and workloads. PFSsim differentiates itself from related work in that it provides a powerful platform featuring a modular design with high flexibility in the modeling of subsystems including the network, clients, data servers and I/O schedulers. It does so by designing the simulator to capture abstractions found in common PFSs. PFSsim also exposes script-based interfaces for detailed configurations. Experiments and validation against real systems considering sub-modules and the entire simulator show that PFSsim is capable of simulating a representative PFS (PVFS2) and of modeling different I/O scheduler algorithms with good fidelity. In addition, the simulation speed is also shown to be acceptable. Show less

Many high-end computing (HEC) centers and commercial data centers adopt parallel file systems (PFSs) as their storage solutions. As the number of applications concurrently accessing a PFS grows in both quantity and variety, it is expected that scheduling algorithms for data access will play an increasingly important role in PFS service quality. However, it is costly and disruptive to thoroughly research scheduling mechanisms in deployed peta- or exascale systems, compounded by the complexity in… Show more

Many high-end computing (HEC) centers and commercial data centers adopt parallel file systems (PFSs) as their storage solutions. As the number of applications concurrently accessing a PFS grows in both quantity and variety, it is expected that scheduling algorithms for data access will play an increasingly important role in PFS service quality. However, it is costly and disruptive to thoroughly research scheduling mechanisms in deployed peta- or exascale systems, compounded by the complexity in scheduling policy implementation and experimental data gathering. While a few parallel file system simulation frameworks have been proposed, their goals have not been in the scheduling algorithm evaluation. In this paper, we propose PFSsim, a simulator designed for the purpose of evaluating I/O scheduling algorithms in PFS. PFSsim is a trace-driven simulator based on the network simulation framework OMNeT++ and the disk system simulator DiskSim. A flexible scheduler module is provided for scheduling algorithm deployment, and the system characteristics are highly configurable. We have simulated PVFS2 on PFSsim, and the experimental results show that PFSsim is capable of simulating the system characteristics and showing the performance of the scheduling algorithms. Show less

Many high-end computing (HEC) centers and commercial data centers adopt parallel file systems (PFSs) as their storage solutions. As the number of applications concurrently accessing a PFS grows in both quantity and variety, it is expected that scheduling algorithms for data access will play an increasingly important role in PFS service quality. However, it is costly and disruptive to thoroughly research scheduling mechanisms in deployed peta- or exascale systems, compounded by the complexity in… Show more

Many high-end computing (HEC) centers and commercial data centers adopt parallel file systems (PFSs) as their storage solutions. As the number of applications concurrently accessing a PFS grows in both quantity and variety, it is expected that scheduling algorithms for data access will play an increasingly important role in PFS service quality. However, it is costly and disruptive to thoroughly research scheduling mechanisms in deployed peta- or exascale systems, compounded by the complexity in scheduling policy implementation and experimental data gathering. While a few parallel file system simulation frameworks have been proposed, their goals have not been in the scheduling algorithm evaluation. In this paper, we propose PFSsim, a simulator designed for the purpose of evaluating I/O scheduling algorithms in PFS. PFSsim is a trace-driven simulator based on the network simulation framework OMNeT++ and the disk system simulator DiskSim. A flexible scheduler module is provided for scheduling algorithm deployment, and the system characteristics are highly configurable. We have simulated PVFS2 on PFSsim, and the experimental results show that PFSsim is capable of simulating the system characteristics and showing the performance of the scheduling algorithms. Show less