Repository navigation
Split the omp model into omp-for and omp-target models to avoid making extra copies of buffers - #43
nicolas-bouton wants to merge 3 commits into
Conversation
Remove occurences of `#pragma omp parallel for`. And doesn't change the behavior of the cmake, same variables need to be passed to enable offloading. * CMakeLists.txt, src/main.cpp, src/omp-target/fasten.hpp, src/omp-target/model.cmake: Here. Signed-off-by: Nicolas Bouton <nicolas.bouton@sipearl.com>
Provide a revised version of the `#pragma omp parallel for` implementation that avoids making extra copies of buffers (like the std-ranges model), reducing memory overhead and improving performance. * CMakeLists.txt, src/main.cpp, src/omp-for/fasten.hpp, src/omp-for/model.cmake: Here. Signed-off-by: Nicolas Bouton <nicolas.bouton@sipearl.com>
* src/kokkos/model.cmake: Here. Signed-off-by: Nicolas Bouton <nicolas.bouton@sipearl.com>
|
Thanks for the PR. The copies are intentional for first-touch NUMA placement and occur outside the timed kernel (that said, the data movement is really small relative to the computation). Splitting the model would also duplicate the kernel, so I’d prefer to keep the current implementation. I've address the Kokkos issue separately in 40254ff |
|
[EDITED] Hello @tom91136, Thank you for reviewing the PR and for addressing the Kokkos issue separately in 40254ff. I would appreciate it if you could reconsider this PR for the following reasons. I understand that the copies in the current The additional copies discussed in this PR are at line 219 which are in fact not deep copies. And the intentional copies are effectively outside of the timed kernel as you pointed out. However, on AArch64-based CPUs, my PR resulted in performance improvements of approximately 10%–20%. More broadly, the PR attempts to align the implementations with the Regarding kernel duplication, I understand the concern. The goal is not only to separate the Finally, I noticed that the Kokkos-related changes in 40254ff appear to overlap with the work included in this PR. Since these changes originated from this contribution, could you please credit me in the commit message with the Co-authored-by trailer? Would you be willing to reconsider the PR, or suggest specific changes that would address these concerns? |
|
Hi, so the Kokkos change (40254ff) does not in fact originate from your contribution, it was a direct copy of https://github.com/UoB-HPC/CloverLeaf/blob/9e67e5bcf264a259e7a873173da7f413fb44a146/kokkos/model.cmake that I made in 2023, you can check the diff yourself. Can you elaborate on where the 10~20% came from in your change, and also which compiler on which hardware and at what PPWI do you observe this uplift? |
|
Understood. miniBUDE has been compiled and executed on aarch64 based CPUs (namely ARM Neoverse N1, ARM Neoverse V1, ARM Neoverse V2) with their appropriate architecture flags with The testcase used is the following: Performance divergence occurs at high number of threads (between 40 to max cores). Unfortunatly, after retesting it, the performance gain is about 5% which is lower than the performance gain claimed in my PR message. |
This MR turns the omp model into 2 models, omp-for and omp-target, avoids making extra copies of buffers for the omp-for model based on the std-range model, whereas the omp-target model is still making extra copies of buffers for the device.
The code from the omp model was used as the code for the omp-target model, since the kernel for this model was not modified, so it is clear in the git diff that the code for this model is not touched.
The Kokkos cmake file was also modified to let the user use an already compiled Kokkos library.