Repository navigation
What is the most efficient way to use CPUs in many different partitions? #7317
Unanswered
IncubatorShokuhou
asked this question in
General
Replies: 1 comment
|
I'll try to answer although I am out of my depth so people should feel free to correct me. Dask distributed is a general interface for interacting with dask clusters. Libraries like dask-jobqueue and dask-kubernetes implement different types of clusters. I think that people tend to use dask-jobqueue to create clusters when running on an HPC system. Then those clusters are passed to distributed. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
As far as I can see, there are several different ways to use multi CPUs in dask framework, e.g. dask-distributed, dask-mpi, dask-jobqueue ,etc.
So I am wondering which is the most efficient way when I have several CPU nodes on different partitions on a HPC， or that they are actually no difference?
All reactions