-
Notifications
You must be signed in to change notification settings - Fork 220
Closed
Description
deepTools uses multiprocessing.cpu_count() to get the number of cpus via deeptools/parserCommon.py.
multiprocessing.cpu_count() return the number of cpu in the machine, But this is not the same as the number of available cpu to the process. For example, you can run in a taskset context or a batch scheduler like slurm.
see:
$ nproc
96
$ taskset -c 1 nproc
1
$ taskset -c 1 python3 -c "import multiprocessing; print(multiprocessing.cpu_count())"
96
I would suggest to use len(os.sched_getaffinity(0)) instead of multiprocessing.cpu_count()
$ python3 -c "import os; print(len(os.sched_getaffinity(0)))"
96
$ taskset -c 1 python3 -c "import os; print(len(os.sched_getaffinity(0)))"
1
NB Mac OSX python does not have os.sched_getaffinity so a portable way to code it would be
try:
num_cpus = len(os.sched_getaffinity(0))
except AttributeError:
num_cpus = multiprocessing.cpu_count()
regards
Eric
Metadata
Metadata
Assignees
Labels
No labels