Skip to content

Fix accelerate launch --cpu never setting the KMP variables - #4222

Merged
SunMarc merged 3 commits into
huggingface:mainfrom
yupengtang:fix-cpu-launch-kmp-env
Sep 8, 2026
Merged

SunMarc merged 3 commits into
huggingface:mainfrom
yupengtang:fix-cpu-launch-kmp-env

Conversation

@yupengtang

Copy link
Copy Markdown
Contributor

What does this PR do?

accelerate launch --cpu no longer sets KMP_AFFINITY and KMP_BLOCKTIME.

#3912 gated those two on CPU mode with:

if parse_flag_from_env(current_env["ACCELERATE_USE_CPU"], False):

parse_flag_from_env takes an environment variable name and does os.environ.get(key, str(default)). Here it is handed the value, the string "True" or "False", so it looks for an environment variable literally called True, does not find one, and returns the default. The guard is False on every launch.

Before #3912 the line was if current_env["ACCELERATE_USE_CPU"]:, which was always True because "False" is a non-empty string, so the variables were set on GPU launches too. That was the bug reported in #3911. The fix replaced one constant with the other.

On main, with prepare_simple_launcher_cmd_env:

accelerate launch --cpu      ACCELERATE_USE_CPU='True'   KMP_AFFINITY=None  KMP_BLOCKTIME=None
accelerate launch (no flag)  ACCELERATE_USE_CPU='False'  KMP_AFFINITY=None  KMP_BLOCKTIME=None

and the mechanism, which is easy to confirm directly:

>>> parse_flag_from_env("True", False)
False
>>> os.environ["True"] = "yes"       # an env var actually named "True"
>>> parse_flag_from_env("True", False)
True

With this PR:

accelerate launch --cpu      ACCELERATE_USE_CPU='True'   KMP_AFFINITY='granularity=fine,compact,1,0'  KMP_BLOCKTIME='1'
accelerate launch (no flag)  ACCELERATE_USE_CPU='False'  KMP_AFFINITY=None                            KMP_BLOCKTIME=None

The flag is now tested the same way the line two above it already tests it (current_env["ACCELERATE_USE_CPU"] = str(args.cpu or args.use_cpu)), and the rest of the launch code does the same thing (commands/launch.py uses args.use_cpu and args.cpu directly throughout). The import that #3912 added for the broken call has no other user in the file, so it goes with it.

Testing

tests/test_cli.py::AccelerateLauncherTester::test_cpu_launch_sets_kmp_env covers both directions, so neither the original #3911 behaviour nor this one can come back. It fails on main with KeyError: 'KMP_AFFINITY' and passes here.

  • pytest tests/test_cli.py: 37 passed, 3 skipped, 1 failed. The failure is test_accelerate_test, which fails identically on main on this machine with FileNotFoundError: 'accelerate-launch', since I run from source rather than an installed console script.
  • ruff check . and ruff format --check . with the 0.13.1 pinned in setup.py: clean for these files. ruff check also reports one unrelated F821 in commands/env.py on main, which Fix accelerate env never reading the MPS chip name聽#4221 fixes.

Ran on Python 3.13, torch 2.11.0, Linux, CPU.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline, Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Who can review?

@SunMarc

huggingface#3912 gated KMP_AFFINITY and KMP_BLOCKTIME on CPU mode with:

    parse_flag_from_env(current_env["ACCELERATE_USE_CPU"], False)

`parse_flag_from_env` takes an environment variable *name*, not a value, and
does `os.environ.get(key, str(default))`. It is handed the string "True" or
"False" here, looks for an environment variable by that name, does not find
one and returns the default. The guard is therefore always False, so the two
variables are never set, including for `accelerate launch --cpu`.

Before huggingface#3912 the condition was `if current_env["ACCELERATE_USE_CPU"]:`, which
was always True because "False" is a non-empty string, so KMP_AFFINITY was set
on GPU launches too. That was the bug huggingface#3912 set out to fix; it swapped one
constant for the other.

Test the flag itself, the way the line above it already does, and drop the
import that was added only for the broken call.

@SunMarc SunMarc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the correct fix

Comment thread src/accelerate/utils/launch.py Outdated
Comment thread src/accelerate/utils/launch.py
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@SunMarc
SunMarc merged commit d8544dd into huggingface:main Sep 8, 2026
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants