Update dynamic env handling to preserve None when USE_DYNAMIC is unset#3567
Merged
Conversation
Contributor
Author
|
@IlyasMoutawwakil can you check this PR? |
Contributor
Author
|
You can try the change by a simple test and also observe the old behavior as it is import unittest
import os
from dataclasses import dataclass, field
from unittest.mock import patch
import copy
def str_to_bool(value):
if value.lower() in ("y", "yes", "t", "true", "on", "1"):
return 1
class KwargsHandler:
def to_dict(self):
return copy.deepcopy(self.__dict__)
def to_kwargs(self):
default_dict = self.__class__().to_dict()
this_dict = self.to_dict()
return {k: v for k, v in this_dict.items() if default_dict[k] != v}
@dataclass
class TorchDynamoPluginCurrent(KwargsHandler):
dynamic: bool = field(default=None, metadata={"help": "Whether to use dynamic shape"})
def __post_init__(self):
prefix = "ACCELERATE_DYNAMO_"
if self.dynamic is None:
self.dynamic = str_to_bool(os.environ.get(prefix + "USE_DYNAMIC", "False")) == 1
def to_dict(self):
dynamo_config = copy.deepcopy(self.__dict__)
return dynamo_config
def to_kwargs(self):
kwargs = super().to_kwargs()
return kwargs
@dataclass
class TorchDynamoPluginModified(KwargsHandler):
dynamic: bool = field(default=None, metadata={"help": "Whether to use dynamic shape"})
def __post_init__(self):
prefix = "ACCELERATE_DYNAMO_"
if self.dynamic is None:
env_value = os.environ.get(prefix + "USE_DYNAMIC")
self.dynamic = (str_to_bool(env_value) == 1) if env_value is not None else None
def to_dict(self):
dynamo_config = copy.deepcopy(self.__dict__)
return dynamo_config
def to_kwargs(self):
kwargs = super().to_kwargs()
return kwargs
class TestTorchDynamoPlugin(unittest.TestCase):
def test_dynamic_current_implementation(self):
"""Test the current implementation of TorchDynamoPlugin.__post_init__"""
# Test case 1: Environment variable set to True
with patch.dict(os.environ, {"ACCELERATE_DYNAMO_USE_DYNAMIC": "True"}):
plugin = TorchDynamoPluginCurrent()
self.assertTrue(plugin.dynamic, "Expected dynamic to be True when env is 'True'")
# Test case 2: Environment variable set to False
with patch.dict(os.environ, {"ACCELERATE_DYNAMO_USE_DYNAMIC": "False"}):
plugin = TorchDynamoPluginCurrent()
self.assertFalse(plugin.dynamic, "Expected dynamic to be False when env is 'False'")
# Test case 3: Environment variable unset (should be None, but current fails)
with patch.dict(os.environ, {}, clear=True):
plugin = TorchDynamoPluginCurrent()
self.assertFalse(plugin.dynamic, "Current implementation incorrectly sets dynamic to False when env is unset")
def test_dynamic_modified_implementation(self):
"""Test the modified implementation of TorchDynamoPlugin.__post_init__"""
# Test case 1: Environment variable set to True
with patch.dict(os.environ, {"ACCELERATE_DYNAMO_USE_DYNAMIC": "True"}):
plugin = TorchDynamoPluginModified()
self.assertTrue(plugin.dynamic, "Expected dynamic to be True when env is 'True'")
# Test case 2: Environment variable set to False
with patch.dict(os.environ, {"ACCELERATE_DYNAMO_USE_DYNAMIC": "False"}):
plugin = TorchDynamoPluginModified()
self.assertFalse(plugin.dynamic, "Expected dynamic to be False when env is 'False'")
# Test case 3: Environment variable unset (should be None)
with patch.dict(os.environ, {}, clear=True):
plugin = TorchDynamoPluginModified()
self.assertIsNone(plugin.dynamic, "Expected dynamic to be None when env is unset")
if __name__ == "__main__":
unittest.main() |
3 tasks
IlyasMoutawwakil
approved these changes
May 14, 2025
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
5 tasks
SunMarc
approved these changes
May 14, 2025
SunMarc
left a comment
Member
There was a problem hiding this comment.
Indeed thanks for spotting this. Evenutually, we also need to fix for USE_DYNAMIC also to align the behavior
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Modified the logic for setting
self.dynamicto explicitly preserveNonewhen theUSE_DYNAMICenvironment variable is not set, aligning with the behavior described in the PyTorch documentation for torch.compile (https://docs.pytorch.org/stable/generated/torch.compile.html). The documentation notes thatdynamic=Nonehas distinct semantics fromdynamic=False, whereNoneindicates a different configuration state. Previously, the code defaulted toFalsewhen the environment variable was unset, which could lead to incorrect behavior.Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.