Skip to content

Conversation

@yoshipon
Copy link
Collaborator

@yoshipon yoshipon commented Dec 9, 2025

This pull request introduces support for multiple learning rate schedulers in the OptimizerLightningModule and OptimizerConfig classes, allowing users to configure and combine different learning rate schedules (e.g., step-wise warmup and epoch-wise plateau) in PyTorch Lightning workflows. The update adds a new LRSchedulerConfig class, deprecates the old single-scheduler interface, and updates documentation and examples to reflect these changes.

Core API Enhancements:

  • Added the LRSchedulerConfig dataclass to encapsulate configuration for individual learning rate schedulers, including generator, interval, monitor, frequency, and other options. (aiaccel/torch/lightning/opt_lightning_module.py)
  • Updated OptimizerConfig to accept a list of LRSchedulerConfig instances via the new schedulers argument, supporting multiple schedulers per optimizer. The old scheduler_generator/scheduler_interval/scheduler_monitor fields are now deprecated in favor of this new approach, and validation ensures only one method is used. (aiaccel/torch/lightning/opt_lightning_module.py) [1] [2]
  • Modified OptimizerLightningModule.configure_optimizers() to return optimizer and scheduler(s) in the format expected by Lightning, supporting multiple schedulers if provided. (aiaccel/torch/lightning/opt_lightning_module.py)
  • Updated the __init__.py exports to include LRSchedulerConfig in the public API. (aiaccel/torch/lightning/__init__.py)

Documentation and Example Updates:

  • Expanded the user guide and API reference to document the new multiple-scheduler feature, including a YAML configuration example demonstrating how to combine different scheduler types. (docs/source/user_guide/torch.rst, docs/source/api_reference/torch.rst) [1] [2]
  • Updated the ResNet50 CIFAR10 example configuration to use the new schedulers field with two schedulers: LinearLR (step) and CosineAnnealingLR (epoch). (examples/torch/image_classification/recipes/resnet50.cifar10/config.yaml)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants