Repository navigation
[trainer] feat: A runnable separate async trainer - #6790
Conversation
There was a problem hiding this comment.
Code Review
This pull request enables separate async training by removing the NotImplementedError in PPOTrainerSeparateAsync and passing start_rank to LLMServerManager to prevent Ray named-actor collisions between hybrid and standalone replicas. Additionally, the on_validate_end method was removed, and TODOs were added regarding auto-offload configuration. Feedback on the changes suggests updating the type annotation of start_rank in _initialize_llm_servers to Optional[int] to match its default value of None.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| return instance | ||
|
|
||
| async def _initialize_llm_servers(self, start_rank: int = 0): | ||
| async def _initialize_llm_servers(self, start_rank: int = None): |
There was a problem hiding this comment.
The type annotation for start_rank is specified as int, but its default value is set to None. To ensure type safety and prevent static analysis or type checker warnings, the type should be updated to Optional[int].
| async def _initialize_llm_servers(self, start_rank: int = None): | |
| async def _initialize_llm_servers(self, start_rank: Optional[int] = None): |
What does this PR do?
See above.
Currently the colocated group never switches to rollouter, similar to the fully async implementation.
Switch and offload strategies will be introduced later.