refactor: split PD disaggregation into separate EngineGroups - #1609
Merged
Merged
Conversation
- Add role and rank_offset fields to EngineGroup so each group knows its worker type and global rank starting point - Refactor init_rollout_engines to accept worker_type/rank_offset instead of computing prefill/decode internally - Update _allocate_rollout_engine_addr_and_ports_normal to use worker_type parameter instead of prefill_limit calculation - Split start_rollout_server to create separate prefill and decode EngineGroups when prefill_num_servers is set - Change health monitors from single to per-group list so each EngineGroup is monitored independently
Move the engine initialization logic from the standalone init_rollout_engines() function into EngineGroup.init_engines(). This eliminates the duplication of role/rank_offset between the function call and the EngineGroup constructor. start_rollout_server now creates EngineGroup with [None] engines and calls group.init_engines() to fill them in. reinit_engines is kept as an alias of init_engines for clarity in fault-recovery paths.
When debug_train_only is set, self.server is None. Add None checks to offload() and onload() to prevent AttributeError.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.