Repository navigation
[docs] add SGLang FP8 ignored layers usage - #6915
Conversation
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: gem-mint <146161229+gem-mint@users.noreply.github.com>
There was a problem hiding this comment.
Code Review
This pull request updates the documentation in docs/low_precision/fp8.md to explain how to skip FP8 weight quantization for selected modules during SGLang FP8 rollout, using either the SGLANG_FP8_IGNORED_LAYERS environment variable or the model's quantization_config. There are no review comments, and I have no feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
Hi. Is this modification related to this PR #6906 ? |
|
@sophiayyya Hi. Yes, this docs PR is a follow-up to #6906. #6906 added SGLang FP8 ignored-layer support in both the SGLang launch path and the actor weight-sync path; this PR documents how users can pass those rules via |
## Summary - Document how to skip selected modules during SGLang FP8 rollout weight quantization. - Add examples for `SGLANG_FP8_IGNORED_LAYERS`, comma-separated entries, and model `quantization_config.ignored_layers` with `re:` patterns. ## Context This docs-only PR follows up on verl-project#6906, which added support for applying the same SGLang FP8 ignored-layer rules when launching SGLang and when syncing updated actor weights into the rollout engine. ## Duplicate Check Before opening this PR, I checked for existing work with: ```bash gh issue view 6906 --repo verl-project/verl --comments gh pr list --repo verl-project/verl --state open --search "6906 in:body" gh pr list --repo verl-project/verl --state open --search "SGLang FP8 ignored layers docs" gh pr list --repo verl-project/verl --state open --search "rollout fp8 ignored_layers" ``` No duplicate open PRs were found. ## Tests ```bash git diff --check ``` Result: passed. ```bash PYTHONIOENCODING=utf-8 PYTHONUTF8=1 python tests/special_sanity/check_docs_time_info.py ``` Result: passed. ```bash PYTHONIOENCODING=utf-8 PYTHONUTF8=1 python tests/special_sanity/validate_structure.py \ --allow-files tests\\test_protocol_on_cpu.py tests\\test_base_config_on_cpu.py tests\\test_protocol_v2_on_cpu.py ``` Result: passed. `pre-commit` was also run during commit. All hooks passed except `validate-structure`, which was skipped for the commit after the manual command above passed with Windows path separators; the default hook allow-list uses POSIX-style paths and does not match those existing root-level tests on Windows. ## AI Assistance This PR was prepared with AI assistance. I reviewed the generated documentation change and validation output. Signed-off-by: gem-mint <146161229+gem-mint@users.noreply.github.com>
Summary
SGLANG_FP8_IGNORED_LAYERS, comma-separated entries, and modelquantization_config.ignored_layerswithre:patterns.Context
This docs-only PR follows up on #6906, which added support for applying the same SGLang FP8 ignored-layer rules when launching SGLang and when syncing updated actor weights into the rollout engine.
Duplicate Check
Before opening this PR, I checked for existing work with:
No duplicate open PRs were found.
Tests
Result: passed.
Result: passed.
Result: passed.
pre-commitwas also run during commit. All hooks passed exceptvalidate-structure, which was skipped for the commit after the manual command above passed with Windows path separators; the default hook allow-list uses POSIX-style paths and does not match those existing root-level tests on Windows.AI Assistance
This PR was prepared with AI assistance. I reviewed the generated documentation change and validation output.