Repository navigation
[ray, doc] fix: make Slurm Ray network interface configurable - #7386
Merged
wuxibin89 merged 1 commit intoAug 13, 2026
Merged
Conversation
Assisted-by: OpenAI Codex Signed-off-by: SkyFishMoon <yangtianyu612@gmail.com>
SkyFishMoon
marked this pull request as ready for review
August 13, 2026 09:09
wuxibin89
approved these changes
Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR makes the network interface used by Ray configurable in the Slurm launch example.
On multi-NIC clusters, users can set
RAY_NETWORK_INTERFACEto resolve the IPv4 address of that interface independently on every allocated node. The example then passes the resolved address toray start --node-ip-addressfor both the head and worker nodes. If the selected interface does not have an IPv4 address on a node, the script exits with a clear error instead of silently using another interface.The existing behavior remains unchanged when
RAY_NETWORK_INTERFACEis not set. The multinode documentation now explains how to use the option and clarifies that Gloo and NCCL have separate interface settings.Checklist Before Starting
RAY_NETWORK_INTERFACE, andnode-ip-addresswith Slurm. No open PR implementing this fix was found.[{modules}] {type}: {description}:[ray, doc] fix: make Slurm Ray network interface configurable.Test
The following checks were run:
The IPv4 resolver was also tested with mocked
srunoutput for both successful CIDR-to-IPv4 extraction and the failure path where the selected interface has no IPv4 address.The change was validated on a four-node Slurm cluster with a 1 Gbps management interface (
lan0) and a 200 Gbps HDR interface (ib0). Before explicitly selecting the interface, Ray actors registered with10.241.*addresses fromlan0. WithRAY_NETWORK_INTERFACE=ib0, the Ray head and workers registered with10.245.*addresses fromib0, and the distributed training job completed successfully.API and Usage Example
The new setting is optional. Existing users do not need to change their launch command.
On a multi-NIC cluster, select the interface when submitting the job:
RAY_NETWORK_INTERFACEcontrols Ray's node addresses only.GLOO_SOCKET_IFNAMEandNCCL_SOCKET_IFNAMEremain explicit because they configure separate communication layers.Design & Code Changes
RAY_NETWORK_INTERFACEenvironment variable to the Slurm example.srunon the host.--node-ip-addresswhen the option is set.Checklist Before Submitting
Important
Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.
pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always.srunoutput and validated on a real four-node Slurm cluster.ci-requestchannel in the verl Slack workspace or the Feishu group.AI assistance disclosure
OpenAI Codex assisted with the implementation and documentation. I reviewed every changed line and ran the validation described above.