Summary

Three small fixes from debugging a 28-node rccl_perf run on a shared cluster:

• cvs/lib/verify_lib.py — surface amdgpu runlist oversubscription as a non-fatal WARN. Per AMD docs (Runlist is getting oversubscribed, Expect reduced ROCm performance), this is a real perf-degrading state — the HW scheduler round-robins queues and inactive queues can block the GPU for ms-scale windows. The collective itself completes correctly, so we don't fail the test, but we do log a WARN line so reviewers know the perf numbers from this run were measured under a degraded HW state and shouldn't be trusted blindly. Implemented via a new module-level warn_patterns_dict scanned alongside err_patterns_dict inside verify_dmesg_for_errors. Backward-compatible — function signature and return shape unchanged, all 7+ existing callers untouched.

• cvs/tests/rccl/rccl_perf.py — flip verify_dmesg_for_errors(..., till_end_flag=False) so the dmesg scan is bounded by each parametrized test's own start_time/end_time window. With till_end_flag=True the scan ran from start_time to the end of the dmesg buffer, so a kernel event from one test (e.g. a scatter_perf segfault) was re-flagged as a failure for every subsequent parametrized test in the run.

• cvs/lib/rccl_lib.py — write the mpirun hostfile to /tmp/rccl_hosts_file_<USER>.txt instead of the shared /tmp/rccl_hosts_file.txt. A leftover hostfile owned by another user blocks the run with Operation not permitted / Permission denied on shared cluster nodes.

Why the WARN approach (not drop-from-failures)

Earlier rev of this PR dropped the oversubscription patterns from the failure regex outright. That made the run "green" but threw away the signal — a perf run in a degraded HW state would now be marked PASS and its bandwidth deltas silently trusted. Demoting to a WARN bucket separates the two questions:

• Did the collective complete correctly? → still a PASS (correct behavior).
• Is this a valid perf measurement? → flagged via WARN in std.log.

Real driver-side errors (Queue preemption failed, Failed to evict process queues, No more SDMA queue to allocate, amdgpu: process pid) still match the FAIL regex and behave as before.

Test plan

• 28-node validation on RCCL 2.27.7 / ROCm 7.2 and RCCL 2.28.3 / ROCm 7.13: previously 7-9 false-positive failures per run, now 14/14 collectives PASS, oversubscription events emit WARN lines in std.log.
• Confirmed Test failure and segfault still fail the correct (and only the correct) test after the per-test window change.
• Confirmed two concurrent users on the same nodes no longer collide on the hostfile.

Refs: AIMVT-175