Skip to content

Windows ROCm 10.1 nightly runtime: hipHostRegister of pageable memory >=1MiB returns success but leaves corrupted state -> background-thread segfault (gfx1100; 7.14 runtime fails cleanly) #8939

Open

Description

@vecStardo

Windows ROCm 10.1 nightly runtime: hipHostRegister of pageable memory ≥1 MiB returns success but leaves corrupted state → background-thread segfault (gfx1100; 7.14 runtime fails cleanly)

Summary

On Windows with the ROCm 10.1 nightly runtime (pip rocm==10.1.0a20260822 from rocm.nightlies.amd.com/whl-multi-arch/), registering a ≥1 MiB pageable host buffer with hipHostRegister returns success but leaves broken state. Shortly afterwards a runtime background thread segfaults (0xC0000005) inside amdhip64_7.dll, killing the process with no catchable error.

The ROCm 7.14 runtime, on the same machine and same driver, handles the identical call by returning a clean hipErrorInvalidArgument (surfacing as a catchable exception) — no crash. Ordinary hipMemcpy from pageable memory works on both runtimes at every size we tested (≤14 MiB, sync/async, streams, double buffering). The defect is specific to the register path of the 10.1 nightly build.

Environment

  • Windows 11 (build 26200), Radeon RX 7900 XTX (gfx1100), Adrenalin 26.8.1
  • Runtime under test: pip rocm==10.1.0a20260822 (_rocm_sdk_core/bin/amdhip64_7.dll)
  • Comparison runtime: ROCm 7.14 bundled with the lc700x/MIGraphX-Windows v7.14.0a20260615.post.2.2 wheel (same DLL name, older build)
  • Workload triggering the registration: MIGraphX run() with a numpy-backed argument (its input path registers host memory for zero-copy)

Reproduction (minimal)

A single-op ONNX (ReLU) whose input buffer crosses the threshold. With the 10.1 nightly runtime:

input size result
128 KB / 256 KB / 512 KB / 768 KB RUN_OK
1 MiB / 2 MiB / 4 MiB / 14 MiB process dies: Exception Code: 0xC0000005, stack entirely inside amdhip64_7.dll background thread (anchored near hsa_system_get_major_extension_table / hipProfilerRegisterChunkCallbackExt offset region; BaseThreadInitThunk at the base)

With the 7.14 runtime: ≤768 KB RUN_OK; ≥1 MiB raises a clean RuntimeError from hip.cpp:128 register_on_gpu (hipErrorInvalidArgument), process survives.

Repro sketch (Python, migraphx wheel):

import migraphx, numpy as np
m = migraphx.parse_onnx("relu_2mb.onnx")          # single ReLU, input float32[524288]
m.compile(migraphx.get_target("gpu"))
shapes = m.get_parameter_shapes()
args = {k: migraphx.argument(np.random.rand(*shapes[k].lens()).astype(np.float32)) for k in shapes}
m.run(args)   # 10.1 nightly: SIGSEGV (0xC0000005) in amdhip64_7.dll background thread

Raw hipMemcpy micro-benchmarks (ctypes, pageable host memory, all sizes ≤14 MiB, sync/async/stream/double-buffered) pass on both runtimes, so plain copies are unaffected.

Workaround

Allocating the buffer with hipHostMalloc instead of registering existing pageable memory works on both runtimes — a 13.5 MB input transfers and runs fine, and with this shim our full 139 MB detection model compiles and runs end-to-end on the 7.14 runtime. But the fake-success-then-crash behavior of the 10.1 nightly register path looks like a genuine runtime (or runtime↔KMD) defect worth fixing: a failure here should be reported honestly, not deferred to a background-thread crash.

Cross-reference

microsoft/WindowsML#15 reports the same GPU (RX 7900 XTX) and the same crash signature — segfault in the bundled amdhip64_7.dll while compiling a large model — on AMD's official WinML MIGraphX EP. Possibly the same underlying Windows HIP runtime weakness.

Happy to provide the exact ONNX repro files, the threshold logs, and the full crash stacks.

Activity

  1. stellaraccident commented on Oct 12, 2026

    @stellaraccident
    Collaborator

    @ROCm/rocm-genies another runtime issue that needs routing. This might be only an issue on the PAL path. If fixed in the ROCR path, then we probably won't go back and fix PAL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions