Skip to content

[Windows][Strix Halo gfx1151] HIP allocations capped at OS-visible RAM when BIOS UMA/VGM = 96 GB (ROCm 10.1, driver 26.10.41.05) #8922

Open

Description

@xyny89

Also filed as ROCm/legacy-rocm-build#6696
Hardware: Ryzen AI Max+ 395 (Radeon 8060S, gfx1151), 128 GB LPDDR5X
OS: Windows 11 Pro, build 26300
Drivers tested: Adrenalin 26.8.1 (32.0.31041.1004) and ROCm Adrenalin 26.10.41.05 (32.0.31041.5005)
Runtime: amdhip64_7.dll 10.0.3684.0 (HIP 7.16.x) from ROCm 10.1 tarball (therock-dist-windows-gfx1151-10.1.0)

Test: Python ctypes against amdhip64_7.dll, allocate 4 GiB blocks with hipMalloc, hipMemset + hipDeviceSynchronize each, stop on 2 consecutive failures. OCL_SET_SVM_SIZE=262144 set (no difference without it). No other GPU workloads running.

Results:

BIOS UMA/VGM Driver / runtime hipMemGetInfo total Max allocated (all touched OK)
1 GB 26.8.1 76.99 GiB 100 GiB (fails at 104 GiB, err 2)
96 GB 26.8.1 107.87 GiB 32 GiB (fails at 36 GiB, err 2)
96 GB 26.10.41.05 107.87 GiB 32 GiB
96 GB 26.10.41.05 + ROCm 10.1 107.87 GiB 28 GiB

At 96 GB VGM (OS sees 31.6 GiB RAM), hipMalloc, hipExtMallocWithFlags (fine-grained) and hipHostMalloc all cap at ~28-32 GiB, i.e. roughly OS-visible RAM, while hipMemGetInfo reports ~75 GiB still free. hipMallocManaged fails at 12 GiB with error 719 (unspecified launch failure).
It looks like HIP allocations go to the WDDM shared/system pool instead of the dedicated carve-out.

Expected: with 96 GB VGM, HIP should be able to use the dedicated 96 GB (plus shared memory), as described in the ROCm 10.1 changelog ("HIP runtime now correctly uses the full unified memory pool available on high-memory APUs").

Additional issues:

  1. hipMemGetInfo underreports at 1 GB VGM (77 GiB total, 100 GiB actually allocatable).
  2. Driver 26.10.41.05 installs System32\amdhip64_7.dll that depends on rocm_kpack.dll, which the driver doesn't install (LoadLibrary error 126 until the ROCm 10.1 bin folder is on PATH).

Related: #5940 (closed as "Fixed in upcoming driver release").

Repro: set OCL_SET_SVM_SIZE=262144 then python hip_alloc_test.py 120

hip_alloc_test.py
"""HIP allocation ceiling test for Windows (Strix Halo gfx1151).
Allocates 4 GiB blocks with hipMalloc until 2 consecutive failures,
memsets + syncs each block, re-touches all at the end, then frees.
Usage:  set OCL_SET_SVM_SIZE=262144  (optional)
        python hip_alloc_test.py [max_gib] [dll_path]
"""
import ctypes, os, sys, time

MAX_GIB = int(sys.argv[1]) if len(sys.argv) > 1 else 120
DLL = sys.argv[2] if len(sys.argv) > 2 else "amdhip64_7.dll"
BLOCK = 4 << 30
GIB = 1 << 30

if os.path.isdir(r"C:\ROCm\10.1\bin"):
    os.add_dll_directory(r"C:\ROCm\10.1\bin")
hip = ctypes.WinDLL(DLL, winmode=0)

def err(code):
    hip.hipGetErrorString.restype = ctypes.c_char_p
    return f"{code} {hip.hipGetErrorString(code).decode()}"

def meminfo():
    free, total = ctypes.c_size_t(), ctypes.c_size_t()
    rc = hip.hipMemGetInfo(ctypes.byref(free), ctypes.byref(total))
    return f"free {free.value/GIB:.2f} / total {total.value/GIB:.2f} GiB" if rc == 0 else f"hipMemGetInfo failed: {err(rc)}"

ver = ctypes.c_int()
hip.hipRuntimeGetVersion(ctypes.byref(ver))
print(f"HIP runtime {ver.value}, OCL_SET_SVM_SIZE={os.environ.get('OCL_SET_SVM_SIZE')}")
print("before:", meminfo())

ptrs, fails = [], 0
while len(ptrs) * BLOCK < MAX_GIB * GIB and fails < 2:
    p = ctypes.c_void_p()
    t = time.time()
    rc = hip.hipMalloc(ctypes.byref(p), ctypes.c_size_t(BLOCK))
    if rc != 0:
        fails += 1
        print(f"block {len(ptrs)+1}: hipMalloc FAIL {err(rc)}")
        continue
    fails = 0
    rc = hip.hipMemset(p, 0xAB, ctypes.c_size_t(BLOCK)) or hip.hipDeviceSynchronize()
    ptrs.append(p)
    print(f"block {len(ptrs)}: OK, total {len(ptrs)*4} GiB, {time.time()-t:.1f}s" + (f", memset/sync {err(rc)}" if rc else ""))

print(f"max allocated: {len(ptrs)*4} GiB")
print("after alloc:", meminfo())
errors = sum(1 for p in ptrs if hip.hipMemset(p, 0x5A, ctypes.c_size_t(BLOCK)) != 0)
rc = hip.hipDeviceSynchronize()
print(f"final touch: {errors} memset errors, sync {err(rc)}")
for p in ptrs:
    hip.hipFree(p)
print("after free:", meminfo())

Activity

  1. xyny89 commented on Oct 11, 2026

    @xyny89
    Author

    Update (same machine, driver 26.10.41.05 + ROCm 10.1 tarball runtime, BIOS UMA/VGM = 0.5 GB, OS sees 127.1 GiB RAM):

    • hipMemGetInfo before: free 98.06 / total 98.22 GiB
    • hipMalloc 4 GiB blocks: 100 GiB max, block 26 (104 GiB) fails with error 2 (out of memory). Identical with ROCm 10.1 bin\amdhip64_7.dll and System32 DLL. At that point Windows still had 13-15 GiB RAM available and 25-28 GiB commit headroom (commit limit 143.1 GiB), so this looks like a fixed per-process driver cap, not memory exhaustion. Same 100 GiB cap as driver 26.8.1 at 1 GB VGM.
    • Regression vs 26.8.1: after allocating 100 GiB, a final hipMemset pass over all blocks returns 0 errors but hipDeviceSynchronize returns error 719 (unspecified launch failure), killing the context. Did not happen with 26.8.1.
    • Also: in one fresh process ~3 min after boot, error 719 was returned immediately by hipMemGetInfo and every hipMalloc (0 bytes allocated); not reproducible in the next process.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions