Store CuArray offset in bytes instead of elements by maleadt · Pull Request #3088 · JuliaGPU/CUDA.jl

maleadt · 2026-04-09T12:17:19Z

The element-based offset was lossy when materializing reinterpret on views with non-aligned offsets (e.g., reinterpreting a view of Float64 as SVector{3,Float64}). The byte offset would get truncated by integer division when converting to the new element count.

The original element-based design came from copying Julia, where wrappers like Reinterpret are never materialized.

Fixes #2980

The element-based offset was lossy when materializing reinterpret on views with non-aligned offsets (e.g., reinterpreting a view of Float64 as SVector{3,Float64}). The byte offset would get truncated by integer division when converting to the new element count. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Ref: JuliaGPU/CUDA.jl#3088

github-actions

CUDA.jl Benchmarks

Details

Benchmark suite	Current: `11e53dc`	Previous: `9bcae25`	Ratio
`array/accumulate/Float32/1d`	`101364` ns	`101038` ns	`1.00`
`array/accumulate/Float32/dims=1`	`77000` ns	`76504` ns	`1.01`
`array/accumulate/Float32/dims=1L`	`1586873` ns	`1584181` ns	`1.00`
`array/accumulate/Float32/dims=2`	`143721.5` ns	`143368` ns	`1.00`
`array/accumulate/Float32/dims=2L`	`658719` ns	`657719` ns	`1.00`
`array/accumulate/Int64/1d`	`119164` ns	`118641` ns	`1.00`
`array/accumulate/Int64/dims=1`	`80485` ns	`80082.5` ns	`1.01`
`array/accumulate/Int64/dims=1L`	`1694663` ns	`1695059.5` ns	`1.00`
`array/accumulate/Int64/dims=2`	`155979` ns	`156167` ns	`1.00`
`array/accumulate/Int64/dims=2L`	`962257` ns	`961945` ns	`1.00`
`array/broadcast`	`20815` ns	`20674` ns	`1.01`
`array/construct`	`1295.8` ns	`1310.5` ns	`0.99`
`array/copy`	`18823` ns	`18665` ns	`1.01`
`array/copyto!/cpu_to_gpu`	`219180` ns	`212874` ns	`1.03`
`array/copyto!/gpu_to_cpu`	`286008` ns	`283181` ns	`1.01`
`array/copyto!/gpu_to_gpu`	`11430` ns	`11289` ns	`1.01`
`array/iteration/findall/bool`	`133237` ns	`132319` ns	`1.01`
`array/iteration/findall/int`	`150552` ns	`149555` ns	`1.01`
`array/iteration/findfirst/bool`	`82526` ns	`81551` ns	`1.01`
`array/iteration/findfirst/int`	`84556` ns	`83592` ns	`1.01`
`array/iteration/findmin/1d`	`88142.5` ns	`87059` ns	`1.01`
`array/iteration/findmin/2d`	`117841` ns	`117673` ns	`1.00`
`array/iteration/logical`	`204543.5` ns	`201621.5` ns	`1.01`
`array/iteration/scalar`	`68753` ns	`67482` ns	`1.02`
`array/permutedims/2d`	`52664` ns	`52629` ns	`1.00`
`array/permutedims/3d`	`52931` ns	`52965` ns	`1.00`
`array/permutedims/4d`	`51745` ns	`51689` ns	`1.00`
`array/random/rand/Float32`	`13244` ns	`13212` ns	`1.00`
`array/random/rand/Int64`	`30572.5` ns	`37213` ns	`0.82`
`array/random/rand!/Float32`	`8761.666666666666` ns	`8755` ns	`1.00`
`array/random/rand!/Int64`	`34751` ns	`34251` ns	`1.01`
`array/random/randn/Float32`	`40235` ns	`41327.5` ns	`0.97`
`array/random/randn!/Float32`	`28763.5` ns	`31478` ns	`0.91`
`array/reductions/mapreduce/Float32/1d`	`35814` ns	`34808.5` ns	`1.03`
`array/reductions/mapreduce/Float32/dims=1`	`40082.5` ns	`49666` ns	`0.81`
`array/reductions/mapreduce/Float32/dims=1L`	`51945` ns	`51871.5` ns	`1.00`
`array/reductions/mapreduce/Float32/dims=2`	`57111` ns	`56886` ns	`1.00`
`array/reductions/mapreduce/Float32/dims=2L`	`69893` ns	`69867` ns	`1.00`
`array/reductions/mapreduce/Int64/1d`	`41780` ns	`43013.5` ns	`0.97`
`array/reductions/mapreduce/Int64/dims=1`	`43162` ns	`46867` ns	`0.92`
`array/reductions/mapreduce/Int64/dims=1L`	`87938` ns	`87803` ns	`1.00`
`array/reductions/mapreduce/Int64/dims=2`	`60322` ns	`60174` ns	`1.00`
`array/reductions/mapreduce/Int64/dims=2L`	`85423` ns	`85361` ns	`1.00`
`array/reductions/reduce/Float32/1d`	`35791` ns	`34927` ns	`1.02`
`array/reductions/reduce/Float32/dims=1`	`41731` ns	`40237` ns	`1.04`
`array/reductions/reduce/Float32/dims=1L`	`52091` ns	`51974` ns	`1.00`
`array/reductions/reduce/Float32/dims=2`	`57571` ns	`56885` ns	`1.01`
`array/reductions/reduce/Float32/dims=2L`	`70351` ns	`70269.5` ns	`1.00`
`array/reductions/reduce/Int64/1d`	`41913` ns	`42812` ns	`0.98`
`array/reductions/reduce/Int64/dims=1`	`51257.5` ns	`44667` ns	`1.15`
`array/reductions/reduce/Int64/dims=1L`	`88036` ns	`87879` ns	`1.00`
`array/reductions/reduce/Int64/dims=2`	`60495` ns	`59725.5` ns	`1.01`
`array/reductions/reduce/Int64/dims=2L`	`85286` ns	`85065` ns	`1.00`
`array/reverse/1d`	`18572` ns	`18432` ns	`1.01`
`array/reverse/1dL`	`69149.5` ns	`69003` ns	`1.00`
`array/reverse/1dL_inplace`	`66167` ns	`66090` ns	`1.00`
`array/reverse/1d_inplace`	`10447.666666666666` ns	`8760.666666666666` ns	`1.19`
`array/reverse/2d`	`21063` ns	`21048` ns	`1.00`
`array/reverse/2dL`	`73123` ns	`73087` ns	`1.00`
`array/reverse/2dL_inplace`	`66309` ns	`66163` ns	`1.00`
`array/reverse/2d_inplace`	`10673` ns	`10357` ns	`1.03`
`array/sorting/1d`	`2736178.5` ns	`2735237` ns	`1.00`
`array/sorting/2d`	`1070156` ns	`1068605` ns	`1.00`
`array/sorting/by`	`3305687` ns	`3303990` ns	`1.00`
`cuda/synchronization/context/auto`	`1226.4` ns	`1177.6` ns	`1.04`
`cuda/synchronization/context/blocking`	`959.0333333333333` ns	`967.125` ns	`0.99`
`cuda/synchronization/context/nonblocking`	`7820.6` ns	`8244.6` ns	`0.95`
`cuda/synchronization/stream/auto`	`1068.2` ns	`1030.9285714285713` ns	`1.04`
`cuda/synchronization/stream/blocking`	`832.5975609756098` ns	`823.7613636363636` ns	`1.01`
`cuda/synchronization/stream/nonblocking`	`7720.9` ns	`7674` ns	`1.01`
`integration/byval/reference`	`144147` ns	`144222` ns	`1.00`
`integration/byval/slices=1`	`145881` ns	`146200.5` ns	`1.00`
`integration/byval/slices=2`	`284780` ns	`284729` ns	`1.00`
`integration/byval/slices=3`	`423479` ns	`423343` ns	`1.00`
`integration/cudadevrt`	`102827` ns	`102656` ns	`1.00`
`integration/volumerhs`	`9439257` ns	`9437972.5` ns	`1.00`
`kernel/indexing`	`13543` ns	`13430` ns	`1.01`
`kernel/indexing_checked`	`14411` ns	`14182` ns	`1.02`
`kernel/launch`	`2341.4444444444443` ns	`2305.1111111111113` ns	`1.02`
`kernel/occupancy`	`718.7450980392157` ns	`703.0316455696203` ns	`1.02`
`kernel/rand`	`18541` ns	`15742.5` ns	`1.18`
`latency/import`	`3828970444.5` ns	`3826995569` ns	`1.00`
`latency/precompile`	`4599947559.5` ns	`4576949294` ns	`1.01`
`latency/ttfp`	`4411491653.5` ns	`4392035739` ns	`1.00`

This comment was automatically generated by workflow using github-action-benchmark.

For arrays of pure singleton unions (e.g. Union{Missing,Nothing}), Base.elsize is 0, so dividing the byte offset by it crashes. Such arrays only ever have a byte offset of 0 (data is empty), so the element offset is also 0. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

codecov · 2026-04-10T07:56:12Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.41%. Comparing base (5f45772) to head (11e53dc).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3088   +/-   ##
=======================================
  Coverage   90.41%   90.41%           
=======================================
  Files         141      141           
  Lines       11993    11993           
=======================================
  Hits        10844    10844           
  Misses       1149     1149

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Store CuArray offset in bytes instead of elements

christiangnrd added a commit to JuliaGPU/Metal.jl that referenced this pull request Apr 9, 2026

Store MtlArray offset in bytes instead of elements

97e45c6

Ref: JuliaGPU/CUDA.jl#3088

github-actions Bot reviewed Apr 9, 2026

View reviewed changes

maleadt mentioned this pull request Apr 10, 2026

Store JLArray offset in bytes instead of elements JuliaGPU/GPUArrays.jl#703

Merged

maleadt merged commit a7fded3 into master Apr 10, 2026
1 of 2 checks passed

maleadt deleted the tb/offset branch April 10, 2026 07:42

maleadt added a commit that referenced this pull request Apr 16, 2026

Merge pull request #3088 from JuliaGPU/tb/offset

9bff14b

Store CuArray offset in bytes instead of elements

maleadt mentioned this pull request Apr 20, 2026

Add preference to use CUDA_Runtime_Discovery for the compiler binaries #3080

Merged

Navigation Menu

Repository navigation

Store CuArray offset in bytes instead of elements - #3088

Store CuArray offset in bytes instead of elements#3088
maleadt merged 2 commits into
masterfrom
tb/offset

maleadt commented Apr 9, 2026

Uh oh!

github-actions Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

codecov Bot commented Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

maleadt commented Apr 9, 2026

Uh oh!

github-actions Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

CUDA.jl Benchmarks

Uh oh!

Uh oh!

codecov Bot commented Apr 10, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot left a comment •

edited

Loading