CUDA.jl

CUDA programming in Julia

The CUDA.jl package is the main programming interface for working with NVIDIA CUDA GPUs using Julia. It features a user-friendly array abstraction, a compiler for writing CUDA kernels in Julia, and wrappers for various CUDA libraries.

Before all, make sure you have a recent NVIDIA driver. On Windows, also make sure you have the Visual C++ redistributable installed. You do not need to install the CUDA Toolkit.

CUDA.jl can be installed with the Julia package manager. From the Julia REPL, type ] to enter the Pkg REPL mode and run:

pkg> add CUDA

Or, equivalently, via the Pkg API:

julia> import Pkg; Pkg.add("CUDA")

For an overview of the CUDA toolchain in use, you can run the following command after importing the package:

julia> using CUDA

julia> CUDA.versioninfo()

This may take a while, as it will precompile the package and download a suitable version of the CUDA toolkit. If your GPU is not fully supported, the above command (or any other command that initializes the toolkit) will issue a warning.

For quick testing, you can also use the juliagpu/cuda.jl container image from the GitHub Container Registry, which provides Julia, a precompiled version of CUDA.jl, and a matching CUDA toolkit. Tags are named after the CUDA major version (cuda12 or cuda13); the bare tag tracks the latest stable release, while <version>-cuda<major> pins to a specific release and dev-cuda<major> tracks the latest main build:

# latest stable release, CUDA 13
docker run -it --rm --gpus=all ghcr.io/juliagpu/cuda.jl:cuda13

# a specific release, CUDA 12
docker run -it --rm --gpus=all ghcr.io/juliagpu/cuda.jl:v6.1.0-cuda12

# latest main build, CUDA 13
docker run -it --rm --gpus=all ghcr.io/juliagpu/cuda.jl:dev-cuda13

For more usage instructions and other information, please refer to the documentation.

From version 6.0 on, CUDA.jl is a meta-package the imports the following registered subpackages:

• CUDACore: the core package with the main CUDA API wrappers and the compiler for writing CUDA kernels in Julia;
• CUDATools: developer tools including the profiler, CUPTI, NVML, and code reflection utilities;
• cuBLAS: wrappers for the cuBLAS library;
• cuFFT: wrappers for the cuFFT library;
• cuRAND: wrappers for the cuRAND library;
• cuSPARSE: wrappers for the cuSPARSE library;
• cuSOLVER: wrappers for the cuSOLVER library.

Other registered subpackages that are not imported by default include:

• cuTENSOR: wrappers for the cuTENSOR library;
• cuStateVec: wrappers for the cuStateVec library;
• cuTensorNet: wrappers for the cuTensorNet library;
• CUDNN: wrappers for the cuDNN library.

While importing CUDA.jl will remain the recommended way to use these packages, you can also import them separately in order to optimize precompilation or load times, or (in the future) reduce the size of the required artifacts.

CUDA.jl and its subpackages are are always released together with version numbers that move in lockstep.

The latest development version of CUDA.jl requires Julia 1.10 or higher. If you are using an older version of Julia, you need to use a previous version of CUDA.jl. This will happen automatically when you install the package using Julia's package manager.

Note that CUDA.jl may not work with a custom build of Julia; it is recommended that you install Julia using the official binaries or juliaup.

CUDA.jl supports many CUDA toolkit versions, and we generally support those toolkits that NVIDIA still supports for their other vendor libraries (like cuDNN). Beyond that, we will try to keep some level of support for older toolkits such that CUDA.jl can be run on embedded devices that are still supported by NVIDIA.

Toolkit support directly impact the devices we support:

Tegra devices are supported as well, although additionally limited by the CUDA toolkit shipped with the last JetPack release for that device. On JetPack 5 and 6, CUDA.jl loads NVIDIA's L4T forward-compatibility driver, so those boards are not restricted to the CUDA version they were flashed with; see the installation guide for the per-generation details.

• 🔵 CI tested: supported and regularly exercised by CI.
• 🟢 Supported: expected to work, but not covered by CI.
• 🟡 Best effort: untested and not supporting all functionality.
• 🔴 Not supported.

CUDA.jl will automatically download and use a CUDA Toolkit that's supported by your NVIDIA driver as well as most of the devices in your system. There are two cases where you may want to select a different version:

• You have a GPU that is not supported by the current CUDA Toolkit. Although CUDA.jl tries to select a runtime that supports most of your GPUs, specific devices may end up being unsupported by the active toolkit. In that case, call e.g. CUDA.set_runtime_version!(v"12.9") to use a specific toolkit which still supports the device in question.
• You want to use a CUDA Toolkit that is already installed on your system. This may be necessary on older systems for which matching runtime and compiler artifacts are unavailable. Call CUDA.set_runtime_version!(local_toolkit=true).

Both these options will be remembered across sessions via a local preference, which can also be set directly (see LocalPreferences.toml).

Much of the software in this ecosystem was developed as part of academic research. If you would like to help support it, please star the repository as such metrics may help us secure funding in the future. If you use our software as part of your research, teaching, or other activities, we would be grateful if you could cite our work. The CITATION.bib file in the root of this repository lists the relevant papers.

Usage questions can be posted on the Julia Discourse forum under the GPU domain and/or in the #gpu channel of the Julia Slack.

Contributions are very welcome, as are feature requests and suggestions. Please open an issue if you encounter any problems.

You can pass special flags to Buildkite to avoid overburdening the GPU CI agents, which are a shared resource. The flag should be added to your top level commit message (not the extended message).

Available flags are:

• [only julia]: test only on the various Julia versions CUDA.jl supports, skipping testing subpackages and the various CUDA toolkits
• [only tests]: don't run the benchmark or documentation steps
• [only subpackages]: skip all the Julia and CUDA toolkit version runs, only test cuTENSOR, cuStateVec, cuTensorNet, and CUDNN
• [only docs]: only build the documentation, skip all tests and benchmarks
• [only special]: only test the special cases of multi-GPU, the various default memory locations, compute sanitizer, and the GPU-less environment
• [only downstream]: test only the "downstream" packages which depend on CUDA.jl and its libraries, such as Enzyme.jl
• [skip downstream]: don't the "downstream" packages
• [skip special]: don't run the above described special tests
• [skip tests]: skip all tests, only run benchmarks and documentation
• [skip benchmarks]: don't run the benchmarks

If you use these flags, make sure you don't exclude tests your changes could actually impact.