Installation   |   Usage   |   Release Notes   |   Documentation   |   How to Contribute   |   License

oneAPI Collective Communications Library (oneCCL) provides an efficient implementation of communication patterns used in deep learning.

oneCCL is governed by the UXL Foundation and is an implementation of the oneAPI specification.

• oneAPI Collective Communications Library (oneCCL)
  • Installation
    • Linux build:
  • Usage
    • Launching Example Application
    • Common problems
  • Governance

oneCCL supports SYCL. In order to enable it, compile with icpx/icx-cl compiler or any compiler supporting find_package(IntelSYCL) in cmake.

To install Level Zero, refer to the instructions in Intel(R) Graphics Compute Runtime repository or to the installation guide for oneAPI users.

git submodule init # The two steps are not required if -DONECCL_USE_SYSTEM_LIBCCL=ON
git submodule update

mkdir build
cd build

cmake .. -DCMAKE_CXX_COMPILER=icpx -DCMAKE_C_COMPILER=icx -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_PREFIX=$(pwd)/_install
make -j install

DONECCL_USE_SYSTEM_LIBCCL option can be used to use libccl implementation from the system rather than compiling it from sources ar deps/libccl To compile legacy libccl code and use it with the flag use following steps:

git clone https://github.com/intel-innersource/libraries.performance.communication.oneccl ccl

git clone https://github.com/intel-innersource/libraries.performance.communication.oneccl-v2 ccl-v2

# Build legacy libccl with SYCL support
cd ccl
mkdir build
cd build
cmake .. -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DCMAKE_EXPORT_COMPILE_COMMANDS=1 && make -j install

# Build legacy libccl for CPU only
cd ..
mkdir build_cpu
cd build_cpu
cmake .. && make -j install
cd ..

# Copy CPU only binaries of legacy libccl
mkdir -p build/_install/lib/ccl/cpu/
cp -r build_cpu/_install/lib build/_install/lib/ccl/cpu/

# Go back to the new repo, assuming both repos were cloned at `~`
cd ~/ccl-v2
mkdir build
cd build
# Source previously build libccl, so CMAKE_PREFIX_PATH is set to `~/ccl/build/_install/lib/cmake`
source ~/ccl/build/_install/env/vars.sh
# Run cmake, please note DONECCL_USE_SYSTEM_LIBCCL=ON to use oneCCL through cmake's `find_package`
cmake .. -DCMAKE_CXX_COMPILER=icpx -DCMAKE_C_COMPILER=icx -DONECCL_USE_SYSTEM_LIBCCL=ON -DCMAKE_INSTALL_PREFIX=$(pwd)/_install
make install -j

if you build this package for ARC or BMG GPU with embedded libccl(DONECCL_USE_SYSTEM_LIBCCL=OFF), you need to add an additional flag during the cmake process. Commands below show the build process.

git submodule init # The two steps are not required if -DONECCL_USE_SYSTEM_LIBCCL=ON
git submodule update

mkdir build
cd build

cmake .. -DCMAKE_CXX_COMPILER=icpx -DCMAKE_C_COMPILER=icx -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_PREFIX=$(pwd)/_install -DCCL_ENABLE_ARCA=ON
make -j install

git submodule init # The two steps are not required if -DONECCL_USE_SYSTEM_LIBCCL=ON
git submodule update

mkdir build
cd build

cmake .. -DCMAKE_CXX_COMPILER=icpx -DCMAKE_C_COMPILER=icx -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_PREFIX=$(pwd)/_install -DCCL_ENABLE_ARCB=ON
make -j install

Use the command:

$ source <install_dir>/env/setvars.sh
$ CCL_LOG_LEVEL=info mpirun -n 2 <install_dir>/examples/sycl/sycl

C API version of oneCCL introduced a new plugin based architecture. Running examples or tests might require adjusting LD_LIBRARY_PATH, so it points to at least one directory with correct plugin. It can be done using vars.sh script or manually like export LD_LIBRARY_PATH=<build_dir>/plugins/legacy/:$LD_LIBRARY_PATH, so test binaries linking to libccl.so.2 through RUNPATH can still discover required plugin.

You can also use CCL_PLUGIN environment variable to set name of the plugin (ONECCL_LEGACY/ONECCL_LEGACY_CPU/ONECCL_NULL) or a whole path to a shared object implementing oneCCL plugin interface (<build_dir>/plugins/legacy/libccl_legacy.so).

The oneCCL project is governed by the UXL Foundation and you can get involved in this project in multiple ways. It is possible to join the Special Interest Groups (SIG) meetings where the group discuss and demonstrates work using the foundation projects. Members can also join the Open Source and Specification Working Group meetings.

You can also join the mailing lists for the UXL Foundation to be informed of when meetings are happening and receive the latest information and discussions.

See CONTRIBUTING for more information.

Distributed under the Apache License 2.0 license. See LICENSE for more information.

See SECURITY for more information.