GBRL is a Python-based Gradient Boosting Trees (GBT) library, similar to popular packages such as XGBoost, CatBoost, but specifically designed and optimized for reinforcement learning (RL). GBRL is implemented in C++/CUDA aimed to seamlessly integrate within popular RL libraries.
GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets. The following diagram illustrates how GBRL uses gradient boosting trees in RL:
GBRL features a shared tree-based structure for policy and value functions, significantly reducing memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
- GBT Tailored for RL: GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets.
- Optimized Actor-Critic Architecture: GBRL features a shared tree-based structure for policy and value functions. This significantly reduces memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
- Hardware Acceleration: GBRL leverages CUDA for hardware-accelerated computation, ensuring efficiency and speed.
- Seamless Integration: GBRL is designed for easy integration with popular RL libraries. We implemented GBT-based actor-critic algorithm implementations (A2C, PPO, and AWR) in stable_baselines3 GBRL_SB3.
The following results, obtained using the GBRL_SB3 repository, demonstrate the performance of PPO with GBRL compared to neural-networks across various scenarios and environments:
- Python 3.9 or higher
GBRL provides pre-compiled binaries for easy installation. Choose one of the following options:
CPU-only installation (default):
pip install gbrl
GPU-enabled installation (requires CUDA 12 runtime libraries):
pip install gbrl-gpu
For further installation details and dependencies see the documentation.
For a detailed usage example, see tutorial.ipynb
- Greedy (Depth-wise) tree building - (CPU/GPU)
- Oblivious (Symmetric) tree building - (CPU/GPU)
- L2 split score - (CPU/GPU)
- Cosine split score - (CPU/GPU)
- Uniform based candidate generation - (CPU/GPU)
- Quantile based candidate generation - (CPU/GPU)
- Supervised learning fitting / Multi-iteration fitting - (CPU/GPU)
- MultiRMSE loss (only)
- Categorical inputs
- Input feature weights - (CPU/GPU)
- Monotonic constraints - (CPU/GPU, oblivious trees; the whole model must use SGD, Adam is not allowed on any output)
- SGD optimizer - (CPU/GPU)
- ADAM optimizer - (CPU only)
- Control Variates (gradient variance reduction technique) - (CPU only)
- Shared Tree for policy and value function - (CPU/GPU)
- Linear and constant learning rate scheduler - (CPU/GPU, linear scheduler GPU only for Oblivious trees)
- Support for up to two different optimizers (e.g., policy/value) - (CPU/GPU if both are SGD)
- SHAP value calculation (optimizer-aware;
return_base=Truegives values that reconstruct the prediction)
For comprehensive documentation, visit the GBRL documentation.
To contribute to GBRL, please review and sign the Contributor License Agreement (CLA) available at: https://github.com/NVlabs/gbrl/blob/master/CLA.md
@inproceedings{
fuhrer2025gradient,
title={Gradient Boosting Reinforcement Learning},
author={Benjamin Fuhrer and Chen Tessler and Gal Dalal},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://arxiv.org/abs/2407.08250}
}
Copyright © 2024-2026, NVIDIA Corporation. All rights reserved.
This work is made available under the NVIDIA MIT License. Click here to view a copy of this license.