drone is a reinforcement learning based drone controller and simulator tool, which is included in PufferLib as a first-party simulator example.

This project was presented at the 2025 and 2026 Warwick AI summits. The slides for these talks are available in this repo: 2026, 2025.

The development environment for this project is packaged as a Docker image, so you don't need to install the PufferLib RL stack on your machine directly. The image is bind-mounted onto your project tree at runtime so you can continue to edit the env, config and resources on the host while training and evaluating inside the container.

We recommend using the tool just to manage and setup this project, as we provide a comprehensive justfile which includes recipies for all commands used to perform common operations across the RL and firmware sides of the project. Many recipies (including setup) will also require uv to be installed. A setup script is provided that installs both tools (if missing), initialises the git submodules and pulls the prebuilt image. After cloning, run ./setup.sh (or just setup) followed by just dev to open a shell inside the container.

git clone https://github.com/tensaur/drone.git
cd drone

./setup.sh            # or `just setup`
just dev              # open a shell inside the container (first time builds the env)
just train hover

If you already have a Linux box with a PufferLib stack setup, and would rather build natively, pass --native to setup.sh (or run just setup-native directly).

There is no automated setup process for Windows machines. We suggest running the project using Windows Subsystem for Linux (WSL).

Once the setup process is complete, the puffer command will be available while the virtual environment is active. This command can be used to train and run the RL policy, as well as recipies provided by the justfile which act as shorthands.

# train
just train [hover|race]
puffer train drone --env.task [hover|race]

# eval
just eval latest [hover|race]
puffer eval drone --load-model-path latest

Further just recipies are provided to run hyperparamater sweeps, configure and flash the firmware, as well as code formatting. You can get details and documentation for all recipies and paramaters by simply running just.

For further details on building and flashing the firmware to hardware (Crazyflie 2.1 Brushless) see the docs here.

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

MIT License

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, shall be licensed as above, without any additional terms or conditions.