rl_nav

Inference scripts for the RL navigation module of the Unified Autonomy Stack. Full documentation at ntnu-arl.github.io/unified_autonomy_stack/rl.

ROS 2 package for reinforcement-learning-based LiDAR navigation on UAVs. Policies are trained in Aerial Gym using Sample Factory (PPO + GRU) and deployed here for real-time inference.

The package focuses on acceleration control: the policy outputs body-frame linear accelerations and yaw rate directly from LiDAR and state observations, without an intermediate velocity controller.

pip install sample-factory torch scipy gymnasium ros2-numpy

ros2-numpy is only required by the real-robot node (lidar_nav_acc_ros2_node.py); the sim node does not need it.

ROS 2 packages: rclpy, sensor_msgs, geometry_msgs, nav_msgs, mavros_msgs, std_msgs

The node is triggered by incoming LiDAR data and publishes acceleration commands at the same rate.

cd scripts/
python3 lidar_nav_acc_ros2_node.py \
    --env=lidar_navigation_task \
    --experiment=magpie_accel_policy4 \
    --train_dir=/path/to/rl_nav/models

cd scripts/
python3 lidar_nav_acc_ros2_sim_node.py \
    --env=lidar_navigation_task \
    --experiment=magpie_accel_policy4 \
    --train_dir=/path/to/rl_nav/models

The --train_dir flag must point to the models/ directory of this repo (the parent of the experiment folder).

Training uses the lidar_navigation_task environment in Aerial Gym with Sample Factory:

python train.py \
    --env=lidar_navigation_task \
    --experiment=<experiment_name> \
    --train_dir=<path>/models \
    --with_wandb=false

To deploy a trained policy:

1. Copy the experiment folder from your Aerial Gym train_dir into models/:

   models/<your_experiment_name>/
   ├── config.json
   └── checkpoint_p0/
       └── best_*.pth
   

2. Update config.json — set "train_dir": "../models" so the checkpoint path resolves correctly. Set "load_checkpoint_kind": "best" to load the best checkpoint instead of the latest.

3. Pass the experiment name on the command line:

   python3 lidar_nav_acc_ros2_node.py \
       --env=lidar_navigation_task \
       --experiment=<your_experiment_name> \
       --train_dir=/path/to/rl_nav/models

4. Tune the action scaling in Config for sim-to-real transfer. These are inference-only parameters and do not exist in the training scripts — adjust them empirically on the real platform:

   ACTION_SCALE = np.array([0.85, 0.5, 0.4, 0.6])  # policy4 values

5. Tune the EMA filter to smooth commands on the real platform. Also inference-only:

   ACTION_FILTER_ALPHA = np.array([0.25, 0.3, 0.001, 0.3])

   Alpha=0 means no filtering (pass-through). Alpha→1 means heavy smoothing.

Network architecture (fixed in standalone_inference.py):

• Encoder: Conv2d(1→16→32→64) on 16×20 LiDAR image, 128-dim output; concatenated with 17-dim state → MLP (256-128-64)
• Core: GRU, 128 hidden units, 1 layer
• Head: Linear to 4 actions with adaptive std

The architecture fields in config.json are read automatically by Sample Factory — do not edit them.

The policy receives a flat 337-dim vector at each step:

The LiDAR grid is produced by spherical binning (480 azimuth × 48 elevation) followed by min-pooling down to 16×20, then inverting depth.

The network outputs 4 values in [-1, 1] which are scaled to physical units:

magpie_accel_policy4 (Feb 2026) — stable acceleration policy tuned for the real Magpie platform. Use this as the starting point. Additional policies are archived under models/ for reference.

The Config class at the top of each node script controls all runtime parameters without touching the network:

class Config:
    ACTION_SCALE   = np.array([0.85, 0.5, 0.4, 0.6])  # physical scaling per channel
    ACTION_FILTER_ALPHA = np.array([0.25, 0.3, 0.001, 0.3])  # EMA (0=off, 1=max smooth)
    LIDAR_MAX_RANGE = 10.0   # metres
    LIDAR_MIN_RANGE = 0.4    # metres
    MEDIAN_FILTER   = True
    MEDIAN_FILTER_KERNEL_SIZE = 7
    RESET_AT_NEW_WP = False   # reset RNN hidden state on each new waypoint
    DEVICE          = "cuda:0"

If you use this code or the trained models in your research, please consider citing:

@misc{dharmadhikari2026unifiedautonomystackblueprint,
      title={The Unified Autonomy Stack: Toward a Blueprint for Generalizable Robot Autonomy}, 
      author={Mihir Dharmadhikari and Nikhil Khedekar and Mihir Kulkarni and Morten Nissov and Martin Jacquet and Angelos Zacharia and Marvin Harms and Albert Gassol Puigjaner and Philipp Weiss and Kostas Alexis},
      year={2026},
      eprint={2605.12735},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2605.12735}, 
}