With great many support devices comes great many tests

OpenWrt Testing is a framework to run tests on OpenWrt devices, emulated or real. Using labgrid to control the devices, the framework offers a simple way to write tests and run them on different hardware.

• An OpenWrt firmware image
• Python and uv
• QEMU

For maximum convenience, clone the repository inside the openwrt.git repository as tests/:

cd /path/to/openwrt.git/
git clone https://github.com/aparcar/openwrt-tests.git tests/

Install required packages to use Labgrid and QEMU:

curl -LsSf https://astral.sh/uv/install.sh | sh

sudo apt-get update
sudo apt-get -y install \
    qemu-system-mips \
    qemu-system-x86 \
    qemu-system-aarch64 \
    make

Verify the installation by running the tests:

make tests/setup V=s

You can run tests via the Makefile or directly using pytest.

You can start runtime and shell tests via the Makefile.

cd /path/to/openwrt.git
make tests/x86-64 V=s

If you don't plan to clone this repository inside the openwrt.git repository, you can still run the tests. Use this command to run tests on malta/be image:

pytest tests/ \
    --lg-env targets/qemu-malta-be.yaml \
    --lg-log \
    --log-cli-level=CONSOLE \
    --lg-colored-steps \
    --firmware ../../openwrt/bin/targets/malta/be/openwrt-malta-be-vmlinux-initramfs.elf

The framework uses pytest to execute commands and evaluate the output. Test cases use the two fixture ssh_command or shell_command. The object offers the function run(cmd) and returns stdout, stderr (SSH only) and the exit code.

The example below runs uname -a and checks that the device is running GNU/Linux

def test_uname(shell_command):
    assert "GNU/Linux" in shell_command.run("uname -a")[0][0]

With labgrid, you can remotely access devices. Key capabilities include power control, console, and SSH access.

To enable remote access, you need SSH access with forwarding enabled on the host exporting the device. For example, to reach the device openwrt-one located in the lab labgrid-aparcar, you must have access to both the labgrid-aparcar host and use the global-coordinator as a jump host:

global-coordinator -> labgrid-aparcar -> openwrt-one

You can request access to existing labs or contribute your own. To do this, submit a pull request modifying the labnet.yaml file.

To access a remote device, configure the following environment variables. Notably, LG_PROXY sets the proxy host (always the lab name):

export LG_IMAGE=~/firmware/openwrt-ath79-generic-tplink_tl-wdr3600-v1-initramfs-kernel.bin # Firmware to boot
export LG_PLACE=aparcar-tplink_tl-wdr3600-v1 # Target device, formatted as <lab>-<device>
export LG_PROXY=labgrid-aparcar # Proxy to use, typically the lab name
export LG_ENV=targets/tplink_tl-wdr3600-v1.yaml # Environment definition

To avoid interference from CI or other developers, lock the device before use:

uv run labgrid-client lock

Once locked, you can power-cycle the device and access its console:

uv run labgrid-client power cycle
uv run labgrid-client console

To bring the device into a specific state—such as booting your firmware defined by LG_IMAGE—use a state definition:

uv run labgrid-client --state shell console

You can also run local tests directly on the remote device:

pytest tests/ --log-cli-level=CONSOLE

Lastly, unlock your device when you're done:

uv run labgrid-client unlock