We have created an interactive Docker container. This is the preferred way of testing since our CI uses the same container, too.

1. Build the container in the repository root with

   docker build -f docker/develop/develop.dockerfile -t cobbler-dev .

2. Run the container with

   docker run --cap-add=NET_ADMIN -it --rm --name cobbler-dev -v $PWD:/code cobbler-dev

3. Inside the container, setup everything with

   ./docker/develop/scripts/setup-supervisor.sh (on newer versions setup-supervisor.sh is enough)

To apply any changes, build Cobbler again and restart the cobblerd service:

make install && supervisorctl restart cobblerd

In case you want to clean the environment without removing and restarting the container:

./docker/develop/scripts/clean-cobbler.sh && supervisorctl restart cobblerd

If you get the error message rm: cannot remove '/var/lib/cobbler/collections/**/*.json': No such file or directory, you may ignore it since you have no persisted changes.

One specific test (test_get_file_device_path) will fail, if your host distribution uses Btrfs as filesystem.

Before running tests, ensure you have installed requirements-tests.txt via e.g. pip.

To run tests:

• Ensure you have pytest installed.
• Ensure you have a running Cobbler instance which is setup correctly.
• From within the tests directory execute pytest if you want to run all tests or execute pytest xx_test.py if you want to run tests on one specific file.

Note that these tests require modifications to a running Cobbler server. They will attempt to clean up any test objects created, but it may be best to not execute these on a production Cobbler server.

Some of the tests will fail outside the docker container due to their specific customization to this container. One specific test will fail, if your host distribution uses Btrfs as filesystem.

This suite is intended for black-box Cobbler testing. Naturally, to run the suite you'll need a fully working Cobbler instance.

The suite itself can be found in the system-tests directory:

images/
listings/
Makefile
prelude
scripts/
tests/
workspace/

You can find the test cases inside the tests directory.

If you're developing a Cobbler inside a VM or container, run the following commands using the main Cobbler Makefile:

# make system-test-env
# make system-test

Target system-test-env is used to bootstrap your VM or container and shall be only used once. Target system-test will run the suite and print the results.

The results will be in the workspace directory.

To run only the specific tests, pass the SYSTESTS variable to make. Globbing is supported:

# make SYSTESTS='basic-profile-rename basic-system-*' system-test

If you just want to check that things work on a particular version, run the following command:

# ./docker/rpms/build-and-install-rpms.sh --with-system-tests el8 \
        docker/rpms/CentOS_8/CentOS8.dockerfile

Writing tests is easy. All tests are Bash scripts with the following common part:

#!/usr/bin/env bash
# Check that Cobbler <test-description>

source ${SYSTESTS_PRELUDE} && prepare

set -x -e -o pipefail

The SYSTESTS_PRELUDE variable expands to system-tests/prelude that contains helper functions and variables. The prepare function is necessary and shall not be omitted.

The coding style simple: tabs, 80-90 characters, ${var} instead of $var. Also if you have some complex scenario, leave a comment.

As you can see all tests run up until the first command failure, hence there is no need for complex failure conditions. Write your test as one big assert. The trace of the test execution will be stored in system-tests/workspace/<test_name>/_output.

Currently tests fall into three groups:

• autoinstall
• basic
• import

The tests in this group check that Cobbler can provision a system. We use QEMU, Linux kernel and a special initramfs that can be found in system-tests/images/dummy. Consult system-tests/Makefile for build instructions. These tests are very time consuming and they are not included in the default set.

Basic tests. Create objects, remove objects, check that files are created properly in the TFTP root, etc. When in doubt, put your tests here.

The tests in this group check that Cobbler can import an installation ISO. Instead of using the real ISO image we copy it's directory structure to system-tests/listings.

To generate listings you need the following packages installed on your system:

• xorriso
• jq

To add a new listing for already existing breed/version pair:

$ make listing ISO="path/to/iso" BREED="foo" DISTRO="f8"
$ git add system-tests/listings/foo/f8/FooLinux-x86_64-minimal.iso/

Where FooLinux-x86_64-minimal.iso is the installation ISO and foo-f8 is the breed/version pair. If you need to add a listing for a new pair, you should additionally run the following commands:

$ ln -s import-@ system-tests/tests/import-<breed>-<version>
$ git add system-tests/tests/import-<breed>-<version>

Currently the tests are getting better but we are not yet satisfied with the state of them. There are bugs which could have been prevented by a better testsuite. Thus we welcome any idea and contribution for testsuite improvements!