|
Hegel 0.11.1
Property-based testing for C++
|
Hegel is a property-based testing library for C++. Hegel is based on Hypothesis, using the libhegel engine.
This guide walks you through the basics of installing Hegel and writing your first tests.
Using CMake:
Hegel requires CMake 3.14 and, by default, a C++20 compiler. The build downloads a small prebuilt shared library (libhegel, Hegel's native engine) for your platform; no other tooling is required.
To consume Hegel from C++17, configure with -DHEGEL_REFLECTION=OFF. This drops the reflect-cpp dependency: you lose default_generator (type-directed derivation for structs), but every other generator and combinator still works. (The designated-initializer parameter API, e.g. integers<int>({.min_value = 0}), then relies on a GCC/Clang C++17 extension.)
You're now ready to write your first test. In a new file:
Now build and run the test. You should see that this test passes.
Let's look at what's happening in more detail. HEGEL_TEST defines a property test as an ordinary function, self_equality. Running the test executes your test body 100 times by default. While you may call the function from main(), if you use a test framework, write the property inside one of its tests instead. We officially support gtest (see Hegel and test frameworks). Please make an issue on Github if Hegel does not integrate well with your framework.
The body receives a TestCase, which provides a TestCase::draw() method for drawing different values. This test draws a random integer and checks that it should be equal to itself. The macro also names the test in Hegel's example database, so a failure found in one run is replayed first in the next.
The name you pass to TestCase::draw() labels the value in the failure report, so a failing run replays each drawn value under the variable it was assigned to (e.g. auto n = 42). Give each draw the name of its variable. TestCase::draw(gen) without a name prints numbered placeholders (auto draw_1 = ...;) instead.
Next, try a test that fails:
This test asserts that any integer is less than 50, which is obviously incorrect. Hegel will find a test case that makes this assertion fail, and then shrink it to find the smallest counterexample. It reports:
The header names the test and where it is defined, the count says how many cases it took to find the failure, and the falsifying value(s). The last line replays that exact failure. See HEGEL_REPRODUCE_FAILURE.
To fix this test, you can constrain the integers you generate with the min_value and max_value parameters:
Run the test again. It should now pass.
Hegel provides a rich library of generators in the hegel::generators namespace that you can use out of the box. There are primitive generators, such as integers, floats, and text, and combinators that allow you to make generators out of other generators, such as vectors and tuples.
For example, you can use vectors to generate a vector of integers:
This test checks that appending an element to a random vector of integers should always increase its length.
You can also define custom generators. For example, say you have a Person struct that we want to generate:
Note that you can feed the results of a draw to subsequent calls. For example, say that you extend the Person struct to include a driving_license boolean field:
Hegel can also derive generators automatically for reflectable structs via default_generator. This uses reflect-cpp to inspect the struct's fields and pick an appropriate generator for each:
Call .override(...) on the returned generator to customize individual fields (see override).
A test body states what must hold by throwing when it does not. Any exception fails the test case, and Hegel then shrinks the values that caused it:
The assertion macros of a test framework work too. See Hegel and test frameworks for GoogleTest.
Hegel groups counterexamples by origin to tell one bug from another. The origin of a thrown exception is its type and the site it was thrown from. Derive the exception from FailureOrigin to group them some other way, but the origin must be stable.
Drawn values print automatically in the failure report (auto x = ...;). Use TestCase::note to add whatever context the values alone do not show:
Notes and drawn values print on the failing replay only. Raise Settings::verbosity to Verbosity::Verbose to see them for every case.
By default Hegel runs 100 test cases. To override this, write a Settings initializer after the test name:
These settings are the test function's default argument; passing a Settings when invoking the test (self_equality({.test_cases = 5})) replaces them for that run.
With a test framework, write the property inside one of its tests and call hegel::test() with the body. You cannot use HEGEL_TEST there.
Outside a test framework, use HEGEL_TEST, since it derives the database key and test location for you. hegel::test() can still be used, but you will have to set Settings::database_key yourself if you want failures persisted to the example database. You will also have to pass in TestLocation to see test location information in the failure output.