Hegel 0.11.1
Property-based testing for C++
Loading...
Searching...
No Matches
Hegel

Hegel is a property-based testing library for C++. Hegel is based on Hypothesis, using the libhegel engine.

Getting started

This guide walks you through the basics of installing Hegel and writing your first tests.

Install Hegel

Using CMake:

include(FetchContent)
FetchContent_Declare(
hegel
GIT_REPOSITORY https://github.com/hegeldev/hegel-cpp.git
GIT_TAG v0.11.1
)
FetchContent_MakeAvailable(hegel)
target_link_libraries(your_target PRIVATE hegel)

Hegel requires CMake 3.14 and, by default, a C++20 compiler. The build downloads a small prebuilt shared library (libhegel, Hegel's native engine) for your platform; no other tooling is required.

To consume Hegel from C++17, configure with -DHEGEL_REFLECTION=OFF. This drops the reflect-cpp dependency: you lose default_generator (type-directed derivation for structs), but every other generator and combinator still works. (The designated-initializer parameter API, e.g. integers<int>({.min_value = 0}), then relies on a GCC/Clang C++17 extension.)

Write your first test

You're now ready to write your first test. In a new file:

#include <hegel/hegel.h>
namespace gs = hegel::generators;
HEGEL_TEST(self_equality)(hegel::TestCase& tc) {
auto n = tc.draw("n", gs::integers<int>());
if (n != n) { // integers should always be equal to themselves
throw std::runtime_error("self-equality failed");
}
}
int main() {
self_equality();
return 0;
}
Handle to the currently-executing test case.
Definition test_case.h:40
#define HEGEL_TEST(name,...)
Define a Hegel property test.
Definition hegel.h:575
Hegel generators.
Definition core.h:17

Now build and run the test. You should see that this test passes.

Let's look at what's happening in more detail. HEGEL_TEST defines a property test as an ordinary function, self_equality. Running the test executes your test body 100 times by default. While you may call the function from main(), if you use a test framework, write the property inside one of its tests instead. We officially support gtest (see Hegel and test frameworks). Please make an issue on Github if Hegel does not integrate well with your framework.

The body receives a TestCase, which provides a TestCase::draw() method for drawing different values. This test draws a random integer and checks that it should be equal to itself. The macro also names the test in Hegel's example database, so a failure found in one run is replayed first in the next.

The name you pass to TestCase::draw() labels the value in the failure report, so a failing run replays each drawn value under the variable it was assigned to (e.g. auto n = 42). Give each draw the name of its variable. TestCase::draw(gen) without a name prints numbered placeholders (auto draw_1 = ...;) instead.

Next, try a test that fails:

HEGEL_TEST(below_50)(hegel::TestCase& tc) {
auto n = tc.draw("n", gs::integers<int>());
if (n >= 50) { // this will fail!
throw std::runtime_error("n should be below 50");
}
}

This test asserts that any integer is less than 50, which is obviously incorrect. Hegel will find a test case that makes this assertion fail, and then shrink it to find the smallest counterexample. It reports:

--- Failure: below_50 (my_test.cpp:3) ----------------------------------
Falsified after 3 test cases (0 discarded):
auto n = 50;
Exception: std::runtime_error: n should be below 50
rerun with: HEGEL_REPRODUCE_FAILURE(below_50, "AAEAAAAACgEAAAAy")
#define HEGEL_REPRODUCE_FAILURE(name, blob,...)
Replay a failing example for a HEGEL_TEST from its blob.
Definition hegel.h:613

The header names the test and where it is defined, the count says how many cases it took to find the failure, and the falsifying value(s). The last line replays that exact failure. See HEGEL_REPRODUCE_FAILURE.

To fix this test, you can constrain the integers you generate with the min_value and max_value parameters:

HEGEL_TEST(below_50)(hegel::TestCase& tc) {
auto n = tc.draw(
"n", gs::integers<int>({.min_value = 0, .max_value = 49}));
if (n >= 50) {
throw std::runtime_error("n should be below 50");
}
}

Run the test again. It should now pass.

Use generators

Hegel provides a rich library of generators in the hegel::generators namespace that you can use out of the box. There are primitive generators, such as integers, floats, and text, and combinators that allow you to make generators out of other generators, such as vectors and tuples.

For example, you can use vectors to generate a vector of integers:

namespace gs = hegel::generators;
HEGEL_TEST(push_back_grows)(hegel::TestCase& tc) {
auto vector = tc.draw("vector", gs::vectors(gs::integers<int>()));
auto initial_length = vector.size();
vector.push_back(tc.draw(gs::integers<int>()));
if (vector.size() <= initial_length) {
throw std::runtime_error("push_back should increase size");
}
}

This test checks that appending an element to a random vector of integers should always increase its length.

You can also define custom generators. For example, say you have a Person struct that we want to generate:

struct Person {
int age;
std::string name;
};
auto generate_person() {
return gs::compose([](const hegel::TestCase& tc) {
int age = tc.draw(gs::integers<int>());
std::string name = tc.draw(gs::text());
return Person{age, name};
});
}
T draw(const generators::Generator< T > &gen) const
Draw a random value from a generator.
Definition core.h:264

Note that you can feed the results of a draw to subsequent calls. For example, say that you extend the Person struct to include a driving_license boolean field:

struct Person {
int age;
std::string name;
bool driving_license;
};
auto generate_person() {
return gs::compose([](const hegel::TestCase& tc) {
int age = tc.draw(gs::integers<int>());
std::string name = tc.draw(gs::text());
bool driving_license =
age >= 18 ? tc.draw(gs::booleans()) : false;
return Person{age, name, driving_license};
});
}

Hegel can also derive generators automatically for reflectable structs via default_generator. This uses reflect-cpp to inspect the struct's fields and pick an appropriate generator for each:

struct Person {
std::string name;
int age;
};
HEGEL_TEST(generate_people)(hegel::TestCase& tc) {
auto p = tc.draw("p", gs::default_generator<Person>());
}

Call .override(...) on the returned generator to customize individual fields (see override).

Check properties

A test body states what must hold by throwing when it does not. Any exception fails the test case, and Hegel then shrinks the values that caused it:

HEGEL_TEST(running_sum_stays_positive)(hegel::TestCase& tc) {
auto l = tc.draw("l", gs::vectors(gs::integers<int>()));
if (lowest_running_sum(l) < 0) {
throw std::runtime_error("running sum went negative");
}
}
--- Failure: running_sum_stays_positive (my_test.cpp:17) ---------------
Falsified after 2 test cases (0 discarded):
auto l = std::vector<int>{-1};
Exception: std::runtime_error: running sum went negative
rerun with: HEGEL_REPRODUCE_FAILURE(running_sum_stays_positive,
"AAMAAAABAQAKAQAAAP8BAA==")

The assertion macros of a test framework work too. See Hegel and test frameworks for GoogleTest.

Hegel groups counterexamples by origin to tell one bug from another. The origin of a thrown exception is its type and the site it was thrown from. Derive the exception from FailureOrigin to group them some other way, but the origin must be stable.

Debug your failing test cases

Drawn values print automatically in the failure report (auto x = ...;). Use TestCase::note to add whatever context the values alone do not show:

HEGEL_TEST(addition_commutes)(hegel::TestCase& tc) {
auto x = tc.draw("x", gs::integers<int>());
auto y = tc.draw("y", gs::integers<int>());
tc.note("x + y = " + std::to_string(x + y) +
", y + x = " + std::to_string(y + x));
if (x + y != y + x) {
throw std::runtime_error("addition is not commutative");
}
}
void note(std::string_view message) const
Record a message that will be printed on the final replay of a failing test case.

Notes and drawn values print on the failing replay only. Raise Settings::verbosity to Verbosity::Verbose to see them for every case.

Change the number of test cases

By default Hegel runs 100 test cases. To override this, write a Settings initializer after the test name:

HEGEL_TEST(self_equality, {.test_cases = 500})(hegel::TestCase& tc) {
auto n = tc.draw("n", gs::integers<int>());
if (n != n) {
throw std::runtime_error("self-equality failed");
}
}

These settings are the test function's default argument; passing a Settings when invoking the test (self_equality({.test_cases = 5})) replaces them for that run.

Hegel and test frameworks

With a test framework, write the property inside one of its tests and call hegel::test() with the body. You cannot use HEGEL_TEST there.

#include <gtest/gtest.h>
#include <hegel/hegel.h>
TEST(Arithmetic, AdditionCommutes) {
auto x = tc.draw("x", gs::integers<int>());
auto y = tc.draw("y", gs::integers<int>());
ASSERT_EQ(x + y, y + x);
});
}
void test(const std::function< void(TestCase &)> &test_fn, const Settings &settings={}, const std::vector< std::string > &failure_blobs={}, const char *caller_file=HEGEL_CALLER_FILE, const char *caller_function=HEGEL_CALLER_FUNCTION, int caller_line=HEGEL_CALLER_LINE)
Run a Hegel test.
--- Failure: Arithmetic.AdditionCommutes (arithmetic_test.cpp:8) ---
Falsified after 2 test cases (0 discarded):
auto x = 51;
auto y = 0;
Exception: hegel::GTestFailure: arithmetic_test.cpp:11: Expected: (x + y) ==
(y + x), actual: 51 vs 0
The exception the GoogleTest integration raises for a test case that fails an assertion.
Definition gtest.h:51

Outside a test framework, use HEGEL_TEST, since it derives the database key and test location for you. hegel::test() can still be used, but you will have to set Settings::database_key yourself if you want failures persisted to the example database. You will also have to pass in TestLocation to see test location information in the failure output.

Learning more

  • Browse the hegel::generators namespace for the full list of available generators.
  • See Settings for more configuration settings to customise how your test runs.