d548be26e3
* Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2.2 KiB
Skipping Test Cases at Runtime
Introduced in Catch2 X.Y.Z.
In some situations it may not be possible to meaningfully execute a test case, for example when the system under test is missing certain hardware capabilities.
If the required conditions can only be determined at runtime, it often doesn't make sense to consider such a test case as either passed or failed, because it simply can not run at all.
To properly express such scenarios, Catch2 allows to explicitly skip test cases, using the SKIP macro:
SKIP( message expression )
Example usage:
TEST_CASE("copy files between drives") {
if(getNumberOfHardDrives() < 2) {
SKIP("at least two hard drives required");
}
// ...
}
This test case is then reported as skipped instead of passed or failed.
The SKIP macro behaves similarly to an explicit FAIL, in that it is the last expression that will be executed:
TEST_CASE("my test") {
printf("foo");
SKIP();
printf("bar"); // not printed
}
However a failed assertion before a SKIP still causes the entire test case to fail:
TEST_CASE("failing test") {
CHECK(1 == 2);
SKIP();
}
Interaction with Sections and Generators
Sections, nested sections as well as test cases with generators can all be individually skipped, with the rest executing as usual:
TEST_CASE("complex test case") {
int value = GENERATE(2, 4, 6);
SECTION("a") {
SECTION("a1") { CHECK(value < 8); }
SECTION("a2") {
if (value == 4) {
SKIP();
}
CHECK(value % 2 == 0);
}
}
}
This test case will report 5 passing assertions; one for each of the three values in section a1, as well as one for each in a2, except for when value == 4.
Note that as soon as one section is skipped, the entire test case will be reported as skipped (unless there is a failing assertion, in which case it will be reported as failed instead).
If all test cases in a run are skipped, Catch2 returns a non-zero exit code by default. This can be overridden using the --allow-running-no-tests flag.