mirror of
https://github.com/catchorg/Catch2.git
synced 2025-08-03 13:55:39 +02:00
Add SUCCEED and FAIL docs next to SKIP docs
This commit is contained in:
114
docs/skipping-passing-failing.md
Normal file
114
docs/skipping-passing-failing.md
Normal file
@@ -0,0 +1,114 @@
|
||||
<a id="top"></a>
|
||||
# Explicitly skipping, passing, and failing tests at runtime
|
||||
|
||||
## Skipping Test Cases at Runtime
|
||||
|
||||
> [Introduced](https://github.com/catchorg/Catch2/pull/2360) in Catch2 X.Y.Z.
|
||||
|
||||
In some situations it may not be possible to meaningfully execute a test case, for example when the system under test is missing certain hardware capabilities.
|
||||
If the required conditions can only be determined at runtime, it often doesn't make sense to consider such a test case as either passed or failed, because it simply can not run at all.
|
||||
To properly express such scenarios, Catch2 allows to explicitly _skip_ test cases, using the `SKIP` macro:
|
||||
|
||||
**SKIP(** [streamable expression] **)**
|
||||
|
||||
Example usage:
|
||||
|
||||
```c++
|
||||
TEST_CASE("copy files between drives") {
|
||||
if(getNumberOfHardDrives() < 2) {
|
||||
SKIP("at least two hard drives required");
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
This test case is then reported as _skipped_ instead of _passed_ or _failed_.
|
||||
|
||||
The `SKIP` macro behaves similarly to an explicit [`FAIL`](logging.md#top), in that it is the last expression that will be executed:
|
||||
|
||||
```c++
|
||||
TEST_CASE("my test") {
|
||||
printf("foo");
|
||||
SKIP();
|
||||
printf("bar"); // not printed
|
||||
}
|
||||
```
|
||||
|
||||
However a failed assertion _before_ a `SKIP` still causes the entire test case to fail:
|
||||
|
||||
```c++
|
||||
TEST_CASE("failing test") {
|
||||
CHECK(1 == 2);
|
||||
SKIP();
|
||||
}
|
||||
```
|
||||
|
||||
### Interaction with Sections and Generators
|
||||
|
||||
Sections, nested sections as well as test cases with [generators](generators.md#top) can all be individually skipped, with the rest executing as usual:
|
||||
|
||||
```c++
|
||||
TEST_CASE("complex test case") {
|
||||
int value = GENERATE(2, 4, 6);
|
||||
SECTION("a") {
|
||||
SECTION("a1") { CHECK(value < 8); }
|
||||
SECTION("a2") {
|
||||
if (value == 4) {
|
||||
SKIP();
|
||||
}
|
||||
CHECK(value % 2 == 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This test case will report 5 passing assertions; one for each of the three values in section `a1`, as well as one for each in `a2`, except for when `value == 4`.
|
||||
|
||||
Note that as soon as one section is skipped, the entire test case will be reported as _skipped_ (unless there is a failing assertion, in which case it will be reported as _failed_ instead).
|
||||
|
||||
If all test cases in a run are skipped, Catch2 returns a non-zero exit code by default.
|
||||
This can be overridden using the [--allow-running-no-tests](command-line.md#no-tests-override) flag.
|
||||
|
||||
|
||||
## Passing and failing test cases
|
||||
|
||||
Test cases can also be explicitly passed or failed, without the use of
|
||||
assertions, and with a specific message. This can be useful to handle
|
||||
complex preconditions/postconditions and give useful error messages
|
||||
when they fail.
|
||||
|
||||
* `SUCCEED( [streamable expression] )`
|
||||
|
||||
`SUCCEED` is morally equivalent with `INFO( [streamable expression] ); REQUIRE( true );`.
|
||||
Note that it does not stop further test execution, so it cannot be used
|
||||
to guard failing assertions from being executed.
|
||||
|
||||
_In practice, `SUCCEED` is usually used as a test placeholder, to avoid
|
||||
[failing a test case due to missing assertions](command-line.md#warnings)._
|
||||
|
||||
```cpp
|
||||
TEST_CASE( "SUCCEED showcase" ) {
|
||||
int I = 1;
|
||||
SUCCEED( "I is " << I );
|
||||
// ... execution continues here ...
|
||||
}
|
||||
```
|
||||
|
||||
* `FAIL( [streamable expression] )`
|
||||
|
||||
`FAIL` is morally equivalent with `INFO( [streamable expression] ); REQUIRE( false );`.
|
||||
|
||||
_In practice, `FAIL` is usually used to stop executing test that is currently
|
||||
known to be broken, but has to be fixed later._
|
||||
|
||||
```cpp
|
||||
TEST_CASE( "FAIL showcase" ) {
|
||||
FAIL( "This test case causes segfault, which breaks CI." );
|
||||
// ... this will not be executed ...
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
---
|
||||
|
||||
[Home](Readme.md#top)
|
Reference in New Issue
Block a user