3rd-party/catch2
1
0
Fork 0
mirror of https://github.com/catchorg/Catch2.git synced 2025-08-24 23:45:40 +02:00
Code Issues Packages Projects Releases Wiki Activity

Make message macros (FAIL, WARN, INFO, etc) thread safe

Browse Source 
This builds on the existing work to make assertion thread safe,
by adding an extra synchronization point in the holder of
`ReusableStringStream`'s stream instances, as those are used to
build the messages, and finishing the move of message scope holders
to be thread-local.
This commit is contained in:
Martin Hořeňovský
2025-08-23 00:15:53 +02:00
parent f4e05a67bb
commit 582200a1f8
7 changed files with 165 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

131
docs/thread-safety.md
View File

@@ -2,7 +2,9 @@
# Thread safety in Catch2
**Contents**<br>
[Using assertion macros from multiple threads](#using-assertion-macros-from-multiple-threads)<br>
[Using assertion macros from spawned threads](#using-assertion-macros-from-spawned-threads)<br>
[Assertion-like message macros and spawned threads](#assertion-like-message-macros-and-spawned-threads)<br>
[Message macros and spawned threads](#message-macros-and-spawned-threads)<br>
[examples](#examples)<br>
[`STATIC_REQUIRE` and `STATIC_CHECK`](#static_require-and-static_check)<br>
[Fatal errors and multiple threads](#fatal-errors-and-multiple-threads)<br>
@@ -10,17 +12,18 @@
> Thread safe assertions were introduced in Catch2 3.9.0
Thread safety in Catch2 is currently limited to all the assertion macros.
Interacting with benchmark macros, message macros (e.g. `INFO` or `CAPTURE`),
sections macros, generator macros, or test case macros is not thread-safe.
The message macros are likely to be made thread-safe in the future, but
the way sections define test runs is incompatible with user being able
to spawn threads arbitrarily, thus that limitation is here to stay.
Thread safety in Catch2 is currently limited to all the assertion macros,
and to message or message-adjacent macros (e.g. `INFO` or `WARN`).
Interacting with benchmark macros, sections macros, generator macros, or
test case macros is not thread-safe. The way sections define paths through
the test is incompatible with user spawning threads arbitrarily, so this
limitation is here to stay.
**Important: thread safety in Catch2 is [opt-in](configuration.md#experimental-thread-safety)**
## Using assertion macros from multiple threads
## Using assertion macros from spawned threads
The full set of Catch2's runtime assertion macros is thread-safe. However,
it is important to keep in mind that their semantics might not support
@@ -30,7 +33,7 @@ Specifically, the `REQUIRE` family of assertion macros have semantics
of stopping the test execution on failure. This is done by throwing
an exception, but since the user-spawned thread will not have the test-level
try-catch block ready to catch the test failure exception, failing a
`REQUIRE` assertion inside this thread will terminate the process.
`REQUIRE` assertion inside user-spawned thread will terminate the process.
The `CHECK` family of assertions does not have this issue, because it
does not try to stop the test execution.
@@ -38,16 +41,32 @@ does not try to stop the test execution.
Note that `CHECKED_IF` and `CHECKED_ELSE` are also thread safe (internally
they are assertion macro + an if).
**`SKIP()`, `FAIL()`, `SUCCEED()` are not assertion macros, and are not
thread-safe.**
## Assertion-like message macros and spawned threads
Similarly to assertion macros, not all assertion-like message macros can
be used from spawned thread.
`SKIP` and `FAIL` macros stop the test execution. Just like with `REQUIRE`,
this means that they cannot be used inside user-spawned threads. `SUCCEED`,
`FAIL_CHECK` and `WARN` do not attempt to stop the test execution and
thus can be used from any thread.
## Message macros and spawned threads
Macros that add extra messages to following assertion, such as `INFO`
or `CAPTURE`, are all thread safe and can be used in any thread. Note
that these messages are per-thread, and thus `INFO` inside a user-spawned
thread will not be seen by the main thread, and vice versa.
## examples
### `REQUIRE` from main thread, `CHECK` from spawned threads
### `REQUIRE` from the main thread, `CHECK` from spawned threads
```cpp
TEST_CASE( "Failed REQUIRE in main thread is fine" ) {
TEST_CASE( "Failed REQUIRE in the main thread is fine" ) {
std::vector<std::jthread> threads;
for ( size_t t = 0; t < 16; ++t) {
threads.emplace_back( []() {
@@ -85,7 +104,7 @@ TEST_CASE( "Successful REQUIRE in spawned thread is fine" ) {
This will also work as expected, because the `REQUIRE` is successful.
```cpp
TEST_CASE( "Failed REQUIRE in spawned thread is fine" ) {
TEST_CASE( "Failed REQUIRE in spawned thread kills the process" ) {
std::vector<std::jthread> threads;
for ( size_t t = 0; t < 16; ++t) {
threads.emplace_back( []() {
@@ -99,12 +118,88 @@ TEST_CASE( "Failed REQUIRE in spawned thread is fine" ) {
This will fail catastrophically and terminate the process.
### INFO across threads
```cpp
TEST_CASE( "messages don't cross threads" ) {
std::jthread t1( [&]() {
for ( size_t i = 0; i < 100; ++i ) {
INFO( "spawned thread #1" );
CHECK( 1 == 1 );
}
} );
std::thread t2( [&]() {
for (size_t i = 0; i < 100; ++i) {
UNSCOPED_INFO( "spawned thread #2" );
}
} );
for (size_t i = 0; i < 100; ++i) {
CHECK( 1 == 2 );
}
}
```
None of the failed checks will show the "spawned thread #1" message, as
that message is for the `t1` thread. If the reporter shows passing
assertions (e.g. due to the tests being run with `-s`), you will see the
"spawned thread #1" message alongside the passing `CHECK( 1 == 1 )` assertion.
The message "spawned thread #2" will never be shown, because there are no
assertions in `t2`.
### FAIL/SKIP from the main thread
```cpp
TEST_CASE( "FAIL in the main thread is fine" ) {
std::vector<std::jthread> threads;
for ( size_t t = 0; t < 16; ++t) {
threads.emplace_back( []() {
for (size_t i = 0; i < 10; ++i) {
CHECK( true );
CHECK( false );
}
} );
}
FAIL();
}
```
This will work as expected, that is, the process will finish running
normally, the test case will fail and there will be 321 total assertions,
160 passing and 161 failing (`FAIL` counts as failed assertion).
However, when the main thread hits `FAIL`, it will wait for the other
threads to finish due to `std::jthread`'s destructor joining the spawned
thread. Due to this, using `SKIP` is not recommended once more threads
are spawned; while the main thread will bail from the test execution,
the spawned threads will keep running and may fail the test case.
### FAIL/SKIP from spawned threads
```cpp
TEST_CASE( "FAIL/SKIP in spawned thread kills the process" ) {
std::vector<std::jthread> threads;
for ( size_t t = 0; t < 16; ++t) {
threads.emplace_back( []() {
for (size_t i = 0; i < 10'000; ++i) {
FAIL();
}
} );
}
}
```
As with failing `REQUIRE`, both `FAIL` and `SKIP` in spawned threads
terminate the process.
## `STATIC_REQUIRE` and `STATIC_CHECK`
None of `STATIC_REQUIRE`, `STATIC_REQUIRE_FALSE`, `STATIC_CHECK`, and
`STATIC_CHECK_FALSE` are currently thread safe. This might be surprising
given that they are a compile-time checks, but they also rely on the
message macros to register the result with reporter at runtime.
All of `STATIC_REQUIRE`, `STATIC_REQUIRE_FALSE`, `STATIC_CHECK`, and
`STATIC_CHECK_FALSE` are thread safe in the delayed evaluation configuration.
## Fatal errors and multiple threads