Add configuration option to make assertions thread-safe

All the previous refactoring to make the assertion fast paths
smaller and faster also allows us to implement the fast paths
just with thread-local and atomic variables, without full mutexes.

However, the performance overhead of thread-safe assertions is
still significant for single threaded usage:

|  slowdown |  Debug  | Release |
|-----------|--------:|--------:|
| fast path |   1.04x |   1.43x |
| slow path |   1.16x |   1.22x |

Thus, we don't make the assertions thread-safe by default, and instead
provide a build-time configuration option that the users can set to get
thread-safe assertions.

This commit is functional, but it still needs some follow-up work:
 * We do not need full seq_cst increments for the atomic counters,
   and using weaker ones can be faster.
 * We brute-force updating the reporter-friendly totals from internal
   atomic counters by doing it everywhere. We should properly trace
   where this is needed instead.
 * Message macros (`INFO`, `UNSCOPED_INFO`, `CAPTURE`, etc) are not
   made thread safe in this commit, but they can be made thread safe
   in the future, by building on top of this work.
 * Add more tests, including with thread-sanitizer, and compiled
   examples to the repository. Right now, these changes have been
   compiled with tsan manually, but these tests are not added to CI.

Closes #2948

docs/configuration.md

@@ -14,8 +14,10 @@
 [Other toggles](#other-toggles)<br>
 [Enabling stringification](#enabling-stringification)<br>
 [Disabling exceptions](#disabling-exceptions)<br>
+[Disabling deprecation warnings](#disabling-deprecation-warnings)<br>
 [Overriding Catch's debug break (`-b`)](#overriding-catchs-debug-break--b)<br>
 [Static analysis support](#static-analysis-support)<br>
+[Experimental thread safety](#experimental-thread-safety)<br>
 
 Catch2 is designed to "just work" as much as possible, and most of the
 configuration options below are changed automatically during compilation,
@@ -314,6 +316,21 @@ no backwards compatibility guarantees._
 are not meant to be runnable, only "scannable".
 
 
+## Experimental thread safety
+
+> Introduced in Catch2 X.Y.Z
+
+Catch2 can optionally support thread-safe assertions, that means, multiple
+user-spawned threads can use the assertion macros at the same time. Due
+to the performance cost this imposes even on single-threaded usage, Catch2
+defaults to non-thread-safe assertions.
+
+    CATCH_CONFIG_EXPERIMENTAL_THREAD_SAFE_ASSERTIONS     // enables thread safe assertions
+    CATCH_CONFIG_NO_EXPERIMENTAL_THREAD_SAFE_ASSERTIONS  // force-disables thread safe assertions
+
+See [the documentation on thread safety in Catch2](thread-safety.md#top)
+for details on which macros are safe and other notes.
+
 
 ---
 

docs/test-fixtures.md

@@ -4,7 +4,7 @@
 **Contents**<br>
 [Non-Templated test fixtures](#non-templated-test-fixtures)<br>
 [Templated test fixtures](#templated-test-fixtures)<br>
-[Signature-based parameterised test fixtures](#signature-based-parametrised-test-fixtures)<br>
+[Signature-based parameterised test fixtures](#signature-based-parameterised-test-fixtures)<br>
 [Template fixtures with types specified in template type lists](#template-fixtures-with-types-specified-in-template-type-lists)<br>
 
 ## Non-Templated test fixtures

docs/thread-safety.md

@@ -0,0 +1,130 @@
+<a id="top"></a>
+# Thread safety in Catch2
+
+**Contents**<br>
+[Using assertion macros from multiple threads](#using-assertion-macros-from-multiple-threads)<br>
+[examples](#examples)<br>
+[`STATIC_REQUIRE` and `STATIC_CHECK`](#static_require-and-static_check)<br>
+[Fatal errors and multiple threads](#fatal-errors-and-multiple-threads)<br>
+[Performance overhead](#performance-overhead)<br>
+
+> Thread safe assertions were introduced in Catch2 X.Y.Z
+
+Thread safety in Catch2 is currently limited to all the assertion macros.
+Interacting with benchmark macros, message macros (e.g. `INFO` or `CAPTURE`),
+sections macros, generator macros, or test case macros is not thread-safe.
+The message macros are likely to be made thread-safe in the future, but
+the way sections define test runs is incompatible with user being able
+to spawn threads arbitrarily, thus that limitation is here to stay.
+
+**Important: thread safety in Catch2 is [opt-in](configuration.md#experimental-thread-safety)**
+
+
+## Using assertion macros from multiple threads
+
+The full set of Catch2's runtime assertion macros is thread-safe. However,
+it is important to keep in mind that their semantics might not support
+being used from user-spawned threads.
+
+Specifically, the `REQUIRE` family of assertion macros have semantics
+of stopping the test execution on failure. This is done by throwing
+an exception, but since the user-spawned thread will not have the test-level
+try-catch block ready to catch the test failure exception, failing a
+`REQUIRE` assertion inside this thread will terminate the process.
+
+The `CHECK` family of assertions does not have this issue, because it
+does not try to stop the test execution.
+
+Note that `CHECKED_IF` and `CHECKED_ELSE` are also thread safe (internally
+they are assertion macro + an if).
+
+**`SKIP()`, `FAIL()`, `SUCCEED()` are not assertion macros, and are not
+thread-safe.**
+
+
+## examples
+
+### `REQUIRE` from main thread, `CHECK` from spawned threads
+
+```cpp
+TEST_CASE( "Failed REQUIRE in main thread is fine" ) {
+    std::vector<std::jthread> threads;
+    for ( size_t t = 0; t < 16; ++t) {
+        threads.emplace_back( []() {
+            for (size_t i = 0; i < 10'000; ++i) {
+                CHECK( true );
+                CHECK( false );
+            }
+        } );
+    }
+
+    REQUIRE( false );
+}
+```
+This will work as expected, that is, the process will finish running
+normally, the test case will fail and there will be the correct count of
+passing and failing assertions (160000 and 160001 respectively). However,
+it is important to understand that when the main thread fails its assertion,
+the spawned threads will keep running.
+
+
+### `REQUIRE` from spawned threads
+
+```cpp
+TEST_CASE( "Successful REQUIRE in spawned thread is fine" ) {
+    std::vector<std::jthread> threads;
+    for ( size_t t = 0; t < 16; ++t) {
+        threads.emplace_back( []() {
+            for (size_t i = 0; i < 10'000; ++i) {
+                REQUIRE( true );
+            }
+        } );
+    }
+}
+```
+This will also work as expected, because the `REQUIRE` is successful.
+
+```cpp
+TEST_CASE( "Failed REQUIRE in spawned thread is fine" ) {
+    std::vector<std::jthread> threads;
+    for ( size_t t = 0; t < 16; ++t) {
+        threads.emplace_back( []() {
+            for (size_t i = 0; i < 10'000; ++i) {
+                REQUIRE( false );
+            }
+        } );
+    }
+}
+```
+This will fail catastrophically and terminate the process.
+
+
+## `STATIC_REQUIRE` and `STATIC_CHECK`
+
+None of `STATIC_REQUIRE`, `STATIC_REQUIRE_FALSE`, `STATIC_CHECK`, and
+`STATIC_CHECK_FALSE` are currently thread safe. This might be surprising
+given that they are a compile-time checks, but they also rely on the
+message macros to register the result with reporter at runtime.
+
+
+## Fatal errors and multiple threads
+
+By default, Catch2 tries to catch fatal errors (POSIX signals/Windows
+Structured Exceptions) and report something useful to the user. This
+always happened on a best-effort basis, but in presence of multiple
+threads and locks the chance of it working decreases. If this starts
+being an issue for you, [you can disable it](configuration.md#other-toggles).
+
+
+## Performance overhead
+
+In the worst case, which is optimized build and assertions using the
+fast path for successful assertions, the performance overhead of using
+the thread-safe assertion implementation can reach 40%. In other cases,
+the overhead will be smaller, between 4% and 20%.
+
+
+
+---
+
+[Home](Readme.md#top)