Commit Graph

4627 Commits

Author SHA1 Message Date
Martin Hořeňovský
4ed3088190 Remove 'thread safe assertions' from limitations.md
We now provide thread safe assertions as a configuration option.
2025-07-24 21:57:49 +02:00
Martin Hořeňovský
bdc634e9f1 Document that the default test run order is now random 2025-07-24 21:55:30 +02:00
Martin Hořeňovský
2a8a8a7210 Add configuration option to make assertions thread-safe
All the previous refactoring to make the assertion fast paths
smaller and faster also allows us to implement the fast paths
just with thread-local and atomic variables, without full mutexes.

However, the performance overhead of thread-safe assertions is
still significant for single threaded usage:

|  slowdown |  Debug  | Release |
|-----------|--------:|--------:|
| fast path |   1.04x |   1.43x |
| slow path |   1.16x |   1.22x |

Thus, we don't make the assertions thread-safe by default, and instead
provide a build-time configuration option that the users can set to get
thread-safe assertions.

This commit is functional, but it still needs some follow-up work:
 * We do not need full seq_cst increments for the atomic counters,
   and using weaker ones can be faster.
 * We brute-force updating the reporter-friendly totals from internal
   atomic counters by doing it everywhere. We should properly trace
   where this is needed instead.
 * Message macros (`INFO`, `UNSCOPED_INFO`, `CAPTURE`, etc) are not
   made thread safe in this commit, but they can be made thread safe
   in the future, by building on top of this work.
 * Add more tests, including with thread-sanitizer, and compiled
   examples to the repository. Right now, these changes have been
   compiled with tsan manually, but these tests are not added to CI.

Closes #2948
2025-07-24 10:10:00 +02:00
Martin Hořeňovský
900a6d5516 Release script also recognizes version placeholders in 'vX.Y.Z' form 2025-07-24 00:02:48 +02:00
Martin Hořeňovský
1aa6fa215c Backfill release version placeholders that used wrong format 2025-07-24 00:02:08 +02:00
Martin Hořeňovský
5aa8d11321 Clear unscoped messages lazily
This improves the fast path performance for successful assertions
by about 7%, at the cost of potentially keeping around the message
allocation longer.
2025-07-21 21:50:51 +02:00
Martin Hořeňovský
9c5c21df82 Simplify removal of messages from RunContext
Instead of doing range-based removals, we only erase specific message,
because there can never be more than 1 message with the specific ID.
2025-07-21 21:50:49 +02:00
Martin Hořeňovský
10aef62f21 Regularize scoped message lifetime to only consider the object's scope
This means that:
1) Scoped messages are always removed at the end of their scope,
   even if the scope ended due to an exception.
2) Scoped messages outlive section end, if that section's scope is
   enclosed in their own.

Previously neither of these were true, which has led to a number
of surprising behaviour, where e.g. this:
```cpp
TEST_CASE() {
    try {
        INFO( "some info" );
        throw std::runtime_error( "ex" );
    } catch (std::exception const&) {}

    REQUIRE( false );
}
```
would print "some info" as the message for the assertion, while this:
```cpp
TEST_CASE() {
    INFO("Hello");
    SECTION("dummy") {}
    REQUIRE(false);
}
```
would not print out "Hello" as the message for the assertion.

This had an underlying reason, in that it was trying to helpfully
keep the messages around in case of unexpected exceptions, so that
code like this:
```cpp
TEST_CASE() {
    auto [input, expected] = GENERATE(...);
    CAPTURE(input);
    auto result = transform(input); // throws
    REQUIRE(result == expected);
}
```
would report the value of `input` when `transform` throws. However,
it was surprising in practice and was causing various issues around
handling of messages in other cases.

Closes #1759
Closes #2019
Closes #2959
2025-07-21 21:50:47 +02:00
Martin Hořeňovský
98b4bbb35e Deprecate comparison operators on MessageInfo 2025-07-21 21:50:45 +02:00
Martin Hořeňovský
8c3ffe05e1 Skeleton for applying deprecation warnings 2025-07-21 21:50:43 +02:00
Martin Hořeňovský
715558fd97 Use the fast path for passing assertions in RunContext::handleNonExpr
`handleNonExpr` is responsible for handling assertions that do not
result in a decomposable expression, e.g. `REQUIRE_THROWS`, or
`REQUIRE_NOTHROW`.

Running benchmark on these two macros specifically, with
`REQUIRE_THROWS([](){ throw 1; }())`, and `REQUIRE_NOTHROW([](){}())`,
we get these speedups:

|         |  Debug | Release |
|---------|--------|---------|
| THROWS  |  3.69x |   2.10x |
| NOTHROW |  1.18x |   1.05x |

Obviously the actual performance improvement is dependent on
how expensive the expression under test is.
2025-07-20 21:01:10 +02:00
Martin Hořeňovský
55b14e1b34 Update last seen line info in the assertionEnded fast path
This costs us about 1% perf in Debug build and 3% in Release build,
but it is worth it for more precise information during unexpected
exceptions or fatal errors.

Given a simple test case like this
```cpp
TEST_CASE("Hard fail") {
    REQUIRE( 1 == 1 );
    REQUIRE( 2 == 2 );
    throw 1;
    REQUIRE( 3 == 3 );
}
```

Catch2 before this change would report the line info from the
`TEST_CASE` macro as the last seen expression before error. With
this change, it will correctly report the line info from the
`REQUIRE(2 == 2)` assertion as the last seen expression before error.
2025-07-20 18:34:21 +02:00
Martin Hořeňovský
ce128f584c Remove assertionPassed fast path from the capture interface
As it is an internal helper for RunContext to speed-up handling of
succesfull assertions, I have no idea why it was added to the
interface back when it was first implemented.
2025-07-20 18:32:25 +02:00
Martin Hořeňovský
db6171a706 Mention ReporterPreferences::shouldReportAllAssertionStarts in docs 2025-07-17 22:57:18 +02:00
Martin Hořeňovský
c22096846c Let reporters opt into fast path in RunContext::assertionStarting
The fast path allows `RunContext` to skip disabling output redirect,
and notifying the reporters, turning `RunContext::notifyAssertionStarted`
into a no-op. This improves the overall performance of assertion handling
significantly, and also prepares ground for future changes around
assertion handling and thread safety.

For simple 10M assertion run, this improves the running time by ~30%
in Debug build and ~40% in Release build.

For backwards-compatibility reasons, the fast path is disabled
by default. However, none of the first party reporters use the
`assertionStarting` event, so all first party reporters are opted-in.
2025-07-17 13:47:44 +02:00
Martin Hořeňovský
ae33e5b99a Remove empty implementations of assertionStarting from reporters 2025-07-17 12:47:46 +02:00
Martin Hořeňovský
0e8112a762 Only track the last line info in RunContext
There were only two places where we used the full `AssertionInfo`
instance in `m_lastAssertionInfo`:
1) when reporting unexpected exception from running a test case
2) when reporting fatal error

because in those two places we do not have access to a real
instance of `AssertionInfo`, but we still need to send one to the
reporters. As a bonus, in both of these places we were already
constructing a fake-ish assertion info, by using the last encountered
source location, but dummying out the other information.

Instead, we only keep track of the last encountered source location,
and construct the dummy `AssertionInfo` on-demand.

This finishes the set of refactoring around `m_lastAssertionInfo`
in `RunContext` and improves the performance of running assertions
by ~5% in both Debug and Release mode.

--------------

Note that this change also causes small difference in output. It
could be avoided by having an invalidation flag and tracking where
the information would be invalidated before, but the difference
includes more precise line location for unexpected errors (both
exceptions and fatals), so I prefer the new output.
2025-07-17 11:27:03 +02:00
Martin Hořeňovský
9be81c0e05 Move resetAssertionInfo into RunContext::reportExpr
Because `RunContext::populateReaction` no longer implicitly depends
on the value of `RunContext::m_lastAssertionInfo`, we don't have to
delay the reset until `RunContext::reportExpr` is finished.
2025-07-16 21:58:42 +02:00
Martin Hořeňovský
efcb76874e Store only lineinfo if m_lastAssertionInfo is going to be reset
`RunContext::resetAssertionInfo` overwrites everything in
`m_lastAssertionInfo` except for the lineInfo, so if the reset
is called at the end of a copying function, we can save work
by copying only the lineInfo.
2025-07-16 21:52:05 +02:00
Martin Hořeňovský
895b8af6bd Avoid needless copy in handling fatal errors 2025-07-16 21:50:20 +02:00
Martin Hořeňovský
066f00acf5 Pass result disposition into RunContext::populateReaction directly
This avoids implicit dataflow through RunContext::m_lastAssertionInfo,
which will be useful in later refactoring.
2025-07-16 21:43:00 +02:00
Martin Hořeňovský
1b72e45354 Cache config::abortAfter and config::shouldDebugBreak in RunContext 2025-07-16 21:29:53 +02:00
Martin Hořeňovský
f7968e9697 Constify various autoregistry globals to avoid clang-tidy complaints
Closes #2582
2025-07-15 15:23:32 +02:00
Martin Hořeňovský
3ebc346bcb Fix Clang-Tidy's readability-static-accessed-through-instance in tests 2025-07-15 15:23:23 +02:00
Martin Hořeňovský
a483b6d7d3 Default to randomized test case order 2025-07-15 12:10:40 +02:00
Martin Hořeňovský
b9c018b38a Use template variables for our trait-likes in Clara 2025-07-15 11:49:39 +02:00
Martin Hořeňovský
050e14dce0 Disable Wmissing-noreturn for test files
Adding unreachable to `FAIL` and `SKIP` made Clang become very good
at figuring out that tests of `FAIL` and `SKIP` will never return
and thus could be marked as [[noreturn]].

To avoid introducing lot of warning suppression noise into the
test files, we just disable it instead.
2025-07-14 12:15:25 +02:00
Martin Hořeňovský
6097bd6ee9 Mark code past FAIL and SKIP as unreachable
This helps the compiler see that the execution does not continue,
which in turn means that it is not an error if the user does not
return value from the function after FAIL/SKIP.

Closes #2941
2025-07-14 10:39:48 +02:00
Martin Hořeňovský
b62413aee3 Add stream flushes after reporters print out the test run metadata.
This ensures that the test run metadata are printed even if the
process dies right after.

Closes #2964
2025-07-10 23:51:31 +02:00
Martin Hořeňovský
3839e27f05 Generate pkg-config files at install time
This enables us to generate pkg-config with proper paths if the
installation prefix is decided at install time, e.g. with
```
cmake --install build-dir --prefix /path/to/somewhere
```

It also means that we can use CMake's generator expression to get
the _real_ final name of the libraries, e.g. including debug postfix.

Closes #2979
2025-07-10 16:39:46 +02:00
Martin Hořeňovský
39c32b9662 Avoid needless string temporary in template product test registration 2025-07-09 12:29:36 +02:00
Clement Courbet
2de12cb05f Allow using only types in TEMPLATE_TEST_CASE_SIG. (#2995)
Right now `TEMPLATE_TEST_CASE_SIG` fails to compile when the signature contains only types:

```
TEMPLATE_TEST_CASE_SIG(
  "TemplateTestSig: compiles with two type parameters",
  "[template][onlytypes]",
  ((typename U, typename V), U, V), (int,int)) {}
```

The trick is to resolve the ambiguity between the two overloads of
`get_wrapper` (`TypeList` and `Nttp`) by making one match more strongly.
We also need to allow `reg_test` to register more than one type.

Add unit tests.

Fixes #2680

---------

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2025-07-08 14:26:25 +02:00
Omar Boukli-Hacene
8dc9f1a124 Move header from internal
Move `catch_test_run_info.hpp` from directory
`src/catch2/internal/` to be directly under `src/catch2/`.

Header `catch_test_run_info.hpp` defines `Catch::TestRunInfo` which is part of the public API, since it is required when overriding `Catch::EventListenerBase`.

Fixes issue caught by Clang Tidy misc-include-cleaner check.
2025-07-07 17:27:48 +02:00
Martin Hořeňovský
038ee6ea13 Drop windows-2019 server from list of GitHub Action runners 2025-07-02 14:13:57 +02:00
Tyson Jones
bd2d918f32 patched >2-digit tag-count padding in base reporter
Passing the CLI argument '--list-tags' outputs a list like:

```
All available tags:
  11  [mytag]
   4  [myothertag]
2 tags
```

However, the width of the tag-count column was previously hardcoded to 2, such that the formatting broke (with the tag names awkwardly misaligned/displaced) for 3+ digit tag counts. This occurred whenever a tag contains one hundred or more tests, and would result in formatting like:

```
All available tags:
  11  [mytag]
  100  [myjumbotag]
   4  [myothertag]
3 tags
```

This patch pre-computes the maximum number of digits among the tag counts, expanding the padding when there are more than 100 tests in a tag (and handling when tags are empty). It now outputs e.g.

```
All available tags:
   11  [mytag]
  100  [myjumbotag]
    4  [myothertag]
3 tags
```
2025-07-02 14:13:07 +02:00
Jean-Michaël Celerier
d134b0cae3 clang: do not issue bogus warnings about integer manipulation in hash functions with fsanitize=undefined/integer (#2965)
With -fsanitize=integer every over/under-flowing integer manipulation triggers a warning.
This is extremely useful as it allows to find some non-obvious bugs such as

    for(size_t i = 0; i < N - 1; i++) { ... }

But it comes with a lot of false positives, for instance with every hash function
doing shifting on unsigned integer. Random number generators are also often detected
with this sanitizer.

This marks a few of these functions as safe in this case.

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2025-07-02 13:30:36 +02:00
Eisuke Kawashima
ceed26842b fix(python): fix invalid escape sequences 2025-06-24 15:32:27 -06:00
Victor Chernyakin
04cbcfa1d1 Vectorize artwork 2025-06-21 13:30:38 -06:00
Eisuke Kawashima
416b075211 fix: make deleted member function public 2025-06-15 13:26:38 -06:00
Eisuke Kawashima
334827eb53 Replace deprecated FindPythonInterp with FindPython3
fix #2755

https://cmake.org/cmake/help/v3.16/module/FindPythonInterp.html
2025-06-15 13:23:27 -06:00
Chris Thrasher
ac207fc90f Use latest release in docs 2025-06-15 13:05:41 -06:00
Eisuke Kawashima
1de7d0ed7b Remove redundant override specifier from functions that are declared final 2025-06-14 18:42:43 -06:00
Chris Thrasher
65cd203926 Stop testing with GCC 7 and 8
Support for GCC 7 and 8 can continue on an as-needed basis. The
goal is not to explicitly break support for such compilers. Chances
are they will continue to compile Catch2 for some time into the
future. Rather, it no longer seems a prudent use of resources to
continuously test with these compilers.
2025-06-14 11:44:33 -06:00
Vertexwahn
3013cb897b Bazel support: Add license annotations 2025-06-01 21:36:26 -06:00
Martin Hořeňovský
74fcff6e5b Use Start-BitsTransfer for downloading OpenCppCoverage instead
The old `Start-FileDownload` was no longer recognized by the pswh
version on my local machine, this updates the OpenCppCoverage install
script to use a still usable cmdlet.
2025-05-02 22:58:36 +02:00
Chris Thrasher
5abfc0aa9c Add return code constants to API 2025-04-29 09:26:28 -06:00
Martin Hořeňovský
1e7b879fae Small include cleanup 2025-04-29 13:15:34 +02:00
ZXShady
4ff57aba42 Use variable templates 2025-04-27 21:14:04 -06:00
Chris Thrasher
dde8220613 Add VS 2025 CI jobs 2025-04-27 11:08:50 -06:00
Chris Thrasher
cecb02e68f Use presets in CI 2025-04-27 11:08:48 -06:00