This means that:
1) Scoped messages are always removed at the end of their scope,
even if the scope ended due to an exception.
2) Scoped messages outlive section end, if that section's scope is
enclosed in their own.
Previously neither of these were true, which has led to a number
of surprising behaviour, where e.g. this:
```cpp
TEST_CASE() {
try {
INFO( "some info" );
throw std::runtime_error( "ex" );
} catch (std::exception const&) {}
REQUIRE( false );
}
```
would print "some info" as the message for the assertion, while this:
```cpp
TEST_CASE() {
INFO("Hello");
SECTION("dummy") {}
REQUIRE(false);
}
```
would not print out "Hello" as the message for the assertion.
This had an underlying reason, in that it was trying to helpfully
keep the messages around in case of unexpected exceptions, so that
code like this:
```cpp
TEST_CASE() {
auto [input, expected] = GENERATE(...);
CAPTURE(input);
auto result = transform(input); // throws
REQUIRE(result == expected);
}
```
would report the value of `input` when `transform` throws. However,
it was surprising in practice and was causing various issues around
handling of messages in other cases.
Closes#1759Closes#2019Closes#2959
`handleNonExpr` is responsible for handling assertions that do not
result in a decomposable expression, e.g. `REQUIRE_THROWS`, or
`REQUIRE_NOTHROW`.
Running benchmark on these two macros specifically, with
`REQUIRE_THROWS([](){ throw 1; }())`, and `REQUIRE_NOTHROW([](){}())`,
we get these speedups:
| | Debug | Release |
|---------|--------|---------|
| THROWS | 3.69x | 2.10x |
| NOTHROW | 1.18x | 1.05x |
Obviously the actual performance improvement is dependent on
how expensive the expression under test is.
This costs us about 1% perf in Debug build and 3% in Release build,
but it is worth it for more precise information during unexpected
exceptions or fatal errors.
Given a simple test case like this
```cpp
TEST_CASE("Hard fail") {
REQUIRE( 1 == 1 );
REQUIRE( 2 == 2 );
throw 1;
REQUIRE( 3 == 3 );
}
```
Catch2 before this change would report the line info from the
`TEST_CASE` macro as the last seen expression before error. With
this change, it will correctly report the line info from the
`REQUIRE(2 == 2)` assertion as the last seen expression before error.
As it is an internal helper for RunContext to speed-up handling of
succesfull assertions, I have no idea why it was added to the
interface back when it was first implemented.
The fast path allows `RunContext` to skip disabling output redirect,
and notifying the reporters, turning `RunContext::notifyAssertionStarted`
into a no-op. This improves the overall performance of assertion handling
significantly, and also prepares ground for future changes around
assertion handling and thread safety.
For simple 10M assertion run, this improves the running time by ~30%
in Debug build and ~40% in Release build.
For backwards-compatibility reasons, the fast path is disabled
by default. However, none of the first party reporters use the
`assertionStarting` event, so all first party reporters are opted-in.
There were only two places where we used the full `AssertionInfo`
instance in `m_lastAssertionInfo`:
1) when reporting unexpected exception from running a test case
2) when reporting fatal error
because in those two places we do not have access to a real
instance of `AssertionInfo`, but we still need to send one to the
reporters. As a bonus, in both of these places we were already
constructing a fake-ish assertion info, by using the last encountered
source location, but dummying out the other information.
Instead, we only keep track of the last encountered source location,
and construct the dummy `AssertionInfo` on-demand.
This finishes the set of refactoring around `m_lastAssertionInfo`
in `RunContext` and improves the performance of running assertions
by ~5% in both Debug and Release mode.
--------------
Note that this change also causes small difference in output. It
could be avoided by having an invalidation flag and tracking where
the information would be invalidated before, but the difference
includes more precise line location for unexpected errors (both
exceptions and fatals), so I prefer the new output.
Because `RunContext::populateReaction` no longer implicitly depends
on the value of `RunContext::m_lastAssertionInfo`, we don't have to
delay the reset until `RunContext::reportExpr` is finished.
`RunContext::resetAssertionInfo` overwrites everything in
`m_lastAssertionInfo` except for the lineInfo, so if the reset
is called at the end of a copying function, we can save work
by copying only the lineInfo.
Adding unreachable to `FAIL` and `SKIP` made Clang become very good
at figuring out that tests of `FAIL` and `SKIP` will never return
and thus could be marked as [[noreturn]].
To avoid introducing lot of warning suppression noise into the
test files, we just disable it instead.
This helps the compiler see that the execution does not continue,
which in turn means that it is not an error if the user does not
return value from the function after FAIL/SKIP.
Closes#2941
Right now `TEMPLATE_TEST_CASE_SIG` fails to compile when the signature contains only types:
```
TEMPLATE_TEST_CASE_SIG(
"TemplateTestSig: compiles with two type parameters",
"[template][onlytypes]",
((typename U, typename V), U, V), (int,int)) {}
```
The trick is to resolve the ambiguity between the two overloads of
`get_wrapper` (`TypeList` and `Nttp`) by making one match more strongly.
We also need to allow `reg_test` to register more than one type.
Add unit tests.
Fixes#2680
---------
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
Move `catch_test_run_info.hpp` from directory
`src/catch2/internal/` to be directly under `src/catch2/`.
Header `catch_test_run_info.hpp` defines `Catch::TestRunInfo` which is part of the public API, since it is required when overriding `Catch::EventListenerBase`.
Fixes issue caught by Clang Tidy misc-include-cleaner check.
Passing the CLI argument '--list-tags' outputs a list like:
```
All available tags:
11 [mytag]
4 [myothertag]
2 tags
```
However, the width of the tag-count column was previously hardcoded to 2, such that the formatting broke (with the tag names awkwardly misaligned/displaced) for 3+ digit tag counts. This occurred whenever a tag contains one hundred or more tests, and would result in formatting like:
```
All available tags:
11 [mytag]
100 [myjumbotag]
4 [myothertag]
3 tags
```
This patch pre-computes the maximum number of digits among the tag counts, expanding the padding when there are more than 100 tests in a tag (and handling when tags are empty). It now outputs e.g.
```
All available tags:
11 [mytag]
100 [myjumbotag]
4 [myothertag]
3 tags
```
With -fsanitize=integer every over/under-flowing integer manipulation triggers a warning.
This is extremely useful as it allows to find some non-obvious bugs such as
for(size_t i = 0; i < N - 1; i++) { ... }
But it comes with a lot of false positives, for instance with every hash function
doing shifting on unsigned integer. Random number generators are also often detected
with this sanitizer.
This marks a few of these functions as safe in this case.
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
`gmtime*` on Windows fails on dates pre 1970, and because we didn't
check the return code, we would then pass invalid `tm` struct to
`strftime` causing it to assert.
Closes#2944
Turns out that even in GCC, the expression in `__builtin_cosntant_p`
can end up evaluated and side-effects executed. To allow users to
work around this bug, I added a configuration option to disable its
use in internal macros.
Related to #2925
While isatty() is a POSIX interface and theoretically could be used
more broadly than on Linux and macOS, use a conservative approach and
use it on any platform that uses GNU libc.
In case the warning -Werror=conversion is active with GCC, the warnings
about "conversion from A to B may change value" lead to a compilation
error. This explicitly convert the values to address these warnings.
The main reason for this is to be able to distinguish between
different errors (or "errors") based on the return code. Before
this change, it was impossible to use the exit code to figure out
whether a test binary failed because all tests were skipped or
because exactly 4 assertions have failed.
This meant that using `catch_discover_tests` and telling it to
check for exit code == 4 to determine skipped tests could lead to
false negatives.
Most of these will not matter in practice due to C++14 imposing
significant limitations on what else we can make constexpr, and we cannot
have references outliving the constexpr context either way.
