The main reason for this is to be able to distinguish between
different errors (or "errors") based on the return code. Before
this change, it was impossible to use the exit code to figure out
whether a test binary failed because all tests were skipped or
because exactly 4 assertions have failed.
This meant that using `catch_discover_tests` and telling it to
check for exit code == 4 to determine skipped tests could lead to
false negatives.
Most of these will not matter in practice due to C++14 imposing
significant limitations on what else we can make constexpr, and we cannot
have references outliving the constexpr context either way.
This rework changes two important things
1) the output redirect is deactivated while control is given to the reporters.
This means that combining reporters that write to stdout with capturing
reporters, e.g. `./tests -s -r console -r junit::out=junit.xml`, no
longer leads to the capturing reporter seeing all the output from
the other reporter captured.
Trying this with the `SelfTest` binary would previously lead to JUnit
spending **hours** trying to escape all of ConsoleReporter's output and
write it to the output file. I actually ended up killing the process
after 3 hours, during which the JUnit reporter wrote something like 50 MBs
of output to a file.
2) The redirect object's lifetime is tied to the `RunContext`, instead
of being constructed for every partial test case run separately.
This has no effect on the basic StreamRedirect, but improves the FileRedirect
significantly. Previously, running many tests in single process with this
redirect (e.g. running `SelfTest -r junit`) would cause later tests to
always fail before starting, due to exceeding the limit of temporary files.
For the current `SelfTest` binary, the old implementation would lead to
**295** test failures from not being able to initiate the redirect. The
new implementation completely eliminates them.
----
There is one downside to the new implementation of FileRedirect, specific
to Linux. Running the `SelfTest` binary on Linux causes 3-4 tests to have
no captured stdout/stderr, even though the tests were supposed to be
writing there (there was no output to the actual stdout/stderr either,
the output was just completely lost).
Since this never happen for smaller test case sets, nor does it reproduce
on other platforms, this implementation is still strictly better than
the old one, and thus it can get reasonably merged.
Move `catch_case_sensitive.hpp` from directory
`src/catch2/internal/internal` to be directly under `src/catch2/`.
Header `catch_case_sensitive.hpp` defines `enum CaseSensitive` which is
part of the public API, since it is exposed by the string matcher
contracts.
Fixes issue caught by Clang Tidy misc-include-cleaner check.
This is done by no longer requiring all assertions to be seen
by the JUnit reporter. Since the JUnit reporter never outputs
all assertions, even with `-s`, the only difference from storing
passing assertions in the `CumulativeReporterBase` was some
bookkeeping in deciding which sections are relevant to the output.
Given `TEST_CASE` that just runs many passing assertions, e.g.
```
TEST_CASE( "PERFORMANCE_TEST_CASE", "[.]" ) {
for (size_t i = 0; i < 1000'000'000; ++i) {
REQUIRE( i == i );
}
}
```
the new JUnit reporter will finish in 5:47, using up ~7.7 MB of RAM.
The old JUnit reporter would finish in 0:30, due to bad_alloc
after using up 14.5 GB of RAM (the system has 16 GB total).
If the total number of assertions is lowered to 10M, the old
implementation uses up ~7.1 GB of RAM and finishes in 12 minutes.
The new implementation still needs only ~7.7 MB of RAM, and finishes
in 4 minutes.
There is a slight downside in that the output is slightly different;
the new implementation will include the `TEST_CASE` level section
in output, even if it does not have its own assertion. In other words,
given a `TEST_CASE` like this
```
TEST_CASE( "JsonWriter", "[JSON][JsonWriter]" ) {
std::stringstream stream;
SECTION( "Newly constructed JsonWriter does nothing" ) {
Catch::JsonValueWriter writer{ stream };
REQUIRE( stream.str() == "" );
}
SECTION( "Calling writeObject will create an empty pair of braces" ) {
{ auto writer = Catch::JsonValueWriter{ stream }.writeObject(); }
REQUIRE( stream.str() == "{\n}" );
}
}
```
the new implementation will output 3 `testcase` tags, 2 for the explicit
`SECTION`s with tests, and 1 for the top level section.
However, this can be worked-around if required, and the performance
improvement is such that it is worth changing the current output,
even if it needs to be fixed in the future.
Closes#2897
This PR introduces a new `TEST_CASE` macro called `TEST_CASE_PERSISTENT_FIXTURE`. `TEST_CASE_PERSISTENT_FIXTURE` offers the same functionality as `TEST_CASE_METHOD` except for one difference. The object on which the test method is invoked is only created once for all invocations of the test case. The object is created just after the `testCaseStarting` event is broadcast and the object is destroyed just before the `testCaseEnding` event is broadcast.
The main motivation for this new functionality is to allow `TEST_CASE`s to do expensive setup and teardown once per `TEST_CASE`, without having to resort to abusing event listeners or static function variables with manual initialization.
Implements #1602
---------
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
Technically we should be able to remove moveability as well, but
then we would need a larger surgery. This already improves the
compile times and binary sizes by a surprising amount.
This PR adds functionality to skip around ANSI escape sequences in catch_textflow so they do not contribute to line length and line wrapping code does not split escape sequences in the middle. I've implemented this by creating a AnsiSkippingString abstraction that has a bidirectional iterator that can skip around escape sequences while iterating. Additionally I refactored Column::const_iterator to be iterator-based rather than index-based so this abstraction is a simple drop-in for std::string.
Currently only color sequences are handled, other escape sequences are left unaffected.
Motivation: Text with ANSI color sequences gets messed up when being output by Catch2 #2833.
`max_digits10` is the number of decimal digits that the type _can_
represent, in other words, the number of decimal digits needed to
disambiguate any two floating point numbers when serialized to
a string.
Closes#1210
When a signal is caught, the destructors of Sections will not be called.
Thus, we must call `sectionEndedEarly` manually for those Sections.
Example test case:
```
TEST_CASE("broken") {
SECTION("section") {
/// Use illegal cpu instruction
__asm__ __volatile__("ud2" : : : "memory");
}
}
```
Remove `#pragma intrinsic(_umul128)` because it doesn't work on
ARM64EC and x64 works without it.
Co-authored-by: Agoston Szepessy <agos@microsoft.com>
The WithinRelMatcher does not listen to the precision specification; it just naively pipes to stringstream. If you specify the precision, that has no impact on the outputted values when the match fails.
This fix makes the WithinRelMatcher listen to the precision (https://github.com/catchorg/Catch2/blob/devel/docs/tostring.md#floating-point-precision) specified by: Catch::StringMaker<float>::precision
The issue was that `capture_by_value` was meant to be specialized
by plain type, e.g. `capture_by_value<std::weak_ordering>`, but
the TMP in Decomposer did not properly throw away `const` (and
`volatile`) qualifiers, before taking the value of `capture_by_value`.
Our implementation should be slightly faster, and has the
advantage of being consistent between platforms. This does not
have immediate user impact, because we currently use random_device
to generate random seed for resampling, but if we decide to change
this in the future, it is one less place to fix.
Now we use intrinsics when possible, and fallback to optimized
implementation in portable C++. The difference is about 4x when
we can use intrinsics and about 2x when we cannot.
This should speed up our Lemire's algorithm implementation nicely.
The previously used `make_unsigned` approach combined with the overload
set of `extendedMult` caused compilation issues on MacOS platform. By
forcing the selection to be one of `std::uintX_t` types we don't need
to complicate the overload set further.
Previously it could be just plain reporter name, e.g. `xml`, but
it could not specify other reporter options. This change is not
particularly useful for the built-in reporters, as it mostly comes
in handy for combining specific custom reporter with custom arguments,
and the built-in reporters do not have those.
The Core Guidelines state "A base class destructor should be either
public and virtual, or protected and non-virtual" so, hopefully, no
static analyser will complain.
INTERFACE should be used on dependencies from other than our source files.
PRIVATE should be used the include happens in our source files.
In this case (src/catch2/internal/catch_debug_console.cpp:20) the include
is from our source file.
I encountered a issue with this when building Catch2 for Android in
combination with BUILD_SHARED_LIBS. Changing the visibility to PRIVATE
fixes the issue.
