This is done by no longer requiring all assertions to be seen
by the JUnit reporter. Since the JUnit reporter never outputs
all assertions, even with `-s`, the only difference from storing
passing assertions in the `CumulativeReporterBase` was some
bookkeeping in deciding which sections are relevant to the output.
Given `TEST_CASE` that just runs many passing assertions, e.g.
```
TEST_CASE( "PERFORMANCE_TEST_CASE", "[.]" ) {
for (size_t i = 0; i < 1000'000'000; ++i) {
REQUIRE( i == i );
}
}
```
the new JUnit reporter will finish in 5:47, using up ~7.7 MB of RAM.
The old JUnit reporter would finish in 0:30, due to bad_alloc
after using up 14.5 GB of RAM (the system has 16 GB total).
If the total number of assertions is lowered to 10M, the old
implementation uses up ~7.1 GB of RAM and finishes in 12 minutes.
The new implementation still needs only ~7.7 MB of RAM, and finishes
in 4 minutes.
There is a slight downside in that the output is slightly different;
the new implementation will include the `TEST_CASE` level section
in output, even if it does not have its own assertion. In other words,
given a `TEST_CASE` like this
```
TEST_CASE( "JsonWriter", "[JSON][JsonWriter]" ) {
std::stringstream stream;
SECTION( "Newly constructed JsonWriter does nothing" ) {
Catch::JsonValueWriter writer{ stream };
REQUIRE( stream.str() == "" );
}
SECTION( "Calling writeObject will create an empty pair of braces" ) {
{ auto writer = Catch::JsonValueWriter{ stream }.writeObject(); }
REQUIRE( stream.str() == "{\n}" );
}
}
```
the new implementation will output 3 `testcase` tags, 2 for the explicit
`SECTION`s with tests, and 1 for the top level section.
However, this can be worked-around if required, and the performance
improvement is such that it is worth changing the current output,
even if it needs to be fixed in the future.
Closes#2897
This PR introduces a new `TEST_CASE` macro called `TEST_CASE_PERSISTENT_FIXTURE`. `TEST_CASE_PERSISTENT_FIXTURE` offers the same functionality as `TEST_CASE_METHOD` except for one difference. The object on which the test method is invoked is only created once for all invocations of the test case. The object is created just after the `testCaseStarting` event is broadcast and the object is destroyed just before the `testCaseEnding` event is broadcast.
The main motivation for this new functionality is to allow `TEST_CASE`s to do expensive setup and teardown once per `TEST_CASE`, without having to resort to abusing event listeners or static function variables with manual initialization.
Implements #1602
---------
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
This PR adds functionality to skip around ANSI escape sequences in catch_textflow so they do not contribute to line length and line wrapping code does not split escape sequences in the middle. I've implemented this by creating a AnsiSkippingString abstraction that has a bidirectional iterator that can skip around escape sequences while iterating. Additionally I refactored Column::const_iterator to be iterator-based rather than index-based so this abstraction is a simple drop-in for std::string.
Currently only color sequences are handled, other escape sequences are left unaffected.
Motivation: Text with ANSI color sequences gets messed up when being output by Catch2 #2833.
`max_digits10` is the number of decimal digits that the type _can_
represent, in other words, the number of decimal digits needed to
disambiguate any two floating point numbers when serialized to
a string.
The WithinRelMatcher does not listen to the precision specification; it just naively pipes to stringstream. If you specify the precision, that has no impact on the outputted values when the match fails.
This fix makes the WithinRelMatcher listen to the precision (https://github.com/catchorg/Catch2/blob/devel/docs/tostring.md#floating-point-precision) specified by: Catch::StringMaker<float>::precision
The issue was that `capture_by_value` was meant to be specialized
by plain type, e.g. `capture_by_value<std::weak_ordering>`, but
the TMP in Decomposer did not properly throw away `const` (and
`volatile`) qualifiers, before taking the value of `capture_by_value`.
Now we use intrinsics when possible, and fallback to optimized
implementation in portable C++. The difference is about 4x when
we can use intrinsics and about 2x when we cannot.
This should speed up our Lemire's algorithm implementation nicely.
The previously used `make_unsigned` approach combined with the overload
set of `extendedMult` caused compilation issues on MacOS platform. By
forcing the selection to be one of `std::uintX_t` types we don't need
to complicate the overload set further.
This was originally motivated by `REQUIRE((a <=> b) == 0)` no
longer compiling using MSVC. After some investigation, I found
that they changed their implementation of the zero literal
detector from the previous pointer-constructor with deleted
other constructors, into one that uses `consteval` constructor
from int.
This breaks the previous detection logic, because now
`is_foo_comparable<std::strong_ordering, int>` is true, but
actually trying to compare them is a compile-time error...
The solution was to make the decomposition `constexpr` and rely
on a late C++20 DR that makes it so that `consteval` propagates
up through the callstack of `constexpr` functions, until it either
runs out of `constexpr` functions, or succeeds.
However, the default handling of types in decomposition is to
take a reference to them. This reference never becomes dangling,
but because the constexpr evaluation engine cannot prove this,
decomposition paths taking references to objects cannot be
actually evaluated at compilation time. Thankfully we already
did have a value-oriented decomposition path for arithmetic types
(as these are common linkage-less types), so we could just
explicitly spell out the `std::foo_ordering` types as also being
supposed to be decomposed by-value.
Two more fun facts about these changes
1) The original motivation of the MSVC change was to avoid
trigering a `Wzero-as-null-pointer-constant` warning. I still
do not believe this was a good decision.
2) Current latest version of MSVC does not actually implement the
aforementioned C++20 DR, so even with this commit, MSVC cannot
compile `REQUIRE((a <=> b) == 0)`.
Without SSE2 enabled, x86 targets will use x87 FPU, which breaks
the tests checking for reproducible results from our random
floating point number generators. The output is still reproducible,
at least between binaries targetting x87, but the tests hardcode
results for the whole pipeline being done in 32/64bit precision.
Closes#2796
By moving to use our `uniform_integer_distribution`, which is
reproducible across different platforms, instead of the stdlib
one which is not, we can provide reproducible results for `float`s
and `double`s. Still no reproducibility for `long double`s, because
those are too different across different platforms.
* Utility for extended mult n x n bits -> 2n bits
* Utility to adapt output from URBG to target (unsigned) integral
type
* Utility to reorder signed values into unsigned type while keeping
the order.
Specifically we add
* `gamma(a, b)`, which returns the magnitude of largest 1-ULP
step in range [a, b].
* `count_equidistant_float(a, b, distance)`, which returns the
number of equi-distant floats in range [a, b].
Technically, the declaration should not have a space between
the quotes and the underscore, because `_foo` is a reserved
identifier, but `""_foo` is not. In general it works, but newer
Clang versions warn about this, because WG21 wants to deprecate
and later remove this form completely.
The basic idea was to reduce the number of things dependent on the `Clock`
type. To that end, I replaced `Duration<Clock>` with `IDuration` typedef
for `std::nanoseconds`, and `FloatDuration<Clock>` with `FDuration`
typedef for `Duration<double, std::nano>`. We can generally assume that
any clock's duration can be expressed in nanoseconds, as long as we insert
`duration_cast`s into the right places.
Note that we cannot remove all dependence on `Clock` as a template
arguments, because functions that actually measure the elapsed time have
to use the Clock.
We also changed some template function arguments to pass plain function
pointers, so that the actual implementation can be placed into a cpp file.
* AssertionEnd does not reset the assertion info yet. That is done after populateReaction. And reset assertion info would also reset the result disposition to normal, so that any uncaught exception would be reported as failure
* Approving test output changes due to added unit tests
* Unit tests to throw std::runtime_error instead of std::exception
* Add a unit test to test incomplete assertion handler
---------
Co-authored-by: Ross <ross.tang@gfo-x.com>
To keep the compilation firewall effect, the implementations
are hidden behind a PIMPL. In this case it is probably not
worth it, but we can inline it later if needed.
Mark Jansen