This event listener performs basic consistency checks (akin to
matching braces) on events that are passed to the listeners
when the `SelfTest` test binary is run.
The current checks are about nesting events (e.g. `testCaseStarting`
cannot be received before `testRunStarting`, `sectionStarting`
can only be received when a test case is active, etc), and matching
up counts of starting/ended events.
The simplicity means that it could be confused by starting/ended
events matching up but being out of order, e.g.
```
* test case A starting
* test case B ended
* test case B starting
* test case A ended
```
would be accepted, even though it is wrong. However, doing full
order checking would be much more implementation work, for relatively
little benefit, so it is left out for now.
With these changes, all these benchmarks
```cpp
BENCHMARK("Empty benchmark") {};
BENCHMARK("Throwing benchmark") {
throw "just a plain literal, bleh";
};
BENCHMARK("Asserting benchmark") {
REQUIRE(1 == 2);
};
BENCHMARK("FAIL'd benchmark") {
FAIL("This benchmark only fails, nothing else");
};
```
report the respective failure and mark the outer `TEST_CASE` as
failed. Previously, the first two would not fail the `TEST_CASE`,
and the latter two would break xml reporter's formatting, because
`benchmarkFailed`, `benchmarkEnded` etc would not be be called
properly in failure cases.
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.
Closes#2152
More specifically, made the actual implementation of string-like
type handling take argument as `Catch::StringRef`, instead of
taking `std::string const&`.
This means that string-like types that are not `std::string` no
longer need to pay for an extra construction of `std::string`
(including the potential allocation), before they can be stringified.
The actual string stringification routine is now also better about
reserving sufficient space.
This test tends to be brittle on Mac CI machines, which are
heavily loaded and bursty. Since the tests are only run as part
of the "extra tests" test set, this increase should not have
a significant impact on the total duration of CI runs.
This includes
* Testing both positive and negative path through the matchers
* Testing them with types whose `begin` and `end` member functions
require ADL
* Testing them with types that return different types from `begin`
and `end`
The old name was a legacy of v2 era, where all headers were stitched
into one. With v3 using separate headers, it is better when they have
proper name.
The problem was that under specific circumstances, namely that none
of their children progressed, `GeneratorTracker` will not progress.
This was changed recently, to allow for code like this, where a
`SECTION` follows a `GENERATE` at the same level:
```cpp
SECTION("A") {}
auto a = GENERATE(1, 2);
SECTION("B") {}
```
However, this interacted badly with `SECTION` filters (`-c foo`),
as they could deactivate all `SECTION`s below a generator, and thus
stop it from progressing forever. This commit makes GeneratorTracker
check whether there are any filters active, and if they are, it checks
whether its section-children can ever run.
Fixes#2025
Most of the changes are completely pointless renaming of constructor
arguments so that they do not use the same name as the type members,
but 🤷Closes#2015
The base was also renamed from `TestEventListenerBase` to
`EventListenerBase`, and modified to derive directly from the
reporter interface, rather than deriving from `StreamingReporterBase`.
Each of the two reporter bases now has its own header file, and
cpp file. Even though this adds another TU to the compilation,
the total CPU time taken by compilation is reduced by about 1%
for debug build and ~0.5% for optimized build of the main library.
(The improvement would be roughly doubles without splitting the TUs,
but the maintainability hit is not worth it.)
The code size of the static library build has also somewhat decreased.
Follow up: Introduce combined TU for reporters, and further split
apart the catch_reporter_streaming_base.hpp header into its
constituent parts, as it still contains a whole bunch of other stuff.
* Added some missing `noexcept`s on custom destructors.
* Fixed `std::move` being called on a const-reference.
* Initialized `ScopedMessage::m_moved` in class definition, instead
of doing so in constructors explicitly.
* Turned some `enum`s into `enum class`es.
* Initialized `StreamingReporterBase::currentTestCaseInfo` in class
definition.
* Some cleanups in SelfTest code.
TAP format requires all results to be reported.
Removed extraneous preferences function (handled by parent)
Incorporated fix from 3d9e7db2e0
Simplified total printing
This means that code such as
```cpp
TEST_CASE() {
SECTION("first") { SUCCEED(); }
auto _ = GENERATE(1, 2);
SECTION("second") { SUCCEED(); }
}
```
will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.
The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.
Closes#1938
A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.
When there are many tests, this produces output spam which makes it hard
to find the test failure output. Nevertheless, it is helpful to be
informed of tests which are unusually slow.
Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed. This allows slow
tests to be visible without mentioning every test.
* Successive executions of the same `GENERATE` macro (e.g. because
of a for loop) no longer lead to multiple nested generators.
* The same line can now contain multiple `GENERATE` macros without
issues.
Fixes#1913
This brings our output inline with GTest's. We do not handle skipped
tests properly, but that should be currently less important than
having the attribute exist with proper value for non-skipped tests.
Thanks @joda-01.
Closes#1899
Doing some benchmarking with ClangBuildAnalyzer suggests that
compiling Catch2's `SelfTest` spends 10% of the time instantiating
`std::unique_ptr` for some interface types required for registering
and running tests.
The lesser compilation overhead of `Catch::Detail::unique_ptr` should
significantly reduce that time.
The compiled implementation was also changed to use the custom impl,
to avoid having to convert between using `std::unique_ptr` and
`Catch::Detail::unique_ptr`. This will likely also improve the compile
times of the implementation, but that is less important than improving
compilation times of the user's TUs with tests.
This simplified variant supports only a subset of the functionality
in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports
single element pointer (no array support) with default deleter.
By removing the support for custom deleters, we also avoid requiring
significant machinery to support EBO, speeding up instantiations of
`unique_ptr<T>` significantly. Catch2 also currently does not need
to support `unique_ptr<T[]>`, so that is not supported either.
