This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.
Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".
The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).
We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
a
n
c
y
.
.
.
```
Fixes#2309
Clang-tidy is smart enough to understand that the conditional is never
updated in the loop body. It will let you get away with it if it can
prove that the conditional is always false, but that is not always
possible.
Here is an example where it's not able to prove it, and thus gives a
false positive. This is a minimal reproduction of an actual case I hit
in production, where `function` is picking the function based on some
`constexpr` logic related to which type argument is currently being
tested.
```
int f();
TEMPLATE_TEST_CASE("reproduction", "", int) {
const auto function = []() {
return f;
}();
const int error = function();
REQUIRE(error == 0); // clang-tidy complains: bugprone-infinite-loop
}
```
I did not choose to add this test to the test suite, since we're not
running `clang-tidy` in CI afaik. To run it manually, simply add the
snippet above somewhere and run clang-tidy with
`--checks=bugprone-infinite-loop`. Or see an example at
https://godbolt.org/z/4v8b8WexP.
The reason we get the infinite loop warning in the first place is the
conditional at the end of this `do`-loop. Ideally, this conditional
would just be `while(false)`, but the actual content of the
`REQUIRE`-statement has been included here too in order to not loose
warnings from signed/unsigned comparisons. In short, if you do
`REQUIRE(i < j)`, where `i` is a negative signed integer and `j` is an
unsigned integer, you're supposed to get a warning from
`-Wsign-compare`. Due to the decomposition in Catch2, you lose this
warning, which is why the content of the `REQUIRE` statement has been
added to the conditional to force the compiler to evaluate the actual
comparison as well.
This was discussed on Discord today, and an alternative approach (which
I don't have time to implement) would be to in the decomposition replace
the comparison operators with `cmp_less` and friends. These are C++20
though, and would have to be implemented manually. I am also not sure
it's a good idea to "magically" change the semantics of `<` when it's
used inside a `REQUIRE` macro.
Another alternative approach would be to trigger this warning in a
different way, by including the content of the `REQUIRE` macro in a
different way which doesn't affect the for loop. But I don't have enough
of an overview here to know where would be a good place and how to test
that I didn't break anything.
* Apply PR #2297 to devel branch
It turns out that Issue #2272 partially affected the devel branch. When
building tests with C++20, the compiler emits a warning that top-level
comma expressions in array subscripts are depricated. Warnings are
treated as errors, so this caused the build to fail.
This commit adds localized warning suppression
in accordance with this recommendation here:
https://github.com/catchorg/Catch2/pull/2297#discussion_r720848392
Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>
* Fixed unknown pragma warning on old versions of gcc & clang
This commit fixes an unkwown pragma warning on older versions of GCC
and Clang. These older versions don't have a warning for depricated use
of the comma subscript. Because warning suppression is localized, and
only refers to the comma subscript warning, it doesn't affect compiler
warnings in other parts of the code.
Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>
* More #warning backwards compatibility fixes
Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>
We used to use whatever precision we ended up having from C++'s
stdlib. However, some relatively popular tools, like Jenkins,
use Maven SureFire XML schema to validate JUnit test reports, and
Maven SureFire schema requires the duration to have at most 3
decimal places.
For compatibility, the JUnit reporter will now respect this
limitation.
Closes#2221
There is no good reason to provide a "add empty line" primitive
for writing XML documents, and the fact that it remains unused
after all the time it was provided only confirms this further.
This decreases code size and improves performance of passing around
`unique_ptr` instances by value somewhat. It virtually guarantees
problems when combining code compiled with Clang and GCC, but that
was never supported anyway.
In b7b346c3e5 this conditional was simplified to just
`while( false)` rather than the current one. Then in 3a33315ff8, that
change was reverted. I found it hard to understand this
complicated conditional, but after some digging in history I found this
comment which used to be here. It was removed in b7b346c3e5, but not
restored together with the revert in 3a33315ff8. Let's revive it.
It used to be part of the experimental benchmarking support, but
since that was replaced with proper benchmarking support with its
own timer facilities, it is now a dead code and useless.
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.
Closes#2131
This change also changes it so that test case macros using a
class name can have same name **and** tags as long as the
used class name differs.
Closes#1915Closes#1999
This event listener performs basic consistency checks (akin to
matching braces) on events that are passed to the listeners
when the `SelfTest` test binary is run.
The current checks are about nesting events (e.g. `testCaseStarting`
cannot be received before `testRunStarting`, `sectionStarting`
can only be received when a test case is active, etc), and matching
up counts of starting/ended events.
The simplicity means that it could be confused by starting/ended
events matching up but being out of order, e.g.
```
* test case A starting
* test case B ended
* test case B starting
* test case A ended
```
would be accepted, even though it is wrong. However, doing full
order checking would be much more implementation work, for relatively
little benefit, so it is left out for now.
They now take `StringRef` as the argument, and are virtual only
in the basic interface.
Also cleaned out the various reporters and their overrides
of these members which were often empty or delegating up.
This means that e.g. for `TEST_CASE` with two sibling `SECTION`s
the event will fire twice, because the `TEST_CASE` will be entered
twice.
Closes#2107 (the event mentioned there already exists, but this
is its counterpart that we also want to provide to users)
