Commit Graph

4378 Commits

Author SHA1 Message Date
Martin Hořeňovský
e86f84b8ef
Mention that run == iteration in benchmarking docs 2021-11-15 23:09:18 +01:00
Martin Hořeňovský
d012735c6e
Add test for runtime behaviour of STATIC_CHECK 2021-11-15 20:06:29 +01:00
Martin Hořeňovský
67caef6f45
Mention STATIC_CHECK in release notes 2021-11-15 13:04:08 +01:00
Morwenn
f41d761674
Add STATIC_CHECK and STATIC_CHECK_FALSE (#2318) 2021-11-15 00:28:27 +01:00
Martin Hořeňovský
edc2f6e8a3
Further refactoring of StreamingReporterBase 2021-11-14 11:41:29 +01:00
Martin Hořeňovský
b2ac27423a
Cleanup visibility in StreamingReporterBase 2021-11-14 11:41:27 +01:00
Martin Hořeňovský
a754cb9062
Make TestRunInfo constexpr 2021-11-14 11:41:26 +01:00
Martin Hořeňovský
5f38cc39fa
JUnit/SonarQube reporters use base's stream member 2021-11-14 11:41:25 +01:00
Martin Hořeňovský
b892ab133c
Cleanup visibility in CumulativeReporterBase 2021-11-14 11:41:24 +01:00
Martin Hořeňovský
0c9fe16537
Opt out JUnit/SonarQube reporter from expanding passing assertions
Closes #1966
2021-11-14 11:27:45 +01:00
Martin Hořeňovský
d02ea5adee
Cumulative reporter base can be customized to not expand assertions 2021-11-14 11:27:44 +01:00
Martin Hořeňovský
9b4e69333f
Small cleanup in tests 2021-11-14 11:27:43 +01:00
Martin Hořeňovský
4d9bfb2951
Avoid static analysis warning in Approx 2021-11-14 11:27:42 +01:00
Martin Hořeňovský
c4df47c246
Approvals normalize paths with both backward and forward slash 2021-11-13 22:10:25 +01:00
Martin Hořeňovský
9200b4078b
Move reporter_registrars.hpp to reporters/ 2021-11-10 23:52:59 +01:00
Martin Hořeňovský
6603f1d972
Use case in names of default reporters 2021-11-10 23:32:01 +01:00
Martin Hořeňovský
62d8913d67
Cumulative reporter base records benchmark results 2021-11-09 11:52:50 +01:00
Martin Hořeňovský
8780425385
Make reporter lookup case insensitive, registration case preserving
Previously registration was case preserving, but lookup used
lowercased reporter name, so a reporter whose name contained
upper case character could not be requested by the user.
2021-11-09 11:50:03 +01:00
Martin Hořeňovský
7800fe9708
Lift toLower(char) to header 2021-11-09 11:44:54 +01:00
Martin Hořeňovský
141e384c60
Fix missing include in reporter_registrars.hpp 2021-11-08 11:32:27 +01:00
Martin Hořeňovský
f1239b2045 Add doxygen doccomments to IStreamingReporter 2021-11-05 14:24:22 +01:00
Martin Hořeňovský
912df7df35
Fix quadratic runtime when linebreaking strings without newlines
The problem was that every line would iterate from current line
start position to the end of the string, looking for a newline
to break on, leading to accidentally quadratic runtime. With this
change, the code only ever searches up to the current line's
length and not more.

Credit to @jorgenpt for the fix suggestion.

Closes #2315
2021-11-04 00:23:56 +01:00
Martin Hořeňovský
931f41b4d6
Add some tests for TextFlow::Column 2021-11-01 22:51:17 +01:00
Martin Hořeňovský
70c4ec78fb
Improve comments and names in TextFlow::Column 2021-11-01 19:14:37 +01:00
Martin Hořeňovský
455ae0c561 Typedef Column::iterator as Column::const_iterator not vice versa 2021-10-31 13:01:41 +01:00
Martin Hořeňovský
2520ad4b6e Return const_iterator from Column::begin/end const
This is what should normally happen, even if it does not change
anything given that `Column::const_iterator` is currently a typedef
for `Column::iterator`.
2021-10-31 12:59:00 +01:00
Martin Hořeňovský
e539e1cb52
Move strings in Clara's result type 2021-10-29 23:04:24 +02:00
Martin Hořeňovský
3c5c86a4e4
Add test for filtering out multiple initial values in filter gen 2021-10-28 11:26:53 +02:00
Martin Hořeňovský
514206df36
Add accept-many Clara lambdas to release notes 2021-10-27 20:54:49 +02:00
Martin Hořeňovský
becab0cf74
Add test sharding to the release notes 2021-10-27 20:49:29 +02:00
Martin Hořeňovský
12d14a3c63
Add support for multiply calling lambda parsers in Clara
Previously a lambda parser in Clara could only be invoked once,
even if it internally was ok with being invoked multiple times.

With this change, a lambda parser can mark itself as `accept_many`,
in which case it will be invoked multiple times if the appropriate
flag was supplied multiple times by the user.
2021-10-27 20:15:28 +02:00
Martin Hořeňovský
f17725a186
Split void_type into its own header and rename it to void_t 2021-10-27 20:01:13 +02:00
Martin Hořeňovský
ec2d5013fb Make testSharding.py test script executable 2021-10-27 17:24:30 +02:00
Martin Hořeňovský
342ef5ca7e Cleanup the shard integration test script 2021-10-27 17:24:30 +02:00
Martin Hořeňovský
5ac1ffe9ee Improve shardIndex/Count cli argument parsing 2021-10-27 17:24:30 +02:00
Ben Dunkin
3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský
6456ee8b01
Return Clara parsing error message by const-ref 2021-10-27 15:10:52 +02:00
Martin Hořeňovský
905bf438ae
Fix bad indentation calculation in the console reporter
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".

The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).

We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
           a
           n
           c
           y
           .
           .
           .
```

Fixes #2309
2021-10-25 15:21:28 +02:00
Martin Hořeňovský
0fdee1c273
Stop declaring compiled Catch2 artifacts as arch independent 2021-10-25 14:46:54 +02:00
Anders Schau Knatten
22750cde0e Disable false positive from clang-tidy
Clang-tidy is smart enough to understand that the conditional is never
updated in the loop body. It will let you get away with it if it can
prove that the conditional is always false, but that is not always
possible.

Here is an example where it's not able to prove it, and thus gives a
false positive. This is a minimal reproduction of an actual case I hit
in production, where `function` is picking the function based on some
`constexpr` logic related to which type argument is currently being
tested.

```
int f();

TEMPLATE_TEST_CASE("reproduction", "", int) {
    const auto function = []() {
        return f;
    }();
    const int error = function();
    REQUIRE(error == 0); // clang-tidy complains: bugprone-infinite-loop
}
```

I did not choose to add this test to the test suite, since we're not
running `clang-tidy` in CI afaik. To run it manually, simply add the
snippet above somewhere and run clang-tidy with
`--checks=bugprone-infinite-loop`. Or see an example at
https://godbolt.org/z/4v8b8WexP.

The reason we get the infinite loop warning in the first place is the
conditional at the end of this `do`-loop. Ideally, this conditional
would just be `while(false)`, but the actual content of the
`REQUIRE`-statement has been included here too in order to not loose
warnings from signed/unsigned comparisons. In short, if you do
`REQUIRE(i < j)`, where `i` is a negative signed integer and `j` is an
unsigned integer, you're supposed to get a warning from
`-Wsign-compare`. Due to the decomposition in Catch2, you lose this
warning, which is why the content of the `REQUIRE` statement has been
added to the conditional to force the compiler to evaluate the actual
comparison as well.

This was discussed on Discord today, and an alternative approach (which
I don't have time to implement) would be to in the decomposition replace
the comparison operators with `cmp_less` and friends. These are C++20
though, and would have to be implemented manually. I am also not sure
it's a good idea to "magically" change the semantics of `<` when it's
used inside a `REQUIRE` macro.

Another alternative approach would be to trigger this warning in a
different way, by including the content of the `REQUIRE` macro in a
different way which doesn't affect the for loop. But I don't have enough
of an overview here to know where would be a good place and how to test
that I didn't break anything.
2021-10-21 22:45:19 +02:00
Martin Hořeňovský
bf5c58adf6
The limit on TEMPLATE* test cases is actually reachable 2021-10-21 15:52:07 +02:00
Alecto Irene Perez
06cf2a4724
Apply PR #2297 to devel branch (#2300)
* Apply PR #2297 to devel branch

It turns out that Issue #2272 partially affected the devel branch. When
building tests with C++20, the compiler emits a warning that top-level
comma expressions in array subscripts are depricated. Warnings are
treated as errors, so this caused the build to fail.

This commit adds localized warning suppression
in accordance with this recommendation here:
https://github.com/catchorg/Catch2/pull/2297#discussion_r720848392

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>

* Fixed unknown pragma warning on old versions of gcc & clang

This commit fixes an unkwown pragma warning on older versions of GCC
and Clang. These older versions don't have a warning for depricated use
of the comma subscript. Because warning suppression is localized, and
only refers to the comma subscript warning, it doesn't affect compiler
warnings in other parts of the code.

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>

* More #warning backwards compatibility fixes

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>
2021-10-21 15:47:21 +02:00
Martin Hořeňovský
4436a60456
Add myself to FUNDING 2021-10-11 10:44:29 +02:00
Martin Hořeňovský
36b4a71ff0
Pick release notes for 2.13.7 2021-10-10 22:23:14 +02:00
Martin Hořeňovský
b406ad52a7
Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-10-10 22:21:39 +02:00
Martin Hořeňovský
de67278e14
JUnit reporter uses only 3 decimal places when reporting durations
We used to use whatever precision we ended up having from C++'s
stdlib. However, some relatively popular tools, like Jenkins,
use Maven SureFire XML schema to validate JUnit test reports, and
Maven SureFire schema requires the duration to have at most 3
decimal places.

For compatibility, the JUnit reporter will now respect this
limitation.

Closes #2221
2021-10-10 22:10:48 +02:00
Martin Hořeňovský
1d9696d22d
Cleanup reporting of rng seed in existing reporters 2021-10-09 00:02:30 +02:00
Martin Hořeňovský
ed1f343a41
Provide random-device option for --rng-seed and make it default 2021-10-08 21:35:45 +02:00
Martin Hořeňovský
200a487cf2
Add generateRandomSeed utility to generate randomness seed 2021-10-08 21:35:41 +02:00
Martin Hořeňovský
fce42b62ad
Generate random rng seed if user did not specify one
Closes #2161
2021-10-08 21:35:29 +02:00