Commit Graph

213 Commits

Author SHA1 Message Date
Martin Hořeňovský
ac93f19437
Improved path normalization in approvalTests.py
Instead of redoing the whole line where path was found, only the
directory part of the path is removed, instead of removing all
of the line before the path starts.

This results in slight change in how junit and sonarqube approvals
come out, and significant change in how TeamCity reporter approvals
come out. This latter difference is the reason for the change,
as now the lines with `testFailed` and `testIgnored` messages
are not completely butchered.
2023-01-01 20:27:59 +01:00
lbckmnn
9c0533a905
Add MessageMatches matcher for exception (#2570) 2022-12-12 00:40:47 +01:00
Martin Hořeňovský
28e651f152
Move SFINAE in decomposer into return type
This is needed so that we can use conjunction and other logical
type traits to workaround issue with older GCC versions (8 and
below), when they run into types that have ambiguous constructor
from `0`, see e.g. #2571.

However, using conjunction and friends in the SFINAE constraint
in the template parameter breaks for C++20 and up, due to the new
comparison operator rewriting rules. With C++20, when the compiler
see `a == b`, it also tries `b == a` and collects overload set
for both of these expressions.

In Catch2, this means that e.g. `REQUIRE( 1 == 2 )` would lead
the compiler to check overloads for both `ExprLhs<int> == int`
and `int == ExprLhs<int>`. Since the overload set and SFINAE
constraints assume that `ExprLhs<T>` is always on the left side,
when the compiler tries to resolve the template parameters, all
hell breaks loose and the compilation fails.

By moving the SFINAE constraints to the return type, the compiler
can discard the switched expression without having to resolve
the complex SFINAE constraints, and thus everything works the
way it is supposed to.

Fixes #2571
2022-12-09 00:40:01 +01:00
Martin Hořeňovský
a20200be7e
Revert "Fix old GCC + types with ambiguous constructor from 0"
This reverts commit 291c502f66.

The issue is that it breaks under C++20 for some reason.
2022-11-22 15:23:03 +01:00
Martin Hořeňovský
291c502f66
Fix old GCC + types with ambiguous constructor from 0
Closes #2571
2022-11-20 17:07:32 +01:00
Martin Hořeňovský
943c6e3dee
Add version field to the XML reporter output
We start at version 2, with version 1 being taken up by the output
from Catch2 v2.
2022-11-10 12:04:51 +01:00
Martin Hořeňovský
3b139ae51a
Decomposer checks for 0 when assuming an int arg was 0 literal 2022-11-05 20:43:52 +01:00
Martin Hořeňovský
ec59cd8736
Support decomposing types that only compare with literal 0
This is primarily done to support new `std::*_ordering` types,
but the refactoring also supports any other type with this
property.

The compilation overhead is surprisingly low. Testing it with
clang on a Linux machine, compiling our SelfTest project takes
only 2-3% longer with these changes than it takes otherwise.

Closes #2555
2022-11-04 19:24:44 +01:00
Martin Hořeňovský
d7f8c36e4c
Add traits for checking whether types are comparable 2022-11-04 19:24:42 +01:00
Martin Hořeňovský
b3dbd83da2
Add test for comparing pointers to NULL with != 2022-11-04 19:24:39 +01:00
Martin Hořeňovský
12b4390169
Fix license rebase error in test helpers 2022-10-28 14:39:56 +02:00
Martin Hořeňovský
3b40cf13eb
Split out parseTestSpec into test-only helpers 2022-10-28 13:27:46 +02:00
Lars Toenning
f1084fb309
Fix references to license file
The license file was renamed with 6a502cc2f5
2022-10-28 11:30:15 +02:00
Martin Hořeňovský
d41da10c54
Change reporters to report filters in round-trippable format 2022-10-27 20:46:59 +02:00
Martin Hořeňovský
e19ed221bd
Provide round-tripping serialization for TestSpec 2022-10-27 18:23:59 +02:00
Martin Hořeňovský
c6dfeb5e7d
Split tests for TestSpec into its own file
They also got slapped with the `[approvals]` tag in the process,
because we have too many approval tests and want less of them,
and these particular tests don't bring much value.

Related to #2090
2022-10-26 21:03:56 +02:00
Martin Hořeňovský
ffa152095c
Report used filters in the SonarQube reporter 2022-10-26 00:05:00 +02:00
Martin Hořeňovský
0ce8c25566
Report used filters in the TAP reporter 2022-10-26 00:04:49 +02:00
Philip Salzmann
6185d0cc0a Use console reporter's totals summary for compact reporter
This changes the compact reporter's summary of test run totals to use
the same format as the console reporter. This means that while output is
no longer on a single line (two instead), it now includes totals for
`failedButOk` test cases and assertions, which were previously missing.
2022-10-25 20:02:26 +02:00
Martin Hořeňovský
a43f67962e
Refactor parsing of shard index/count in cmdline handling
This worsens the message for negative numbers a bit, but
simplifies the code enough that this is still a win.
2022-10-23 00:10:05 +02:00
Martin Hořeňovský
f1361ef624
Refactor how the RNG seed is parsed when handling cmdline 2022-10-22 22:18:30 +02:00
Martin Hořeňovský
d7341b5dc1
Add parseUInt utility function
There is an increasing number of places where Catch2 wants to parse
strings into numbers, but being stuck in C++14 world, we do not
have good stdlib facilities to do this (`strtoul` and `stoul`
are both bad).
2022-10-21 10:53:50 +02:00
Martin Hořeňovský
77f7c0104d
Fix fibonacci impl in benchmark tests
The previous implementation was always 1 number ahead. Fixing this
is not important at all, but it was bugging me.
2022-10-18 10:44:18 +02:00
Martin Hořeňovský
9c9f35068e
Normalize C++ namespace in JUnit's reporter classname field
Closes #2468
2022-07-17 19:15:20 +02:00
Raphael Schaller
1bd233866c Add AllTrue, AnyTrue, NoneTrue matchers 2022-07-16 16:16:05 +02:00
Martin Hořeňovský
caf1264588 Fixes for matcher testing helpers when testing bools 2022-07-16 16:16:05 +02:00
Martin Hořeňovský
ff9506cedd
Only test SEH handling with MSVC
MinGW doesn't support `__try` and friends at all, while Clang
only supports it partially, and the test would require some
changes to make it work there. Since this is only a test, we can
afford to keep it MSVC-only.

Closes #2447
2022-06-06 00:29:01 +02:00
Martin Hořeňovský
7a2a6c632f
Make Random*Generators be seedable
This is a necessary step to fix issue with combining multiple
random generators in one test case, with different section nesting.
2022-05-17 17:57:21 +02:00
Martin Hořeňovský
5d7883b551
Remove leftover comment in tests 2022-05-16 21:50:52 +02:00
Martin Hořeňovský
48f3226974
Add support for retrieving generator's element as string 2022-05-16 16:34:06 +02:00
Martin Hořeňovský
af8b54ecd5
Generators now keep track of the index of current element
This might become potentially useful in the future, when we want
to provide the ability to forward jump generators, to be able to
simply reproduce specific input to a test.

I am not yet sure how it would work, but the basic idea is that
it could be similar to the `-c` switch for selecting specific
`SECTION` in a test case.
2022-05-16 11:56:21 +02:00
Martin Hořeňovský
33aeb603fe
Add check for registering multiple reporters under same name 2022-05-15 09:54:27 +02:00
Martin Hořeňovský
fc3d11b1d1
Add Wfloat-equal to default-enabled warnings
This is kinda messy, because there is no good way to signal to
the compiler that some code uses direct comparison of floating
point numbers intentionally, so instead we have to use diagnostic
pragmas.

We also have to over-suppress the test files, because Clang (and
possibly GCC) still issue warnings from template instantiation
even if the instantion site is under warning suppression, so the
template definition has to be under warning suppression as well.

Closes #2406
2022-05-14 15:47:20 +02:00
Martin Hořeňovský
1ef65d60f1
Tiny fixup in Misc.tests.cpp 2022-05-11 22:56:30 +02:00
Martin Hořeňovský
5a49285e9c
Add --list-listeners option 2022-05-10 20:57:59 +02:00
Martin Hořeňovský
d60fbe49be
Slight improvement for reporter test 2022-05-10 20:57:50 +02:00
Martin Hořeňovský
d9b0a38f81
Report Catch2's version in the xml reporter output 2022-05-03 19:46:35 +02:00
Martin Hořeňovský
91ea25e51a
Fix use of u8-literal that would break in C++20 2022-05-03 13:54:07 +02:00
Martin Hořeňovský
e2d07d35f4
Add -Wmissing-prototypes to common warnings 2022-05-03 13:54:05 +02:00
Martin Hořeňovský
198808a24e
Misc cleanups from exploring Clang warnings 2022-05-03 13:53:55 +02:00
Martin Hořeňovský
198713e5dc
Cleanup of unused functions
The cleanup also found out that custom translation for std-derived
exceptions test wasn't running properly, and fixed that.
We cannot enable the warning globally, because the tests contain
some functions that are unused by design -- e.g. when checking
stringification priority of StringMaker vs range fallback and so
on.
2022-05-02 21:32:24 +02:00
Martin Hořeňovský
07cdef2096
Fix uninit memory in Clara tests 2022-04-24 13:31:19 +02:00
Martin Hořeňovský
5baa29b6b9
Add test for %debug stream name 2022-04-23 23:34:27 +02:00
Daniel Feist
291b35b389
Added --skip-benchmarks command-line option. (#2408)
Closes #2392

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-04-23 23:14:49 +02:00
Martin Hořeňovský
17a04f88d9
Always report rng seed from builtin reporters
Not all reporters use a format that supports this, so TeamCity
and Automake reporters still do not report it. The console
reporter now reports it even on successful runs, where before
it only reported the rng seed in the header, which was showed
either for failed run, or for run with `-s`.

CLoses #2065
2022-04-19 10:32:13 +02:00
Martin Hořeňovský
7b93a2014c
Rename catch_stream -> catch_reusable_string_stream
After everything else was split out, this name much reflects the
actual contents of the file(s).
2022-04-16 16:34:08 +02:00
Martin Hořeňovský
98bb638fb2
Split out IStream out of catch_stream.hpp 2022-04-16 16:33:57 +02:00
Martin Hořeňovský
a243cbae52
Introduce type to handle reporter specs with defaults filled-in 2022-04-13 15:02:25 +02:00
Martin Hořeňovský
79d1e82381
Store IStream instance owning ptrs in reporter instances
This ended up being a surprisingly large refactoring, motivated
by removing a `const_cast` from `Config`'s handling of reporter
streams, forced by previous commit.
2022-04-13 15:00:50 +02:00
Martin Hořeňovský
4f09f1120b
Make IStream::stream non-const
This way it makes much more sense from logically-const point
of view, and also means that concrete implementations don't
have to always have a `mutable` keyword on the stream member.
2022-04-13 14:59:39 +02:00