Commit Graph

57 Commits

Author SHA1 Message Date
Martin Hořeňovský
ac93f19437
Improved path normalization in approvalTests.py
Instead of redoing the whole line where path was found, only the
directory part of the path is removed, instead of removing all
of the line before the path starts.

This results in slight change in how junit and sonarqube approvals
come out, and significant change in how TeamCity reporter approvals
come out. This latter difference is the reason for the change,
as now the lines with `testFailed` and `testIgnored` messages
are not completely butchered.
2023-01-01 20:27:59 +01:00
lbckmnn
9c0533a905
Add MessageMatches matcher for exception (#2570) 2022-12-12 00:40:47 +01:00
Martin Hořeňovský
c6dfeb5e7d
Split tests for TestSpec into its own file
They also got slapped with the `[approvals]` tag in the process,
because we have too many approval tests and want less of them,
and these particular tests don't bring much value.

Related to #2090
2022-10-26 21:03:56 +02:00
Martin Hořeňovský
d7341b5dc1
Add parseUInt utility function
There is an increasing number of places where Catch2 wants to parse
strings into numbers, but being stuck in C++14 world, we do not
have good stdlib facilities to do this (`strtoul` and `stoul`
are both bad).
2022-10-21 10:53:50 +02:00
Raphael Schaller
1bd233866c Add AllTrue, AnyTrue, NoneTrue matchers 2022-07-16 16:16:05 +02:00
Martin Hořeňovský
5baa29b6b9
Add test for %debug stream name 2022-04-23 23:34:27 +02:00
Martin Hořeňovský
797c3e7318
Refactor tests 2022-03-31 22:30:38 +02:00
Daniel Feist
78e33ce51f
Added TestCaseInfoHasher and tests. (#2394)
Test case hashing includes tags and class name

As the hasher involves more code now, it was split out into its own file
and it got its own set of tests.

Closes #2304
2022-03-31 16:46:41 +02:00
Martin Hořeňovský
6227ca317e
Add tests for ColourGuard 2022-03-24 16:02:56 +01:00
Martin Hořeňovský
06f74a0f8e
Add %stderr and %stdout as recognized magic stream names 2022-03-07 10:59:27 +01:00
Martin Hořeňovský
715cd25081
Add IStream::isStdOut 2022-02-22 15:45:51 +01:00
Martin Hořeňovský
4ff8b27bb6
Cleanups output normalization in ApprovalTests
Removed:
 * NaN normalization
 * INFINITY normalization
 * errno normalization
 * Completely unused duration regex

Tests using these macros should be tagged `[approvals]`
so they are not run as part of approval tests.

Also simplified regex for the test's executable filename,
and hidden some tests relying on nullptr normalization.
2022-01-27 21:11:51 +01:00
Martin Hořeňovský
b6ff2c3dda
Error out reporter registration for reporter names with '::' 2022-01-09 19:01:14 +01:00
Martin Hořeňovský
8baf9c05a3 Add more unit tests
* Call order of listeners/reporters in multireporter
* Adding listeners/reporters properly updates reporter preferences
2022-01-01 14:10:52 +01:00
Martin Hořeňovský
c9c3b74805
Remove some obsolete tests 2021-12-31 15:12:53 +01:00
Martin Hořeňovský
f00b6e2019
Add more tests for various less common tag-related cases 2021-12-28 00:22:53 +01:00
Martin Hořeňovský
cbb6764fb1
Add CaseInsensitiveEqualTo comparison type 2021-12-26 18:54:47 +01:00
Martin Hořeňovský
3cc0c033e4
Allow enabling multiple warnings in one invocation 2021-12-18 20:50:05 +01:00
Martin Hořeňovský
12d14a3c63
Add support for multiply calling lambda parsers in Clara
Previously a lambda parser in Clara could only be invoked once,
even if it internally was ok with being invoked multiple times.

With this change, a lambda parser can mark itself as `accept_many`,
in which case it will be invoked multiple times if the appropriate
flag was supplied multiple times by the user.
2021-10-27 20:15:28 +02:00
Martin Hořeňovský
5ac1ffe9ee Improve shardIndex/Count cli argument parsing 2021-10-27 17:24:30 +02:00
Martin Hořeňovský
905bf438ae
Fix bad indentation calculation in the console reporter
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".

The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).

We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
           a
           n
           c
           y
           .
           .
           .
```

Fixes #2309
2021-10-25 15:21:28 +02:00
Martin Hořeňovský
b406ad52a7
Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-10-10 22:21:39 +02:00
Martin Hořeňovský
200a487cf2
Add generateRandomSeed utility to generate randomness seed 2021-10-08 21:35:41 +02:00
Martin Hořeňovský
426954032f
Rename Contains string matcher builder to ContainsSubstring
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.

Closes #2131
2021-09-27 21:28:33 +02:00
Martin Hořeňovský
f02c2678a1
Use StringRef for className in TestCaseInfo 2021-09-27 19:07:45 +02:00
Martin Hořeňovský
21b99d6f58
Add StringRef::compare for three way comparison 2021-09-27 18:41:17 +02:00
Martin Hořeňovský
d42e7a23a0
Change startsWith(char) to take StringRef as argument 2021-09-27 14:55:28 +02:00
Martin Hořeňovský
3d1cf95b32
Fix ulp distance calculation for numbers with different signs
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.

Closes #2152
2021-07-27 21:04:41 +02:00
Martin Hořeňovský
bf61a418cb
Remove the ill-conceived compilation perf tests using real tests 2021-06-20 19:15:02 +02:00
Martin Hořeňovský
aee31d0620
Disallow empty tags in test case specification 2021-06-01 00:17:13 +02:00
Martin Hořeňovský
5741de9ccd
Add explicit test for serialization of boolean attributes in XML 2021-05-31 08:41:28 +02:00
Martin Hořeňovský
9137e591fa
Cleanup and optimize stringifying of string-like types
More specifically, made the actual implementation of string-like
type handling take argument as `Catch::StringRef`, instead of
taking `std::string const&`.

This means that string-like types that are not `std::string` no
longer need to pay for an extra construction of `std::string`
(including the potential allocation), before they can be stringified.

The actual string stringification routine is now also better about
reserving sufficient space.
2021-05-18 23:15:22 +02:00
Martin Hořeňovský
5eb7aa4f90
Add test for dots in tag names 2021-05-12 21:43:14 +02:00
Jozef Grajciar
eb911aa995
Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187)
Resolves #1390

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
Martin Hořeňovský
971b1fc32a
--list-* flags write to target specified by the -o flag
Also added tests for the default implementations of list* reporter
helpers.

Closes #2061
2021-01-26 18:36:54 +01:00
Martin Hořeňovský
77643ce2e5 Add more comprehensive tests for the quantifier matchers
This includes
* Testing both positive and negative path through the matchers
* Testing them with types whose `begin` and `end` member functions
require ADL
* Testing them with types that return different types from `begin`
and `end`
2020-12-27 20:20:55 +01:00
Uriel García Rivas
552af8920d Added AnyMatch, AllMatch and NoneMatch 2020-12-27 20:20:55 +01:00
Martin Hořeňovský
135103bacf
Arg::parse accepts plain Args as argument 2020-11-07 18:00:27 +01:00
Sean Middleditch
31d4831245
Support sentinel-based ranges in default stringify (#2004) 2020-09-07 14:23:47 +02:00
Martin Hořeňovský
ac54ba7e12
Add test for shortcircuiting behaviour of generic matcher combinators 2020-07-26 21:24:05 +02:00
Martin Hořeňovský
4565b826cf
Modify generator tracking to allow GENERATEs between SECTIONs
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-26 11:35:06 +02:00
Martin Hořeňovský
0e77adee05
Add explicit test for shortcircuiting behaviour of combined matchers 2020-07-22 21:36:14 +02:00
Martin Hořeňovský
de53773e46
Fix copy paste error in 7-arg TEMPLATE_TEST_CASE_SIG implementation
Closes #1954
2020-07-22 21:18:52 +02:00
Martin Hořeňovský
480f3f418b
Improved generator tracking
* Successive executions of the same `GENERATE` macro (e.g. because
of a for loop) no longer lead to multiple nested generators.
* The same line can now contain multiple `GENERATE` macros without
issues.

Fixes #1913
2020-07-22 20:44:22 +02:00
Martin Hořeňovský
41bbaa6d57
Implement a simplified variant of std::unique_ptr<T>
This simplified variant supports only a subset of the functionality
in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports
single element pointer (no array support) with default deleter.

By removing the support for custom deleters, we also avoid requiring
significant machinery to support EBO, speeding up instantiations of
`unique_ptr<T>` significantly. Catch2 also currently does not need
to support `unique_ptr<T[]>`, so that is not supported either.
2020-05-31 15:08:47 +02:00
Martin Hořeňovský
33c58dad41
Remove duplicated test for #1027 2020-05-03 09:58:46 +02:00
Martin Hořeňovský
53434a2f32
Support bitand and bitor in REQUIRE/CHECK
This means that bit-flag-like types with conversion to bool can be
asserted on, like so `REQUIRE(var & Flags::AddNewline)`.
2020-05-03 07:45:04 +02:00
Martin Hořeňovský
dd35430a2b
Add more tests for test spec parser
Originally the tests were from #1912, but as it turned out, the issue
was somewhere else. Still, the inputs provided were interesting, so
they are now part of our test suite.
2020-05-02 18:24:07 +02:00
Martin Hořeňovský
87a8b61d5a
Fix bug in test spec parser handling of escaping in ORed patterns
It did not clear out all of its internal state when switching from
one pattern to another, so when it should've escaped `,`, it took
its position from its position in the original user-provided string,
rather than its position in the current pattern.

Fixes #1905
2020-05-01 09:56:34 +02:00
Martin Hořeňovský
5198fd3c9a
Fix compilation error when lambdas are used in assertions
This is a partial revert of b7b346c3e5.
2020-05-01 09:24:26 +02:00