Commit Graph

64 Commits

Author SHA1 Message Date
Martin Hořeňovský
17a04f88d9
Always report rng seed from builtin reporters
Not all reporters use a format that supports this, so TeamCity
and Automake reporters still do not report it. The console
reporter now reports it even on successful runs, where before
it only reported the rng seed in the header, which was showed
either for failed run, or for run with `-s`.

CLoses #2065
2022-04-19 10:32:13 +02:00
Martin Hořeňovský
423e1d2ebb
Integrate the new reporter specs into Catch2
This means that the CLI interface now uses the new key-value oriented
reporter spec, the common reporter base creates the colour implementation
based on the reporter-specific configuration, and it also stores the
custom configuration options for each reporter instance.

Closes #339 as it allows per-reporter forcing of ansi colour codes.
2022-04-07 12:50:08 +02:00
Martin Hořeňovský
797c3e7318
Refactor tests 2022-03-31 22:30:38 +02:00
Daniel Feist
78e33ce51f
Added TestCaseInfoHasher and tests. (#2394)
Test case hashing includes tags and class name

As the hasher involves more code now, it was split out into its own file
and it got its own set of tests.

Closes #2304
2022-03-31 16:46:41 +02:00
Martin Hořeňovský
6227ca317e
Add tests for ColourGuard 2022-03-24 16:02:56 +01:00
Martin Hořeňovský
06f74a0f8e
Add %stderr and %stdout as recognized magic stream names 2022-03-07 10:59:27 +01:00
Martin Hořeňovský
715cd25081
Add IStream::isStdOut 2022-02-22 15:45:51 +01:00
Martin Hořeňovský
4ff8b27bb6
Cleanups output normalization in ApprovalTests
Removed:
 * NaN normalization
 * INFINITY normalization
 * errno normalization
 * Completely unused duration regex

Tests using these macros should be tagged `[approvals]`
so they are not run as part of approval tests.

Also simplified regex for the test's executable filename,
and hidden some tests relying on nullptr normalization.
2022-01-27 21:11:51 +01:00
Martin Hořeňovský
b6ff2c3dda
Error out reporter registration for reporter names with '::' 2022-01-09 19:01:14 +01:00
Martin Hořeňovský
8baf9c05a3 Add more unit tests
* Call order of listeners/reporters in multireporter
* Adding listeners/reporters properly updates reporter preferences
2022-01-01 14:10:52 +01:00
Martin Jeřábek
ccd67b293d Add support for multiple parallel reporters
This requires a bunch of different changes across the reporter
subsystem.

* We need to handle multiple reporters and their differing
  preferences in `ListeningReporter`, e.g. what to do when
  we mix reporters that capture and don't capture stdout.
* We need to change how the reporter is given output and
  how we parse reporter's output destination from CLI.
* Approval tests need to handle multireporter option
2022-01-01 14:02:23 +01:00
Martin Hořeňovský
c9c3b74805
Remove some obsolete tests 2021-12-31 15:12:53 +01:00
Martin Jeřábek
8711b63a0a
Add tests for StringRef's copy operations being shallow 2021-12-29 20:20:29 +01:00
Martin Hořeňovský
f00b6e2019
Add more tests for various less common tag-related cases 2021-12-28 00:22:53 +01:00
Martin Hořeňovský
cbb6764fb1
Add CaseInsensitiveEqualTo comparison type 2021-12-26 18:54:47 +01:00
Martin Hořeňovský
3cc0c033e4
Allow enabling multiple warnings in one invocation 2021-12-18 20:50:05 +01:00
Martin Hořeňovský
6603f1d972
Use case in names of default reporters 2021-11-10 23:32:01 +01:00
Martin Hořeňovský
3c5c86a4e4
Add test for filtering out multiple initial values in filter gen 2021-10-28 11:26:53 +02:00
Martin Hořeňovský
12d14a3c63
Add support for multiply calling lambda parsers in Clara
Previously a lambda parser in Clara could only be invoked once,
even if it internally was ok with being invoked multiple times.

With this change, a lambda parser can mark itself as `accept_many`,
in which case it will be invoked multiple times if the appropriate
flag was supplied multiple times by the user.
2021-10-27 20:15:28 +02:00
Martin Hořeňovský
5ac1ffe9ee Improve shardIndex/Count cli argument parsing 2021-10-27 17:24:30 +02:00
Ben Dunkin
3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský
905bf438ae
Fix bad indentation calculation in the console reporter
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".

The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).

We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
           a
           n
           c
           y
           .
           .
           .
```

Fixes #2309
2021-10-25 15:21:28 +02:00
Martin Hořeňovský
b406ad52a7
Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-10-10 22:21:39 +02:00
Martin Hořeňovský
200a487cf2
Add generateRandomSeed utility to generate randomness seed 2021-10-08 21:35:41 +02:00
Martin Hořeňovský
426954032f
Rename Contains string matcher builder to ContainsSubstring
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.

Closes #2131
2021-09-27 21:28:33 +02:00
Martin Hořeňovský
f02c2678a1
Use StringRef for className in TestCaseInfo 2021-09-27 19:07:45 +02:00
Martin Hořeňovský
21b99d6f58
Add StringRef::compare for three way comparison 2021-09-27 18:41:17 +02:00
Martin Hořeňovský
d42e7a23a0
Change startsWith(char) to take StringRef as argument 2021-09-27 14:55:28 +02:00
Martin Hořeňovský
3d1cf95b32
Fix ulp distance calculation for numbers with different signs
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.

Closes #2152
2021-07-27 21:04:41 +02:00
Martin Hořeňovský
bf61a418cb
Remove the ill-conceived compilation perf tests using real tests 2021-06-20 19:15:02 +02:00
Martin Hořeňovský
aee31d0620
Disallow empty tags in test case specification 2021-06-01 00:17:13 +02:00
Martin Hořeňovský
5741de9ccd
Add explicit test for serialization of boolean attributes in XML 2021-05-31 08:41:28 +02:00
Martin Hořeňovský
9137e591fa
Cleanup and optimize stringifying of string-like types
More specifically, made the actual implementation of string-like
type handling take argument as `Catch::StringRef`, instead of
taking `std::string const&`.

This means that string-like types that are not `std::string` no
longer need to pay for an extra construction of `std::string`
(including the potential allocation), before they can be stringified.

The actual string stringification routine is now also better about
reserving sufficient space.
2021-05-18 23:15:22 +02:00
Martin Hořeňovský
5eb7aa4f90
Add test for dots in tag names 2021-05-12 21:43:14 +02:00
Jozef Grajciar
eb911aa995
Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187)
Resolves #1390

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
Martin Hořeňovský
971b1fc32a
--list-* flags write to target specified by the -o flag
Also added tests for the default implementations of list* reporter
helpers.

Closes #2061
2021-01-26 18:36:54 +01:00
Martin Hořeňovský
77643ce2e5 Add more comprehensive tests for the quantifier matchers
This includes
* Testing both positive and negative path through the matchers
* Testing them with types whose `begin` and `end` member functions
require ADL
* Testing them with types that return different types from `begin`
and `end`
2020-12-27 20:20:55 +01:00
Uriel García Rivas
552af8920d Added AnyMatch, AllMatch and NoneMatch 2020-12-27 20:20:55 +01:00
Martin Hořeňovský
135103bacf
Arg::parse accepts plain Args as argument 2020-11-07 18:00:27 +01:00
Sean Middleditch
31d4831245
Support sentinel-based ranges in default stringify (#2004) 2020-09-07 14:23:47 +02:00
Martin Hořeňovský
5ca68829e1
Refactor how shortcircuiting of old style matchers is tested 2020-07-26 21:31:29 +02:00
Martin Hořeňovský
ac54ba7e12
Add test for shortcircuiting behaviour of generic matcher combinators 2020-07-26 21:24:05 +02:00
Martin Hořeňovský
4565b826cf
Modify generator tracking to allow GENERATEs between SECTIONs
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-26 11:35:06 +02:00
Martin Hořeňovský
0e77adee05
Add explicit test for shortcircuiting behaviour of combined matchers 2020-07-22 21:36:14 +02:00
Martin Hořeňovský
de53773e46
Fix copy paste error in 7-arg TEMPLATE_TEST_CASE_SIG implementation
Closes #1954
2020-07-22 21:18:52 +02:00
Martin Hořeňovský
480f3f418b
Improved generator tracking
* Successive executions of the same `GENERATE` macro (e.g. because
of a for loop) no longer lead to multiple nested generators.
* The same line can now contain multiple `GENERATE` macros without
issues.

Fixes #1913
2020-07-22 20:44:22 +02:00
Martin Hořeňovský
317145514f
Add op+(StringRef, StringRef) -> std::string 2020-05-31 22:30:41 +02:00
Martin Hořeňovský
41bbaa6d57
Implement a simplified variant of std::unique_ptr<T>
This simplified variant supports only a subset of the functionality
in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports
single element pointer (no array support) with default deleter.

By removing the support for custom deleters, we also avoid requiring
significant machinery to support EBO, speeding up instantiations of
`unique_ptr<T>` significantly. Catch2 also currently does not need
to support `unique_ptr<T[]>`, so that is not supported either.
2020-05-31 15:08:47 +02:00
Martin Hořeňovský
33c58dad41
Remove duplicated test for #1027 2020-05-03 09:58:46 +02:00
Martin Hořeňovský
53434a2f32
Support bitand and bitor in REQUIRE/CHECK
This means that bit-flag-like types with conversion to bool can be
asserted on, like so `REQUIRE(var & Flags::AddNewline)`.
2020-05-03 07:45:04 +02:00