Commit Graph

80 Commits

Author SHA1 Message Date
Martin Hořeňovský 715cd25081
Add IStream::isStdOut 2022-02-22 15:45:51 +01:00
Martin Hořeňovský fc5552d27b
Push down handling of default reporter to Config's constructor
This simplifies the handling of default reporter in console
parsing, at the cost of making `Config`'s constructor responsible
for more things.
2022-02-17 23:02:30 +01:00
Martin Hořeňovský 4ff8b27bb6
Cleanups output normalization in ApprovalTests
Removed:
 * NaN normalization
 * INFINITY normalization
 * errno normalization
 * Completely unused duration regex

Tests using these macros should be tagged `[approvals]`
so they are not run as part of approval tests.

Also simplified regex for the test's executable filename,
and hidden some tests relying on nullptr normalization.
2022-01-27 21:11:51 +01:00
Martin Hořeňovský b6ff2c3dda
Error out reporter registration for reporter names with '::' 2022-01-09 19:01:14 +01:00
Martin Hořeňovský 8baf9c05a3 Add more unit tests
* Call order of listeners/reporters in multireporter
* Adding listeners/reporters properly updates reporter preferences
2022-01-01 14:10:52 +01:00
Martin Jeřábek ccd67b293d Add support for multiple parallel reporters
This requires a bunch of different changes across the reporter
subsystem.

* We need to handle multiple reporters and their differing
  preferences in `ListeningReporter`, e.g. what to do when
  we mix reporters that capture and don't capture stdout.
* We need to change how the reporter is given output and
  how we parse reporter's output destination from CLI.
* Approval tests need to handle multireporter option
2022-01-01 14:02:23 +01:00
Martin Hořeňovský c9c3b74805
Remove some obsolete tests 2021-12-31 15:12:53 +01:00
Martin Jeřábek 8711b63a0a
Add tests for StringRef's copy operations being shallow 2021-12-29 20:20:29 +01:00
Martin Hořeňovský f00b6e2019
Add more tests for various less common tag-related cases 2021-12-28 00:22:53 +01:00
Martin Hořeňovský 45577a1f4c
Refactor implementation of case-insensitivity in tags
By not materializing the lower cased tags ahead of time, we
save allocations at the cost of worsened performance when comparing
two tags.

Since there are rarely many tags, and commonly they are not
compared even if present, this is almost always a win. The new
implementation also improves the robustness of the code
responsible for handling tags in a case-insensitive manner.
2021-12-28 00:20:55 +01:00
Martin Hořeňovský cbb6764fb1
Add CaseInsensitiveEqualTo comparison type 2021-12-26 18:54:47 +01:00
Martin Hořeňovský 3cc0c033e4
Allow enabling multiple warnings in one invocation 2021-12-18 20:50:05 +01:00
Morwenn f41d761674
Add STATIC_CHECK and STATIC_CHECK_FALSE (#2318) 2021-11-15 00:28:27 +01:00
Martin Hořeňovský 6603f1d972
Use case in names of default reporters 2021-11-10 23:32:01 +01:00
Martin Hořeňovský 3c5c86a4e4
Add test for filtering out multiple initial values in filter gen 2021-10-28 11:26:53 +02:00
Martin Hořeňovský 12d14a3c63
Add support for multiply calling lambda parsers in Clara
Previously a lambda parser in Clara could only be invoked once,
even if it internally was ok with being invoked multiple times.

With this change, a lambda parser can mark itself as `accept_many`,
in which case it will be invoked multiple times if the appropriate
flag was supplied multiple times by the user.
2021-10-27 20:15:28 +02:00
Martin Hořeňovský 5ac1ffe9ee Improve shardIndex/Count cli argument parsing 2021-10-27 17:24:30 +02:00
Ben Dunkin 3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský 905bf438ae
Fix bad indentation calculation in the console reporter
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".

The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).

We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
           a
           n
           c
           y
           .
           .
           .
```

Fixes #2309
2021-10-25 15:21:28 +02:00
Martin Hořeňovský b406ad52a7
Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-10-10 22:21:39 +02:00
Martin Hořeňovský 1d9696d22d
Cleanup reporting of rng seed in existing reporters 2021-10-09 00:02:30 +02:00
Martin Hořeňovský 200a487cf2
Add generateRandomSeed utility to generate randomness seed 2021-10-08 21:35:41 +02:00
Martin Hořeňovský 426954032f
Rename Contains string matcher builder to ContainsSubstring
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.

Closes #2131
2021-09-27 21:28:33 +02:00
Martin Hořeňovský f02c2678a1
Use StringRef for className in TestCaseInfo 2021-09-27 19:07:45 +02:00
Martin Hořeňovský 21b99d6f58
Add StringRef::compare for three way comparison 2021-09-27 18:41:17 +02:00
Martin Hořeňovský d42e7a23a0
Change startsWith(char) to take StringRef as argument 2021-09-27 14:55:28 +02:00
Martin Hořeňovský e5938007f7
Completely remove the testGroup events
This was done because they were 1:1 mapping to testRun events, and
I could not think of a reasonable way to make them their own thing.
2021-09-07 20:00:22 +02:00
Martin Hořeňovský 3d1cf95b32
Fix ulp distance calculation for numbers with different signs
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.

Closes #2152
2021-07-27 21:04:41 +02:00
Martin Hořeňovský bf61a418cb
Remove the ill-conceived compilation perf tests using real tests 2021-06-20 19:15:02 +02:00
Martin Hořeňovský 816f69416b
Use TestCaseInfo in tests directly 2021-06-01 00:17:19 +02:00
Martin Hořeňovský aee31d0620
Disallow empty tags in test case specification 2021-06-01 00:17:13 +02:00
Martin Hořeňovský 5741de9ccd
Add explicit test for serialization of boolean attributes in XML 2021-05-31 08:41:28 +02:00
Martin Hořeňovský 9137e591fa
Cleanup and optimize stringifying of string-like types
More specifically, made the actual implementation of string-like
type handling take argument as `Catch::StringRef`, instead of
taking `std::string const&`.

This means that string-like types that are not `std::string` no
longer need to pay for an extra construction of `std::string`
(including the potential allocation), before they can be stringified.

The actual string stringification routine is now also better about
reserving sufficient space.
2021-05-18 23:15:22 +02:00
Martin Hořeňovský 5eb7aa4f90
Add test for dots in tag names 2021-05-12 21:43:14 +02:00
Jozef Grajciar eb911aa995
Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187)
Resolves #1390

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
Martin Hořeňovský f1d7a10e06
Remove StringRef::isNullTerminated and StringRef::c_str
Both of them were fundamentally unsafe to use and shouldn't be used
at all.
2021-03-26 21:11:44 +01:00
Martin Hořeňovský 971b1fc32a
--list-* flags write to target specified by the -o flag
Also added tests for the default implementations of list* reporter
helpers.

Closes #2061
2021-01-26 18:36:54 +01:00
Roman Proskuryakov 0acb371b92
Fix Wold-style-cast error (#2125)
* Add Wold-style-cast to cmake flags
* Fix old style cast in catch_stats.hpp
* Fix old style cast in catch_stats.cpp
2020-12-28 14:00:19 +01:00
Martin Hořeňovský 77643ce2e5 Add more comprehensive tests for the quantifier matchers
This includes
* Testing both positive and negative path through the matchers
* Testing them with types whose `begin` and `end` member functions
require ADL
* Testing them with types that return different types from `begin`
and `end`
2020-12-27 20:20:55 +01:00
Uriel García Rivas 552af8920d Added AnyMatch, AllMatch and NoneMatch 2020-12-27 20:20:55 +01:00
Martin Hořeňovský fefa001bb6
Fixup approvals 2020-11-07 20:16:53 +01:00
Martin Hořeňovský 135103bacf
Arg::parse accepts plain Args as argument 2020-11-07 18:00:27 +01:00
Sean Middleditch 31d4831245
Support sentinel-based ranges in default stringify (#2004) 2020-09-07 14:23:47 +02:00
Martin Hořeňovský cc18bd719d
Sweep out Wshadow
Most of the changes are completely pointless renaming of constructor
arguments so that they do not use the same name as the type members,
but 🤷

Closes #2015
2020-09-06 13:11:42 +02:00
Martin Hořeňovský 5ca68829e1
Refactor how shortcircuiting of old style matchers is tested 2020-07-26 21:31:29 +02:00
Martin Hořeňovský ac54ba7e12
Add test for shortcircuiting behaviour of generic matcher combinators 2020-07-26 21:24:05 +02:00
Gavin S 87d0197cbd
Update catch_reporter_tap.hpp
TAP format requires all results to be reported.
Removed extraneous preferences function (handled by parent)
Incorporated fix from 3d9e7db2e0
Simplified total printing
2020-07-26 14:00:10 +02:00
Martin Hořeňovský 4565b826cf
Modify generator tracking to allow GENERATEs between SECTIONs
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-26 11:35:06 +02:00
Ryan Pavlik 7f58840163
Add OverallResultsCases element to XML reporter 2020-07-24 22:30:28 +02:00
Martin Hořeňovský 0e77adee05
Add explicit test for shortcircuiting behaviour of combined matchers 2020-07-22 21:36:14 +02:00