Commit Graph

129 Commits

Author SHA1 Message Date
Martin Hořeňovský fc5552d27b
Push down handling of default reporter to Config's constructor
This simplifies the handling of default reporter in console
parsing, at the cost of making `Config`'s constructor responsible
for more things.
2022-02-17 23:02:30 +01:00
Martin Hořeňovský 4ff8b27bb6
Cleanups output normalization in ApprovalTests
Removed:
 * NaN normalization
 * INFINITY normalization
 * errno normalization
 * Completely unused duration regex

Tests using these macros should be tagged `[approvals]`
so they are not run as part of approval tests.

Also simplified regex for the test's executable filename,
and hidden some tests relying on nullptr normalization.
2022-01-27 21:11:51 +01:00
Martin Hořeňovský b6ff2c3dda
Error out reporter registration for reporter names with '::' 2022-01-09 19:01:14 +01:00
Martin Hořeňovský 8baf9c05a3 Add more unit tests
* Call order of listeners/reporters in multireporter
* Adding listeners/reporters properly updates reporter preferences
2022-01-01 14:10:52 +01:00
Martin Jeřábek ccd67b293d Add support for multiple parallel reporters
This requires a bunch of different changes across the reporter
subsystem.

* We need to handle multiple reporters and their differing
  preferences in `ListeningReporter`, e.g. what to do when
  we mix reporters that capture and don't capture stdout.
* We need to change how the reporter is given output and
  how we parse reporter's output destination from CLI.
* Approval tests need to handle multireporter option
2022-01-01 14:02:23 +01:00
Martin Hořeňovský c9c3b74805
Remove some obsolete tests 2021-12-31 15:12:53 +01:00
Martin Jeřábek 8711b63a0a
Add tests for StringRef's copy operations being shallow 2021-12-29 20:20:29 +01:00
Martin Hořeňovský f00b6e2019
Add more tests for various less common tag-related cases 2021-12-28 00:22:53 +01:00
Martin Hořeňovský 45577a1f4c
Refactor implementation of case-insensitivity in tags
By not materializing the lower cased tags ahead of time, we
save allocations at the cost of worsened performance when comparing
two tags.

Since there are rarely many tags, and commonly they are not
compared even if present, this is almost always a win. The new
implementation also improves the robustness of the code
responsible for handling tags in a case-insensitive manner.
2021-12-28 00:20:55 +01:00
Martin Hořeňovský cbb6764fb1
Add CaseInsensitiveEqualTo comparison type 2021-12-26 18:54:47 +01:00
Martin Hořeňovský 3cc0c033e4
Allow enabling multiple warnings in one invocation 2021-12-18 20:50:05 +01:00
Morwenn f41d761674
Add STATIC_CHECK and STATIC_CHECK_FALSE (#2318) 2021-11-15 00:28:27 +01:00
Martin Hořeňovský 6603f1d972
Use case in names of default reporters 2021-11-10 23:32:01 +01:00
Martin Hořeňovský 3c5c86a4e4
Add test for filtering out multiple initial values in filter gen 2021-10-28 11:26:53 +02:00
Martin Hořeňovský 12d14a3c63
Add support for multiply calling lambda parsers in Clara
Previously a lambda parser in Clara could only be invoked once,
even if it internally was ok with being invoked multiple times.

With this change, a lambda parser can mark itself as `accept_many`,
in which case it will be invoked multiple times if the appropriate
flag was supplied multiple times by the user.
2021-10-27 20:15:28 +02:00
Martin Hořeňovský 5ac1ffe9ee Improve shardIndex/Count cli argument parsing 2021-10-27 17:24:30 +02:00
Ben Dunkin 3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský 905bf438ae
Fix bad indentation calculation in the console reporter
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".

The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).

We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
           a
           n
           c
           y
           .
           .
           .
```

Fixes #2309
2021-10-25 15:21:28 +02:00
Martin Hořeňovský b406ad52a7
Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-10-10 22:21:39 +02:00
Martin Hořeňovský 1d9696d22d
Cleanup reporting of rng seed in existing reporters 2021-10-09 00:02:30 +02:00
Martin Hořeňovský 200a487cf2
Add generateRandomSeed utility to generate randomness seed 2021-10-08 21:35:41 +02:00
Martin Hořeňovský 426954032f
Rename Contains string matcher builder to ContainsSubstring
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.

Closes #2131
2021-09-27 21:28:33 +02:00
Martin Hořeňovský f02c2678a1
Use StringRef for className in TestCaseInfo 2021-09-27 19:07:45 +02:00
Martin Hořeňovský 21b99d6f58
Add StringRef::compare for three way comparison 2021-09-27 18:41:17 +02:00
Martin Hořeňovský d42e7a23a0
Change startsWith(char) to take StringRef as argument 2021-09-27 14:55:28 +02:00
Martin Hořeňovský e5938007f7
Completely remove the testGroup events
This was done because they were 1:1 mapping to testRun events, and
I could not think of a reasonable way to make them their own thing.
2021-09-07 20:00:22 +02:00
Martin Hořeňovský 3d1cf95b32
Fix ulp distance calculation for numbers with different signs
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.

Closes #2152
2021-07-27 21:04:41 +02:00
Martin Hořeňovský bf61a418cb
Remove the ill-conceived compilation perf tests using real tests 2021-06-20 19:15:02 +02:00
Martin Hořeňovský 816f69416b
Use TestCaseInfo in tests directly 2021-06-01 00:17:19 +02:00
Martin Hořeňovský aee31d0620
Disallow empty tags in test case specification 2021-06-01 00:17:13 +02:00
Martin Hořeňovský 5741de9ccd
Add explicit test for serialization of boolean attributes in XML 2021-05-31 08:41:28 +02:00
Martin Hořeňovský 9137e591fa
Cleanup and optimize stringifying of string-like types
More specifically, made the actual implementation of string-like
type handling take argument as `Catch::StringRef`, instead of
taking `std::string const&`.

This means that string-like types that are not `std::string` no
longer need to pay for an extra construction of `std::string`
(including the potential allocation), before they can be stringified.

The actual string stringification routine is now also better about
reserving sufficient space.
2021-05-18 23:15:22 +02:00
Martin Hořeňovský 5eb7aa4f90
Add test for dots in tag names 2021-05-12 21:43:14 +02:00
Jozef Grajciar eb911aa995
Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187)
Resolves #1390

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
Martin Hořeňovský f1d7a10e06
Remove StringRef::isNullTerminated and StringRef::c_str
Both of them were fundamentally unsafe to use and shouldn't be used
at all.
2021-03-26 21:11:44 +01:00
Martin Hořeňovský 971b1fc32a
--list-* flags write to target specified by the -o flag
Also added tests for the default implementations of list* reporter
helpers.

Closes #2061
2021-01-26 18:36:54 +01:00
Roman Proskuryakov 0acb371b92
Fix Wold-style-cast error (#2125)
* Add Wold-style-cast to cmake flags
* Fix old style cast in catch_stats.hpp
* Fix old style cast in catch_stats.cpp
2020-12-28 14:00:19 +01:00
Martin Hořeňovský 77643ce2e5 Add more comprehensive tests for the quantifier matchers
This includes
* Testing both positive and negative path through the matchers
* Testing them with types whose `begin` and `end` member functions
require ADL
* Testing them with types that return different types from `begin`
and `end`
2020-12-27 20:20:55 +01:00
Uriel García Rivas 552af8920d Added AnyMatch, AllMatch and NoneMatch 2020-12-27 20:20:55 +01:00
Martin Hořeňovský fefa001bb6
Fixup approvals 2020-11-07 20:16:53 +01:00
Martin Hořeňovský 135103bacf
Arg::parse accepts plain Args as argument 2020-11-07 18:00:27 +01:00
Sean Middleditch 31d4831245
Support sentinel-based ranges in default stringify (#2004) 2020-09-07 14:23:47 +02:00
Martin Hořeňovský cc18bd719d
Sweep out Wshadow
Most of the changes are completely pointless renaming of constructor
arguments so that they do not use the same name as the type members,
but 🤷

Closes #2015
2020-09-06 13:11:42 +02:00
Martin Hořeňovský 5ca68829e1
Refactor how shortcircuiting of old style matchers is tested 2020-07-26 21:31:29 +02:00
Martin Hořeňovský ac54ba7e12
Add test for shortcircuiting behaviour of generic matcher combinators 2020-07-26 21:24:05 +02:00
Gavin S 87d0197cbd
Update catch_reporter_tap.hpp
TAP format requires all results to be reported.
Removed extraneous preferences function (handled by parent)
Incorporated fix from 3d9e7db2e0
Simplified total printing
2020-07-26 14:00:10 +02:00
Martin Hořeňovský 4565b826cf
Modify generator tracking to allow GENERATEs between SECTIONs
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-26 11:35:06 +02:00
Ryan Pavlik 7f58840163
Add OverallResultsCases element to XML reporter 2020-07-24 22:30:28 +02:00
Martin Hořeňovský 0e77adee05
Add explicit test for shortcircuiting behaviour of combined matchers 2020-07-22 21:36:14 +02:00
Martin Hořeňovský de53773e46
Fix copy paste error in 7-arg TEMPLATE_TEST_CASE_SIG implementation
Closes #1954
2020-07-22 21:18:52 +02:00
Martin Hořeňovský 480f3f418b
Improved generator tracking
* Successive executions of the same `GENERATE` macro (e.g. because
of a for loop) no longer lead to multiple nested generators.
* The same line can now contain multiple `GENERATE` macros without
issues.

Fixes #1913
2020-07-22 20:44:22 +02:00
Martin Hořeňovský f3fe2dcb11
Add status attribute to JUnit's section reporting
This brings our output inline with GTest's. We do not handle skipped
tests properly, but that should be currently less important than
having the attribute exist with proper value for non-skipped tests.

Thanks @joda-01.

Closes #1899
2020-07-12 21:22:15 +02:00
Martin Hořeňovský 317145514f
Add op+(StringRef, StringRef) -> std::string 2020-05-31 22:30:41 +02:00
Martin Hořeňovský 41bbaa6d57
Implement a simplified variant of std::unique_ptr<T>
This simplified variant supports only a subset of the functionality
in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports
single element pointer (no array support) with default deleter.

By removing the support for custom deleters, we also avoid requiring
significant machinery to support EBO, speeding up instantiations of
`unique_ptr<T>` significantly. Catch2 also currently does not need
to support `unique_ptr<T[]>`, so that is not supported either.
2020-05-31 15:08:47 +02:00
Martin Hořeňovský 895d0a0696
Small improvements for StringRef
* `operator[]` is constexpr
* `operator<<` and `operator+=` are hidden friends
2020-05-10 07:22:11 +02:00
Martin Hořeňovský 33c58dad41
Remove duplicated test for #1027 2020-05-03 09:58:46 +02:00
Martin Hořeňovský 5b8cccaf6a
Add support for bitwise xor to the decomposer 2020-05-03 07:52:47 +02:00
Martin Hořeňovský 53434a2f32
Support bitand and bitor in REQUIRE/CHECK
This means that bit-flag-like types with conversion to bool can be
asserted on, like so `REQUIRE(var & Flags::AddNewline)`.
2020-05-03 07:45:04 +02:00
schallerr 2a93a65bc2
Support custom allocators in vector Matchers (#1909)
-- Combined with f4fc2dab2c
   during cherry-picking.
2020-05-03 07:40:04 +02:00
Martin Hořeňovský dd35430a2b
Add more tests for test spec parser
Originally the tests were from #1912, but as it turned out, the issue
was somewhere else. Still, the inputs provided were interesting, so
they are now part of our test suite.
2020-05-02 18:24:07 +02:00
Martin Hořeňovský 87a8b61d5a
Fix bug in test spec parser handling of escaping in ORed patterns
It did not clear out all of its internal state when switching from
one pattern to another, so when it should've escaped `,`, it took
its position from its position in the original user-provided string,
rather than its position in the current pattern.

Fixes #1905
2020-05-01 09:56:34 +02:00
Martin Hořeňovský 5198fd3c9a
Fix compilation error when lambdas are used in assertions
This is a partial revert of b7b346c3e5.
2020-05-01 09:24:26 +02:00
Martin Hořeňovský f7d7aa9eb2
Fix and extend tests for composing generic matchers 2020-03-29 14:03:12 +02:00
Martin Hořeňovský 3a3efebd16
Add IsEmpty and SizeIs matchers for ranges/containers
`SizeIs` can accept both `size_t` and a matcher. In the first case,
it checks whether the size of the range is equal to specified size.
In the second case, it checks whether the provided matcher accepts
the size of the range.
2020-03-27 10:24:08 +01:00
Martin Hořeňovský 007efc173a
Add generic Contains matcher
It matches a range iff the range contains a specific element,
or an element in the range matches the provided matcher.
2020-03-27 10:22:25 +01:00
Martin Hořeňovský cf6575576f
Start fixing up Matchers: namespaces, composition ops
This commit also forbids composing lvalues of composed matchers, as
per previous deprecation notice. I do not expect this to be contentious
in practice, because there was a bug in that usage for years, and
nobody complained.
2020-02-20 13:03:30 +01:00
Tristan Stenner 6a2c025bfc
Add command line option 'never' to --wait-for-keypress (#1866)
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-02-16 16:11:11 +01:00
Martin Hořeňovský f8794634c2
Fix significant bug with storing composed matchers
Given that in the 2 or so years that matchers are thing nobody complained,
it seems that people do not actually write this sort of code, and the
possibility will be removed in v3. However, to avoid correctness bugs,
we will have to support this weird code in v2.
2020-02-16 16:02:31 +01:00
khyperia 3c7e737a7b
Allow configuring of benchmark warmup time 2020-02-16 15:44:23 +01:00
melak47 17c4b2d093
Feature: generic matchers (#1843)
This commit extends the Matchers feature with the ability to have type-independent (e.g. templated) matchers. This is done by adding a new base type that Matchers can extend, `MatcherGenericBase`, and overloads of operators `!`, `&&` and `||` that handle matchers extending `MatcherGenericBase` in a special manner.

These new matchers can also take their arguments as values and non-const references.

Closes #1307 
Closes #1553 
Closes #1554 

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-02-16 11:19:10 +01:00
Martin Hořeňovský 2945b80f61
Add more constexpr to StringRef 2020-02-13 15:01:03 +01:00
Martin Hořeňovský 63b7d6f98e
Improve erasure of test numbers in TAP approvals 2020-02-13 14:22:18 +01:00
Martin Hořeňovský c50ba09cde
Split [.foo] into [.][foo] when parsing test specs
b77cec05c0 fixed this problem for tagging tests, so that a test
case tagged with `[.foo]` would be parsed as tagged with `[.][foo]`.
This does the same for the test spec parsing.

Fixes #1798
2020-02-13 13:35:10 +01:00
Martin Hořeňovský cd7d7a1c67
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle
Now that Catch2 is a proper library, we can always build the full
library (comparatively minor slowdown) and the user can avoid
including benchmarking headers to avoid the compilation slowdown.
2020-02-06 11:36:46 +01:00
Martin Hořeňovský 34bc56340d
Normalize TAP approvals to avoid massive diffs for every change 2020-02-03 20:29:36 +01:00
Martin Hořeňovský c3a5e21648
Move Approx out of the Detail namespace 2020-02-03 15:14:59 +01:00
Martin Hořeňovský 1327946785
Add all in-repo reporters to approval tests 2020-02-02 15:04:19 +01:00
Martin Hořeňovský d63681f707
Remove testing of CATCH_CONFIG_FALLBACK_STRINGIFIER from SelfTest
This removes a potential ODR violation and the configuration option
is tested in its own separate binary anyway.
2020-01-21 15:14:24 +01:00
Martin Hořeňovský 0fea081ad1
Move tests from projects/ to tests/ 2019-12-05 16:00:20 +01:00