Commit Graph

147 Commits

Author SHA1 Message Date
Martin Hořeňovský
905bf438ae
Fix bad indentation calculation in the console reporter
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".

The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).

We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
           a
           n
           c
           y
           .
           .
           .
```

Fixes #2309
2021-10-25 15:21:28 +02:00
Alecto Irene Perez
06cf2a4724
Apply PR #2297 to devel branch (#2300)
* Apply PR #2297 to devel branch

It turns out that Issue #2272 partially affected the devel branch. When
building tests with C++20, the compiler emits a warning that top-level
comma expressions in array subscripts are depricated. Warnings are
treated as errors, so this caused the build to fail.

This commit adds localized warning suppression
in accordance with this recommendation here:
https://github.com/catchorg/Catch2/pull/2297#discussion_r720848392

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>

* Fixed unknown pragma warning on old versions of gcc & clang

This commit fixes an unkwown pragma warning on older versions of GCC
and Clang. These older versions don't have a warning for depricated use
of the comma subscript. Because warning suppression is localized, and
only refers to the comma subscript warning, it doesn't affect compiler
warnings in other parts of the code.

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>

* More #warning backwards compatibility fixes

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>
2021-10-21 15:47:21 +02:00
Martin Hořeňovský
b406ad52a7
Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-10-10 22:21:39 +02:00
Martin Hořeňovský
1d9696d22d
Cleanup reporting of rng seed in existing reporters 2021-10-09 00:02:30 +02:00
Martin Hořeňovský
200a487cf2
Add generateRandomSeed utility to generate randomness seed 2021-10-08 21:35:41 +02:00
Martin Hořeňovský
13670f535f
Add more tests for XmlWrite::write* members 2021-10-02 14:39:52 +02:00
Martin Hořeňovský
426954032f
Rename Contains string matcher builder to ContainsSubstring
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.

Closes #2131
2021-09-27 21:28:33 +02:00
Martin Hořeňovský
f02c2678a1
Use StringRef for className in TestCaseInfo 2021-09-27 19:07:45 +02:00
Martin Hořeňovský
21b99d6f58
Add StringRef::compare for three way comparison 2021-09-27 18:41:17 +02:00
Martin Hořeňovský
d42e7a23a0
Change startsWith(char) to take StringRef as argument 2021-09-27 14:55:28 +02:00
Martin Hořeňovský
7bb00a42be
Remove stub of obsolete test 2021-09-27 14:45:39 +02:00
Martin Hořeňovský
fb4153e05e
Multiple tests can have same name as long as their tags differ
This change also changes it so that test case macros using a
class name can have same name **and** tags as long as the
used class name differs.

Closes #1915
Closes #1999
2021-09-25 21:37:03 +02:00
Martin Hořeňovský
3f8cae8025
Add consistency-checking event listener to SelfTest
This event listener performs basic consistency checks (akin to
matching braces) on events that are passed to the listeners
when the `SelfTest` test binary is run.

The current checks are about nesting events (e.g. `testCaseStarting`
cannot be received before `testRunStarting`, `sectionStarting`
can only be received when a test case is active, etc), and matching
up counts of starting/ended events.

The simplicity means that it could be confused by starting/ended
events matching up but being out of order, e.g.
```
* test case A starting
* test case B ended
* test case B starting
* test case A ended
```
would be accepted, even though it is wrong. However, doing full
order checking would be much more implementation work, for relatively
little benefit, so it is left out for now.
2021-09-15 23:38:43 +02:00
Martin Hořeňovský
10fb93cce8
Add new reporter event called for each test case enter/exit
This means that e.g. for `TEST_CASE` with two sibling `SECTION`s
the event will fire twice, because the `TEST_CASE` will be entered
twice.

Closes #2107 (the event mentioned there already exists, but this
is its counterpart that we also want to provide to users)
2021-09-14 13:19:45 +02:00
Martin Hořeňovský
e5938007f7
Completely remove the testGroup events
This was done because they were 1:1 mapping to testRun events, and
I could not think of a reasonable way to make them their own thing.
2021-09-07 20:00:22 +02:00
Martin Hořeňovský
ff0a5227ca
Make Approx::operator() const
Closes #2273
2021-08-26 21:27:43 +02:00
Martin Hořeňovský
1cbbc5d2cb
Add test for !shouldfail and failed benchmarks 2021-08-21 00:06:31 +02:00
Martin Hořeňovský
0a3f511cfe
Fix inconsistencies in reporting benchmarking failures
With these changes, all these benchmarks
```cpp
BENCHMARK("Empty benchmark") {};
BENCHMARK("Throwing benchmark") {
    throw "just a plain literal, bleh";
};
BENCHMARK("Asserting benchmark") {
    REQUIRE(1 == 2);
};
BENCHMARK("FAIL'd benchmark") {
    FAIL("This benchmark only fails, nothing else");
};
```

report the respective failure and mark the outer `TEST_CASE` as
failed. Previously, the first two would not fail the `TEST_CASE`,
and the latter two would break xml reporter's formatting, because
`benchmarkFailed`, `benchmarkEnded` etc would not be be called
properly in failure cases.
2021-08-07 15:16:11 +02:00
Martin Hořeňovský
3d1cf95b32
Fix ulp distance calculation for numbers with different signs
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.

Closes #2152
2021-07-27 21:04:41 +02:00
Martin Hořeňovský
bf61a418cb
Remove the ill-conceived compilation perf tests using real tests 2021-06-20 19:15:02 +02:00
AlCash07
c77ba5314a Fix decomposing in presence of universal ADL-found operators
Closes #2121
2021-06-08 23:36:06 +02:00
Martin Hořeňovský
65c9a1d31a
Add test for comparing immovable types 2021-06-07 20:05:03 +02:00
Martin Hořeňovský
fa31d58934
Don't use removed-in-C++20 allocator parts in CustomAllocator tests 2021-06-07 17:44:39 +02:00
Martin Hořeňovský
816f69416b
Use TestCaseInfo in tests directly 2021-06-01 00:17:19 +02:00
Martin Hořeňovský
aee31d0620
Disallow empty tags in test case specification 2021-06-01 00:17:13 +02:00
Martin Hořeňovský
5741de9ccd
Add explicit test for serialization of boolean attributes in XML 2021-05-31 08:41:28 +02:00
Dimitrij Mijoski
86f86c4c23 Fix compiling with C++17 + Clang 5 + libstdc++ v5
This basically tests the combination where the compiler supports most
of C++17 but the library does not.
2021-05-26 00:34:23 +02:00
Martin Hořeňovský
9137e591fa
Cleanup and optimize stringifying of string-like types
More specifically, made the actual implementation of string-like
type handling take argument as `Catch::StringRef`, instead of
taking `std::string const&`.

This means that string-like types that are not `std::string` no
longer need to pay for an extra construction of `std::string`
(including the potential allocation), before they can be stringified.

The actual string stringification routine is now also better about
reserving sufficient space.
2021-05-18 23:15:22 +02:00
Martin Hořeňovský
5eb7aa4f90
Add test for dots in tag names 2021-05-12 21:43:14 +02:00
Jozef Grajciar
eb911aa995
Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187)
Resolves #1390

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
Martin Hořeňovský
f1d7a10e06
Remove StringRef::isNullTerminated and StringRef::c_str
Both of them were fundamentally unsafe to use and shouldn't be used
at all.
2021-03-26 21:11:44 +01:00
Martin Hořeňovský
3afea8128a
Increase timing window for min duration tests to 1s
This test tends to be brittle on Mac CI machines, which are
heavily loaded and bursty. Since the tests are only run as part
of the "extra tests" test set, this increase should not have
a significant impact on the total duration of CI runs.
2021-02-20 23:09:02 +01:00
Martin Hořeňovský
8b27041fbe
Add CMake option to generate surrogate TUs in development build
A surrogate TU is TU that includes 1 specific header, and does
nothing else. This is useful to verify that the header is
self-sufficient and does not require other headers to be included
for compilation to succeed.

Closes #2106
Closes #2166 (this is a better solution)
2021-02-19 23:22:39 +01:00
Martin Hořeňovský
971b1fc32a
--list-* flags write to target specified by the -o flag
Also added tests for the default implementations of list* reporter
helpers.

Closes #2061
2021-01-26 18:36:54 +01:00
Martin Hořeňovský
1d269211bd
Split CATCH_CONFIG_WCHAR into its own header
Part of #2041
2020-12-28 20:51:49 +01:00
Roman Proskuryakov
0acb371b92
Fix Wold-style-cast error (#2125)
* Add Wold-style-cast to cmake flags
* Fix old style cast in catch_stats.hpp
* Fix old style cast in catch_stats.cpp
2020-12-28 14:00:19 +01:00
Martin Hořeňovský
77643ce2e5 Add more comprehensive tests for the quantifier matchers
This includes
* Testing both positive and negative path through the matchers
* Testing them with types whose `begin` and `end` member functions
require ADL
* Testing them with types that return different types from `begin`
and `end`
2020-12-27 20:20:55 +01:00
Uriel García Rivas
552af8920d Added AnyMatch, AllMatch and NoneMatch 2020-12-27 20:20:55 +01:00
Martin Hořeňovský
a091853f4a
Rename file catch_matchers_floating -> catch_matchers_floating_point
The old name was a legacy of v2 era, where all headers were stitched
into one. With v3 using separate headers, it is better when they have
proper name.
2020-12-13 18:12:20 +01:00
Martin Hořeňovský
19cbdebb0e
Lenghten timing threshold in --min-duration integration test 2020-11-28 21:00:09 +01:00
Martin Hořeňovský
fefa001bb6
Fixup approvals 2020-11-07 20:16:53 +01:00
Martin Hořeňovský
135103bacf
Arg::parse accepts plain Args as argument 2020-11-07 18:00:27 +01:00
Martin Hořeňovský
7b9bf633be
Tighten the test for matching filenames-as-tags 2020-10-23 23:02:09 +02:00
Martin Hořeňovský
4c8454b5ec
Fix potential infinite loops in generators combined with section filter
The problem was that under specific circumstances, namely that none
of their children progressed, `GeneratorTracker` will not progress.
This was changed recently, to allow for code like this, where a
`SECTION` follows a `GENERATE` at the same level:

```cpp
SECTION("A") {}
auto a = GENERATE(1, 2);
SECTION("B") {}
```

However, this interacted badly with `SECTION` filters (`-c foo`),
as they could deactivate all `SECTION`s below a generator, and thus
stop it from progressing forever. This commit makes GeneratorTracker
check whether there are any filters active, and if they are, it checks
whether its section-children can ever run.

Fixes #2025
2020-10-23 21:21:15 +02:00
Martin Hořeňovský
8878f90323
Fix matching of non-lowercase filename tags
The bug was caused by forgetting to lower-case the filename tag
for matching against test spec.

Fixes #2064
2020-10-22 16:20:41 +02:00
Martin Hořeňovský
923bcc5d6f
Special case --list-tests --verbosity quiet
The new output (mostly) follows the old `--list-test-names-only`
format, with the exception of no longer supporting line output
for `--verbosity high`.

Fixes #2051
2020-10-20 15:09:48 +02:00
Martin Hořeňovský
6ffac61719
Fix types in generators: decay types, properly constrain forwarding
Fixes #2040
Closes #2012
2020-10-20 10:57:37 +02:00
Martin Hořeňovský
60dfec559f
Provide 1 .hpp + 1 .cpp distribution of Catch2
This commits also adds a script that does the amalgamation of headers
and .cpp files into the distributable version, removes the old
`generateSingleHeader` script, and also adds a very simple compilation
test for the amalgamated distribution.
2020-09-09 13:02:50 +02:00
Sean Middleditch
31d4831245
Support sentinel-based ranges in default stringify (#2004) 2020-09-07 14:23:47 +02:00
Martin Hořeňovský
2d4f8ac8e6
Disable problematic test on Windows 2020-09-06 20:41:58 +02:00