Commit Graph

921 Commits

Author SHA1 Message Date
Xo Wang 33794a204c Fix clang analyzer warning about FilterGenerator
Refactor FilterGenerator to remove ctor call to overridden method next()
in order to address clang static analyzer diagnostic:

catch2-src/single_include/catch2/catch.hpp:4166:42: note: Call to virtual method 'FilterGenerator::next' during construction bypasses virtual dispatch
                auto has_initial_value = next();
                                         ^~~~~~
2021-11-24 11:30:03 +01:00
Martin Hořeňovský f45dac8fc1 Add broken test 2021-11-24 11:30:03 +01:00
Alecto Irene Perez dba29b60d6
Fixed #2272: Compilation failure with C++20 (#2297)
* Suppressed warning for comma-in-indexing-operator in tests that check
  that specific behaviour.
* Made deprecated (and removed) allocator usings conditional on the tests
  being compiled with old version of MSVC that still requires them.

Fixes #2272
2021-10-04 21:11:03 +02:00
Rupert Nash 3d01f3ae32 Backport changes from 7bea1e2ac36ac54b648ae5c9d381a59bc69db912 to fix #2273 for 2.x 2021-09-18 21:24:03 +02:00
Martin Hořeňovský eea3e9a5b5
Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-07-27 23:16:01 +02:00
Dimitrij Mijoski 531a149ae7 Fix compiling v2.x with C++17 + Clang 5 + libstdc++ v5
This basically tests the combination where the compiler supports most
of C++17 but the library does not.
2021-05-25 23:50:45 +02:00
Martin Hořeňovský 765ac08f08
Fix potential infinite loops in generators combined with section filter
The problem was that under specific circumstances, namely that none
of their children progressed, `GeneratorTracker` will not progress.
This was changed recently, to allow for code like this, where a
`SECTION` follows a `GENERATE` at the same level:

```cpp
SECTION("A") {}
auto a = GENERATE(1, 2);
SECTION("B") {}
```

However, this interacted badly with `SECTION` filters (`-c foo`),
as they could deactivate all `SECTION`s below a generator, and thus
stop it from progressing forever. This commit makes GeneratorTracker
check whether there are any filters active, and if they are, it checks
whether its section-children can ever run.

Fixes #2025
2020-10-31 18:04:15 +01:00
Martin Hořeňovský e6f5e05ebc
Improve detection of std::uncaught_exceptions support
The problem was that Catch2 did not reliably include `<exception>`
before it checked for the feature test macro for
`std::uncaught_exceptions`. To avoid overhead of including
`<exception>` everywhere, the configuration check was split out
into a separate header.

Closes #2021
2020-10-06 11:07:53 +02:00
Sean Middleditch 284672cc84
Support sentinel-based ranges in default stringify (#2004) 2020-08-18 10:34:47 +02:00
Martin Hořeňovský b79a83e4aa
Modify generator tracking to allow GENERATEs between SECTIONs
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-11 23:16:07 +02:00
Martin Hořeňovský 89fe35d515
Fix how testRandomOrder.py builds tag arguments 2020-07-11 21:32:10 +02:00
Martin Hořeňovský 115d6a1c40
Increase tolerances in --min-duration tests
The underpowered and oversubscribed CI servers are hell.
2020-07-07 11:36:56 +02:00
Martin Hořeňovský 6f7c191513
--min-duration is overriden by -d no 2020-07-06 20:33:08 +02:00
Martin Hořeňovský 4eb9d37e05
Refactor tests for duration reporting threshold 2020-07-06 20:02:20 +02:00
John Bytheway 53d8af8e96 Test for --min-duration 2020-07-06 11:35:02 +02:00
John Bytheway 46fde0c597 Add --min-duration option
A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.

When there are many tests, this produces output spam which makes it hard
to find the test failure output.  Nevertheless, it is helpful to be
informed of tests which are unusually slow.

Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed.  This allows slow
tests to be visible without mentioning every test.
2020-07-06 11:35:02 +02:00
Ryan Pavlik ee4538c0c6 Add OverallResultsCases element to XML reporter 2020-06-28 22:36:08 +02:00
Martin Hořeňovský 1806b21545
Add explicit test for shortcircuiting behaviour of combined matchers 2020-06-14 21:48:08 +02:00
Martin Hořeňovský cbbebb65b6
Fix copy paste error in 7-arg TEMPLATE_TEST_CASE_SIG implementation
Closes #1954
2020-06-13 15:46:59 +02:00
Martin Hořeňovský 9500ded83b
Improved generator tracking
* Successive executions of the same `GENERATE` macro (e.g. because
of a for loop) no longer lead to multiple nested generators.
* The same line can now contain multiple `GENERATE` macros without
issues.

Fixes #1913
2020-06-01 19:06:51 +02:00
Martin Hořeňovský f133277910
Add status attribute to JUnit's section reporting
This brings our output inline with GTest's. We do not handle skipped
tests properly, but that should be currently less important than
having the attribute exist with proper value for non-skipped tests.

Thanks @joda-01.

Closes #1899
2020-05-15 21:00:19 +02:00
Martin Hořeňovský 5c9f09e94a
Add support for bitwise xor to the decomposer 2020-04-21 19:27:12 +02:00
Martin Hořeňovský f4fc2dab2c
Fixup template type argument inference for vector matchers 2020-04-21 19:09:45 +02:00
Martin Hořeňovský 4e4171420d
Support bitand and bitor in REQUIRE/CHECK
This means that bit-flag-like types with conversion to bool can be
asserted on, like so `REQUIRE(var & Flags::AddNewline)`.
2020-04-21 11:00:08 +02:00
Martin Hořeňovský 37cbf4a4fe
Add more tests for test spec parser
Originally the tests were from #1912, but as it turned out, the issue
was somewhere else. Still, the inputs provided were interesting, so
they are now part of our test suite.
2020-04-17 21:19:18 +02:00
schallerr 38f897c887
Support custom allocators in vector Matchers (#1909) 2020-04-16 15:36:54 +02:00
Martin Hořeňovský a2fc7cf8c0
Randomize test for subset invariant random ordering of tests
Also removed the iterative checking that seeds 1-100 do not create
the same output, because it used too much runtime.
2020-04-14 16:38:10 +02:00
John Bytheway da9e3eec65 Add test for consistent random ordering 2020-04-14 12:47:36 +02:00
Martin Hořeňovský 5d32ce26f4
Fix bug in test spec parser handling of escaping in ORed patterns
It did not clear out all of its internal state when switching from
one pattern to another, so when it should've escaped `,`, it took
its position from its position in the original user-provided string,
rather than its position in the current pattern.

Fixes #1905
2020-04-12 18:48:52 +02:00
Joel Uckelman e5c9a58d66 Fixed typo in "benchmark name" column width calculation. Closes #1885. 2020-03-26 10:31:35 +01:00
Martin Hořeňovský 3a33315ff8
Fix compilation error when lambdas are used in assertions
This is a partial revert of b7b346c3e5.
2020-03-19 13:32:45 +01:00
Martin Hořeňovský b7b346c3e5
Make warnings in assertions fire for GCC/Clang again
The old code caused warnings to fire under MSVC, and Clang <3.8.
I could not find a GCC version where it worked, but I assume that it
did at some point.

This new code causes all of MSVC, GCC, Clang, in current versions,
to emit signed/unsigned comparison warning in test like this:

```cpp
TEST_CASE() {
    int32_t i = -1;
    uint32_t j = 1;
    REQUIRE(i != j);
}
```

Where previously only MSVC would emit the warning.

Fixes #1880
2020-03-19 09:45:41 +01:00
Tristan Stenner 200b8b6fc0
Add command line option 'never' to --wait-for-keypress (#1866)
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-02-15 20:42:57 +01:00
offa 36170d60ca stdio.h replaced with cstdio. 2020-02-08 14:27:40 +01:00
Martin Hořeňovský f20a9dbc6e
Fix significant bug with storing composed matchers
Given that in the 2 or so years that matchers are thing nobody complained,
it seems that people do not actually write this sort of code, and the
possibility will be removed in v3. However, to avoid correctness bugs,
we will have to support this weird code in v2.
2020-02-01 21:06:07 +01:00
Martin Hořeňovský 6a3d0dc176
Add a test for custom debug break macros
See #1846
2020-02-01 20:18:05 +01:00
khyperia ccb1f70629 Make CATCH_BREAK_INTO_DEBUGGER be user-configurable 2020-02-01 17:01:50 +01:00
dmsteck 481f54b357
Make hidden tags behave identically (#1847)
Add both `[.]` and `[!hide]` tags when registering a hidden test case, as per documentation.

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-01-31 21:44:06 +01:00
offa 2d172dc688 Some refactorings:
- Overrides added
 - usages of push_back() replaced with emplace_back()
 - Loop variable made const-refernce
 - NULL replaced with nullptr
 - Names used in the declaration and definition unified
 - size() replaced with empty
 - Identical cases merged
2020-01-25 09:01:04 +01:00
Joe Burzinski 6c9a255dc2 Fix forwarding in SingleValueGenerator and generator creation
Fixes #1809
2019-12-15 20:50:43 +01:00
Joe Burzinski cfba9dce97 Fix wrong namespacing of benchmarking constructor helpers 2019-11-21 16:22:04 +01:00
Martin Hořeňovský d7b8c3ace3
Minor cleanup in the benchmarking tests 2019-11-15 11:59:35 +01:00
Martin Hořeňovský 930f49a641
Split [.foo] into [.][foo] when parsing test specs
b77cec05c0 fixed this problem for tagging tests, so that a test
case tagged with `[.foo]` would be parsed as tagged with `[.][foo]`.
This does the same for the test spec parsing.

Fixes #1798
2019-11-05 23:28:47 +01:00
Martin Hořeňovský c409dccee5
Cleanup tests for C++20 2019-11-04 10:42:34 +01:00
Martin Hořeňovský 776a4686c7
Warning fixes in examples and tests 2019-10-31 14:29:59 +01:00
Martin Hořeňovský 74e0e737a6
Remove useless test 2019-10-29 11:44:24 +01:00
Martin Hořeňovský 849f2848bd
Rework how warning suppression in macros is done
Previously, each warning suppression was self-contained, with its
own pair of `SUPPRESS_X_WARNING` and `UNSUPPRESS_X_WARNING` macros.
This had the obvious advantage of being self-containing, but it
also meant that if we needed to suppress more than one warning
in a single place, then we would manipulate the compiler's warning
state multiple times, even though logically we would only need one
layer.

The new way of suppressing warnings in macros is to push compiler's
warning state with `CATCH_INTERNAL_START_WARNINGS_SUPPRESSION` macro,
then disable whatever macros we need with the
`CATCH_INTERNAL_SUPPRESS_X_WARNINGS` macro, and then return to the
previous state using `CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION`.
2019-10-27 21:07:21 +01:00
Alexander Stein 2fbd66c51c Improve JUnit failure report
The JUnit report is improved in that:
* The message shows the testing condition, not the result
* The actual message has similar output than the console one
2019-10-27 18:02:14 +01:00
sp-dani-garcia 51b29ced1a Add SonarQube Generic Test Data reporter
It outputs reports in the `Generic Execution Test Data` format, see
https://docs.sonarqube.org/latest/analysis/generic-test/, specifically
https://docs.sonarqube.org/latest/analysis/generic-test/#header-2

Close #1738 (this is a cherry-pick and fixup of that PR)
2019-10-27 10:02:41 +01:00
Manuel Drews 4327baba40 Enable `range` generator for floating point types 2019-10-26 20:52:09 +02:00