Commit Graph

71 Commits

Author SHA1 Message Date
Chris Thrasher
bfe3ff8f19 Specify minimum C++ version for amalgamated test build
This target previously did not specify its minimum C++ standard.
AppleClang was emitting warnings about the use of C++11 features
which is fixed by telling CMake that this target requires C++14.
Because this target does not link to the existing CMake targets
it never inherited that C++ standard requirement.
2024-03-13 21:12:59 +01:00
Chris Thrasher
792c3b7549 Stop repeating conditional in endif()
This is not necessary so we can remove it. Most conditionals already
omit this anyways.
2024-03-12 22:55:12 +01:00
Martin Jeřábek
cde3509664 Fix various useful clang-tidy warnings
* bugprone-branch-clone
* bugprone-copy-constructor-init
* bugprone-empty-catch
* bugprone-sizeof-expression
* bugprone-switch-missing-default-case
* bugprone-unused-local-non-trivial-variable
* clang-analyzer-core.uninitialized.Assign
* clang-analyzer-cplusplus.Move
* clang-analyzer-optin.cplusplus.VirtualCall
* modernize-loop-convert
* modernize-raw-string-literal
* modernize-use-equals-default
* modernize-use-override
* modernize-use-using
* performance-avoid-endl
* performance-inefficient-string-concatenation
* performance-inefficient-vector-operation
* performance-noexcept-move-constructor
* performance-unnecessary-value-param (and improve generator example)
* readability-duplicate-include
* readability-inconsistent-declaration-parameter-name
* readability-non-const-parameter
* readability-redundant-casting
* readability-redundant-member-init
* readability-redundant-smartptr-get
* readability-static-accessed-through-instance
* unused variable in amalgamted tests
2024-03-01 21:24:45 +01:00
Martin Hořeňovský
683c85772f
Clean up explanation in tests 2023-07-12 11:49:43 +02:00
Martin Hořeňovský
d84777c9cb
Fix assertionStarting events being sent after the expr is evaluated
Closes #2678
2023-05-06 11:48:41 +02:00
Philip Salzmann
d548be26e3
Add new SKIP macro for skipping tests at runtime (#2360)
* Add new SKIP macro for skipping tests at runtime

This adds a new `SKIP` macro for dynamically skipping tests at runtime.
The "skipped" status of a test case is treated as a first-class citizen,
like "succeeded" or "failed", and is reported with a new color on the
console.

* Don't show "skipped assertions" in console/compact reporters

Also extend skip tests to cover a few more use cases.

* Return exit code 4 if all test cases are skipped

* Use LightGrey for the skip colour

This isn't great, but is better than the deep blue that was borderline
invisible on dark backgrounds. The fix is to redo the colouring
a bit, including introducing light-blue that is actually visible.

* Add support for explicit skips in all reporters

* --allow-running-no-tests also allows all tests to be skipped

* Add docs for SKIP macro, deprecate IEventListener::skipTest

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
Martin Hořeňovský
d1ef461471
Update basic cost estimates for approvals/benchmarks 2022-11-06 21:23:03 +01:00
Martin Hořeňovský
82cec69e93
Don't set C++ standard in CMakeLists for tests 2022-10-29 21:04:58 +02:00
Lars Toenning
f1084fb309
Fix references to license file
The license file was renamed with 6a502cc2f5
2022-10-28 11:30:15 +02:00
Philip Salzmann
6185d0cc0a Use console reporter's totals summary for compact reporter
This changes the compact reporter's summary of test run totals to use
the same format as the console reporter. This means that while output is
no longer on a single line (two instead), it now includes totals for
`failedButOk` test cases and assertions, which were previously missing.
2022-10-25 20:02:26 +02:00
Martin Hořeňovský
80d58a791d
Add support for Bazel's sharding env variables
Closes #2491
2022-10-21 10:53:53 +02:00
Martin Hořeňovský
9d08689845
Add support for Bazel's TESTBRIDGE_TEST_ONLY env var
Closes #2490
2022-10-20 21:34:32 +02:00
Dimitrij Mijoski
98d37da03e
Raise the minimum CMake version to 3.10 (#2523) 2022-09-16 17:56:00 +02:00
Brandon Jones
7e4ec432d0 Change Bazel XML support to depend upon BAZEL_TEST 2022-06-17 16:36:14 +02:00
Sergio Losilla
5efd327dd4
Fix crash when running with --list-listeners and no registered listeners (#2442)
Co-authored-by: Sergio Losilla <sergio.losilla@nt-bnct.com>
2022-06-02 16:11:12 +02:00
Martin Hořeňovský
316025a0d8
Properly handle startup reporter registration errors 2022-05-15 10:35:12 +02:00
Martin Hořeňovský
d2cb934d28
Add -Wmissing-variable-declarations to common warnings 2022-05-03 13:54:03 +02:00
Martin Hořeňovský
c65e5b6514
Add 'uses-python' label to tests using Python 2022-04-24 13:31:28 +02:00
Martin Hořeňovský
05e85c5652
Split out Catch::cout/cerr/clog into their own file 2022-04-16 16:33:50 +02:00
Martin Hořeňovský
c742ea9ad9
Some IWYU include cleanups 2022-04-14 17:16:01 +02:00
Martin Hořeňovský
79d1e82381
Store IStream instance owning ptrs in reporter instances
This ended up being a surprisingly large refactoring, motivated
by removing a `const_cast` from `Config`'s handling of reporter
streams, forced by previous commit.
2022-04-13 15:00:50 +02:00
Lukasz Okraszewski
cb551b4f6d
Add default reporter for Bazel integration (#2399)
When the added Bazel configuration flag is enabled,
a default JUnit reporter will be added if the XML
envrioment variable is defined.
Fix include paths for generated config header.
Enable Bazel config by default when building with
Bazel.


Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-04-08 18:01:59 +02:00
Martin Hořeňovský
423e1d2ebb
Integrate the new reporter specs into Catch2
This means that the CLI interface now uses the new key-value oriented
reporter spec, the common reporter base creates the colour implementation
based on the reporter-specific configuration, and it also stores the
custom configuration options for each reporter instance.

Closes #339 as it allows per-reporter forcing of ansi colour codes.
2022-04-07 12:50:08 +02:00
Martin Hořeňovský
4acc520f76
Event listeners no longer take reporter config in constructor
This also required splitting out Listener factory from
the reporter factory hierarchy. In return, the listener
factories only need to take in `IConfig const*`, which
opens up further refactorings down the road in the colour
selection and implementation.
2022-03-18 00:36:18 +01:00
John Beard
f9facc1881 Add variadic CAPTURE tests
This makes sure that CAPTURE works when called with variadic arguments,
and also works that way when disabled.

The underlying fix to #2316 is not applicable (CAPTURE is already
variadic when disabled).

This is a port of 5e94498ed0 to the
devel branch.
2022-03-05 16:06:54 +01:00
Martin Hořeňovský
79e4cd1af4
Cleanup summary comments in ExtraTests 2022-01-29 00:40:03 +01:00
Martin Hořeňovský
635db2785f
Use the new licence header in ExtraTests's cpp files 2022-01-29 00:07:44 +01:00
Martin Hořeňovský
e53a75b425
Add integration test for listener-reporter event ordering 2022-01-28 00:00:52 +01:00
Martin Hořeňovský
88b28ab592
Extra compilation checks for compiling without exceptions 2022-01-01 23:21:37 +01:00
Martin Hořeňovský
ef3374ed81
Add more macros to the DisabledMacros test 2022-01-01 16:45:28 +01:00
Martin Hořeňovský
899554bff2 Add test for handling of reporter's assertion preferences 2022-01-01 14:15:44 +01:00
Martin Hořeňovský
b4efa4751a Add test for handling of Listeners requesting stdout capture 2022-01-01 14:13:48 +01:00
Martin Hořeňovský
6b55f5d780
Don't send full stdout so far to testCasePartialEnded event 2022-01-01 12:21:41 +01:00
Martin Hořeňovský
9f2dca5384
Exit with non-0 return code if no tests were run
A new flag, `--allow-running-no-tests` was added to override this
behaviour if exit code of 0 was desired.

This change also made `-w NoTests` obsolete, and so it has been
removed.
2021-12-18 20:50:00 +01:00
Martin Hořeňovský
d012735c6e
Add test for runtime behaviour of STATIC_CHECK 2021-11-15 20:06:29 +01:00
Morwenn
f41d761674
Add STATIC_CHECK and STATIC_CHECK_FALSE (#2318) 2021-11-15 00:28:27 +01:00
Martin Hořeňovský
9200b4078b
Move reporter_registrars.hpp to reporters/ 2021-11-10 23:52:59 +01:00
Martin Hořeňovský
62d8913d67
Cumulative reporter base records benchmark results 2021-11-09 11:52:50 +01:00
Martin Hořeňovský
8780425385
Make reporter lookup case insensitive, registration case preserving
Previously registration was case preserving, but lookup used
lowercased reporter name, so a reporter whose name contained
upper case character could not be requested by the user.
2021-11-09 11:50:03 +01:00
Ben Dunkin
3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský
fb4153e05e
Multiple tests can have same name as long as their tags differ
This change also changes it so that test case macros using a
class name can have same name **and** tags as long as the
used class name differs.

Closes #1915
Closes #1999
2021-09-25 21:37:03 +02:00
Martin Hořeňovský
10fb93cce8
Add new reporter event called for each test case enter/exit
This means that e.g. for `TEST_CASE` with two sibling `SECTION`s
the event will fire twice, because the `TEST_CASE` will be entered
twice.

Closes #2107 (the event mentioned there already exists, but this
is its counterpart that we also want to provide to users)
2021-09-14 13:19:45 +02:00
Martin Hořeňovský
3afea8128a
Increase timing window for min duration tests to 1s
This test tends to be brittle on Mac CI machines, which are
heavily loaded and bursty. Since the tests are only run as part
of the "extra tests" test set, this increase should not have
a significant impact on the total duration of CI runs.
2021-02-20 23:09:02 +01:00
Martin Hořeňovský
19cbdebb0e
Lenghten timing threshold in --min-duration integration test 2020-11-28 21:00:09 +01:00
Martin Hořeňovský
60dfec559f
Provide 1 .hpp + 1 .cpp distribution of Catch2
This commits also adds a script that does the amalgamation of headers
and .cpp files into the distributable version, removes the old
`generateSingleHeader` script, and also adds a very simple compilation
test for the amalgamated distribution.
2020-09-09 13:02:50 +02:00
Martin Hořeňovský
2d4f8ac8e6
Disable problematic test on Windows 2020-09-06 20:41:58 +02:00
Martin Hořeňovský
90d6fd849e
Increase tolerances in --min-duration tests
The underpowered and oversubscribed CI servers are hell.
2020-07-26 10:48:03 +02:00
Martin Hořeňovský
13917c44b4
--min-duration is overriden by -d no 2020-07-26 10:48:01 +02:00
Martin Hořeňovský
e6d947f6d4
Refactor tests for duration reporting threshold 2020-07-26 10:47:58 +02:00
Martin Hořeňovský
db32550898
Remove catch_default_main.hpp
There are two reasons for this:

1) It is highly unlikely that someone has use for this header,
which has no customization points and only provides simplest
possible main, and cannot link the static library which also
provides a default main implementation.
2) It being a header was causing extra complications with
the convenience headers, and our checking script. This would either
require special handling in the checking script, or would break user's
of the main convenience header.

All in all, it is simpler and better in the long term to remove it,
than to fix its problems.
2020-05-09 18:00:49 +02:00