Compare commits

..

215 Commits
v3.3.2 ... v2.x

Author SHA1 Message Date
Martin Hořeňovský
ee1450f268 Merge pull request #2862 from rikyoz/backport-clang-tidy-bugprone-chained-comparison
Fix clang-tidy `bugprone-chained-comparison` warnings on v2.x
2024-04-25 14:29:32 +02:00
Oz
a2b2e1f707 Fix clang-tidy bugprone-chained-comparison warnings on v2.x 2024-04-22 22:05:38 +02:00
morinmorin
a7782d1d7c Add workaround for unguarded use of __has_extension (for v2.x) 2024-03-12 22:58:19 +01:00
Martin Hořeňovský
d4b0b34561 Fix lowercase namespace in own-main example
Fixes #2715
2023-07-13 14:02:19 +02:00
Martin Sternevald
c359076e8a Fix missing include causing compiler error
catch_registry_hub.cpp:65:17: error: use of undeclared identifier 'CATCH_INTERNAL_ERROR'
                CATCH_INTERNAL_ERROR("Attempted to register active exception under CATCH_CONFIG_DISABLE_EXCEPTIONS!");
2022-12-10 23:35:59 +01:00
alvinhochun
20ace55034 Allow ANSI colour to be compiled on Windows
This enables building with `CATCH_CONFIG_COLOUR_ANSI` on Windows. The changes are taken from v3 commit 0e176c318b.
2022-10-25 12:10:28 +02:00
Martin Hořeňovský
182c910b4b v2.13.10 2022-10-16 11:02:17 +02:00
Thomas Gamper
02123776f2 fixes catchorg/Catch2#2401 2022-08-21 19:51:36 +02:00
David Matson
14bc25be00 Explicitly specify __cdecl on main/wmain for MSVC (#2487)
Fixes #2486

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-08-06 21:15:53 +02:00
Martin Hořeňovský
20d413b8b6 Fix comma-subscript warning suppression to only target GCC 10.1+
Fixes #2416
2022-06-06 01:04:12 +02:00
Martin Hořeňovský
e33de8fc05 Update FetchContent docs to use 2.13.9 2022-05-31 12:31:02 +02:00
Martin Hořeňovský
1127d847a1 Use better instruction for CATCH_TRAP on ARM Macs
The old instruction would cause the debugger to be stuck at the
triggering source line forever, while the new one should have the
expected semantics, where the debugger can then single-step,
continue. or generally do things, afterwards.

Closes #2422
2022-05-15 23:20:32 +02:00
Marc Mutz
ca455815fd ObjectStorage: port away from std::aligned_storage
It's deprecated in C++23. Just use alignas on a char array, wrapped in
a struct to avoid decaying to char*, which is the canonical
implementation of aligned_storage:
https://en.cppreference.com/w/cpp/types/aligned_storage#Possible_implementation

Fixes #2419

Catch3 is not affected.
2022-05-10 00:06:45 +02:00
Martin Hořeňovský
d71b4617e9 Disable Werror for unknown pragmas in tests
Closes #2375
2022-04-17 21:12:38 +02:00
Martin Hořeňovský
62fd660583 v2.13.9 2022-04-12 22:38:17 +02:00
Martin Mekota
0aa4dbae2e Fix filename as tag bug
When a test file isn't provided with "/" in the path,
the filename isn't registered as a tag.
Fixing by providing a conditon for this edge case.

Closes #2328
2022-03-18 22:32:54 +01:00
John Beard
ff151d2833 Fix: CAPTURE not variadic when disabled (#2378)
Closes: #2316
2022-03-04 15:29:22 +01:00
Mikhail Matrosov
958944d27a Allow to specify CATCH_CONFIG_DEFAULT_REPORTER for static library with main() (#2346) 2022-01-25 21:54:19 +01:00
Martin Hořeňovský
216713a406 v2.13.8 2022-01-03 21:21:39 +01:00
Bernhard Manfred Gruber
e9e4117016 Remove double-underscores in macros
Such identifiers are reserved by the C++ standard.
Fixes part of #578.
2021-12-16 14:34:04 +01:00
Dan Raviv
8a06a6dce8 Fix typo 2021-12-10 16:01:55 +01:00
Xo Wang
33794a204c Fix clang analyzer warning about FilterGenerator
Refactor FilterGenerator to remove ctor call to overridden method next()
in order to address clang static analyzer diagnostic:

catch2-src/single_include/catch2/catch.hpp:4166:42: note: Call to virtual method 'FilterGenerator::next' during construction bypasses virtual dispatch
                auto has_initial_value = next();
                                         ^~~~~~
2021-11-24 11:30:03 +01:00
Martin Hořeňovský
f45dac8fc1 Add broken test 2021-11-24 11:30:03 +01:00
Martin Hořeňovský
f2f0dcc511 Add VS2015 + char literals in GENERATE issue to known-limitations
Closes #2207
2021-11-15 23:27:17 +01:00
Alecto Irene Perez
dba29b60d6 Fixed #2272: Compilation failure with C++20 (#2297)
* Suppressed warning for comma-in-indexing-operator in tests that check
  that specific behaviour.
* Made deprecated (and removed) allocator usings conditional on the tests
  being compiled with old version of MSVC that still requires them.

Fixes #2272
2021-10-04 21:11:03 +02:00
Rupert Nash
3d01f3ae32 Backport changes from 7bea1e2ac36ac54b648ae5c9d381a59bc69db912 to fix #2273 for 2.x 2021-09-18 21:24:03 +02:00
Biswapriyo Nath
85c9544fa4 pkgconfig: Add missing entries
This adds prefix, exec_prefix and libdir fields
2021-09-10 20:08:03 +02:00
Jørgen P. Tjernø
4b9780201b Fix warning suppressions leaking under clang.exe
When running clang.exe under Windows, catch.hpp leaks warning
suppressions because it uses `#pragma warning(push)` & `#pragma
warning(pop)` around warning suppressions like `#pragma clang diagnostic
ignore "-Wunused-variable"`, instead of using `#pragma clang diagnostic
push` and `#pragma clang diagnostic pop`.

This fixes that by only defining
`CATCH_INTERNAL_START_WARNINGS_SUPPRESSION` and
`CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION` to be the cl.exe variants if
`defined(_MSC_VER) && !defined(__clang__)`.
2021-08-30 23:26:51 +02:00
Martin Hořeňovský
c4e3767e26 v2.13.7 2021-07-28 20:30:51 +02:00
Martin Hořeňovský
eea3e9a5b5 Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-07-27 23:16:01 +02:00
Martin Hořeňovský
7727c15290 Remove Conan builds from travis-CI 2021-06-07 20:02:30 +02:00
Dimitrij Mijoski
531a149ae7 Fix compiling v2.x with C++17 + Clang 5 + libstdc++ v5
This basically tests the combination where the compiler supports most
of C++17 but the library does not.
2021-05-25 23:50:45 +02:00
Martin Hořeňovský
9683570be7 Tiny optimization in JUnit reporter 2021-05-23 11:31:33 +02:00
Martin Hořeňovský
581c46249a JUnit reporter uses only 3 decimal places when reporting durations
We used to use whatever precision we ended up having from C++'s
stdlib. However, some relatively popular tools, like Jenkins,
use Maven SureFire XML schema to validate JUnit test reports, and
Maven SureFire schema requires the duration to have at most 3
decimal places.

For compatibility, the JUnit reporter will now respect this
limitation.

Closes #2221
2021-05-22 23:45:43 +02:00
Evgeny Proydakov
b15b862d86 Fixed noexcept benchmark build for gcc. 2021-05-21 20:49:16 +02:00
Martin Hořeňovský
6971476563 Ensure that <iterator> is included before back_inserter is used
Closes #2231
2021-05-21 20:20:00 +02:00
Martin Hořeňovský
5c88067bd3 v2.13.6 2021-04-16 20:14:58 +02:00
Georg Schwab
86a4d704bc fixed inconsistent semicolon expansion in catch_discover_tests (Bug #2214) 2021-04-16 17:26:08 +02:00
Matteo Beniamino
469a717395 Fixed [dis]engage_platform declarations mismatch 2021-04-13 19:50:32 +02:00
Martin Hořeňovský
42e368dd0a v2.13.5 2021-04-10 23:48:32 +02:00
Clare Macrae
1ff1f2741d Prevent Catch2 v2 tests running if loaded as a sub-project
See #2202
2021-04-09 23:54:13 +02:00
Julien Brianceau
269f96e2bc Fix typos in the code base
Note that only documentation and comments are impacted by this change.
2021-04-08 18:06:36 +02:00
Martin Hořeňovský
cec35630fb Don't build Catch2WithMain target by default for v2
The `Catch2WithMain` target was added experimentally for v2.13.4
to provide potentially better compilation (and link) times to users.
However, having it compiled by default causes worse experience for
people who do not use it, and for the v2 versions the single include
distribution is still the main one.

Closes #2142
2021-04-05 18:04:31 +02:00
Martin Hořeňovský
b8b03da534 Mangle doccoments to avoid breaking single include stitching 2021-04-04 18:09:29 +02:00
Martin Hořeňovský
8f277a54c0 Significantly refactor fatal error handling
Because new glibc has changed `MINSIGSTKSZ` to be a syscall instead
of being constant, the signal posix handling needed changes, as it
used the value in constexpr context, for deciding size of an array.
It would be simple to fix it by having the handler determine the
signal handling stack size and allocate the memory every time the
handler is being installed, but that would add another allocation
and a syscall every time a test case is entered.

Instead, I split apart the idea of preparing fatal error handlers,
and engaging them, so that the memory can be allocated only once
and still be guarded by RAII.

Also turns out that Catch2's use of `MINSIGSTKSZ` was wrong, and
we should've been using `SIGSTKSZ` the whole time, which we use now.

Closes #2178
2021-04-04 18:09:26 +02:00
Pavel Kamenov
4cb3220a8a Add lcc to the list of unwanted compilers that mimic gcc 2021-04-04 14:05:39 +02:00
Scott Hutchinson
b025a007b9 Wrap all std::min and std::max calls in parentheses 2021-02-20 23:28:20 +01:00
Rob Boehne
68975e3ff3 [Issue 2154] Correct error when building with IBM's latest XLC (#2155)
* [Issue 2154] Correct error when building with IBM's latest XLC compiler with xlclang++ front-end.

On AIX, the XLC 16.1.0.1 compiler considers the call to `std::abs` ambigious, so it needs help with a static_cast to the type of the template argument.

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-01-21 22:50:49 +01:00
Martin Hořeňovský
bcb9ea8cb5 Merge pull request #2157 from tdegeus/patch
Making target detection on Mac more robust
2021-01-21 15:35:23 +01:00
Tom de Geus
d4c9494eb5 Making target detection on Mac more robust 2021-01-20 21:12:18 +01:00
Mathieu Mirmont
bed285af07 Make the build reproducible with g++-8 and clang-10
Make the build reproducible by removing references to the source
directory from the Catch2WithMain static library.
2021-01-13 18:09:24 +01:00
Martin Hořeňovský
de6fe184a9 v2.13.4 2020-12-29 15:03:40 +01:00
Martin Hořeňovský
fe3dddcc6d Fix updateVersionPlaceholder when the placeholder starts the line 2020-12-29 15:02:25 +01:00
Reinhold Gschweicher
18ab353e55 Add deprecation warning in ParseAndCatchTests
Parsing C++ with regex in CMake is error prone and regularly leads to silently
dropped (not run) test cases.

Going forward the function `catch_discover_tests` from `contrib/CMake.cmake`
should be used.

For more information see https://github.com/catchorg/Catch2/issues/2092#issuecomment-747342765
2020-12-21 13:15:36 +01:00
Reinhold Gschweicher
2375a7f5b7 Fix Catch.cmake helper by setting variable globally
Set `_CATCH_DISCOVER_TESTS_SCRIPT` helper variable globally. Otherwise in a
scoped call (like `add_subdirectory()`) the variable gets lost. This lost
variable results in a post build error with not much information to lead to the
root of the problem.

This enables the usage of the helper script with the following example structure

- CMakeLists.txt (project root with `add_subdirectory(external/catch2)`
- external/catch2
  - CMakeLists.txt (contents listed below)
  - contrib/Catch.cmake
  - contrib/CatchAddTests.cmake
  - catch2/catch.hpp
- tests
  - CMakeLists.txt (add tests with `catch_discover_tests(${PROJECT_NAME})`)

contents of project specific helper `external/catch2/CMakeLists.txt`
```cmake
cmake_minimum_required (VERSION 3.1...${CMAKE_VERSION})
project(Catch2 LANGUAGES CXX VERSION 2.13.3)
add_library(Catch2 INTERFACE)
target_include_directories(Catch2
  INTERFACE
    $<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}>
)
 # provide a namespaced alias for clients to 'link' against if catch is included as a sub-project
add_library(Catch2::Catch2 ALIAS Catch2)
include(contrib/Catch.cmake)
```
2020-12-17 18:50:04 +01:00
Martin Hořeňovský
fac517d571 Fix conan package build 2020-11-26 15:05:58 +01:00
Florian Berchtold
578f8b8006 Add bazel support 2020-11-18 21:37:12 +01:00
Deniz Evrenci
92f8b01dfa Add the static library Catch2WithMain
It should provide a shared impl for all targets that need to link
against Catch2's implementation. However, due to limitations of
C++ linking and Catch2's v2 implementation, this is only experimental
and might not work under some circumstances.
2020-11-18 21:37:10 +01:00
Martin Hořeňovský
dc7e705672 Small cleanups for the reworked test case order hashing 2020-11-17 21:08:33 +01:00
Sergio Losilla
3ba745552b Modified hash to make it more independent of the choice of test names. 2020-11-17 20:58:16 +01:00
Martin Hořeňovský
ff349a50bf v2.13.3 2020-10-31 18:21:23 +01:00
Martin Hořeňovský
aca2472d40 Stop uploading new releases to wandbox 2020-10-31 18:21:00 +01:00
Martin Hořeňovský
765ac08f08 Fix potential infinite loops in generators combined with section filter
The problem was that under specific circumstances, namely that none
of their children progressed, `GeneratorTracker` will not progress.
This was changed recently, to allow for code like this, where a
`SECTION` follows a `GENERATE` at the same level:

```cpp
SECTION("A") {}
auto a = GENERATE(1, 2);
SECTION("B") {}
```

However, this interacted badly with `SECTION` filters (`-c foo`),
as they could deactivate all `SECTION`s below a generator, and thus
stop it from progressing forever. This commit makes GeneratorTracker
check whether there are any filters active, and if they are, it checks
whether its section-children can ever run.

Fixes #2025
2020-10-31 18:04:15 +01:00
Martin Hořeňovský
8dd25b0410 Fix bunch of links to master to point to v2.x branch
Fixes #2063
Related to #2050
2020-10-21 15:30:35 +02:00
Reinhold Gschweicher
69297ceeb6 Consider CMP0110 add_test() policy
CMake 3.19 introduces new add_test() behavior guarded with the policy
CMP0110.

See: https://cmake.org/cmake/help/latest/policy/CMP0110.html

Update the helper script ParseAndAddCatchTests to consider the policy and
handle the test case name accordingly.
2020-10-20 23:25:45 +02:00
Reinhold Gschweicher
2d30df3500 Fix indentation in ParseAndAddCatchTests
Consistently use 4 spaces instead of tabs
2020-10-20 13:48:46 +02:00
Reinhold Gschweicher
77b2a7daea Fix CMake regex to add tests
Fix regex that requires two string arguments in the form of
TEST_CASE("a", "b") resulting in not finding TEST_CASE("a") entries.

See https://regex101.com/r/JygOND/1

Fixes: https://github.com/catchorg/Catch2/issues/2055
2020-10-20 13:48:14 +02:00
laoshanxi
e905edb10f Add AppMesh to Open Source projects using Catch 2020-10-20 13:48:14 +02:00
Martin Hořeňovský
87074da73e v2.13.2 2020-10-07 11:43:02 +02:00
Martin Hořeňovský
e6f5e05ebc Improve detection of std::uncaught_exceptions support
The problem was that Catch2 did not reliably include `<exception>`
before it checked for the feature test macro for
`std::uncaught_exceptions`. To avoid overhead of including
`<exception>` everywhere, the configuration check was split out
into a separate header.

Closes #2021
2020-10-06 11:07:53 +02:00
Rémy Salim
44900d5371 Add WORKING_DIRECTORY to CatchAddTests.cmake commands 2020-10-02 23:29:40 +02:00
Martin Hořeňovský
7c75ecaef4 Workaround AppleClang bug by renaming TestHasher constructor argument
As far as I understand the standard, if there is a function called
`rng` in the global namespace, and a function argument called `rng`,
then the argument should shadow the function. This then means that
uses of `rng` inside the function should refer to the argument.

This is not the case for AppleClang 12.0.0. Luckily the workaround
is simple enough; just rename the argument. Given that the function
is 3 lines and uncomplicated, the change of the name doesn't really
affect readability.

Still, WTF AppleClang?

Closes #2030
2020-10-02 23:22:49 +02:00
Matt Godbolt
fba0feeba1 Add missing syntax highlighting tag 2020-09-24 22:43:07 +02:00
Ansel Sermersheim
3e8800beb1 Support template test cases in ParseAndAddCatchTests
* Change regex to allow parentheses inside the test macro for a type list
* Append a wildcard to the CTestName if the test case is a template
* Also change the regular expression so parentheses are allowed in names
  (fixes #1848)
2020-09-24 22:42:32 +02:00
Martin Hořeňovský
314a05d809 Merge pull request #2023 from globberwops/feature/add-reporter-and-output-dir
Add REPORTER and OUTPUT_DIR args to catch_discover_tests
2020-09-23 22:47:21 +02:00
Martin Stump
eadf4e32b6 Add REPORTER and OUTPUT_* args 2020-09-23 19:12:06 +02:00
Florian Berchtold
f9620d11be Docu/Show how to use CMake FetchContent (#2028) 2020-09-20 18:09:33 +02:00
Will Pazner
00db4fb497 Disable __builtin_constant_p when compiling with nvcc 2020-09-18 16:07:18 +02:00
kotaiadam
2fc83e7e9c fixes bug in example - undeclared identifier
j was not declared in `SECTION("two")`
2020-09-12 11:40:30 +02:00
Travis Wilson
a2e5ce2418 Make experimental capture work on Windows with read-write temp file behavior 2020-09-10 20:14:18 +02:00
Martin Hořeňovský
fd9f5ac661 v2.13.1 2020-09-07 12:34:55 +02:00
Dawid Kurek
f0dc4d9be0 Remove superfluous values
The `0`s were needed for the expansion of parameter pack for the sake of
expander. Now when we have `index++` it is no more needed.
2020-08-27 11:15:58 +02:00
Sean Middleditch
284672cc84 Support sentinel-based ranges in default stringify (#2004) 2020-08-18 10:34:47 +02:00
mattkurz
2b34b5c7d0 Fix typo in generators docs 2020-08-03 23:33:32 +02:00
Gregory Bond
708487d201 Issue 1992: Better c++17 std::byte detection
This fixed buld errors for ubuntu-16 + clang and similar situations
where C++17 is supported in the compiler but not the default
C++ standard library.
2020-08-03 14:38:56 +02:00
James Touton
6859c683e0 Don't apply global settings when configuring as a subproject. 2020-08-01 19:42:30 +02:00
Karthik Nishanth
de3a208e16 Update ParseAndAddCatchTests.cmake 2020-08-01 19:40:29 +02:00
Reinhold Gschweicher
229cc4823c Fix CMake add test helper for CMake 3.18.0
With CMake 3.18.0 the `add_test(NAME)` handling changed. The escaped
double quotes confuse the new call. Work around this upstream change.

fixes: https://github.com/catchorg/Catch2/issues/1984
2020-07-17 23:56:33 +02:00
Martin Hořeňovský
7f21cc6c55 v2.13.0 2020-07-12 20:28:38 +02:00
Martin Hořeňovský
4c85248179 Fixup whitespace in TAPReporter 2020-07-12 20:27:25 +02:00
Martin Hořeňovský
b4e2237152 Remove pointless CompactReporter::getPreferences override 2020-07-12 20:06:14 +02:00
Martin Hořeňovský
40937b67c7 Merge pull request #1983 from s7726/patch-1
Update catch_reporter_tap.hpp
2020-07-12 20:00:03 +02:00
Martin Hořeňovský
0d5b131394 Improve documentation for --min-duration 2020-07-12 16:27:55 +02:00
Martin Hořeňovský
ad3b90553b Document GENERATE's new usage between SECTIONs 2020-07-12 16:24:32 +02:00
Gavin S
b1c45652e5 Update catch_reporter_tap.hpp
TAP format requires all results to be reported.
Removed extraneous preferences function (handled by parent)
Incorporated fix from 3d9e7db2e0
Simplified total printing
2020-07-12 01:17:37 -07:00
Martin Hořeňovský
b79a83e4aa Modify generator tracking to allow GENERATEs between SECTIONs
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-11 23:16:07 +02:00
Martin Hořeňovský
89fe35d515 Fix how testRandomOrder.py builds tag arguments 2020-07-11 21:32:10 +02:00
Martin Hořeňovský
115d6a1c40 Increase tolerances in --min-duration tests
The underpowered and oversubscribed CI servers are hell.
2020-07-07 11:36:56 +02:00
Martin Hořeňovský
91576352f9 Fixup single_include to be consistent with last tag
Closes #1975
2020-07-07 11:32:05 +02:00
Martin Hořeňovský
6f7c191513 --min-duration is overriden by -d no 2020-07-06 20:33:08 +02:00
Martin Hořeňovský
4eb9d37e05 Refactor tests for duration reporting threshold 2020-07-06 20:02:20 +02:00
John Bytheway
53d8af8e96 Test for --min-duration 2020-07-06 11:35:02 +02:00
John Bytheway
46fde0c597 Add --min-duration option
A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.

When there are many tests, this produces output spam which makes it hard
to find the test failure output.  Nevertheless, it is helpful to be
informed of tests which are unusually slow.

Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed.  This allows slow
tests to be visible without mentioning every test.
2020-07-06 11:35:02 +02:00
Martin Hořeňovský
de103db696 Merge pull request #1973 from echuber2/patch-1
Escaping "*" ("times") to fix Markdown presentation
2020-07-06 11:34:22 +02:00
Eric Huber
fedc3a7b7b Escaping literal "*" ("times") to fix markdown 2020-07-05 17:04:00 -05:00
Martin Hořeňovský
c299133a31 v2.12.4 2020-07-05 11:51:30 +02:00
Martin Hořeňovský
09b8017ea3 Update FUNDING file (-Patreon, +PayPal) 2020-07-04 20:06:27 +02:00
George Rhoten
bad3c93049 Fix for macOS on ARM 2020-07-01 19:28:50 +02:00
Martin Hořeňovský
0f05c034c2 v2.12.3 2020-06-29 20:50:39 +02:00
Ryan Pavlik
ee4538c0c6 Add OverallResultsCases element to XML reporter 2020-06-28 22:36:08 +02:00
Martin Hořeňovský
dc7a20fc74 Rewrite the contributing documentation 2020-06-27 17:44:08 +02:00
Martin Hořeňovský
018dc0b74f Replace a TODO comment in examples 2020-06-26 20:04:05 +02:00
Richard Ash
ad1940f336 Add an example of using GENERATE(table())
There are some examples on issue #850 of using this feature, but they
are not easily found from the documentation. Adding them here as an
example makes them more findable and ensures they keep working if the
API changes.
2020-06-23 22:55:03 +02:00
Richard Ash
5399921622 Add notes on compiling the examples.
This took me some time to figure out so document for others.
2020-06-20 21:41:10 +02:00
offa
e815acddf8 Clang-format configuration added.
Some notes on the configuration options chosen:

* We want `AllowShortEnumsOnASingleLine` set to `false`, but that
option is clang-format-11 and up, which is not out yet.
* `IndentPPDirectives` is currently inconsistent, but `AfterHash`
is the preferred style in new code.
* `NamespaceIndentation` is a mess, but `All` is closer to the effect
we want than `Inner`.
* `SpacesInParentheses` set to `true` is not ideal due to it also
introducing extra spaces in preprocessor expressions, but using it
is much closer to the current style than not.

All in all, using this setting globally would reformat pretty much
every line of code in the codebase, but it is as close as possible
to the bespoke style currently used. Still, it should only be used
on the diffs.

Closes #1182
2020-06-19 09:49:10 +02:00
Richard Ash
e7189f1e4f Make scripts/updateDocumentToC.py executable.
On systems where the file system has excute permissions, this script was
not marked as executable in a clean git checkout and so could be run
without first changing the permissions. Fixed by setting the relevant
git flag.
2020-06-18 21:28:08 +02:00
Martin Hořeňovský
1806b21545 Add explicit test for shortcircuiting behaviour of combined matchers 2020-06-14 21:48:08 +02:00
Martin Hořeňovský
288416f501 Devirtualize NameAndLocation query on trackers 2020-06-13 19:26:17 +02:00
Martin Hořeňovský
cbbebb65b6 Fix copy paste error in 7-arg TEMPLATE_TEST_CASE_SIG implementation
Closes #1954
2020-06-13 15:46:59 +02:00
Martin Hořeňovský
fb08596b1b Clarify documentation about nested generators
Closes #1947
2020-06-13 11:12:12 +02:00
Matthias Blankertz
0614a4acb3 Hide std::exception_ptr and friends if exceptions disabled
Some compilers, e.g. the Green Hills C++ compiler, react badly to the
appearance of std::exception_ptr, std::current_exception,
std::rethrow_exception and std::uncaught_exception(s). To allow usage of
Catch2 with these compilers when exceptions are disabled, hide the usage
of std::exception_ptr etc. when compiling with
CATCH_CONFIG_DISABLE_EXCEPTIONS.
2020-06-12 23:23:55 +02:00
Martin Hořeňovský
0f12995501 Fix compilation of examples 2020-06-01 21:27:58 +02:00
Martin Hořeňovský
0807fdb175 Replace stray tabs with spaces 2020-06-01 19:06:51 +02:00
Martin Hořeňovský
9500ded83b Improved generator tracking
* Successive executions of the same `GENERATE` macro (e.g. because
of a for loop) no longer lead to multiple nested generators.
* The same line can now contain multiple `GENERATE` macros without
issues.

Fixes #1913
2020-06-01 19:06:51 +02:00
bogdasar1985
6c6ebe374a fixing UB 2020-05-29 14:56:40 +02:00
Martin Hořeňovský
b1b5cb8122 v2.12.2 2020-05-25 15:13:18 +02:00
Martin Hořeňovský
77dc8cfc45 Really fix use of std::result_of when invoke_result is available
Closes #1934
2020-05-22 10:05:34 +02:00
Martin Hořeňovský
ddc9f4c61d Avoid using std::result_of when std::invoke_result is available
Closes #1934
2020-05-21 21:39:19 +02:00
Martin Hořeňovský
bed47374ce Remove obsolete comment in UnorderedEquals vector matcher 2020-05-18 14:29:50 +02:00
Valentin Tolmer
0e9bae1cdb Create a BUILD file for compatibility with bazel
With this change, it's much easier for bazel projects to depend on
Catch. They just need to add:
  - In the workspace:
  ```
http_archive(
    name = "com_github_catchorg_catch2",
    urls = ["https://github.com/catchorg/Catch2/archive/v2.12.1.tar.gz"],
    strip_prefix = "Catch2-2.12.1",
    sha256 = "e5635c082282ea518a8dd7ee89796c8026af8ea9068cd7402fb1615deacd91c3",
)
```
  Or the appropriate version/sha256.
  - For the tests, assuming that `test_main.cc` contains the
  `CATCH_CONFIG_MAIN`:
  ```
cc_library(
    name = "test_main",
    srcs = ["test_main.cc"],
    deps = ["@com_github_catchorg_catch2//:catch2"],
)
```
2020-05-17 13:18:22 +02:00
Martin Hořeňovský
f133277910 Add status attribute to JUnit's section reporting
This brings our output inline with GTest's. We do not handle skipped
tests properly, but that should be currently less important than
having the attribute exist with proper value for non-skipped tests.

Thanks @joda-01.

Closes #1899
2020-05-15 21:00:19 +02:00
Martin Hořeňovský
f764ee3d30 Document that user can only provide main in TU with CONFIG_RUNNER
Closes #1851
2020-05-15 15:57:27 +02:00
Natsu
c190061001 Fix compilation failure when using libstdc++10 (#1929)
The issue is caused by deleted `std::__detail::begin` declared in `bits/iterator_concepts.h`. This would be found by ADL, and because it is deleted, compilation would fail. This change makes it so that we SFINAE on `begin(std::declval<T>())` and `end(std::declval<T>())` being well-formed.
2020-05-15 11:30:12 +02:00
Billy Robert O'Neal III
b1dcdc5032 Fix invalid isspace call detected by PREfast
D:\vcpkg\toolsrc\include\catch2\catch.hpp(11285): warning C6330: 'char' passed as _Param_(1) when 'unsigned char' is required in call to 'isspace'.
D:\vcpkg\toolsrc\include\catch2\catch.hpp(11288): warning C6330: 'char' passed as _Param_(1) when 'unsigned char' is required in call to 'isspace'.

ISO/IEC 9899:2011:
"7.4 Character handling <ctype.h>"/1
[...] In all cases the argument is an int, the value of which shall be
representable as an unsigned char or shall equal the value of the macro
EOF. If the argument has any other value, the behavior is undefined.

This means if isspace was passed a character like ñ it could corrupt
memory without the static_cast to treat it as a positive value after
integral promotion (and C libraries commonly use the int index supplied
as a key into a table which result in out of bounds access if the
resulting int is negative).
2020-05-12 14:07:22 +02:00
Martin Hořeňovský
f0e596e252 Silence clang-tidy's hicpp-vararg (alias of coreguidelines vararg)
Ideally, clang-tidy would be smart that if one alias of a warning
is suppressed, then the other one is suppressed as well, but as of
right now, it isn't. This means that for now we have to suppress
both aliases of this warning. Opened upstream issue to fix this:
https://bugs.llvm.org/show_bug.cgi?id=45859

Obviously, ideally clang-tidy would also not warn that we are calling
a vararg function when it is an unevaluated magic builtin, but that
also is not happening right now and I opened an issue for it:
https://bugs.llvm.org/show_bug.cgi?id=45860

Closes #1921
2020-05-09 18:23:12 +02:00
Martin Hořeňovský
594cab31ed Upload conan releases to catch2 remote (instead of Catch2)
At some point we moved over to catch2:catchorg (notice lowercase `c`)
instead of Catch2:catchorg, but we kept uploading the released
packages to the upper-cased repository... Time to fix this, and then
merge them again.
2020-05-06 20:49:26 +02:00
Gareth Lloyd
89f5f84351 Provide path of the cmake scripts to conan 2020-04-23 18:11:41 +02:00
Martin Hořeňovský
2e61d38c7c v2.12.1
--- Fixes ---
* Vector matchers now support initializer list literals better

--- Improvements ---
* Added support for `^` (bitwise xor) to `CHECK` and `REQUIRE`
2020-04-21 19:30:38 +02:00
Martin Hořeňovský
5c9f09e94a Add support for bitwise xor to the decomposer 2020-04-21 19:27:12 +02:00
Martin Hořeňovský
f4fc2dab2c Fixup template type argument inference for vector matchers 2020-04-21 19:09:45 +02:00
Martin Hořeňovský
cfb6956698 v2.12.0
--- Improvements ---
* Running tests in random order (`--order rand`) has been reworked significantly (#1908)
  * Given same seed, all platforms now produce the same order
  * Given same seed, the relative order of tests does not change if you select only a subset of them
* Vector matchers support custom allocators (#1909)
* `|` and `&` (bitwise or and bitwise and) are now supported in `CHECK` and `REQUIRE`
  * The resulting type must be convertible to `bool`

--- Fixes ---
* Fixed computation of benchmarking column widths in ConsoleReporter (#1885, #1886)
* Suppressed clang-tidy's `cppcoreguidelines-pro-type-vararg` in assertions (#1901)
  * It was a false positive trigered by the new warning support workaround
* Fixed bug in test specification parser handling of OR'd patterns using escaping (#1905)

--- Miscellaneous ---
* Worked around IBM XL's codegen bug (#1907)
  * It would emit code for _destructors_ of temporaries in an unevaluated context
* Improved detection of stdlib's support for `std::uncaught_exceptions` (#1911)
2020-04-21 16:33:15 +02:00
khyperia
e59fc2c3b3 Fix build with CATCH_CONFIG_DISABLE_EXCEPTIONS enabled 2020-04-21 15:31:15 +02:00
Martin Hořeňovský
4e4171420d Support bitand and bitor in REQUIRE/CHECK
This means that bit-flag-like types with conversion to bool can be
asserted on, like so `REQUIRE(var & Flags::AddNewline)`.
2020-04-21 11:00:08 +02:00
Martin Hořeňovský
37cbf4a4fe Add more tests for test spec parser
Originally the tests were from #1912, but as it turned out, the issue
was somewhere else. Still, the inputs provided were interesting, so
they are now part of our test suite.
2020-04-17 21:19:18 +02:00
schallerr
38f897c887 Support custom allocators in vector Matchers (#1909) 2020-04-16 15:36:54 +02:00
Martin Hořeňovský
fa4a93e051 Update documentation for --order 2020-04-15 16:20:05 +02:00
John Else
6fbe5efc71 Use macro to determine whether std::uncaught_exceptions is available
Catch assumes std::uncaught_exceptions is available whenever C++17 is
available, but for macOS versions older than 10.12 this is not the case.

Instead of checking the C++ version, use a macro to check whether the
feature is available.
2020-04-14 23:03:58 +02:00
Martin Hořeňovský
bad0fb51f8 Refactor implementation of hashed random order test sorting 2020-04-14 16:39:48 +02:00
Martin Hořeňovský
a2fc7cf8c0 Randomize test for subset invariant random ordering of tests
Also removed the iterative checking that seeds 1-100 do not create
the same output, because it used too much runtime.
2020-04-14 16:38:10 +02:00
John Bytheway
da9e3eec65 Add test for consistent random ordering 2020-04-14 12:47:36 +02:00
John Bytheway
f696ab836b Change random test shuffling technique
Previously a random test ordering was obtained by applying std::shuffle
to the tests in declaration order.  This has two problems:

- It depends on the declaration order, so the order in which the tests
  will be run will be platform-specific.
- When trying to debug accidental inter-test dependencies, it is helpful
  to be able to find a minimal subset of tests which exhibits the issue.
  However, any change to the set of tests being run will completely
  change the test ordering, making it difficult or impossible to reduce
  the set of tests being run in any reasonably efficient manner.

Therefore, change the randomization approach to resolve both these
issues.

Generate a random value based on the user-provided RNG seed.  Convert
every test case to an integer by hashing a combination of that value
with the test name.  Sort the test cases by this integer.

The test names and RNG are platform-independent, so this should be
consistent across platforms.  Also, removing one test does not change
the integer value associated with the remaining tests, so they remain in
the same order.

To hash, use the FNV-1a hash, except with the basis being our randomly
selected value rather than the fixed basis set in the algorithm.  Cannot
use std::hash, because it is important that the result be
platform-independent.
2020-04-14 12:47:36 +02:00
Martin Hořeňovský
5d32ce26f4 Fix bug in test spec parser handling of escaping in ORed patterns
It did not clear out all of its internal state when switching from
one pattern to another, so when it should've escaped `,`, it took
its position from its position in the original user-provided string,
rather than its position in the current pattern.

Fixes #1905
2020-04-12 18:48:52 +02:00
Andrew Gaspar
035a062596 Remove usage of __builtin_constant_p under IBM XL 2020-04-10 20:55:23 +02:00
Moritz Haase
d399a308d0 Suppress clang-tidy warning about vararg usage in assertion macros
CATCH_INTERNAL_IGNORE_BUT_WARN() introduced with b7b346c triggers
clang-tidy warning 'cppcoreguidelines-pro-type-vararg' for every usage
of assertion macros like CHECK() and REQUIRE(). Silence it via NOLINT
in the '#if defined(__clang__)' block only, as clang-tidy honors those.
2020-04-02 20:45:57 +02:00
Phoebe
b8ce814ee6 Add vcpkg installation instructions (#1898)
* Add vcpkg installation instructions

* Add index
2020-03-31 19:12:10 +02:00
Mark Gillard
6260962108 Added toml++ to opensource-users.md 2020-03-29 14:21:57 +02:00
Mark Gillard
b4c8967ac5 Fix alphabetical ordering of opensource-users.md 2020-03-29 14:21:57 +02:00
pi1024e
7900fb3abb C-header updates 2020-03-28 18:00:42 +01:00
Invincible
01bdfe3312 Change PARSE_CATCH_TESTS_ADD_FIXTURE_IN_TEST_NAME conditional.
When no TEST_CASE_METHOD function, there is no fixture to get.
2020-03-27 09:55:06 +01:00
Joel Uckelman
e5c9a58d66 Fixed typo in "benchmark name" column width calculation. Closes #1885. 2020-03-26 10:31:35 +01:00
Martin Hořeňovský
255aa5f2af v2.11.3 2020-03-19 13:50:19 +01:00
Martin Hořeňovský
3a33315ff8 Fix compilation error when lambdas are used in assertions
This is a partial revert of b7b346c3e5.
2020-03-19 13:32:45 +01:00
Martin Hořeňovský
5f94c8dafb v2.11.2 2020-03-19 12:37:51 +01:00
Martin Hořeňovský
695a356f93 Move all scripts to Python3 2020-03-19 12:36:30 +01:00
Martin Hořeňovský
020b655635 Update wandbox script to use https endpoint and Python3 2020-03-19 12:32:34 +01:00
Martin Hořeňovský
5561d0342d Different approach to proper colouring of filters 2020-03-19 11:08:01 +01:00
Till Hofmann
3a15315a37 Switch back to uncolored output after printing filters
After printing the list of filters, switch back from yellow to black
before printing a newline to avoid the remaining output to be colored in
yellow.
2020-03-19 11:08:01 +01:00
Martin Hořeňovský
b7b346c3e5 Make warnings in assertions fire for GCC/Clang again
The old code caused warnings to fire under MSVC, and Clang <3.8.
I could not find a GCC version where it worked, but I assume that it
did at some point.

This new code causes all of MSVC, GCC, Clang, in current versions,
to emit signed/unsigned comparison warning in test like this:

```cpp
TEST_CASE() {
    int32_t i = -1;
    uint32_t j = 1;
    REQUIRE(i != j);
}
```

Where previously only MSVC would emit the warning.

Fixes #1880
2020-03-19 09:45:41 +01:00
Julius Hülsmann
9e09d79946 Update tutorial.md
Fix: typo; remove trailing ","
2020-03-18 15:36:19 +01:00
Julius Hülsmann
7048c2c269 Update slow-compiles.md
Add missing "."
2020-03-18 15:35:56 +01:00
magnus.ornebring
6f77d59b16 Add king as user 2020-03-11 09:32:16 +01:00
khyperia
022b61ff92 Don't cast nanoseconds to integers when writing output 2020-03-08 12:10:44 +01:00
khyperia
87b5bf77bc include <iterator> in catch_stats.hpp
needed for std::back_inserter on some platforms
2020-02-24 19:36:34 +01:00
Martin Hořeňovský
81d52c4a5f Specialize CATCH_TRAP() for iOS + thumb instruction set combo
Fixes #1862
2020-02-15 21:45:09 +01:00
Tristan Stenner
200b8b6fc0 Add command line option 'never' to --wait-for-keypress (#1866)
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-02-15 20:42:57 +01:00
offa
e70fd2a4b9 Variables made const-ref. 2020-02-08 14:27:40 +01:00
offa
36170d60ca stdio.h replaced with cstdio. 2020-02-08 14:27:40 +01:00
Clare Macrae
917e92c162 Add link to list of commercial user projects 2020-02-07 15:33:55 +01:00
Clare Macrae
f54950744f Fix typo in link in docs 2020-02-07 15:33:35 +01:00
Martin Hořeňovský
b3b07215d1 Merge pull request #1854 from neheb/patch-1
catch_compiler_capabilities.h: use proper math define
2020-02-03 11:26:31 +01:00
Martin Hořeňovský
2652bb86e3 Cleanup nextafter workaround 2020-02-03 10:05:23 +01:00
Martin Hořeňovský
1715b6b923 Check for Windows instead of WIN32 for wmain entry point
Closes #1849
2020-02-03 09:33:42 +01:00
Martin Hořeňovský
f20a9dbc6e Fix significant bug with storing composed matchers
Given that in the 2 or so years that matchers are thing nobody complained,
it seems that people do not actually write this sort of code, and the
possibility will be removed in v3. However, to avoid correctness bugs,
we will have to support this weird code in v2.
2020-02-01 21:06:07 +01:00
Martin Hořeňovský
273c3f85eb Add missing ToC entry in release notes 2020-02-01 20:24:53 +01:00
Martin Hořeňovský
6a3d0dc176 Add a test for custom debug break macros
See #1846
2020-02-01 20:18:05 +01:00
khyperia
ccb1f70629 Make CATCH_BREAK_INTO_DEBUGGER be user-configurable 2020-02-01 17:01:50 +01:00
khyperia
4a5bc0f39a Make CATCH_BREAK_INTO_DEBUGGER be user-configurable 2020-02-01 12:20:40 +01:00
Rosen Penev
f96e89e016 catch_compiler_capabilities.h: use proper math define
C++11 math requires _GLIBCXX_USE_C99_MATH_TR1 to be true with gcc/clang.

Also fixes an issue with uClibc-ng where __UCLIBC__ is defined in features.h but
that is not included here and is thus no-op.
2020-01-31 22:20:59 -08:00
dmsteck
481f54b357 Make hidden tags behave identically (#1847)
Add both `[.]` and `[!hide]` tags when registering a hidden test case, as per documentation.

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-01-31 21:44:06 +01:00
offa
2d172dc688 Some refactorings:
- Overrides added
 - usages of push_back() replaced with emplace_back()
 - Loop variable made const-refernce
 - NULL replaced with nullptr
 - Names used in the declaration and definition unified
 - size() replaced with empty
 - Identical cases merged
2020-01-25 09:01:04 +01:00
Daniel Griscom
587a20b312 Removed unneeded 'using uchar = unsigned char' 2020-01-24 14:26:22 +01:00
Martin Hořeňovský
bff44f8b01 Merge pull request #1831 from matt77hias/master
Added <random> include for std::uniform_int_distribution
2020-01-14 14:05:50 +01:00
Matthias Moulin
4ea74ff966 Added <random> include for std::uniform_int_distribution 2020-01-14 12:48:09 +01:00
Martin Hořeňovský
d10b9bd02e v2.11.1 2019-12-28 21:24:04 +01:00
Martin Hořeňovský
55794e9b27 Avoid detecting Clang as having MSVC's traditional preprocessor
Fixes #1806
2019-12-28 18:57:24 +01:00
ptc-tgamper
fa6211bfc2 catch_debugger.h - implement break into debugger assembler instructions for iOS 2019-12-23 21:26:13 +01:00
ptc-tgamper
4e90f910dc catch_console_colour.cpp - adjust useColourOnPlatform for iOS 2019-12-23 21:26:13 +01:00
ptc-tgamper
0c59cc83cf catch_debugger.cpp - debugger detection is identical on Mac OS X and iOS 2019-12-23 21:26:13 +01:00
Martin Hořeňovský
e4004e0adb Provide const overload of ObjectStorage::stored_object()
Fixes #1820
2019-12-23 21:22:32 +01:00
Joe Burzinski
6c9a255dc2 Fix forwarding in SingleValueGenerator and generator creation
Fixes #1809
2019-12-15 20:50:43 +01:00
Joe Burzinski
9a8963133f Update single header generation script to warn about unused headers 2019-11-21 16:22:04 +01:00
Joe Burzinski
cfba9dce97 Fix wrong namespacing of benchmarking constructor helpers 2019-11-21 16:22:04 +01:00
Martin Hořeňovský
a537ccae22 Suppress using-namespace lint in GENERATE* macros
Closes #1799
2019-11-16 17:39:28 +01:00
Martin Hořeňovský
e1c9d5569d v2.11.0 2019-11-15 15:06:17 +01:00
Martin Hořeňovský
d512decaac Mention the sonarqube reporter in release process 2019-11-15 14:59:35 +01:00
Martin Hořeňovský
f23f96883a Ensure the full benchmarking support is present in the single header
Fixes #1800
2019-11-15 11:59:43 +01:00
Martin Hořeňovský
d7b8c3ace3 Minor cleanup in the benchmarking tests 2019-11-15 11:59:35 +01:00
Martin Hořeňovský
32733e08c0 Use newer version of Python on AppVeyor 2019-11-13 20:58:15 +01:00
Martin Hořeňovský
930f49a641 Split [.foo] into [.][foo] when parsing test specs
b77cec05c0 fixed this problem for tagging tests, so that a test
case tagged with `[.foo]` would be parsed as tagged with `[.][foo]`.
This does the same for the test spec parsing.

Fixes #1798
2019-11-05 23:28:47 +01:00
Martin Hořeňovský
c409dccee5 Cleanup tests for C++20 2019-11-04 10:42:34 +01:00
Martin Hořeňovský
95bfb33167 Forbid copying ReusableStringStream
Copying a `ReusableStringStream` would lead to "double free" of
the stream, and thus it could be used in multiple places at the
same time, breaking the output.
2019-11-04 00:05:35 +01:00
Martin Hořeňovský
59d2d08c0f Merge pull request #1794 from cericks0n/patch-1
Fix error when period of steady_clock is not nano
2019-11-03 18:22:56 +01:00
cericks0n
fa6d52e2a3 Fix error when period of steady_clock is not nano
On systems where std::chrono::steady_clock::period is not std::nano, benchmark tests fail to compile due to trying to convert analysis.samples from a vector of duration<double, clock::period> to a vector of std::chrono::duration<double, std::nano>.
2019-11-01 15:52:38 -05:00
Martin Hořeňovský
5ac348cd6e Fix compilation error in examples 2019-10-31 16:08:37 +01:00
Martin Hořeňovský
776a4686c7 Warning fixes in examples and tests 2019-10-31 14:29:59 +01:00
699 changed files with 58305 additions and 147287 deletions

View File

@@ -1,10 +0,0 @@
build --enable_platform_specific_config
build:gcc9 --cxxopt=-std=c++2a
build:gcc11 --cxxopt=-std=c++2a
build:clang13 --cxxopt=-std=c++17
build:vs2019 --cxxopt=/std:c++17
build:vs2022 --cxxopt=/std:c++17
build:windows --config=vs2022
build:linux --config=gcc11

View File

@@ -1,45 +1,25 @@
---
Language: Cpp
Standard: c++14
# Note that we cannot use IncludeIsMainRegex functionality, because it
# does not support includes in angle brackets (<>)
SortIncludes: true
IncludeBlocks: Regroup
IncludeCategories:
- Regex: <catch2/.*\.hpp>
Priority: 1
- Regex: <.*/.*\.hpp>
Priority: 2
- Regex: <.*>
Priority: 3
AllowShortBlocksOnASingleLine: Always
AllowShortEnumsOnASingleLine: false
AllowShortFunctionsOnASingleLine: All
AllowShortIfStatementsOnASingleLine: WithoutElse
AllowShortLambdasOnASingleLine: Inline
AccessModifierOffset: "-4"
AccessModifierOffset: '-4'
AlignEscapedNewlines: Left
AllowAllConstructorInitializersOnNextLine: "true"
BinPackArguments: "false"
BinPackParameters: "false"
AllowAllConstructorInitializersOnNextLine: 'true'
BinPackArguments: 'false'
BinPackParameters: 'false'
BreakConstructorInitializers: AfterColon
ConstructorInitializerAllOnOneLineOrOnePerLine: "true"
DerivePointerAlignment: "false"
FixNamespaceComments: "true"
IndentCaseLabels: "false"
ConstructorInitializerAllOnOneLineOrOnePerLine: 'true'
DerivePointerAlignment: 'false'
FixNamespaceComments: 'true'
IncludeBlocks: Regroup
IndentCaseLabels: 'false'
IndentPPDirectives: AfterHash
IndentWidth: "4"
IndentWidth: '4'
Language: Cpp
NamespaceIndentation: All
PointerAlignment: Left
SpaceBeforeCtorInitializerColon: "false"
SpaceInEmptyParentheses: "false"
SpacesInParentheses: "true"
TabWidth: "4"
SpaceBeforeCtorInitializerColon: 'false'
SpaceInEmptyParentheses: 'false'
SpacesInParentheses: 'true'
Standard: Cpp11
TabWidth: '4'
UseTab: Never
AlwaysBreakTemplateDeclarations: Yes
SpaceAfterTemplateKeyword: true
SortUsingDeclarations: true
ReflowComments: true
...

View File

@@ -26,8 +26,8 @@ class BuilderSettings(object):
""" Set Catch2 repository to be used on upload.
The upload server address could be customized by env var
CONAN_UPLOAD. If not defined, the method will check the branch name.
Only devel or CONAN_STABLE_BRANCH_PATTERN will be accepted.
The devel branch will be pushed to testing channel, because it does
Only master or CONAN_STABLE_BRANCH_PATTERN will be accepted.
The master branch will be pushed to testing channel, because it does
not match the stable pattern. Otherwise it will upload to stable
channel.
"""
@@ -49,7 +49,7 @@ class BuilderSettings(object):
def reference(self):
""" Read project version from branch create Conan reference
"""
return os.getenv("CONAN_REFERENCE", "catch2/{}".format(self._version))
return os.getenv("CONAN_REFERENCE", "Catch2/{}".format(self._version))
@property
def channel(self):
@@ -85,7 +85,7 @@ if __name__ == "__main__":
reference=settings.reference,
channel=settings.channel,
upload=settings.upload,
upload_only_when_stable=False,
upload_only_when_stable=settings.upload_only_when_stable,
stable_branch_pattern=settings.stable_branch_pattern,
login_username=settings.login_username,
username=settings.username,

View File

@@ -1,12 +1,11 @@
cmake_minimum_required(VERSION 3.2.0)
project(test_package CXX)
include("${CMAKE_BINARY_DIR}/conanbuildinfo.cmake")
conan_basic_setup()
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup(TARGETS)
find_package(Catch2 REQUIRED CONFIG)
add_executable(${PROJECT_NAME} test_package.cpp)
target_link_libraries(${PROJECT_NAME} Catch2::Catch2WithMain)
set_target_properties(${PROJECT_NAME} PROPERTIES CXX_STANDARD 14)
target_link_libraries(${PROJECT_NAME} CONAN_PKG::Catch2)
set_target_properties(${PROJECT_NAME} PROPERTIES CXX_STANDARD 11)

View File

@@ -6,7 +6,7 @@ import os
class TestPackageConan(ConanFile):
settings = "os", "compiler", "build_type", "arch"
generators = "cmake_find_package_multi", "cmake"
generators = "cmake"
def build(self):
cmake = CMake(self)
@@ -14,7 +14,6 @@ class TestPackageConan(ConanFile):
cmake.build()
def test(self):
assert os.path.isfile(os.path.join(
self.deps_cpp_info["catch2"].rootpath, "licenses", "LICENSE.txt"))
assert os.path.isfile(os.path.join(self.deps_cpp_info["Catch2"].rootpath, "licenses", "LICENSE.txt"))
bin_path = os.path.join("bin", "test_package")
self.run("%s -s" % bin_path, run_environment=True)

View File

@@ -1,4 +1,6 @@
#include <catch2/catch_test_macros.hpp>
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
int Factorial( int number ) {
return number <= 1 ? 1 : Factorial( number - 1 ) * number;

1
.github/FUNDING.yml vendored
View File

@@ -1,2 +1 @@
github: "horenmar"
custom: "https://www.paypal.me/horenmar"

View File

@@ -1,24 +0,0 @@
name: Bazel build
on: [push, pull_request]
jobs:
build_and_test_ubuntu:
name: Linux Ubuntu 22.04 Bazel build <GCC 11.2.0>
runs-on: ubuntu-22.04
strategy:
matrix:
compilation_mode: [fastbuild, dbg, opt]
steps:
- uses: actions/checkout@v3
- name: Mount bazel cache
uses: actions/cache@v3
with:
path: "/home/runner/.cache/bazel"
key: bazel-ubuntu22-gcc11
- name: Build Catch2
run: |
bazelisk build --compilation_mode=${{matrix.compilation_mode}} //...

View File

@@ -1,43 +0,0 @@
name: Linux builds (meson)
on: [push, pull_request]
jobs:
build:
name: meson ${{matrix.cxx}}, C++${{matrix.std}}, ${{matrix.build_type}}
runs-on: ubuntu-22.04
strategy:
matrix:
cxx:
- g++-11
- clang++-11
build_type: [debug, release]
std: [14, 17]
include:
- cxx: clang++-11
other_pkgs: clang-11
steps:
- uses: actions/checkout@v2
- name: Prepare environment
run: sudo apt-get install -y meson ninja-build ${{matrix.other_pkgs}}
- name: Configure build
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: -std=c++${{matrix.std}} ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
meson -Dbuildtype=${{matrix.build_type}} ${{runner.workspace}}/meson-build
- name: Build tests + lib
working-directory: ${{runner.workspace}}/meson-build
run: ninja
- name: Run tests
working-directory: ${{runner.workspace}}/meson-build
# Hardcode 2 cores we know are there
run: |
meson test --verbose

View File

@@ -1,104 +0,0 @@
# The builds in this file are more complex (e.g. they need custom CMake
# configuration) and thus are unsuitable to the simple build matrix
# approach used in simple-builds
name: Linux builds (complex)
on: [push, pull_request]
jobs:
build:
name: ${{matrix.build_description}}, ${{matrix.cxx}}, C++${{matrix.std}} ${{matrix.build_type}}
runs-on: ubuntu-20.04
strategy:
matrix:
# We add builds one by one in this case, because there are no
# dimensions that are shared across the builds
include:
# Single surrogate header build
- cxx: clang++-10
build_description: Surrogates build
build_type: Debug
std: 14
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_SURROGATES=ON
# Extras and examples with gcc-7
- cxx: g++-7
build_description: Extras + Examples
build_type: Debug
std: 14
other_pkgs: g++-7
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
- cxx: g++-7
build_description: Extras + Examples
build_type: Release
std: 14
other_pkgs: g++-7
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
# Extras and examples with Clang-10
- cxx: clang++-10
build_description: Extras + Examples
build_type: Debug
std: 17
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
- cxx: clang++-10
build_description: Extras + Examples
build_type: Release
std: 17
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
# Configure tests with Clang-10
- cxx: clang++-10
build_description: CMake configuration tests
build_type: Debug
std: 14
other_pkgs: clang-10
cmake_configurations: -DCATCH_ENABLE_CONFIGURE_TESTS=ON
# Valgrind test Clang-10
- cxx: clang++-10
build_description: Valgrind tests
build_type: Debug
std: 14
other_pkgs: clang-10 valgrind
cmake_configurations: -DMEMORYCHECK_COMMAND=`which valgrind` -DMEMORYCHECK_COMMAND_OPTIONS="-q --track-origins=yes --leak-check=full --num-callers=50 --show-leak-kinds=definite --error-exitcode=1"
other_ctest_args: -T memcheck -LE uses-python
steps:
- uses: actions/checkout@v2
- name: Prepare environment
run: sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
- name: Configure build
working-directory: ${{runner.workspace}}
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
cmake -Bbuild -H$GITHUB_WORKSPACE \
-DCMAKE_BUILD_TYPE=${{matrix.build_type}} \
-DCMAKE_CXX_STANDARD=${{matrix.std}} \
-DCMAKE_CXX_STANDARD_REQUIRED=ON \
-DCMAKE_CXX_EXTENSIONS=OFF \
-DCATCH_DEVELOPMENT_BUILD=ON \
${{matrix.cmake_configurations}} \
-G Ninja
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: ninja
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2 ${{matrix.other_ctest_args}}

View File

@@ -1,122 +0,0 @@
name: Linux builds (basic)
on: [push, pull_request]
jobs:
build:
name: ${{matrix.cxx}}, C++${{matrix.std}}, ${{matrix.build_type}}
runs-on: ubuntu-20.04
strategy:
matrix:
cxx:
- g++-5
- g++-6
- g++-7
- g++-8
- g++-9
- g++-10
- clang++-6.0
- clang++-7
- clang++-8
- clang++-9
- clang++-10
build_type: [Debug, Release]
std: [14]
include:
- cxx: g++-5
other_pkgs: g++-5
- cxx: g++-6
other_pkgs: g++-6
- cxx: g++-7
other_pkgs: g++-7
- cxx: g++-8
other_pkgs: g++-8
- cxx: g++-9
other_pkgs: g++-9
- cxx: g++-10
other_pkgs: g++-10
- cxx: clang++-6.0
other_pkgs: clang-6.0
- cxx: clang++-7
other_pkgs: clang-7
- cxx: clang++-8
other_pkgs: clang-8
- cxx: clang++-9
other_pkgs: clang-9
- cxx: clang++-10
other_pkgs: clang-10
# Clang 6 + C++17
# does not work with the default libstdc++ version thanks
# to a disagreement on variant implementation.
# - cxx: clang++-6.0
# build_type: Debug
# std: 17
# other_pkgs: clang-6.0
# - cxx: clang++-6.0
# build_type: Release
# std: 17
# other_pkgs: clang-6.0
# Clang 10 + C++17
- cxx: clang++-10
build_type: Debug
std: 17
other_pkgs: clang-10
- cxx: clang++-10
build_type: Release
std: 17
other_pkgs: clang-10
- cxx: clang++-10
build_type: Debug
std: 20
other_pkgs: clang-10
- cxx: clang++-10
build_type: Release
std: 20
other_pkgs: clang-10
- cxx: g++-10
build_type: Debug
std: 20
other_pkgs: g++-10
- cxx: g++-10
build_type: Release
std: 20
other_pkgs: g++-10
steps:
- uses: actions/checkout@v2
- name: Add repositories for older GCC
run: |
sudo apt-add-repository 'deb http://azure.archive.ubuntu.com/ubuntu/ bionic main'
sudo apt-add-repository 'deb http://azure.archive.ubuntu.com/ubuntu/ bionic universe'
if: ${{ matrix.cxx == 'g++-5' || matrix.cxx == 'g++-6' }}
- name: Prepare environment
run: sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
- name: Configure build
working-directory: ${{runner.workspace}}
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
cmake -Bbuild -H$GITHUB_WORKSPACE \
-DCMAKE_BUILD_TYPE=${{matrix.build_type}} \
-DCMAKE_CXX_STANDARD=${{matrix.std}} \
-DCMAKE_CXX_STANDARD_REQUIRED=ON \
-DCMAKE_CXX_EXTENSIONS=OFF \
-DCATCH_DEVELOPMENT_BUILD=ON \
-G Ninja
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: ninja
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2

View File

@@ -1,52 +0,0 @@
name: Mac builds
on: [push, pull_request]
jobs:
build:
# macos-12 updated to a toolchain that crashes when linking the
# test binary. This seems to be a known bug in that version,
# and will eventually get fixed in an update. After that, we can go
# back to newer macos images.
runs-on: macos-11
strategy:
matrix:
cxx:
- g++-11
- clang++
build_type: [Debug, Release]
std: [14, 17]
include:
- build_type: Debug
examples: ON
extra_tests: ON
steps:
- uses: actions/checkout@v2
- name: Configure build
working-directory: ${{runner.workspace}}
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
cmake -Bbuild -H$GITHUB_WORKSPACE \
-DCMAKE_BUILD_TYPE=${{matrix.build_type}} \
-DCMAKE_CXX_STANDARD=${{matrix.std}} \
-DCMAKE_CXX_STANDARD_REQUIRED=ON \
-DCATCH_DEVELOPMENT_BUILD=ON \
-DCATCH_BUILD_EXAMPLES=${{matrix.examples}} \
-DCATCH_BUILD_EXTRA_TESTS=${{matrix.examples}}
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: make -j 2
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2

View File

@@ -1,36 +0,0 @@
name: Check header guards
on: [push, pull_request]
jobs:
build:
# Set the type of machine to run on
runs-on: ubuntu-20.04
steps:
- name: Checkout source code
uses: actions/checkout@v2
- name: Setup Dependencies
uses: actions/setup-python@v2
with:
python-version: '3.7'
- name: Install checkguard
run: pip install guardonce
- name: Check that include guards are properly named
run: |
wrong_files=$(checkguard -r src/catch2/ -p "name | append _INCLUDED | upper")
if [[ $wrong_files ]]; then
echo "Files with wrong header guard:"
echo $wrong_files
exit 1
fi
- name: Check that there are no duplicated filenames
run: |
./tools/scripts/checkDuplicateFilenames.py
- name: Check that all source files have the correct license header
run: |
./tools/scripts/checkLicense.py

View File

@@ -1,37 +0,0 @@
name: Windows builds (basic)
on: [push, pull_request]
jobs:
build:
name: ${{matrix.os}}, ${{matrix.std}}, ${{matrix.build_type}}, ${{matrix.platform}}
runs-on: ${{matrix.os}}
strategy:
matrix:
os: [windows-2019, windows-2022]
platform: [Win32, x64]
build_type: [Debug, Release]
std: [14, 17]
steps:
- uses: actions/checkout@v2
- name: Configure build
working-directory: ${{runner.workspace}}
run: |
cmake -S $Env:GITHUB_WORKSPACE `
-B ${{runner.workspace}}/build `
-DCMAKE_CXX_STANDARD=${{matrix.std}} `
-A ${{matrix.platform}} `
--preset all-tests
- name: Build tests
working-directory: ${{runner.workspace}}
run: cmake --build build --config ${{matrix.build_type}} --parallel %NUMBER_OF_PROCESSORS%
shell: cmd
- name: Run tests
working-directory: ${{runner.workspace}}/build
env:
CTEST_OUTPUT_ON_FAILURE: 1
run: ctest -C ${{matrix.build_type}} -j %NUMBER_OF_PROCESSORS%
shell: cmd

16
.gitignore vendored
View File

@@ -1,5 +1,4 @@
*.build
!meson.build
*.pbxuser
*.mode1v3
*.ncb
@@ -12,6 +11,11 @@ Release
xcuserdata
CatchSelfTest.xcscheme
Breakpoints.xcbkptlist
projects/VS2010/TestCatch/_UpgradeReport_Files/
projects/VS2010/TestCatch/TestCatch/TestCatch.vcxproj.filters
projects/VisualStudio/TestCatch/UpgradeLog.XML
projects/CMake/.idea
projects/CMake/cmake-build-debug
UpgradeLog.XML
Resources/DWARF
projects/Generated
@@ -21,17 +25,7 @@ DerivedData
Build
.idea
.vs
.vscode
cmake-build-*
benchmark-dir
.conan/test_package/build
bazel-*
build-fuzzers
debug-build
.vscode
msvc-sln*
# Currently we use Doxygen for dep graphs and the full docs are only slowly
# being filled in, so we definitely do not want git to deal with the docs.
docs/doxygen
*.cache
compile_commands.json

335
.travis.yml Normal file
View File

@@ -0,0 +1,335 @@
language: cpp
branches:
except:
- /dev-appveyor.*/
common_sources: &all_sources
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty
- llvm-toolchain-trusty-3.9
- llvm-toolchain-trusty-4.0
- llvm-toolchain-xenial-5.0
- llvm-toolchain-xenial-6.0
matrix:
include:
# 1/ Linux Clang Builds
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.5']
env: COMPILER='clang++-3.5'
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.6']
env: COMPILER='clang++-3.6'
# Clang 3.7 is intentionally skipped as we cannot get it easily on
# TravisCI container
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['lcov', 'clang-3.8']
env: COMPILER='clang++-3.8'
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.9']
env: COMPILER='clang++-3.9'
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-4.0']
env: COMPILER='clang++-4.0'
- os: linux
dist: xenial
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-5.0']
env: COMPILER='clang++-5.0'
- os: linux
dist: xenial
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0']
env: COMPILER='clang++-6.0'
# 2/ Linux GCC Builds
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-4.8']
env: COMPILER='g++-4.8'
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-4.9']
env: COMPILER='g++-4.9'
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-5']
env: COMPILER='g++-5'
- os: linux
compiler: gcc
addons: &gcc6
apt:
sources: *all_sources
packages: ['g++-6']
env: COMPILER='g++-6'
- os: linux
compiler: gcc
addons: &gcc7
apt:
sources: *all_sources
packages: ['g++-7']
env: COMPILER='g++-7'
- os: linux
compiler: gcc
addons: &gcc8
apt:
sources: *all_sources
packages: ['g++-8']
env: COMPILER='g++-8'
# 3b/ Linux C++14 Clang builds
# Note that we need newer libstdc++ for C++14 support
- os: linux
compiler: clang
addons:
apt:
packages: ['clang-3.8', 'libstdc++-6-dev']
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty
env: COMPILER='clang++-3.8' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.9', 'libstdc++-6-dev']
env: COMPILER='clang++-3.9' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-4.0', 'libstdc++-6-dev']
env: COMPILER='clang++-4.0' CPP14=1
- os: linux
dist: xenial
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-5.0', 'libstdc++-6-dev']
env: COMPILER='clang++-5.0' CPP14=1
- os: linux
dist: xenial
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0', 'libstdc++-6-dev']
env: COMPILER='clang++-6.0' CPP14=1
# 4a/ Linux C++14 GCC builds
- os: linux
compiler: gcc
addons: *gcc6
env: COMPILER='g++-6' CPP14=1
- os: linux
compiler: gcc
addons: *gcc7
env: COMPILER='g++-7' CPP14=1
- os: linux
compiler: gcc
addons: *gcc8
env: COMPILER='g++-8' CPP14=1
# 5/ OSX Clang Builds
- os: osx
osx_image: xcode7.3
compiler: clang
env: COMPILER='clang++'
- os: osx
osx_image: xcode8
compiler: clang
env: COMPILER='clang++'
- os: osx
osx_image: xcode9
compiler: clang
env: COMPILER='clang++'
- os: osx
osx_image: xcode9.1
compiler: clang
env: COMPILER='clang++'
- os: osx
osx_image: xcode9.1
compiler: clang
env: COMPILER='clang++' CPP14=1
# 6/ Special builds -- examples, coverage, valgrind, etc.
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['lcov', 'g++-7']
env: COMPILER='g++-7' CPP14=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
- os: linux
compiler: clang
addons:
apt:
packages: ['clang-3.8', 'lcov']
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty
env: COMPILER='clang++-3.8' EXAMPLES=1 COVERAGE=1 EXTRAS=1
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['valgrind', 'lcov', 'g++-7']
env: COMPILER='g++-7' CPP14=1 VALGRIND=1
- os: osx
osx_image: xcode9.1
compiler: clang
env: COMPILER='clang++' CPP14=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
# 7/ C++17 builds
- os: linux
compiler: gcc
addons: *gcc7
env: COMPILER='g++-7' CPP17=1
- os: linux
compiler: gcc
addons: *gcc7
env: COMPILER='g++-7' EXAMPLES=1 COVERAGE=1 EXTRAS=1 CPP17=1
- os: linux
dist: xenial
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-5.0']
env: COMPILER='clang++-5.0' CPP17=1
- os: linux
dist: xenial
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0', 'libstdc++-8-dev']
env: COMPILER='clang++-6.0' CPP17=1
- os: linux
dist: xenial
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0', 'libstdc++-8-dev']
env: COMPILER='clang++-6.0' CPP17=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
install:
- DEPS_DIR="${TRAVIS_BUILD_DIR}/deps"
- mkdir -p ${DEPS_DIR} && cd ${DEPS_DIR}
- |
if [[ "${TRAVIS_OS_NAME}" == "linux" ]]; then
CMAKE_URL="http://cmake.org/files/v3.8/cmake-3.8.2-Linux-x86_64.tar.gz"
mkdir cmake && travis_retry wget --no-check-certificate --quiet -O - ${CMAKE_URL} | tar --strip-components=1 -xz -C cmake
export PATH=${DEPS_DIR}/cmake/bin:${PATH}
elif [[ "${TRAVIS_OS_NAME}" == "osx" ]]; then
which cmake || brew install cmake;
fi
before_script:
- export CXX=${COMPILER}
- cd ${TRAVIS_BUILD_DIR}
# Regenerate single header file, so it is tested in the examples...
- python scripts/generateSingleHeader.py
- |
if [[ ${CPP17} -eq 1 ]]; then
export CPP_STANDARD=17
elif [[ ${CPP14} -eq 1 ]]; then
export CPP_STANDARD=14
else
export CPP_STANDARD=11
fi
# Use Debug builds for running Valgrind and building examples
- cmake -H. -BBuild-Debug -DCMAKE_BUILD_TYPE=Debug -Wdev -DCATCH_USE_VALGRIND=${VALGRIND} -DCATCH_BUILD_EXAMPLES=${EXAMPLES} -DCATCH_ENABLE_COVERAGE=${COVERAGE} -DCATCH_BUILD_EXTRA_TESTS=${EXTRAS} -DCMAKE_CXX_STANDARD=${CPP_STANDARD} -DCMAKE_CXX_STANDARD_REQUIRED=On -DCMAKE_CXX_EXTENSIONS=OFF
# Don't bother with release build for coverage build
- cmake -H. -BBuild-Release -DCMAKE_BUILD_TYPE=Release -Wdev -DCMAKE_CXX_STANDARD=${CPP_STANDARD} -DCMAKE_CXX_STANDARD_REQUIRED=On -DCMAKE_CXX_EXTENSIONS=OFF
script:
- cd Build-Debug
- make -j 2
- CTEST_OUTPUT_ON_FAILURE=1 ctest -j 2
# Coverage collection does not work for OS X atm
- |
if [[ "${TRAVIS_OS_NAME}" == "linux" ]] && [[ "${COVERAGE}" == "1" ]]; then
make gcov
make lcov
bash <(curl -s https://codecov.io/bash) -X gcov || echo "Codecov did not collect coverage reports"
fi
- # Go to release build
- cd ../Build-Release
- make -j 2
- CTEST_OUTPUT_ON_FAILURE=1 ctest -j 2

View File

@@ -1,92 +1,17 @@
load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
# Load the cc_library rule.
load("@rules_cc//cc:defs.bzl", "cc_library")
expand_template(
name = "catch_user_config",
out = "catch2/catch_user_config.hpp",
substitutions = {
"@CATCH_CONFIG_CONSOLE_WIDTH@": "80",
"@CATCH_CONFIG_DEFAULT_REPORTER@": "console",
"#cmakedefine CATCH_CONFIG_ANDROID_LOGWRITE": "",
"#cmakedefine CATCH_CONFIG_BAZEL_SUPPORT": "#define CATCH_CONFIG_BAZEL_SUPPORT",
"#cmakedefine CATCH_CONFIG_COLOUR_WIN32": "",
"#cmakedefine CATCH_CONFIG_COUNTER": "",
"#cmakedefine CATCH_CONFIG_CPP11_TO_STRING": "",
"#cmakedefine CATCH_CONFIG_CPP17_BYTE": "",
"#cmakedefine CATCH_CONFIG_CPP17_OPTIONAL": "",
"#cmakedefine CATCH_CONFIG_CPP17_STRING_VIEW": "",
"#cmakedefine CATCH_CONFIG_CPP17_UNCAUGHT_EXCEPTIONS": "",
"#cmakedefine CATCH_CONFIG_CPP17_VARIANT": "",
"#cmakedefine CATCH_CONFIG_DISABLE_EXCEPTIONS_CUSTOM_HANDLER": "",
"#cmakedefine CATCH_CONFIG_DISABLE_EXCEPTIONS": "",
"#cmakedefine CATCH_CONFIG_DISABLE_STRINGIFICATION": "",
"#cmakedefine CATCH_CONFIG_DISABLE": "",
"#cmakedefine CATCH_CONFIG_ENABLE_ALL_STRINGMAKERS": "",
"#cmakedefine CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_ENABLE_PAIR_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_ENABLE_TUPLE_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_EXPERIMENTAL_REDIRECT": "",
"#cmakedefine CATCH_CONFIG_FALLBACK_STRINGIFIER @CATCH_CONFIG_FALLBACK_STRINGIFIER@": "",
"#cmakedefine CATCH_CONFIG_FAST_COMPILE": "",
"#cmakedefine CATCH_CONFIG_GETENV": "",
"#cmakedefine CATCH_CONFIG_GLOBAL_NEXTAFTER": "",
"#cmakedefine CATCH_CONFIG_NO_ANDROID_LOGWRITE": "",
"#cmakedefine CATCH_CONFIG_NO_COLOUR_WIN32": "",
"#cmakedefine CATCH_CONFIG_NO_COUNTER": "",
"#cmakedefine CATCH_CONFIG_NO_CPP11_TO_STRING": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_BYTE": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_OPTIONAL": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_STRING_VIEW": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_VARIANT": "",
"#cmakedefine CATCH_CONFIG_NO_GETENV": "",
"#cmakedefine CATCH_CONFIG_NO_GLOBAL_NEXTAFTER": "",
"#cmakedefine CATCH_CONFIG_NO_POSIX_SIGNALS": "",
"#cmakedefine CATCH_CONFIG_NO_USE_ASYNC": "",
"#cmakedefine CATCH_CONFIG_NO_WCHAR": "",
"#cmakedefine CATCH_CONFIG_NO_WINDOWS_SEH": "",
"#cmakedefine CATCH_CONFIG_NOSTDOUT": "",
"#cmakedefine CATCH_CONFIG_POSIX_SIGNALS": "",
"#cmakedefine CATCH_CONFIG_PREFIX_ALL": "",
"#cmakedefine CATCH_CONFIG_SHARED_LIBRARY": "",
"#cmakedefine CATCH_CONFIG_USE_ASYNC": "",
"#cmakedefine CATCH_CONFIG_WCHAR": "",
"#cmakedefine CATCH_CONFIG_WINDOWS_CRTDBG": "",
"#cmakedefine CATCH_CONFIG_WINDOWS_SEH": "",
},
template = "src/catch2/catch_user_config.hpp.in",
)
# Generated header library, modifies the include prefix to account for
# generation path so that we can include <catch2/catch_user_config.hpp>
# correctly.
cc_library(
name = "catch2_generated",
hdrs = ["catch2/catch_user_config.hpp"],
include_prefix = ".", # to manipulate -I of dependenices
visibility = ["//visibility:public"],
)
# Static library, without main.
# Header-only rule to export catch2/catch.hpp.
cc_library(
name = "catch2",
srcs = glob(
["src/catch2/**/*.cpp"],
exclude = ["src/catch2/internal/catch_main.cpp"],
),
hdrs = glob(["src/catch2/**/*.hpp"]),
includes = ["src/"],
linkstatic = True,
hdrs = ["single_include/catch2/catch.hpp"],
includes = ["single_include/"],
visibility = ["//visibility:public"],
deps = [":catch2_generated"],
)
# Static library, with main.
cc_library(
name = "catch2_main",
srcs = ["src/catch2/internal/catch_main.cpp"],
includes = ["src/"],
linkstatic = True,
name = "catch2_with_main",
srcs = ["src/catch_with_main.cpp"],
visibility = ["//visibility:public"],
deps = [":catch2"],
deps = ["//:catch2"],
)

View File

@@ -1,79 +0,0 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
##
# This file contains options that are materialized into the Catch2
# compiled library. All of them default to OFF, as even the positive
# forms correspond to the user _forcing_ them to ON, while being OFF
# means that Catch2 can use its own autodetection.
#
# For detailed docs look into docs/configuration.md
macro(AddOverridableConfigOption OptionBaseName)
option(CATCH_CONFIG_${OptionBaseName} "Read docs/configuration.md for details" OFF)
option(CATCH_CONFIG_NO_${OptionBaseName} "Read docs/configuration.md for details" OFF)
endmacro()
macro(AddConfigOption OptionBaseName)
option(CATCH_CONFIG_${OptionBaseName} "Read docs/configuration.md for details" OFF)
endmacro()
set(_OverridableOptions
"ANDROID_LOGWRITE"
"BAZEL_SUPPORT"
"COLOUR_WIN32"
"COUNTER"
"CPP11_TO_STRING"
"CPP17_BYTE"
"CPP17_OPTIONAL"
"CPP17_STRING_VIEW"
"CPP17_UNCAUGHT_EXCEPTIONS"
"CPP17_VARIANT"
"GLOBAL_NEXTAFTER"
"POSIX_SIGNALS"
"USE_ASYNC"
"WCHAR"
"WINDOWS_SEH"
"GETENV"
)
foreach(OptionName ${_OverridableOptions})
AddOverridableConfigOption(${OptionName})
endforeach()
set(_OtherConfigOptions
"DISABLE_EXCEPTIONS"
"DISABLE_EXCEPTIONS_CUSTOM_HANDLER"
"DISABLE"
"DISABLE_STRINGIFICATION"
"ENABLE_ALL_STRINGMAKERS"
"ENABLE_OPTIONAL_STRINGMAKER"
"ENABLE_PAIR_STRINGMAKER"
"ENABLE_TUPLE_STRINGMAKER"
"ENABLE_VARIANT_STRINGMAKER"
"EXPERIMENTAL_REDIRECT"
"FAST_COMPILE"
"NOSTDOUT"
"PREFIX_ALL"
"WINDOWS_CRTDBG"
)
foreach(OptionName ${_OtherConfigOptions})
AddConfigOption(${OptionName})
endforeach()
set(CATCH_CONFIG_SHARED_LIBRARY ${BUILD_SHARED_LIBS})
set(CATCH_CONFIG_DEFAULT_REPORTER "console" CACHE STRING "Read docs/configuration.md for details. The name of the reporter should be without quotes.")
set(CATCH_CONFIG_CONSOLE_WIDTH "80" CACHE STRING "Read docs/configuration.md for details. Must form a valid integer literal.")
# There is no good way to both turn this into a CMake cache variable,
# and keep reasonable default semantics inside the project. Thus we do
# not define it and users have to provide it as an outside variable.
#set(CATCH_CONFIG_FALLBACK_STRINGIFIER "" CACHE STRING "Read docs/configuration.md for details.")

View File

@@ -1,120 +0,0 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
include(CheckCXXCompilerFlag)
function(add_cxx_flag_if_supported_to_targets flagname targets)
string(MAKE_C_IDENTIFIER ${flagname} flag_identifier )
check_cxx_compiler_flag("${flagname}" HAVE_FLAG_${flag_identifier})
if (HAVE_FLAG_${flag_identifier})
foreach(target ${targets})
target_compile_options(${target} PRIVATE ${flagname})
endforeach()
endif()
endfunction()
# Assumes that it is only called for development builds, where warnings
# and Werror is desired, so it also enables Werror.
function(add_warnings_to_targets targets)
LIST(LENGTH targets TARGETS_LEN)
# For now we just assume 2 possibilities: msvc and msvc-like compilers,
# and other.
if (MSVC)
foreach(target ${targets})
# Force MSVC to consider everything as encoded in utf-8
target_compile_options( ${target} PRIVATE /utf-8 )
# Enable Werror equivalent
if (CATCH_ENABLE_WERROR)
target_compile_options( ${target} PRIVATE /WX )
endif()
# MSVC is currently handled specially
if ( CMAKE_CXX_COMPILER_ID MATCHES "MSVC" )
STRING(REGEX REPLACE "/W[0-9]" "/W4" CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS}) # override default warning level
target_compile_options( ${target} PRIVATE /w44265 /w44061 /w44062 /w45038 )
endif()
endforeach()
endif()
if (NOT MSVC)
set(CHECKED_WARNING_FLAGS
"-Wabsolute-value"
"-Wall"
"-Wc++20-compat"
"-Wcall-to-pure-virtual-from-ctor-dtor"
"-Wcast-align"
"-Wcatch-value"
"-Wdangling"
"-Wdeprecated"
"-Wdeprecated-register"
"-Wexceptions"
"-Wexit-time-destructors"
"-Wextra"
"-Wextra-semi"
"-Wfloat-equal"
"-Wglobal-constructors"
"-Winit-self"
"-Wmisleading-indentation"
"-Wmismatched-new-delete"
"-Wmismatched-return-types"
"-Wmismatched-tags"
"-Wmissing-braces"
"-Wmissing-declarations"
"-Wmissing-noreturn"
"-Wmissing-prototypes"
"-Wmissing-variable-declarations"
"-Wnull-dereference"
"-Wold-style-cast"
"-Woverloaded-virtual"
"-Wparentheses"
"-Wpedantic"
"-Wreorder"
"-Wreturn-std-move"
"-Wshadow"
"-Wstrict-aliasing"
"-Wsuggest-destructor-override"
"-Wsuggest-override"
"-Wundef"
"-Wuninitialized"
"-Wunneeded-internal-declaration"
"-Wunreachable-code"
"-Wunused"
"-Wunused-function"
"-Wunused-parameter"
"-Wvla"
"-Wweak-vtables"
# This is a useful warning, but our tests sometimes rely on
# functions being present, but not picked (e.g. various checks
# for stringification implementation ordering).
# Ergo, we should use it every now and then, but we cannot
# enable it by default.
# "-Wunused-member-function"
)
foreach(warning ${CHECKED_WARNING_FLAGS})
add_cxx_flag_if_supported_to_targets(${warning} "${targets}")
endforeach()
if (CATCH_ENABLE_WERROR)
foreach(target ${targets})
# Enable Werror equivalent
target_compile_options( ${target} PRIVATE -Werror )
endforeach()
endif()
endif()
endfunction()
# Adds flags required for reproducible build to the target
# Currently only supports GCC and Clang
function(add_build_reproducibility_settings target)
# Make the build reproducible on versions of g++ and clang that supports -ffile-prefix-map
if((CMAKE_CXX_COMPILER_ID STREQUAL "GNU") OR (CMAKE_CXX_COMPILER_ID MATCHES "Clang"))
add_cxx_flag_if_supported_to_targets("-ffile-prefix-map=${CATCH_DIR}/=" "${target}")
endif()
endfunction()

26
CMake/MiscFunctions.cmake Normal file
View File

@@ -0,0 +1,26 @@
#checks that the given hard-coded list contains all headers + sources in the given folder
function(CheckFileList LIST_VAR FOLDER)
set(MESSAGE " should be added to the variable ${LIST_VAR}")
set(MESSAGE "${MESSAGE} in ${CMAKE_CURRENT_LIST_FILE}\n")
file(GLOB GLOBBED_LIST "${FOLDER}/*.cpp"
"${FOLDER}/*.hpp"
"${FOLDER}/*.h")
list(REMOVE_ITEM GLOBBED_LIST ${${LIST_VAR}})
foreach(EXTRA_ITEM ${GLOBBED_LIST})
string(REPLACE "${CATCH_DIR}/" "" RELATIVE_FILE_NAME "${EXTRA_ITEM}")
message(AUTHOR_WARNING "The file \"${RELATIVE_FILE_NAME}\"${MESSAGE}")
endforeach()
endfunction()
function(CheckFileListRec LIST_VAR FOLDER)
set(MESSAGE " should be added to the variable ${LIST_VAR}")
set(MESSAGE "${MESSAGE} in ${CMAKE_CURRENT_LIST_FILE}\n")
file(GLOB_RECURSE GLOBBED_LIST "${FOLDER}/*.cpp"
"${FOLDER}/*.hpp"
"${FOLDER}/*.h")
list(REMOVE_ITEM GLOBBED_LIST ${${LIST_VAR}})
foreach(EXTRA_ITEM ${GLOBBED_LIST})
string(REPLACE "${CATCH_DIR}/" "" RELATIVE_FILE_NAME "${EXTRA_ITEM}")
message(AUTHOR_WARNING "The file \"${RELATIVE_FILE_NAME}\"${MESSAGE}")
endforeach()
endfunction()

View File

@@ -1,10 +0,0 @@
includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@
libdir=@CMAKE_INSTALL_FULL_LIBDIR@
pkg_version=@Catch2_VERSION@
Name: Catch2-With-Main
Description: A modern, C++-native test framework for C++14 and above (links in default main)
Version: ${pkg_version}
Requires: catch2 = ${pkg_version}
Cflags: -I${includedir}
Libs: -L${libdir} -lCatch2Main

View File

@@ -4,8 +4,7 @@ includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@
libdir=@CMAKE_INSTALL_FULL_LIBDIR@
Name: Catch2
Description: A modern, C++-native, test framework for C++14 and above
Description: A modern, C++-native, header-only, test framework for C++11
URL: https://github.com/catchorg/Catch2
Version: @Catch2_VERSION@
Cflags: -I${includedir}
Libs: -L${libdir} -lCatch2

View File

@@ -1,28 +1,13 @@
cmake_minimum_required(VERSION 3.10)
cmake_minimum_required(VERSION 3.5)
# detect if Catch is being bundled,
# disable testsuite in that case
if(NOT DEFINED PROJECT_NAME)
set(NOT_SUBPROJECT ON)
set(NOT_SUBPROJECT ON)
else()
set(NOT_SUBPROJECT OFF)
set(NOT_SUBPROJECT OFF)
endif()
option(CATCH_INSTALL_DOCS "Install documentation alongside library" ON)
option(CATCH_INSTALL_EXTRAS "Install extras (CMake scripts, debugger helpers) alongside library" ON)
option(CATCH_DEVELOPMENT_BUILD "Build tests, enable warnings, enable Werror, etc" OFF)
include(CMakeDependentOption)
cmake_dependent_option(CATCH_BUILD_TESTING "Build the SelfTest project" ON "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_EXAMPLES "Build code examples" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_EXTRA_TESTS "Build extra tests" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_FUZZERS "Build fuzzers" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_COVERAGE "Generate coverage for codecov.io" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_WERROR "Enables Werror during build" ON "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_SURROGATES "Enable generating and building surrogate TUs for the main headers" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_CONFIGURE_TESTS "Enable CMake configuration tests. WARNING: VERY EXPENSIVE" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
# Catch2's build breaks if done in-tree. You probably should not build
# things in tree anyway, but we can allow projects that include Catch2
# as a subproject to build in-tree as long as it is not in our tree.
@@ -30,63 +15,44 @@ if (CMAKE_BINARY_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
message(FATAL_ERROR "Building in-source is not supported! Create a build dir and remove ${CMAKE_SOURCE_DIR}/CMakeCache.txt")
endif()
project(Catch2
VERSION 3.3.2 # CML version placeholder, don't delete
LANGUAGES CXX
# HOMEPAGE_URL is not supported until CMake version 3.12, which
# we do not target yet.
# HOMEPAGE_URL "https://github.com/catchorg/Catch2"
DESCRIPTION "A modern, C++-native, unit test framework."
)
project(Catch2 LANGUAGES CXX VERSION 2.13.10)
# Provide path for scripts. We first add path to the scripts we don't use,
# but projects including us might, and set the path up to parent scope.
# Then we also add path that we use to configure the project, but is of
# no use to top level projects.
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/extras")
if (NOT NOT_SUBPROJECT)
set(CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH}" PARENT_SCOPE)
endif()
# Provide path for scripts
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/CMake")
include(GNUInstallDirs)
include(CMakePackageConfigHelpers)
include(CatchConfigOptions)
if(CATCH_DEVELOPMENT_BUILD)
include(CTest)
endif()
# This variable is used in some subdirectories, so we need it here, rather
# than later in the install block
set(CATCH_CMAKE_CONFIG_DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/Catch2")
option(CATCH_USE_VALGRIND "Perform SelfTests with Valgrind" OFF)
option(CATCH_BUILD_TESTING "Build SelfTest project" ON)
option(CATCH_BUILD_EXAMPLES "Build documentation examples" OFF)
option(CATCH_BUILD_EXTRA_TESTS "Build extra tests" OFF)
option(CATCH_BUILD_STATIC_LIBRARY "Builds static library from the main implementation. EXPERIMENTAL" OFF)
option(CATCH_ENABLE_COVERAGE "Generate coverage for codecov.io" OFF)
option(CATCH_ENABLE_WERROR "Enable all warnings as errors" ON)
option(CATCH_INSTALL_DOCS "Install documentation alongside library" ON)
option(CATCH_INSTALL_HELPERS "Install contrib alongside library" ON)
# We have some Windows builds that test `wmain` entry point,
# and we need this change to be present in all binaries that
# are built during these tests, so this is required here, before
# the subdirectories are added.
if(CATCH_TEST_USE_WMAIN)
# define some folders
set(CATCH_DIR ${CMAKE_CURRENT_SOURCE_DIR})
set(SELF_TEST_DIR ${CATCH_DIR}/projects/SelfTest)
set(BENCHMARK_DIR ${CATCH_DIR}/projects/Benchmark)
set(HEADER_DIR ${CATCH_DIR}/include)
if(USE_WMAIN)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /ENTRY:wmainCRTStartup")
endif()
# Basic paths
set(CATCH_DIR ${CMAKE_CURRENT_SOURCE_DIR})
set(SOURCES_DIR ${CATCH_DIR}/src/catch2)
set(SELF_TEST_DIR ${CATCH_DIR}/tests/SelfTest)
set(BENCHMARK_DIR ${CATCH_DIR}/tests/Benchmark)
set(EXAMPLES_DIR ${CATCH_DIR}/examples)
# We need to bring-in the variables defined there to this scope
add_subdirectory(src)
# Build tests only if requested
if (BUILD_TESTING AND CATCH_BUILD_TESTING AND NOT_SUBPROJECT)
find_package(PythonInterp 3 REQUIRED)
if (NOT PYTHONINTERP_FOUND)
message(FATAL_ERROR "Python not found, but required for tests")
if(NOT_SUBPROJECT)
include(CTest)
set_property(GLOBAL PROPERTY USE_FOLDERS ON)
if(BUILD_TESTING AND CATCH_BUILD_TESTING)
find_package(PythonInterp)
if (NOT PYTHONINTERP_FOUND)
message(FATAL_ERROR "Python not found, but required for tests")
endif()
add_subdirectory(projects)
endif()
add_subdirectory(tests)
endif()
if(CATCH_BUILD_EXAMPLES)
@@ -94,21 +60,75 @@ if(CATCH_BUILD_EXAMPLES)
endif()
if(CATCH_BUILD_EXTRA_TESTS)
add_subdirectory(tests/ExtraTests)
add_subdirectory(projects/ExtraTests)
endif()
if(CATCH_BUILD_FUZZERS)
add_subdirectory(fuzzing)
# add catch as a 'linkable' target
add_library(Catch2 INTERFACE)
# depend on some obvious c++11 features so the dependency is transitively added dependents
target_compile_features(Catch2
INTERFACE
cxx_alignas
cxx_alignof
cxx_attributes
cxx_auto_type
cxx_constexpr
cxx_defaulted_functions
cxx_deleted_functions
cxx_final
cxx_lambdas
cxx_noexcept
cxx_override
cxx_range_for
cxx_rvalue_references
cxx_static_assert
cxx_strong_enums
cxx_trailing_return_types
cxx_unicode_literals
cxx_user_literals
cxx_variadic_macros
)
target_include_directories(Catch2
INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}/single_include>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)
if (ANDROID)
target_link_libraries(Catch2 INTERFACE log)
endif()
if (CATCH_DEVELOPMENT_BUILD)
add_warnings_to_targets("${CATCH_WARNING_TARGETS}")
endif()
# provide a namespaced alias for clients to 'link' against if catch is included as a sub-project
add_library(Catch2::Catch2 ALIAS Catch2)
# Hacky support for compiling the impl into a static lib
if (CATCH_BUILD_STATIC_LIBRARY)
add_library(Catch2WithMain ${CMAKE_CURRENT_LIST_DIR}/src/catch_with_main.cpp)
target_link_libraries(Catch2WithMain PUBLIC Catch2)
add_library(Catch2::Catch2WithMain ALIAS Catch2WithMain)
# Make the build reproducible on versions of g++ and clang that supports -ffile-prefix-map
if(("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND NOT ${CMAKE_CXX_COMPILER_VERSION} VERSION_LESS 8) OR
("${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang" AND NOT ${CMAKE_CXX_COMPILER_VERSION} VERSION_LESS 10))
target_compile_options(Catch2WithMain PRIVATE "-ffile-prefix-map=${CMAKE_SOURCE_DIR}=.")
endif()
if (CATCH_CONFIG_DEFAULT_REPORTER)
target_compile_definitions(Catch2WithMain PRIVATE CATCH_CONFIG_DEFAULT_REPORTER=${CATCH_CONFIG_DEFAULT_REPORTER})
endif()
endif(CATCH_BUILD_STATIC_LIBRARY)
# Only perform the installation steps when Catch is not being used as
# a subproject via `add_subdirectory`, or the destinations will break,
# see https://github.com/catchorg/Catch2/issues/1373
if (NOT_SUBPROJECT)
include(CMakePackageConfigHelpers)
set(CATCH_CMAKE_CONFIG_DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/Catch2")
configure_package_config_file(
${CMAKE_CURRENT_LIST_DIR}/CMake/Catch2Config.cmake.in
${CMAKE_CURRENT_BINARY_DIR}/Catch2Config.cmake
@@ -116,11 +136,58 @@ if (NOT_SUBPROJECT)
${CATCH_CMAKE_CONFIG_DESTINATION}
)
# Workaround lack of generator expressions in install(TARGETS
set(InstallationTargets Catch2)
if (TARGET Catch2WithMain)
list(APPEND InstallationTargets Catch2WithMain)
endif()
# create and install an export set for catch target as Catch2::Catch
install(
TARGETS
${InstallationTargets}
EXPORT
Catch2Targets
DESTINATION
${CMAKE_INSTALL_LIBDIR}
)
install(
EXPORT
Catch2Targets
NAMESPACE
Catch2::
DESTINATION
${CATCH_CMAKE_CONFIG_DESTINATION}
)
# By default, FooConfigVersion is tied to architecture that it was
# generated on. Because Catch2 is header-only, it is arch-independent
# and thus Catch2ConfigVersion should not be tied to the architecture
# it was generated on.
#
# CMake does not provide a direct customization point for this in
# `write_basic_package_version_file`, but it can be accomplished
# indirectly by temporarily redefining `CMAKE_SIZEOF_VOID_P` to an
# empty string. Note that just undefining the variable could be
# insufficient in cases where the variable was already in CMake cache
set(CATCH2_CMAKE_SIZEOF_VOID_P ${CMAKE_SIZEOF_VOID_P})
set(CMAKE_SIZEOF_VOID_P "")
write_basic_package_version_file(
"${CMAKE_CURRENT_BINARY_DIR}/Catch2ConfigVersion.cmake"
COMPATIBILITY
SameMajorVersion
)
set(CMAKE_SIZEOF_VOID_P ${CATCH2_CMAKE_SIZEOF_VOID_P})
install(
DIRECTORY
"single_include/"
DESTINATION
"${CMAKE_INSTALL_INCLUDEDIR}"
)
install(
FILES
@@ -137,29 +204,28 @@ if (NOT_SUBPROJECT)
docs/
DESTINATION
"${CMAKE_INSTALL_DOCDIR}"
PATTERN "doxygen" EXCLUDE
)
endif()
if(CATCH_INSTALL_EXTRAS)
# Install CMake scripts
install(
FILES
"extras/ParseAndAddCatchTests.cmake"
"extras/Catch.cmake"
"extras/CatchAddTests.cmake"
DESTINATION
${CATCH_CMAKE_CONFIG_DESTINATION}
)
# Install debugger helpers
install(
FILES
"extras/gdbinit"
"extras/lldbinit"
DESTINATION
${CMAKE_INSTALL_DATAROOTDIR}/Catch2
)
if(CATCH_INSTALL_HELPERS)
# Install CMake scripts
install(
FILES
"contrib/ParseAndAddCatchTests.cmake"
"contrib/Catch.cmake"
"contrib/CatchAddTests.cmake"
DESTINATION
${CATCH_CMAKE_CONFIG_DESTINATION}
)
# Install debugger helpers
install(
FILES
"contrib/gdbinit"
"contrib/lldbinit"
DESTINATION
${CMAKE_INSTALL_DATAROOTDIR}/Catch2
)
endif()
## Provide some pkg-config integration
@@ -172,15 +238,9 @@ if (NOT_SUBPROJECT)
${CMAKE_CURRENT_BINARY_DIR}/catch2.pc
@ONLY
)
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/CMake/catch2-with-main.pc.in
${CMAKE_CURRENT_BINARY_DIR}/catch2-with-main.pc
@ONLY
)
install(
FILES
"${CMAKE_CURRENT_BINARY_DIR}/catch2.pc"
"${CMAKE_CURRENT_BINARY_DIR}/catch2-with-main.pc"
DESTINATION
${PKGCONFIG_INSTALL_DIR}
)

View File

@@ -1,25 +0,0 @@
{
"version": 3,
"configurePresets": [
{
"name": "basic-tests",
"displayName": "Basic development build",
"description": "Enables development build with basic tests that are cheap to build and run",
"cacheVariables": {
"CATCH_DEVELOPMENT_BUILD": "ON"
}
},
{
"name": "all-tests",
"inherits": "basic-tests",
"displayName": "Full development build",
"description": "Enables development build with examples and ALL tests",
"cacheVariables": {
"CATCH_BUILD_EXAMPLES": "ON",
"CATCH_BUILD_EXTRA_TESTS": "ON",
"CATCH_BUILD_SURROGATES": "ON",
"CATCH_ENABLE_CONFIGURE_TESTS": "ON"
}
}
]
}

2484
Doxyfile

File diff suppressed because it is too large Load Diff

100
README.md
View File

@@ -1,92 +1,28 @@
<a id="top"></a>
![Catch2 logo](data/artwork/catch2-logo-small-with-background.png)
![catch logo](artwork/catch2-logo-small.png)
[![Github Releases](https://img.shields.io/github/release/catchorg/catch2.svg)](https://github.com/catchorg/catch2/releases)
[![Linux build status](https://github.com/catchorg/Catch2/actions/workflows/linux-simple-builds.yml/badge.svg)](https://github.com/catchorg/Catch2/actions/workflows/linux-simple-builds.yml)
[![Linux build status](https://github.com/catchorg/Catch2/actions/workflows/linux-other-builds.yml/badge.svg)](https://github.com/catchorg/Catch2/actions/workflows/linux-other-builds.yml)
[![MacOS build status](https://github.com/catchorg/Catch2/actions/workflows/mac-builds.yml/badge.svg)](https://github.com/catchorg/Catch2/actions/workflows/mac-builds.yml)
[![Build Status](https://ci.appveyor.com/api/projects/status/github/catchorg/Catch2?svg=true&branch=devel)](https://ci.appveyor.com/project/catchorg/catch2)
[![Code Coverage](https://codecov.io/gh/catchorg/Catch2/branch/devel/graph/badge.svg)](https://codecov.io/gh/catchorg/Catch2)
[![Try online](https://img.shields.io/badge/try-online-blue.svg)](https://godbolt.org/z/EdoY15q9G)
[![Build Status](https://travis-ci.org/catchorg/Catch2.svg?branch=v2.x)](https://travis-ci.org/catchorg/Catch2)
[![Build status](https://ci.appveyor.com/api/projects/status/github/catchorg/Catch2?svg=true)](https://ci.appveyor.com/project/catchorg/catch2)
[![codecov](https://codecov.io/gh/catchorg/Catch2/branch/v2.x/graph/badge.svg)](https://codecov.io/gh/catchorg/Catch2)
[![Try online](https://img.shields.io/badge/try-online-blue.svg)](https://wandbox.org/permlink/6JUH8Eybx4CtvkJS)
[![Join the chat in Discord: https://discord.gg/4CWS9zD](https://img.shields.io/badge/Discord-Chat!-brightgreen.svg)](https://discord.gg/4CWS9zD)
## What is Catch2?
<a href="https://github.com/catchorg/Catch2/releases/download/v2.13.10/catch.hpp">The latest version of the single header can be downloaded directly using this link</a>
Catch2 is mainly a unit testing framework for C++, but it also
provides basic micro-benchmarking features, and simple BDD macros.
## Catch2 is released!
Catch2's main advantage is that using it is both simple and natural.
Test names do not have to be valid identifiers, assertions look like
normal C++ boolean expressions, and sections provide a nice and local way
to share set-up and tear-down code in tests.
If you've been using an earlier version of Catch, please see the
Breaking Changes section of [the release notes](https://github.com/catchorg/Catch2/releases/tag/v2.0.1)
before moving to Catch2. You might also like to read [this blog post](https://levelofindirection.com/blog/catch2-released.html) for more details.
**Example unit test**
```cpp
#include <catch2/catch_test_macros.hpp>
#include <cstdint>
uint32_t factorial( uint32_t number ) {
return number <= 1 ? number : factorial(number-1) * number;
}
TEST_CASE( "Factorials are computed", "[factorial]" ) {
REQUIRE( factorial( 1) == 1 );
REQUIRE( factorial( 2) == 2 );
REQUIRE( factorial( 3) == 6 );
REQUIRE( factorial(10) == 3'628'800 );
}
```
**Example microbenchmark**
```cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/benchmark/catch_benchmark.hpp>
#include <cstdint>
uint64_t fibonacci(uint64_t number) {
return number < 2 ? number : fibonacci(number - 1) + fibonacci(number - 2);
}
TEST_CASE("Benchmark Fibonacci", "[!benchmark]") {
REQUIRE(fibonacci(5) == 5);
REQUIRE(fibonacci(20) == 6'765);
BENCHMARK("fibonacci 20") {
return fibonacci(20);
};
REQUIRE(fibonacci(25) == 75'025);
BENCHMARK("fibonacci 25") {
return fibonacci(25);
};
}
```
_Note that benchmarks are not run by default, so you need to run it explicitly
with the `[!benchmark]` tag._
## Catch2 v3 has been released!
You are on the `devel` branch, where the v3 version is being developed.
v3 brings a bunch of significant changes, the big one being that Catch2
is no longer a single-header library. Catch2 now behaves as a normal
library, with multiple headers and separately compiled implementation.
The documentation is slowly being updated to take these changes into
account, but this work is currently still ongoing.
For migrating from the v2 releases to v3, you should look at [our
documentation](docs/migrate-v2-to-v3.md#top). It provides a simple
guidelines on getting started, and collects most common migration
problems.
For the previous major version of Catch2 [look into the `v2.x` branch
here on GitHub](https://github.com/catchorg/Catch2/tree/v2.x).
## What's the Catch?
Catch2 is a multi-paradigm test framework for C++. which also supports
Objective-C (and maybe C).
It is primarily distributed as a single header file, although certain
extensions may require additional headers.
## How to use it
This documentation comprises these three parts:
@@ -95,9 +31,7 @@ This documentation comprises these three parts:
* [Tutorial](docs/tutorial.md#top) - getting started
* [Reference section](docs/Readme.md#top) - all the details
## More
* Issues and bugs can be raised on the [Issue tracker on GitHub](https://github.com/catchorg/Catch2/issues)
* For discussion or questions please use [our Discord](https://discord.gg/4CWS9zD)
* See who else is using Catch2 in [Open Source Software](docs/opensource-users.md#top)
or [commercially](docs/commercial-users.md#top).
* For discussion or questions please use [the dedicated Google Groups forum](https://groups.google.com/forum/?fromgroups#!forum/catch-forum) or our [Discord](https://discord.gg/4CWS9zD)
* See [who else is using Catch2](docs/opensource-users.md#top)

View File

@@ -1,19 +0,0 @@
# Security Policy
## Supported Versions
* Versions 1.x (branch Catch1.x) are no longer supported.
* Versions 2.x (branch v2.x) are currently supported.
* `devel` branch serves for stable-ish development and is supported,
but branches `devel-*` are considered short lived and are not supported separately.
## Reporting a Vulnerability
Due to its nature as a _unit_ test framework, Catch2 shouldn't interact
with untrusted inputs and there shouldn't be many security vulnerabilities
in it.
However, if you find one you send email to martin <dot> horenovsky <at>
gmail <dot> com. If you want to encrypt the email, my pgp key is
`E29C 46F3 B8A7 5028 6079 3B7D ECC9 C20E 314B 2360`.

0
WORKSPACE Normal file
View File

View File

@@ -1,15 +0,0 @@
workspace(name = "catch2")
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "bazel_skylib",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.3.0/bazel-skylib-1.3.0.tar.gz",
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.3.0/bazel-skylib-1.3.0.tar.gz",
],
sha256 = "74d544d96f4a5bb630d465ca8bbcfe231e3594e5aae57e1edbf17a6eb3ca2506",
)
load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
bazel_skylib_workspace()

View File

@@ -1,37 +1,93 @@
version: "{build}-{branch}"
# version string format -- This will be overwritten later anyway
version: "{build}"
# We need a more up to date pip because Python 2.7 is EOL soon
init:
- set PATH=C:\Python35\Scripts;%PATH%
# If we ever get a backlog larger than clone_depth, builds will fail
# spuriously. I do not think we will ever get 20 deep commits deep though.
clone_depth: 20
# We want to build everything, except for branches that are explicitly
# for messing around with travis.
branches:
except:
- /dev-travis.+/
os:
- Visual Studio 2017
- Visual Studio 2015
# We need a more up to date pip because Python 2.7 is EOL soon
init:
- set PATH=C:\Python35;C:\Python35\Scripts;%PATH%
environment:
matrix:
- additional_flags: "/permissive- /std:c++latest"
wmain: 0
- additional_flags: ""
wmain: 0
- additional_flags: "/D_UNICODE /DUNICODE"
wmain: 1
coverage: 0
# Have a coverage dimension
- additional_flags: ""
wmain: 0
coverage: 1
# Have an examples dimension
- additional_flags: ""
wmain: 0
examples: 1
matrix:
exclude:
- os: Visual Studio 2015
additional_flags: "/permissive- /std:c++latest"
- os: Visual Studio 2015
additional_flags: "/D_UNICODE /DUNICODE"
# Exclude unwanted coverage configurations
- coverage: 1
platform: Win32
- coverage: 1
os: Visual Studio 2015
- coverage: 1
configuration: Release
# Exclude unwanted examples configurations
- examples: 1
platform: Win32
- examples: 1
os: Visual Studio 2015
- examples: 1
configuration: Release
install:
- ps: if (($env:CONFIGURATION) -eq "Debug" -And ($env:coverage) -eq "1" ) { pip --disable-pip-version-check install codecov }
# This removes our changes to PATH. Keep this step last!
- ps: if (($env:CONFIGURATION) -eq "Debug" -And ($env:coverage) -eq "1" ) { .\tools\misc\installOpenCppCoverage.ps1 }
- ps: if (($env:CONFIGURATION) -eq "Debug" -And ($env:coverage) -eq "1" ) { .\misc\installOpenCppCoverage.ps1 }
# Win32 and x64 are CMake-compatible solution platform names.
# This allows us to pass %PLATFORM% to CMake -A.
platform:
- Win32
- x64
# build Configurations, i.e. Debug, Release, etc.
configuration:
- Debug
- Release
#Cmake will autodetect the compiler, but we set the arch
before_build:
# We need to modify PATH again, because it was reset since the "init" step
- set PATH=C:\Python35;C:\Python35\Scripts;%PATH%
- set CXXFLAGS=%additional_flags%
# If we are building examples/extra-tests, we need to regenerate the amalgamated files
- cmd: if "%examples%"=="1" ( python .\tools\scripts\generateAmalgamatedFiles.py )
# Indirection because appveyor doesn't handle multiline batch scripts properly
# https://stackoverflow.com/questions/37627248/how-to-split-a-command-over-multiple-lines-in-appveyor-yml/37647169#37647169
# https://help.appveyor.com/discussions/questions/3888-multi-line-cmd-or-powershell-warning-ignore
- cmd: .\tools\misc\appveyorBuildConfigurationScript.bat
- cmd: .\misc\appveyorBuildConfigurationScript.bat
# build with MSBuild
@@ -42,42 +98,4 @@ build:
test_script:
- set CTEST_OUTPUT_ON_FAILURE=1
- cmd: .\tools\misc\appveyorTestRunScript.bat
# Sadly we cannot use the standard "dimensions" based approach towards
# specifying the different builds, as there is no way to add one-offs
# builds afterwards. This means that we will painfully specify each
# build explicitly.
environment:
matrix:
- FLAVOR: VS 2019 x64 Debug Coverage Examples
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
examples: 1
coverage: 1
platform: x64
configuration: Debug
- FLAVOR: VS 2019 x64 Debug WMain
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
wmain: 1
additional_flags: "/D_UNICODE /DUNICODE"
platform: x64
configuration: Debug
- FLAVOR: VS 2019 x64 Debug Latest Strict
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
additional_flags: "/permissive- /std:c++latest"
platform: x64
configuration: Debug
- FLAVOR: VS 2017 x64 Debug
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
platform: x64
configuration: Debug
- FLAVOR: VS 2017 x64 Release Coverage
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
coverage: 1
platform: x64
configuration: Debug
- cmd: .\misc\appveyorTestRunScript.bat

View File

Before

Width:  |  Height:  |  Size: 10 KiB

After

Width:  |  Height:  |  Size: 10 KiB

View File

Before

Width:  |  Height:  |  Size: 33 KiB

After

Width:  |  Height:  |  Size: 33 KiB

View File

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

View File

@@ -10,13 +10,16 @@ coverage:
default:
target: 80%
ignore:
- "projects/SelfTest"
- "**/catch_reporter_tap.hpp"
- "**/catch_reporter_automake.hpp"
- "**/catch_reporter_teamcity.hpp"
- "**/catch_reporter_sonarqube.hpp"
- "**/external/clara.hpp"
- "tests"
codecov:
branch: devel
max_report_age: off
branch: v2.x
comment:
layout: "diff"

View File

@@ -1,60 +1,30 @@
#!/usr/bin/env python
from conans import ConanFile, CMake, tools
from conans import ConanFile, CMake
class CatchConan(ConanFile):
name = "catch2"
description = "A modern, C++-native, framework for unit-tests, TDD and BDD"
topics = ("conan", "catch2", "unit-test", "tdd", "bdd")
name = "Catch2"
description = "A modern, C++-native, header-only, framework for unit-tests, TDD and BDD"
topics = ("conan", "catch2", "header-only", "unit-test", "tdd", "bdd")
url = "https://github.com/catchorg/Catch2"
homepage = url
license = "BSL-1.0"
exports = "LICENSE.txt"
exports_sources = ("src/*", "CMakeLists.txt", "CMake/*", "extras/*")
settings = "os", "compiler", "build_type", "arch"
exports_sources = ("single_include/*", "CMakeLists.txt", "CMake/*", "contrib/*", "src/*")
generators = "cmake"
def _configure_cmake(self):
def package(self):
cmake = CMake(self)
cmake.definitions["BUILD_TESTING"] = "OFF"
cmake.definitions["CATCH_INSTALL_DOCS"] = "OFF"
cmake.definitions["CATCH_INSTALL_EXTRAS"] = "ON"
cmake.configure(build_folder="build")
return cmake
def build(self):
# We need this workaround until the toolchains feature
# to inject stuff like MD/MT
line_to_replace = 'list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/CMake")'
tools.replace_in_file("CMakeLists.txt", line_to_replace,
'''{}
include("{}/conanbuildinfo.cmake")
conan_basic_setup()'''.format(line_to_replace, self.install_folder.replace("\\", "/")))
cmake = self._configure_cmake()
cmake.build()
def package(self):
self.copy(pattern="LICENSE.txt", dst="licenses")
cmake = self._configure_cmake()
cmake.definitions["CATCH_INSTALL_HELPERS"] = "ON"
cmake.configure(build_folder='build')
cmake.install()
def package_info(self):
lib_suffix = "d" if self.settings.build_type == "Debug" else ""
self.copy(pattern="LICENSE.txt", dst="licenses")
self.cpp_info.names["cmake_find_package"] = "Catch2"
self.cpp_info.names["cmake_find_package_multi"] = "Catch2"
# Catch2
self.cpp_info.components["catch2base"].names["cmake_find_package"] = "Catch2"
self.cpp_info.components["catch2base"].names["cmake_find_package_multi"] = "Catch2"
self.cpp_info.components["catch2base"].names["pkg_config"] = "Catch2"
self.cpp_info.components["catch2base"].libs = ["Catch2" + lib_suffix]
self.cpp_info.components["catch2base"].builddirs.append("lib/cmake/Catch2")
# Catch2WithMain
self.cpp_info.components["catch2main"].names["cmake_find_package"] = "Catch2WithMain"
self.cpp_info.components["catch2main"].names["cmake_find_package_multi"] = "Catch2WithMain"
self.cpp_info.components["catch2main"].names["pkg_config"] = "Catch2WithMain"
self.cpp_info.components["catch2main"].libs = ["Catch2Main" + lib_suffix]
self.cpp_info.components["catch2main"].requires = ["catch2base"]
def package_id(self):
self.info.header_only()
def package_info(self):
self.cpp_info.builddirs.append("lib/cmake/Catch2")

View File

@@ -116,13 +116,6 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
``--out dir/<test_name>suffix``. This can be used to add a file extension to
the output e.g. ".xml".
``DL_PATHS path...``
Specifies paths that need to be set for the dynamic linker to find shared
libraries/DLLs when running the test executable (PATH/LD_LIBRARY_PATH respectively).
These paths will both be set when retrieving the list of test cases from the
test executable and when the tests are executed themselves. This requires
cmake/ctest >= 3.22.
#]=======================================================================]
#------------------------------------------------------------------------------
@@ -131,7 +124,7 @@ function(catch_discover_tests TARGET)
""
""
"TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;REPORTER;OUTPUT_DIR;OUTPUT_PREFIX;OUTPUT_SUFFIX"
"TEST_SPEC;EXTRA_ARGS;PROPERTIES;DL_PATHS"
"TEST_SPEC;EXTRA_ARGS;PROPERTIES"
${ARGN}
)
@@ -142,12 +135,6 @@ function(catch_discover_tests TARGET)
set(_TEST_LIST ${TARGET}_TESTS)
endif()
if (_DL_PATHS)
if(${CMAKE_VERSION} VERSION_LESS "3.22.0")
message(FATAL_ERROR "The DL_PATHS option requires at least cmake 3.22")
endif()
endif()
## Generate a unique name based on the extra arguments
string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS} ${_REPORTER} ${_OUTPUT_DIR} ${_OUTPUT_PREFIX} ${_OUTPUT_SUFFIX}")
string(SUBSTRING ${args_hash} 0 7 args_hash)
@@ -177,7 +164,6 @@ function(catch_discover_tests TARGET)
-D "TEST_OUTPUT_DIR=${_OUTPUT_DIR}"
-D "TEST_OUTPUT_PREFIX=${_OUTPUT_PREFIX}"
-D "TEST_OUTPUT_SUFFIX=${_OUTPUT_SUFFIX}"
-D "TEST_DL_PATHS=${_DL_PATHS}"
-D "CTEST_FILE=${ctest_tests_file}"
-P "${_CATCH_DISCOVER_TESTS_SCRIPT}"
VERBATIM

View File

@@ -10,19 +10,10 @@ set(reporter ${TEST_REPORTER})
set(output_dir ${TEST_OUTPUT_DIR})
set(output_prefix ${TEST_OUTPUT_PREFIX})
set(output_suffix ${TEST_OUTPUT_SUFFIX})
set(dl_paths ${TEST_DL_PATHS})
set(script)
set(suite)
set(tests)
if(WIN32)
set(dl_paths_variable_name PATH)
elseif(APPLE)
set(dl_paths_variable_name DYLD_LIBRARY_PATH)
else()
set(dl_paths_variable_name LD_LIBRARY_PATH)
endif()
function(add_command NAME)
set(_args "")
# use ARGV* instead of ARGN, because ARGN splits arrays into multiple arguments
@@ -44,19 +35,18 @@ if(NOT EXISTS "${TEST_EXECUTABLE}")
"Specified test executable '${TEST_EXECUTABLE}' does not exist"
)
endif()
if(dl_paths)
cmake_path(CONVERT "${dl_paths}" TO_NATIVE_PATH_LIST paths)
set(ENV{${dl_paths_variable_name}} "${paths}")
endif()
execute_process(
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-tests --verbosity quiet
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-test-names-only
OUTPUT_VARIABLE output
RESULT_VARIABLE result
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
)
if(NOT ${result} EQUAL 0)
# Catch --list-test-names-only reports the number of tests, so 0 is... surprising
if(${result} EQUAL 0)
message(WARNING
"Test executable '${TEST_EXECUTABLE}' contains no tests!\n"
)
elseif(${result} LESS 0)
message(FATAL_ERROR
"Error running test executable '${TEST_EXECUTABLE}':\n"
" Result: ${result}\n"
@@ -73,7 +63,11 @@ execute_process(
RESULT_VARIABLE reporters_result
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
)
if(NOT ${reporters_result} EQUAL 0)
if(${reporters_result} EQUAL 0)
message(WARNING
"Test executable '${TEST_EXECUTABLE}' contains no reporters!\n"
)
elseif(${reporters_result} LESS 0)
message(FATAL_ERROR
"Error running test executable '${TEST_EXECUTABLE}':\n"
" Result: ${reporters_result}\n"
@@ -100,13 +94,6 @@ if(output_dir AND NOT IS_ABSOLUTE ${output_dir})
endif()
endif()
if(dl_paths)
foreach(path ${dl_paths})
cmake_path(NATIVE_PATH path native_path)
list(APPEND environment_modifications "${dl_paths_variable_name}=path_list_prepend:${native_path}")
endforeach()
endif()
# Parse output
foreach(line ${output})
set(test ${line})
@@ -137,14 +124,6 @@ foreach(line ${output})
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
${properties}
)
if(environment_modifications)
add_command(set_tests_properties
"${prefix}${test}${suffix}"
PROPERTIES
ENVIRONMENT_MODIFICATION "${environment_modifications}")
endif()
list(APPEND tests "${prefix}${test}${suffix}")
endforeach()

Binary file not shown.

Before

Width:  |  Height:  |  Size: 25 KiB

View File

@@ -4,40 +4,38 @@
To get the most out of Catch2, start with the [tutorial](tutorial.md#top).
Once you're up and running consider the following reference material.
**Writing tests:**
Writing tests:
* [Assertion macros](assertions.md#top)
* [Matchers (asserting complex properties)](matchers.md#top)
* [Comparing floating point numbers](comparing-floating-point-numbers.md#top)
* [Matchers](matchers.md#top)
* [Logging macros](logging.md#top)
* [Test cases and sections](test-cases-and-sections.md#top)
* [Test fixtures](test-fixtures.md#top)
* [Explicitly skipping, passing, and failing tests at runtime](skipping-passing-failing.md#top)
* [Reporters (output customization)](reporters.md#top)
* [Reporters](reporters.md#top)
* [Event Listeners](event-listeners.md#top)
* [Data Generators (value parameterized tests)](generators.md#top)
* [Data Generators](generators.md#top)
* [Other macros](other-macros.md#top)
* [Micro benchmarking](benchmarks.md#top)
**Fine tuning:**
Fine tuning:
* [Supplying your own main()](own-main.md#top)
* [Compile-time configuration](configuration.md#top)
* [String Conversions](tostring.md#top)
**Running:**
Running:
* [Command line](command-line.md#top)
**Odds and ends:**
* [Frequently Asked Questions (FAQ)](faq.md#top)
* [Best practices and other tips](usage-tips.md#top)
Odds and ends:
* [CMake integration](cmake-integration.md#top)
* [Tooling integration (CI, test runners, other)](ci-and-misc.md#top)
* [Known limitations](limitations.md#top)
* [CI and other miscellaneous pieces](ci-and-misc.md#top)
**Other:**
* [Why Catch2?](why-catch.md#top)
* [Migrating from v2 to v3](migrate-v2-to-v3.md#top)
* [Open Source Projects using Catch2](opensource-users.md#top)
* [Commercial Projects using Catch2](commercial-users.md#top)
FAQ:
* [Why are my tests slow to compile?](slow-compiles.md#top)
* [Known limitations](limitations.md#top)
Other:
* [Why Catch?](why-catch.md#top)
* [Open Source Projects using Catch](opensource-users.md#top)
* [Commercial Projects using Catch](commercial-users.md#top)
* [Contributing](contributing.md#top)
* [Release Notes](release-notes.md#top)
* [Deprecations and incoming changes](deprecations.md#top)

View File

@@ -3,7 +3,6 @@
**Contents**<br>
[Natural Expressions](#natural-expressions)<br>
[Floating point comparisons](#floating-point-comparisons)<br>
[Exceptions](#exceptions)<br>
[Matcher expressions](#matcher-expressions)<br>
[Thread Safety](#thread-safety)<br>
@@ -20,7 +19,7 @@ Most of these macros come in two forms:
The ```REQUIRE``` family of macros tests an expression and aborts the test case if it fails.
The ```CHECK``` family are equivalent but execution continues in the same test case even if the assertion fails. This is useful if you have a series of essentially orthogonal assertions and it is useful to see all the results rather than stopping at the first failure.
* **REQUIRE(** _expression_ **)** and
* **REQUIRE(** _expression_ **)** and
* **CHECK(** _expression_ **)**
Evaluates the expression and records the result. If an exception is thrown, it is caught, reported, and counted as a failure. These are the macros you will use most of the time.
@@ -32,78 +31,98 @@ CHECK( thisReturnsTrue() );
REQUIRE( i == 42 );
```
Expressions prefixed with `!` cannot be decomposed. If you have a type
that is convertible to bool and you want to assert that it evaluates to
false, use the two forms below:
* **REQUIRE_FALSE(** _expression_ **)** and
* **REQUIRE_FALSE(** _expression_ **)** and
* **CHECK_FALSE(** _expression_ **)**
Note that there is no reason to use these forms for plain bool variables,
because there is no added value in decomposing them.
Evaluates the expression and records the _logical NOT_ of the result. If an exception is thrown it is caught, reported, and counted as a failure.
(these forms exist as a workaround for the fact that ! prefixed expressions cannot be decomposed).
Example:
```cpp
Status ret = someFunction();
REQUIRE_FALSE(ret); // ret must evaluate to false, and Catch2 will print
// out the value of ret if possibly
```
REQUIRE_FALSE( thisReturnsFalse() );
```
Do note that "overly complex" expressions cannot be decomposed and thus will not compile. This is done partly for practical reasons (to keep the underlying expression template machinery to minimum) and partly for philosophical reasons (assertions should be simple and deterministic).
### Other limitations
Note that expressions containing either of the binary logical operators,
`&&` or `||`, cannot be decomposed and will not compile. The reason behind
this is that it is impossible to overload `&&` and `||` in a way that
keeps their short-circuiting semantics, and expression decomposition
relies on overloaded operators to work.
Simple example of an issue with overloading binary logical operators
is a common pointer idiom, `p && p->foo == 2`. Using the built-in `&&`
operator, `p` is only dereferenced if it is not null. With overloaded
`&&`, `p` is always dereferenced, thus causing a segfault if
`p == nullptr`.
If you want to test expression that contains `&&` or `||`, you have two
options.
1) Enclose it in parentheses. Parentheses force evaluation of the expression
before the expression decomposition can touch it, and thus it cannot
be used.
2) Rewrite the expression. `REQUIRE(a == 1 && b == 2)` can always be split
into `REQUIRE(a == 1); REQUIRE(b == 2);`. Alternatively, if this is a
common pattern in your tests, think about using [Matchers](#matcher-expressions).
instead. There is no simple rewrite rule for `||`, but I generally
believe that if you have `||` in your test expression, you should rethink
your tests.
Examples:
* `CHECK(a == 1 && b == 2);`
This expression is too complex because of the `&&` operator. If you want to check that 2 or more properties hold, you can either put the expression into parenthesis, which stops decomposition from working, or you need to decompose the expression into two assertions: `CHECK( a == 1 ); CHECK( b == 2);`
* `CHECK( a == 2 || b == 1 );`
This expression is too complex because of the `||` operator. If you want to check that one of several properties hold, you can put the expression into parenthesis (unlike with `&&`, expression decomposition into several `CHECK`s is not possible).
## Floating point comparisons
### Floating point comparisons
Comparing floating point numbers is complex, and [so it has its own
documentation page](comparing-floating-point-numbers.md#top).
When comparing floating point numbers - especially if at least one of them has been computed - great care must be taken to allow for rounding errors and inexact representations.
Catch provides a way to perform tolerant comparisons of floating point values through use of a wrapper class called `Approx`. `Approx` can be used on either side of a comparison expression. It overloads the comparisons operators to take a tolerance into account. Here's a simple example:
```cpp
REQUIRE( performComputation() == Approx( 2.1 ) );
```
Catch also provides a user-defined literal for `Approx`; `_a`. It resides in
the `Catch::literals` namespace and can be used like so:
```cpp
using namespace Catch::literals;
REQUIRE( performComputation() == 2.1_a );
```
`Approx` is constructed with defaults that should cover most simple cases.
For the more complex cases, `Approx` provides 3 customization points:
* __epsilon__ - epsilon serves to set the coefficient by which a result
can differ from `Approx`'s value before it is rejected.
_By default set to `std::numeric_limits<float>::epsilon()*100`._
* __margin__ - margin serves to set the the absolute value by which
a result can differ from `Approx`'s value before it is rejected.
_By default set to `0.0`._
* __scale__ - scale is used to change the magnitude of `Approx` for relative check.
_By default set to `0.0`._
#### epsilon example
```cpp
Approx target = Approx(100).epsilon(0.01);
100.0 == target; // Obviously true
200.0 == target; // Obviously still false
100.5 == target; // True, because we set target to allow up to 1% difference
```
#### margin example
```cpp
Approx target = Approx(100).margin(5);
100.0 == target; // Obviously true
200.0 == target; // Obviously still false
104.0 == target; // True, because we set target to allow absolute difference of at most 5
```
#### scale
Scale can be useful if the computation leading to the result worked
on different scale than is used by the results. Since allowed difference
between Approx's value and compared value is based primarily on Approx's value
(the allowed difference is computed as
`(Approx::scale + Approx::value) * epsilon`), the resulting comparison could
need rescaling to be correct.
## Exceptions
* **REQUIRE_NOTHROW(** _expression_ **)** and
* **REQUIRE_NOTHROW(** _expression_ **)** and
* **CHECK_NOTHROW(** _expression_ **)**
Expects that no exception is thrown during evaluation of the expression.
* **REQUIRE_THROWS(** _expression_ **)** and
* **REQUIRE_THROWS(** _expression_ **)** and
* **CHECK_THROWS(** _expression_ **)**
Expects that an exception (of any type) is be thrown during evaluation of the expression.
* **REQUIRE_THROWS_AS(** _expression_, _exception type_ **)** and
* **REQUIRE_THROWS_AS(** _expression_, _exception type_ **)** and
* **CHECK_THROWS_AS(** _expression_, _exception type_ **)**
Expects that an exception of the _specified type_ is thrown during evaluation of the expression. Note that the _exception type_ is extended with `const&` and you should not include it yourself.
* **REQUIRE_THROWS_WITH(** _expression_, _string or string matcher_ **)** and
* **REQUIRE_THROWS_WITH(** _expression_, _string or string matcher_ **)** and
* **CHECK_THROWS_WITH(** _expression_, _string or string matcher_ **)**
Expects that an exception is thrown that, when converted to a string, matches the _string_ or _string matcher_ provided (see next section for Matchers).
@@ -139,8 +158,8 @@ REQUIRE_NOTHROW([&](){
To support Matchers a slightly different form is used. Matchers have [their own documentation](matchers.md#top).
* **REQUIRE_THAT(** _lhs_, _matcher expression_ **)** and
* **CHECK_THAT(** _lhs_, _matcher expression_ **)**
* **REQUIRE_THAT(** _lhs_, _matcher expression_ **)** and
* **CHECK_THAT(** _lhs_, _matcher expression_ **)**
Matchers can be composed using `&&`, `||` and `!` operators.

View File

@@ -1,7 +1,11 @@
<a id="top"></a>
# Authoring benchmarks
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
_Note that benchmarking support is disabled by default and to enable it,
you need to define `CATCH_CONFIG_ENABLE_BENCHMARKING`. For more details,
see the [compile-time configuration documentation](configuration.md#top)._
Writing benchmarks is not easy. Catch simplifies certain aspects but you'll
always need to take care about various aspects. Understanding a few things about
@@ -11,8 +15,7 @@ First off, let's go over some terminology that will be used throughout this
guide.
- *User code*: user code is the code that the user provides to be measured.
- *Run*: one run is one execution of the user code. Sometimes also referred
to as an _iteration_.
- *Run*: one run is one execution of the user code.
- *Sample*: one sample is one data point obtained by measuring the time it takes
to perform a certain number of runs. One sample can consist of more than one
run if the clock available does not have enough resolution to accurately

View File

@@ -1,15 +1,14 @@
<a id="top"></a>
# Tooling integration (CI, test runners and so on)
# CI and other odd pieces
**Contents**<br>
[Continuous Integration systems](#continuous-integration-systems)<br>
[Bazel test runner integration](#bazel-test-runner-integration)<br>
[Other reporters](#other-reporters)<br>
[Low-level tools](#low-level-tools)<br>
[CMake](#cmake)<br>
This page talks about Catch2's integration with other related tooling,
like Continuous Integration and 3rd party test runners.
This page talks about how Catch integrates with Continuous Integration
Build Systems may refer to low-level tools, like CMake, or larger systems that run on servers, like Jenkins or TeamCity. This page will talk about both.
## Continuous Integration systems
@@ -18,9 +17,9 @@ Probably the most important aspect to using Catch with a build server is the use
Two of these reporters are built in (XML and JUnit) and the third (TeamCity) is included as a separate header. It's possible that the other two may be split out in the future too - as that would make the core of Catch smaller for those that don't need them.
### XML Reporter
```-r xml```
```-r xml```
The XML Reporter writes in an XML format that is specific to Catch.
The XML Reporter writes in an XML format that is specific to Catch.
The advantage of this format is that it corresponds well to the way Catch works (especially the more unusual features, such as nested sections) and is a fully streaming format - that is it writes output as it goes, without having to store up all its results before it can start writing.
@@ -35,6 +34,19 @@ The advantage of this format is that the JUnit Ant schema is widely understood b
The disadvantage is that this schema was designed to correspond to how JUnit works - and there is a significant mismatch with how Catch works. Additionally the format is not streamable (because opening elements hold counts of failed and passing tests as attributes) - so the whole test run must complete before it can be written.
## Other reporters
Other reporters are not part of the single-header distribution and need
to be downloaded and included separately. All reporters are stored in
`single_include` directory in the git repository, and are named
`catch_reporter_*.hpp`. For example, to use the TeamCity reporter you
need to download `single_include/catch_reporter_teamcity.hpp` and include
it after Catch itself.
```cpp
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
#include "catch_reporter_teamcity.hpp"
```
### TeamCity Reporter
```-r teamcity```
@@ -57,29 +69,19 @@ Because of the incremental nature of Catch's test suites and ability to run spec
```-r sonarqube```
[SonarQube Generic Test Data](https://docs.sonarqube.org/latest/analysis/generic-test/) XML format for tests metrics.
## Bazel test runner integration
Catch2 understands some of the environment variables Bazel uses to control
test execution. Specifically it understands
* JUnit output path via `XML_OUTPUT_FILE`
* Test filtering via `TESTBRIDGE_TEST_ONLY`
* Test sharding via `TEST_SHARD_INDEX`, `TEST_TOTAL_SHARDS`, and `TEST_SHARD_STATUS_FILE`
> Support for `XML_OUTPUT_FILE` was [introduced](https://github.com/catchorg/Catch2/pull/2399) in Catch2 3.0.1
> Support for `TESTBRIDGE_TEST_ONLY` and sharding was introduced in Catch2 3.2.0
This integration is enabled via either a [compile time configuration
option](configuration.md#bazel-support), or via `BAZEL_TEST` environment
variable set to "1".
> Support for `BAZEL_TEST` was [introduced](https://github.com/catchorg/Catch2/pull/2459) in Catch2 3.1.0
## Low-level tools
### Precompiled headers (PCHs)
Catch offers prototypal support for being included in precompiled headers, but because of its single-header nature it does need some actions by the user:
* The precompiled header needs to define `CATCH_CONFIG_ALL_PARTS`
* The implementation file needs to
* undefine `TWOBLUECUBES_SINGLE_INCLUDE_CATCH_HPP_INCLUDED`
* define `CATCH_CONFIG_IMPL_ONLY`
* define `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`
* include "catch.hpp" again
### CodeCoverage module (GCOV, LCOV...)
If you are using GCOV tool to get testing coverage of your code, and are not sure how to integrate it with CMake and Catch, there should be an external example over at https://github.com/fkromer/catch_cmake_coverage
@@ -93,7 +95,7 @@ can use `pkg-config` to get its include path: `pkg-config --cflags catch2`.
### gdb and lldb scripts
Catch2's `extras` folder also contains two simple debugger scripts,
Catch2's `contrib` folder also contains two simple debugger scripts,
`gdbinit` for `gdb` and `lldbinit` for `lldb`. If loaded into their
respective debugger, these will tell it to step over Catch2's internals
when stepping through code.

View File

@@ -2,10 +2,9 @@
# CMake integration
**Contents**<br>
[CMake targets](#cmake-targets)<br>
[CMake target](#cmake-target)<br>
[Automatic test registration](#automatic-test-registration)<br>
[CMake project options](#cmake-project-options)<br>
[`CATCH_CONFIG_*` customization options in CMake](#catch_config_-customization-options-in-cmake)<br>
[Installing Catch2 from git repository](#installing-catch2-from-git-repository)<br>
[Installing Catch2 from vcpkg](#installing-catch2-from-vcpkg)<br>
@@ -16,33 +15,26 @@ integration points for our users.
2) Catch2's repository contains CMake scripts for automatic registration
of `TEST_CASE`s in CTest
## CMake targets
## CMake target
Catch2's CMake build exports two targets, `Catch2::Catch2`, and
`Catch2::Catch2WithMain`. If you do not need custom `main` function,
you should be using the latter (and only the latter). Linking against
it will add the proper include paths and link your target together with
2 static libraries that implement Catch2 and its main respectively.
If you need custom `main`, you should link only against `Catch2::Catch2`.
Catch2's CMake build exports an interface target `Catch2::Catch2`. Linking
against it will add the proper include path and all necessary capabilities
to the resulting binary.
This means that if Catch2 has been installed on the system, it should
be enough to do
This means that if Catch2 has been installed on the system, it should be
enough to do:
```cmake
find_package(Catch2 3 REQUIRED)
# These tests can use the Catch2-provided main
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)
# These tests need their own main
add_executable(custom-main-tests test.cpp test-main.cpp)
target_link_libraries(custom-main-tests PRIVATE Catch2::Catch2)
find_package(Catch2 REQUIRED)
target_link_libraries(tests Catch2::Catch2)
```
These targets are also provided when Catch2 is used as a subdirectory.
Assuming Catch2 has been cloned to `lib/Catch2`, you only need to replace
the `find_package` call with `add_subdirectory(lib/Catch2)` and the snippet
above still works.
This target is also provided when Catch2 is used as a subdirectory.
Assuming that Catch2 has been cloned to `lib/Catch2`:
```cmake
add_subdirectory(lib/Catch2)
target_link_libraries(tests Catch2::Catch2)
```
Another possibility is to use [FetchContent](https://cmake.org/cmake/help/latest/module/FetchContent.html):
```cmake
@@ -51,31 +43,27 @@ Include(FetchContent)
FetchContent_Declare(
Catch2
GIT_REPOSITORY https://github.com/catchorg/Catch2.git
GIT_TAG v3.0.1 # or a later release
GIT_TAG v2.13.9 # or a later release
)
FetchContent_MakeAvailable(Catch2)
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)
target_link_libraries(tests Catch2::Catch2)
```
## Automatic test registration
Catch2's repository also contains three CMake scripts that help users
Catch2's repository also contains two CMake scripts that help users
with automatically registering their `TEST_CASE`s with CTest. They
can be found in the `extras` folder, and are
can be found in the `contrib` folder, and are
1) `Catch.cmake` (and its dependency `CatchAddTests.cmake`)
2) `ParseAndAddCatchTests.cmake` (deprecated)
3) `CatchShardTests.cmake` (and its dependency `CatchShardTestsImpl.cmake`)
If Catch2 has been installed in system, both of these can be used after
doing `find_package(Catch2 REQUIRED)`. Otherwise you need to add them
to your CMake module path.
<a id="catch_discover_tests"></a>
### `Catch.cmake` and `CatchAddTests.cmake`
`Catch.cmake` provides function `catch_discover_tests` to get tests from
@@ -90,29 +78,17 @@ cmake_minimum_required(VERSION 3.5)
project(baz LANGUAGES CXX VERSION 0.0.1)
find_package(Catch2 REQUIRED)
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2)
add_executable(foo test.cpp)
target_link_libraries(foo Catch2::Catch2)
include(CTest)
include(Catch)
catch_discover_tests(tests)
catch_discover_tests(foo)
```
When using `FetchContent`, `include(Catch)` will fail unless
`CMAKE_MODULE_PATH` is explicitly updated to include the extras
directory.
```cmake
# ... FetchContent ...
#
list(APPEND CMAKE_MODULE_PATH ${catch2_SOURCE_DIR}/extras)
include(CTest)
include(Catch)
catch_discover_tests(tests)
```
#### Customization
`catch_discover_tests` can be given several extra arguments:
`catch_discover_tests` can be given several extra argumets:
```cmake
catch_discover_tests(target
[TEST_SPEC arg1...]
@@ -202,7 +178,7 @@ the output file name e.g. ".xml".
### `ParseAndAddCatchTests.cmake`
⚠ This script is [deprecated](https://github.com/catchorg/Catch2/pull/2120)
in Catch2 2.13.4 and superseded by the above approach using `catch_discover_tests`.
in Catch 2.13.4 and superseded by the above approach using `catch_discover_tests`.
See [#2092](https://github.com/catchorg/Catch2/issues/2092) for details.
`ParseAndAddCatchTests` works by parsing all implementation files
@@ -222,12 +198,12 @@ cmake_minimum_required(VERSION 3.5)
project(baz LANGUAGES CXX VERSION 0.0.1)
find_package(Catch2 REQUIRED)
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2)
add_executable(foo test.cpp)
target_link_libraries(foo Catch2::Catch2)
include(CTest)
include(ParseAndAddCatchTests)
ParseAndAddCatchTests(tests)
ParseAndAddCatchTests(foo)
```
@@ -237,7 +213,7 @@ ParseAndAddCatchTests(tests)
* `PARSE_CATCH_TESTS_VERBOSE` -- When `ON`, the script prints debug
messages. Defaults to `OFF`.
* `PARSE_CATCH_TESTS_NO_HIDDEN_TESTS` -- When `ON`, hidden tests (tests
tagged with either of `[.]` or `[.foo]`) will not be registered.
tagged with any of `[!hide]`, `[.]` or `[.foo]`) will not be registered.
Defaults to `OFF`.
* `PARSE_CATCH_TESTS_ADD_FIXTURE_IN_TEST_NAME` -- When `ON`, adds fixture
class name to the test name in CTest. Defaults to `ON`.
@@ -259,68 +235,10 @@ unset(OptionalCatchTestLauncher)
ParseAndAddCatchTests(bar)
```
### `CatchShardTests.cmake`
> `CatchShardTests.cmake` was introduced in Catch2 3.1.0.
`CatchShardTests.cmake` provides a function
`catch_add_sharded_tests(TEST_BINARY)` that splits tests from `TEST_BINARY`
into multiple shards. The tests in each shard and their order is randomized,
and the seed changes every invocation of CTest.
Currently there are 3 customization points for this script:
* SHARD_COUNT - number of shards to split target's tests into
* REPORTER - reporter spec to use for tests
* TEST_SPEC - test spec used for filtering tests
Example usage:
```
include(CatchShardTests)
catch_add_sharded_tests(foo-tests
SHARD_COUNT 4
REPORTER "xml::out=-"
TEST_SPEC "A"
)
catch_add_sharded_tests(tests
SHARD_COUNT 8
REPORTER "xml::out=-"
TEST_SPEC "B"
)
```
This registers total of 12 CTest tests (4 + 8 shards) to run shards
from `foo-tests` test binary, filtered by a test spec.
_Note that this script is currently a proof-of-concept for reseeding
shards per CTest run, and thus does not support (nor does it currently
aim to support) all customization points from
[`catch_discover_tests`](#catch_discover_tests)._
## CMake project options
Catch2's CMake project also provides some options for other projects
that consume it. These are:
* `BUILD_TESTING` -- When `ON` and the project is not used as a subproject,
Catch2's test binary will be built. Defaults to `ON`.
* `CATCH_INSTALL_DOCS` -- When `ON`, Catch2's documentation will be
included in the installation. Defaults to `ON`.
* `CATCH_INSTALL_EXTRAS` -- When `ON`, Catch2's extras folder (the CMake
scripts mentioned above, debugger helpers) will be included in the
installation. Defaults to `ON`.
* `CATCH_DEVELOPMENT_BUILD` -- When `ON`, configures the build for development
of Catch2. This means enabling test projects, warnings and so on.
Defaults to `OFF`.
Enabling `CATCH_DEVELOPMENT_BUILD` also enables further configuration
customization options:
that consume it. These are
* `CATCH_BUILD_TESTING` -- When `ON`, Catch2's SelfTest project will be
built. Defaults to `ON`. Note that Catch2 also obeys `BUILD_TESTING` CMake
@@ -328,40 +246,12 @@ variable, so _both_ of them need to be `ON` for the SelfTest to be built,
and either of them can be set to `OFF` to disable building SelfTest.
* `CATCH_BUILD_EXAMPLES` -- When `ON`, Catch2's usage examples will be
built. Defaults to `OFF`.
* `CATCH_BUILD_EXTRA_TESTS` -- When `ON`, Catch2's extra tests will be
built. Defaults to `OFF`.
* `CATCH_BUILD_FUZZERS` -- When `ON`, Catch2 fuzzing entry points will
be built. Defaults to `OFF`.
* `CATCH_ENABLE_WERROR` -- When `ON`, adds `-Werror` or equivalent flag
to the compilation. Defaults to `ON`.
* `CATCH_BUILD_SURROGATES` -- When `ON`, each header in Catch2 will be
compiled separately to ensure that they are self-sufficient.
Defaults to `OFF`.
## `CATCH_CONFIG_*` customization options in CMake
> CMake support for `CATCH_CONFIG_*` options was introduced in Catch2 3.0.1
Due to the new separate compilation model, all the options from the
[Compile-time configuration docs](configuration.md#top) can also be set
through Catch2's CMake. To set them, define the option you want as `ON`,
e.g. `-DCATCH_CONFIG_NOSTDOUT=ON`.
Note that setting the option to `OFF` doesn't disable it. To force disable
an option, you need to set the `_NO_` form of it to `ON`, e.g.
`-DCATCH_CONFIG_NO_COLOUR_WIN32=ON`.
To summarize the configuration option behaviour with an example:
| `-DCATCH_CONFIG_COLOUR_WIN32` | `-DCATCH_CONFIG_NO_COLOUR_WIN32` | Result |
|-------------------------------|----------------------------------|-------------|
| `ON` | `ON` | error |
| `ON` | `OFF` | force-on |
| `OFF` | `ON` | force-off |
| `OFF` | `OFF` | auto-detect |
* `CATCH_INSTALL_DOCS` -- When `ON`, Catch2's documentation will be
included in the installation. Defaults to `ON`.
* `CATCH_INSTALL_HELPERS` -- When `ON`, Catch2's contrib folder will be
included in the installation. Defaults to `ON`.
* `BUILD_TESTING` -- When `ON` and the project is not used as a subproject,
Catch2's test binary will be built. Defaults to `ON`.
## Installing Catch2 from git repository

View File

@@ -15,11 +15,11 @@
[Warnings](#warnings)<br>
[Reporting timings](#reporting-timings)<br>
[Load test names to run from a file](#load-test-names-to-run-from-a-file)<br>
[Just test names](#just-test-names)<br>
[Specify the order test cases are run](#specify-the-order-test-cases-are-run)<br>
[Specify a seed for the Random Number Generator](#specify-a-seed-for-the-random-number-generator)<br>
[Identify framework and version according to the libIdentify standard](#identify-framework-and-version-according-to-the-libidentify-standard)<br>
[Wait for key before continuing](#wait-for-key-before-continuing)<br>
[Skip all benchmarks](#skip-all-benchmarks)<br>
[Specify the number of benchmark samples to collect](#specify-the-number-of-benchmark-samples-to-collect)<br>
[Specify the number of resamples for bootstrapping](#specify-the-number-of-resamples-for-bootstrapping)<br>
[Specify the confidence-interval for bootstrapping](#specify-the-confidence-interval-for-bootstrapping)<br>
@@ -29,15 +29,14 @@
[Specify the section to run](#specify-the-section-to-run)<br>
[Filenames as tags](#filenames-as-tags)<br>
[Override output colouring](#override-output-colouring)<br>
[Test Sharding](#test-sharding)<br>
[Allow running the binary without tests](#allow-running-the-binary-without-tests)<br>
[Output verbosity](#output-verbosity)<br>
Catch works quite nicely without any command line options at all - but for those times when you want greater control the following options are available.
Click one of the following links to take you straight to that option - or scroll on to browse the available options.
<a href="#specifying-which-tests-to-run"> ` <test-spec> ...`</a><br />
<a href="#usage"> ` -h, -?, --help`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -l, --list-tests`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -t, --list-tags`</a><br />
<a href="#showing-results-for-successful-tests"> ` -s, --success`</a><br />
<a href="#breaking-into-the-debugger"> ` -b, --break`</a><br />
<a href="#eliding-assertions-expected-to-throw"> ` -e, --nothrow`</a><br />
@@ -56,25 +55,18 @@ Click one of the following links to take you straight to that option - or scroll
</br>
<a href="#listing-available-tests-tags-or-reporters"> ` --list-tests`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-tags`</a><br />
<a href="#list-test-names-only"> ` --list-test-names-only`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-reporters`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-listeners`</a><br />
<a href="#order"> ` --order`</a><br />
<a href="#rng-seed"> ` --rng-seed`</a><br />
<a href="#libidentify"> ` --libidentify`</a><br />
<a href="#wait-for-keypress"> ` --wait-for-keypress`</a><br />
<a href="#skip-benchmarks"> ` --skip-benchmarks`</a><br />
<a href="#benchmark-samples"> ` --benchmark-samples`</a><br />
<a href="#benchmark-resamples"> ` --benchmark-resamples`</a><br />
<a href="#benchmark-confidence-interval"> ` --benchmark-confidence-interval`</a><br />
<a href="#benchmark-no-analysis"> ` --benchmark-no-analysis`</a><br />
<a href="#benchmark-warmup-time"> ` --benchmark-warmup-time`</a><br />
<a href="#colour-mode"> ` --colour-mode`</a><br />
<a href="#test-sharding"> ` --shard-count`</a><br />
<a href="#test-sharding"> ` --shard-index`</a><br />
<a href=#no-tests-override> ` --allow-running-no-tests`</a><br />
<a href=#output-verbosity> ` --verbosity`</a><br />
<a href="#use-colour"> ` --use-colour`</a><br />
</br>
@@ -101,8 +93,7 @@ Inclusions and exclusions are evaluated in left-to-right order.
Test case examples:
```
thisTestOnly Matches the test case called, 'thisTestOnly'
<pre>thisTestOnly Matches the test case called, 'thisTestOnly'
"this test only" Matches the test case called, 'this test only'
these* Matches all cases starting with 'these'
exclude:notThis Matches all tests except, 'notThis'
@@ -110,9 +101,8 @@ exclude:notThis Matches all tests except, 'notThis'
~*private* Matches all tests except those that contain 'private'
a* ~ab* abc Matches all tests that start with 'a', except those that
start with 'ab', except 'abc', which is included
~[tag1] Matches all tests except those tagged with '[tag1]'
-# [#somefile] Matches all tests from the file 'somefile.cpp'
```
</pre>
Names within square brackets are interpreted as tags.
A series of tags form an AND expression whereas a comma-separated sequence forms an OR expression. e.g.:
@@ -126,48 +116,18 @@ Test names containing special characters, such as `,` or `[` can specify them on
<a id="choosing-a-reporter-to-use"></a>
## Choosing a reporter to use
<pre>-r, --reporter &lt;reporter[::key=value]*&gt;</pre>
<pre>-r, --reporter &lt;reporter></pre>
Reporters are how the output from Catch2 (results of assertions, tests,
benchmarks and so on) is formatted and written out. The default reporter
is called the "Console" reporter and is intended to provide relatively
verbose and human-friendly output.
A reporter is an object that formats and structures the output of running tests, and potentially summarises the results. By default a console reporter is used that writes, IDE friendly, textual output. Catch comes bundled with some alternative reporters, but more can be added in client code.<br />
The bundled reporters are:
Reporters are also individually configurable. To pass configuration options
to the reporter, you append `::key=value` to the reporter specification
as many times as you want, e.g. `--reporter xml::out=someFile.xml`.
The keys must either be prefixed by "X", in which case they are not parsed
by Catch2 and are only passed down to the reporter, or one of options
hardcoded into Catch2. Currently there are only 2,
["out"](#sending-output-to-a-file), and ["colour-mode"](#colour-mode).
_Note that the reporter might still check the X-prefixed options for
validity, and throw an error if they are wrong._
> Support for passing arguments to reporters through the `-r`, `--reporter` flag was introduced in Catch2 3.0.1
There are multiple built-in reporters, you can see what they do by using the
[`--list-reporters`](command-line.md#listing-available-tests-tags-or-reporters)
flag. If you need a reporter providing custom format outside of the already
provided ones, look at the ["write your own reporter" part of the reporter
documentation](reporters.md#writing-your-own-reporter).
This option may be passed multiple times to use multiple (different)
reporters at the same time. See the [reporter documentation](reporters.md#multiple-reporters)
for details on what the resulting behaviour is. Also note that at most one
reporter can be provided without the output-file part of reporter spec.
This reporter will use the "default" output destination, based on
the [`-o`, `--out`](#sending-output-to-a-file) option.
> Support for using multiple different reporters at the same time was [introduced](https://github.com/catchorg/Catch2/pull/2183) in Catch2 3.0.1
_Note: There is currently no way to escape `::` in the reporter spec,
and thus the reporter names, or configuration keys and values, cannot
contain `::`. As `::` in paths is relatively obscure (unlike ':'), we do
not consider this an issue._
<pre>-r console
-r compact
-r xml
-r junit
</pre>
The JUnit reporter is an xml format that follows the structure of the JUnit XML Report ANT task, as consumed by a number of third-party tools, including Continuous Integration servers such as Hudson. If not otherwise needed, the standard XML reporter is preferred as this is a streaming reporter, whereas the Junit reporter needs to hold all its results until the end so it can write the overall results into attributes of the root node.
<a id="breaking-into-the-debugger"></a>
## Breaking into the debugger
@@ -197,62 +157,24 @@ Sometimes this results in a flood of failure messages and you'd rather just see
<a id="listing-available-tests-tags-or-reporters"></a>
## Listing available tests, tags or reporters
```
--list-tests
--list-tags
<pre>-l, --list-tests
-t, --list-tags
--list-reporters
--list-listeners
```
</pre>
> The `--list*` options became customizable through reporters in Catch2 3.0.1
```-l``` or ```--list-tests``` will list all registered tests, along with any tags.
If one or more test-specs have been supplied too then only the matching tests will be listed.
> The `--list-listeners` option was added in Catch2 3.0.1
```-t``` or ```--list-tags``` lists all available tags, along with the number of test cases they match. Again, supplying test specs limits the tags that match.
`--list-tests` lists all registered tests matching specified test spec.
Usually this listing also includes tags, and potentially also other
information, like source location, based on verbosity and reporter's design.
`--list-tags` lists all tags from registered tests matching specified test
spec. Usually this also includes number of tests cases they match and
similar information.
`--list-reporters` lists all available reporters and their descriptions.
`--list-listeners` lists all registered listeners and their descriptions.
The [`--verbosity` argument](#output-verbosity) modifies the level of detail provided by the default `--list*` options
as follows:
| Option | `normal` (default) | `quiet` | `high` |
|--------------------|---------------------------------|---------------------|-----------------------------------------|
| `--list-tests` | Test names and tags | Test names only | Same as `normal`, plus source code line |
| `--list-tags` | Tags and counts | Same as `normal` | Same as `normal` |
| `--list-reporters` | Reporter names and descriptions | Reporter names only | Same as `normal` |
| `--list-listeners` | Listener names and descriptions | Same as `normal` | Same as `normal` |
```--list-reporters``` lists the available reporters.
<a id="sending-output-to-a-file"></a>
## Sending output to a file
<pre>-o, --out &lt;filename&gt;
<pre>-o, --out &lt;filename>
</pre>
Use this option to send all output to a file, instead of stdout. You can
use `-` as the filename to explicitly send the output to stdout (this is
useful e.g. when using multiple reporters).
> Support for `-` as the filename was introduced in Catch2 3.0.1
Filenames starting with "%" (percent symbol) are reserved by Catch2 for
meta purposes, e.g. using `%debug` as the filename opens stream that
writes to platform specific debugging/logging mechanism.
Catch2 currently recognizes 3 meta streams:
* `%debug` - writes to platform specific debugging/logging output
* `%stdout` - writes to stdout
* `%stderr` - writes to stderr
> Support for `%stdout` and `%stderr` was introduced in Catch2 3.0.1
Use this option to send all output to a file. By default output is sent to stdout (note that uses of stdout and stderr *from within test cases* are redirected and included in the report - so even stderr will effectively end up on stdout).
<a id="naming-a-test-run"></a>
## Naming a test run
@@ -283,24 +205,16 @@ This option transforms tabs and newline characters into ```\t``` and ```\n``` re
## Warnings
<pre>-w, --warn &lt;warning name></pre>
You can think of Catch2's warnings as the equivalent of `-Werror` (`/WX`)
flag for C++ compilers. It turns some suspicious occurrences, like a section
without assertions, into errors. Because these might be intended, warnings
are not enabled by default, but user can opt in.
You can enable multiple warnings at the same time.
There are currently two warnings implemented:
Enables reporting of suspicious test states. There are currently two
available warnings
```
NoAssertions // Fail test case / leaf section if no assertions
// (e.g. `REQUIRE`) is encountered.
UnmatchedTestSpec // Fail test run if any of the CLI test specs did
// not match any tests.
NoAssertions // Fail test case / leaf section if no assertions
// (e.g. `REQUIRE`) is encountered.
NoTests // Return non-zero exit code when no test cases were run
// Also calls reporter's noMatchingTestCases method
```
> `UnmatchedTestSpec` was introduced in Catch2 3.0.1.
<a id="reporting-timings"></a>
## Reporting timings
@@ -310,10 +224,10 @@ When set to ```yes``` Catch will report the duration of each test case, in milli
<pre>-D, --min-duration &lt;value></pre>
> `--min-duration` was [introduced](https://github.com/catchorg/Catch2/pull/1910) in Catch2 2.13.0
> `--min-duration` was [introduced](https://github.com/catchorg/Catch2/pull/1910) in Catch 2.13.0
When set, Catch will report the duration of each test case that took more
than &lt;value> seconds, in milliseconds. This option is overridden by both
than &lt;value> seconds, in milliseconds. This option is overriden by both
`-d yes` and `-d no`, so that either all durations are reported, or none
are.
@@ -322,13 +236,15 @@ are.
## Load test names to run from a file
<pre>-f, --input-file &lt;filename></pre>
Provide the name of a file that contains a list of test case names,
one per line. Blank lines are skipped.
Provide the name of a file that contains a list of test case names - one per line. Blank lines are skipped and anything after the comment character, ```#```, is ignored.
A useful way to generate an initial instance of this file is to combine
the [`--list-tests`](#listing-available-tests-tags-or-reporters) flag with
the [`--verbosity quiet`](#output-verbosity) option. You can also
use test specs to filter this list down to what you want first.
A useful way to generate an initial instance of this file is to use the <a href="#list-test-names-only">list-test-names-only</a> option. This can then be manually curated to specify a specific subset of tests - or in a specific order.
<a id="list-test-names-only"></a>
## Just test names
<pre>--list-test-names-only</pre>
This option lists all available tests in a non-indented form, one on each line. This makes it ideal for saving to a file and feeding back into the <a href="#input-file">```-f``` or ```--input-file```</a> option.
<a id="order"></a>
@@ -339,8 +255,8 @@ Test cases are ordered one of three ways:
### decl
Declaration order (this is the default order if no --order argument is provided).
Tests in the same translation unit are sorted using their declaration orders,
different TUs are sorted in an implementation (linking) dependent order.
Tests in the same TU are sorted using their declaration orders, different
TUs are in an implementation (linking) dependent order.
### lex
@@ -349,36 +265,23 @@ Lexicographic order. Tests are sorted by their name, their tags are ignored.
### rand
Randomly ordered. The order is dependent on Catch2's random seed (see
Randomly sorted. The order is dependent on Catch2's random seed (see
[`--rng-seed`](#rng-seed)), and is subset invariant. What this means
is that as long as the random seed is fixed, running only some tests
(e.g. via tag) does not change their relative order.
> The subset stability was introduced in Catch2 v2.12.0
Since the random order was made subset stable, we promise that given
the same random seed, the order of test cases will be the same across
different platforms, as long as the tests were compiled against identical
version of Catch2. We reserve the right to change the relative order
of tests cases between Catch2 versions, but it is unlikely to happen often.
<a id="rng-seed"></a>
## Specify a seed for the Random Number Generator
<pre>--rng-seed &lt;'time'|'random-device'|number&gt;</pre>
<pre>--rng-seed &lt;'time'|number&gt;</pre>
Sets the seed for random number generators used by Catch2. These are used
e.g. to shuffle tests when user asks for tests to be in random order.
Using `time` as the argument asks Catch2 generate the seed through call
to `std::time(nullptr)`. This provides very weak randomness and multiple
runs of the binary can generate the same seed if they are started close
to each other.
Using `random-device` asks for `std::random_device` to be used instead.
If your implementation provides working `std::random_device`, it should
be preferred to using `time`. Catch2 uses `std::random_device` by default.
Sets a seed for the random number generator using ```std::srand()```.
If a number is provided this is used directly as the seed so the random pattern is repeatable.
Alternatively if the keyword ```time``` is provided then the result of calling ```std::time(0)``` is used and so the pattern becomes unpredictable. In some cases, you might need to pass the keyword ```time``` in double quotes instead of single quotes.
In either case the actual value for the seed is printed as part of Catch's output so if an issue is discovered that is sensitive to test ordering the ordering can be reproduced - even if it was originally seeded from ```std::time(0)```.
<a id="libidentify"></a>
## Identify framework and version according to the libIdentify standard
@@ -393,21 +296,11 @@ See [The LibIdentify repo for more information and examples](https://github.com/
Will cause the executable to print a message and wait until the return/ enter key is pressed before continuing -
either before running any tests, after running all tests - or both, depending on the argument.
<a id="skip-benchmarks"></a>
## Skip all benchmarks
<pre>--skip-benchmarks</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/2408) in Catch2 3.0.1.
This flag tells Catch2 to skip running all benchmarks. Benchmarks in this
case mean code blocks in `BENCHMARK` and `BENCHMARK_ADVANCED` macros, not
test cases with the `[!benchmark]` tag.
<a id="benchmark-samples"></a>
## Specify the number of benchmark samples to collect
<pre>--benchmark-samples &lt;# of samples&gt;</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
When running benchmarks a number of "samples" is collected. This is the base data for later statistical analysis.
Per sample a clock resolution dependent number of iterations of the user code is run, which is independent of the number of samples. Defaults to 100.
@@ -416,7 +309,7 @@ Per sample a clock resolution dependent number of iterations of the user code is
## Specify the number of resamples for bootstrapping
<pre>--benchmark-resamples &lt;# of resamples&gt;</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
After the measurements are performed, statistical [bootstrapping] is performed
on the samples. The number of resamples for that bootstrapping is configurable
@@ -431,7 +324,7 @@ defaults to 95%).
## Specify the confidence-interval for bootstrapping
<pre>--benchmark-confidence-interval &lt;confidence-interval&gt;</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
The confidence-interval is used for statistical bootstrapping on the samples to
calculate the upper and lower bounds of mean and standard deviation.
@@ -441,7 +334,7 @@ Must be between 0 and 1 and defaults to 0.95.
## Disable statistical analysis of collected benchmark samples
<pre>--benchmark-no-analysis</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
When this flag is specified no bootstrapping or any other statistical analysis is performed.
Instead the user code is only measured and the plain mean from the samples is reported.
@@ -450,7 +343,7 @@ Instead the user code is only measured and the plain mean from the samples is re
## Specify the amount of time in milliseconds spent on warming up each test
<pre>--benchmark-warmup-time</pre>
> [Introduced](https://github.com/catchorg/Catch2/pull/1844) in Catch2 2.11.2.
> [Introduced](https://github.com/catchorg/Catch2/pull/1844) in Catch 2.11.2.
Configure the amount of time spent warming up each test.
@@ -499,7 +392,7 @@ There are some limitations of this feature to be aware of:
- Code outside of sections being skipped will still be executed - e.g. any set-up code in the TEST_CASE before the
start of the first section.</br>
- At time of writing, wildcards are not supported in section names.
- If you specify a section without narrowing to a test case first then all test cases will be executed
- If you specify a section without narrowing to a test case first then all test cases will be executed
(but only matching sections within them).
@@ -507,81 +400,21 @@ start of the first section.</br>
## Filenames as tags
<pre>-#, --filenames-as-tags</pre>
This option adds an extra tag to all test cases. The tag is `#` followed
by the unqualified filename the test case is defined in, with the _last_
extension stripped out.
When this option is used then every test is given an additional tag which is formed of the unqualified
filename it is found in, with any extension stripped, prefixed with the `#` character.
For example, tests within the file `tests\SelfTest\UsageTests\BDD.tests.cpp`
will be given the `[#BDD.tests]` tag.
So, for example, tests within the file `~\Dev\MyProject\Ferrets.cpp` would be tagged `[#Ferrets]`.
<a id="colour-mode"></a>
<a id="use-colour"></a>
## Override output colouring
<pre>--colour-mode &lt;ansi|win32|none|default&gt;</pre>
<pre>--use-colour &lt;yes|no|auto&gt;</pre>
> The `--colour-mode` option replaced the old `--colour` option in Catch2 3.0.1
Catch2 support two different ways of colouring terminal output, and by
default it attempts to make a good guess on which implementation to use
(and whether to even use it, e.g. Catch2 tries to avoid writing colour
codes when writing the results into a file).
`--colour-mode` allows the user to explicitly select what happens.
* `--colour-mode ansi` tells Catch2 to always use ANSI colour codes, even
when writing to a file
* `--colour-mode win32` tells Catch2 to use colour implementation based
on Win32 terminal API
* `--colour-mode none` tells Catch2 to disable colours completely
* `--colour-mode default` lets Catch2 decide
`--colour-mode default` is the default setting.
<a id="test-sharding"></a>
## Test Sharding
<pre>--shard-count <#number of shards>, --shard-index <#shard index to run></pre>
> [Introduced](https://github.com/catchorg/Catch2/pull/2257) in Catch2 3.0.1.
When `--shard-count <#number of shards>` is used, the tests to execute
will be split evenly in to the given number of sets, identified by indices
starting at 0. The tests in the set given by
`--shard-index <#shard index to run>` will be executed. The default shard
count is `1`, and the default index to run is `0`.
_Shard index must be less than number of shards. As the name suggests,
it is treated as an index of the shard to run._
Sharding is useful when you want to split test execution across multiple
processes, as is done with the [Bazel test sharding](https://docs.bazel.build/versions/main/test-encyclopedia.html#test-sharding).
<a id="no-tests-override"></a>
## Allow running the binary without tests
<pre>--allow-running-no-tests</pre>
> Introduced in Catch2 3.0.1.
By default, Catch2 test binaries return non-0 exit code if no tests were run,
e.g. if the binary was compiled with no tests, the provided test spec matched no
tests, or all tests [were skipped at runtime](skipping-passing-failing.md#top). This flag
overrides that, so a test run with no tests still returns 0.
## Output verbosity
```
-v, --verbosity <quiet|normal|high>
```
Changing verbosity might change how many details Catch2's reporters output.
However, you should consider changing the verbosity level as a _suggestion_.
Not all reporters support all verbosity levels, e.g. because the reporter's
format cannot meaningfully change. In that case, the verbosity level is
ignored.
Verbosity defaults to _normal_.
Catch colours output for terminals, but omits colouring when it detects that
output is being sent to a pipe. This is done to avoid interfering with automated
processing of output.
`--use-colour yes` forces coloured output, `--use-colour no` disables coloured
output. The default behaviour is `--use-colour auto`.
---

View File

@@ -1,23 +1,22 @@
<a id="top"></a>
# Commercial users of Catch2
# Commercial users of Catch
Catch2 is also widely used in proprietary code bases. This page contains
some of them that are willing to share this information.
If you want to add your organisation, please check that there is no issue
with you sharing this fact.
As well as [Open Source](opensource-users.md#top) users Catch is widely used within proprietary code bases too.
Many organisations like to keep this information internal, and that's fine,
but if you're more open it would be great if we could list the names of as
many organisations as possible that use Catch somewhere in their codebase.
Enterprise environments often tend to be far more conservative in their tool adoption -
and being aware that other companies are using Catch can ease the path in.
So if you are aware of Catch usage in your organisation, and are fairly confident there is no issue with sharing this
fact then please let us know - either directly, via a PR or
[issue](https://github.com/philsquared/Catch/issues), or on the [forums](https://groups.google.com/forum/?fromgroups#!forum/catch-forum).
- Bloomberg
- [Bloomlife](https://bloomlife.com)
- [Inscopix Inc.](https://www.inscopix.com/)
- Locksley.CZ
- [Makimo](https://makimo.pl/)
- NASA
- [Nexus Software Systems](https://nexwebsites.com)
- [Inscopix Inc.](https://www.inscopix.com/)
- [Makimo](https://makimo.pl/)
- [UX3D](https://ux3d.io)
- [King](https://king.com)
---
[Home](Readme.md#top)

View File

@@ -1,192 +0,0 @@
<a id="top"></a>
# Comparing floating point numbers with Catch2
If you are not deeply familiar with them, floating point numbers can be
unintuitive. This also applies to comparing floating point numbers for
(in)equality.
This page assumes that you have some understanding of both FP, and the
meaning of different kinds of comparisons, and only goes over what
functionality Catch2 provides to help you with comparing floating point
numbers. If you do not have this understanding, we recommend that you first
study up on floating point numbers and their comparisons, e.g. by [reading
this blog post](https://codingnest.com/the-little-things-comparing-floating-point-numbers/).
## Floating point matchers
```
#include <catch2/matchers/catch_matchers_floating_point.hpp>
```
[Matchers](matchers.md#top) are the preferred way of comparing floating
point numbers in Catch2. We provide 3 of them:
* `WithinAbs(double target, double margin)`,
* `WithinRel(FloatingPoint target, FloatingPoint eps)`, and
* `WithinULP(FloatingPoint target, uint64_t maxUlpDiff)`.
> `WithinRel` matcher was introduced in Catch2 2.10.0
As with all matchers, you can combine multiple floating point matchers
in a single assertion. For example, to check that some computation matches
a known good value within 0.1% or is close enough (no different to 5
decimal places) to zero, we would write this assertion:
```cpp
REQUIRE_THAT( computation(input),
Catch::Matchers::WithinRel(expected, 0.001)
|| Catch::Matchers::WithinAbs(0, 0.000001) );
```
### WithinAbs
`WithinAbs` creates a matcher that accepts floating point numbers whose
difference with `target` is less-or-equal to the `margin`. Since `float`
can be converted to `double` without losing precision, only `double`
overload exists.
```cpp
REQUIRE_THAT(1.0, WithinAbs(1.2, 0.2));
REQUIRE_THAT(0.f, !WithinAbs(1.0, 0.5));
// Notice that infinity == infinity for WithinAbs
REQUIRE_THAT(INFINITY, WithinAbs(INFINITY, 0));
```
### WithinRel
`WithinRel` creates a matcher that accepts floating point numbers that
are _approximately equal_ to the `target` with a tolerance of `eps.`
Specifically, it matches if
`|arg - target| <= eps * max(|arg|, |target|)` holds. If you do not
specify `eps`, `std::numeric_limits<FloatingPoint>::epsilon * 100`
is used as the default.
```cpp
// Notice that WithinRel comparison is symmetric, unlike Approx's.
REQUIRE_THAT(1.0, WithinRel(1.1, 0.1));
REQUIRE_THAT(1.1, WithinRel(1.0, 0.1));
// Notice that inifnity == infinity for WithinRel
REQUIRE_THAT(INFINITY, WithinRel(INFINITY));
```
### WithinULP
`WithinULP` creates a matcher that accepts floating point numbers that
are no more than `maxUlpDiff`
[ULPs](https://en.wikipedia.org/wiki/Unit_in_the_last_place)
away from the `target` value. The short version of what this means
is that there is no more than `maxUlpDiff - 1` representable floating
point numbers between the argument for matching and the `target` value.
When using the ULP matcher in Catch2, it is important to keep in mind
that Catch2 interprets ULP distance slightly differently than
e.g. `std::nextafter` does.
Catch2's ULP calculation obeys these relations:
* `ulpDistance(-x, x) == 2 * ulpDistance(x, 0)`
* `ulpDistance(-0, 0) == 0` (due to the above)
* `ulpDistance(DBL_MAX, INFINITY) == 1`
* `ulpDistancE(NaN, x) == infinity`
**Important**: The WithinULP matcher requires the platform to use the
[IEEE-754](https://en.wikipedia.org/wiki/IEEE_754) representation for
floating point numbers.
```cpp
REQUIRE_THAT( -0.f, WithinULP( 0.f, 0 ) );
```
## `Approx`
```
#include <catch2/catch_approx.hpp>
```
**We strongly recommend against using `Approx` when writing new code.**
You should be using floating point matchers instead.
Catch2 provides one more way to handle floating point comparisons. It is
`Approx`, a special type with overloaded comparison operators, that can
be used in standard assertions, e.g.
```cpp
REQUIRE(0.99999 == Catch::Approx(1));
```
`Approx` supports four comparison operators, `==`, `!=`, `<=`, `>=`, and can
also be used with strong typedefs over `double`s. It can be used for both
relative and margin comparisons by using its three customization points.
Note that the semantics of this is always that of an _or_, so if either
the relative or absolute margin comparison passes, then the whole comparison
passes.
The downside to `Approx` is that it has a couple of issues that we cannot
fix without breaking backwards compatibility. Because Catch2 also provides
complete set of matchers that implement different floating point comparison
methods, `Approx` is left as-is, is considered deprecated, and should
not be used in new code.
The issues are
* All internal computation is done in `double`s, leading to slightly
different results if the inputs were floats.
* `Approx`'s relative margin comparison is not symmetric. This means
that `Approx( 10 ).epsilon(0.1) != 11.1` but `Approx( 11.1 ).epsilon(0.1) == 10`.
* By default, `Approx` only uses relative margin comparison. This means
that `Approx(0) == X` only passes for `X == 0`.
### Approx details
If you still want/need to know more about `Approx`, read on.
Catch2 provides a UDL for `Approx`; `_a`. It resides in the `Catch::literals`
namespace, and can be used like this:
```cpp
using namespace Catch::literals;
REQUIRE( performComputation() == 2.1_a );
```
`Approx` has three customization points for the comparison:
* **epsilon** - epsilon sets the coefficient by which a result
can differ from `Approx`'s value before it is rejected.
_Defaults to `std::numeric_limits<float>::epsilon()*100`._
```cpp
Approx target = Approx(100).epsilon(0.01);
100.0 == target; // Obviously true
200.0 == target; // Obviously still false
100.5 == target; // True, because we set target to allow up to 1% difference
```
* **margin** - margin sets the absolute value by which
a result can differ from `Approx`'s value before it is rejected.
_Defaults to `0.0`._
```cpp
Approx target = Approx(100).margin(5);
100.0 == target; // Obviously true
200.0 == target; // Obviously still false
104.0 == target; // True, because we set target to allow absolute difference of at most 5
```
* **scale** - scale is used to change the magnitude of `Approx` for the relative check.
_By default, set to `0.0`._
Scale could be useful if the computation leading to the result worked
on a different scale than is used by the results. Approx's scale is added
to Approx's value when computing the allowed relative margin from the
Approx's value.
---
[Home](Readme.md#top)

View File

@@ -2,26 +2,40 @@
# Compile-time configuration
**Contents**<br>
[main()/ implementation](#main-implementation)<br>
[Reporter / Listener interfaces](#reporter--listener-interfaces)<br>
[Prefixing Catch macros](#prefixing-catch-macros)<br>
[Terminal colour](#terminal-colour)<br>
[Console width](#console-width)<br>
[stdout](#stdout)<br>
[Fallback stringifier](#fallback-stringifier)<br>
[Default reporter](#default-reporter)<br>
[Bazel support](#bazel-support)<br>
[C++11 toggles](#c11-toggles)<br>
[C++17 toggles](#c17-toggles)<br>
[Other toggles](#other-toggles)<br>
[Windows header clutter](#windows-header-clutter)<br>
[Enabling stringification](#enabling-stringification)<br>
[Disabling exceptions](#disabling-exceptions)<br>
[Overriding Catch's debug break (`-b`)](#overriding-catchs-debug-break--b)<br>
Catch2 is designed to "just work" as much as possible, and most of the
configuration options below are changed automatically during compilation,
according to the detected environment. However, this detection can also
be overridden by users, using macros documented below, and/or CMake options
with the same name.
Catch is designed to "just work" as much as possible. For most people the only configuration needed is telling Catch which source file should host all the implementation code (```CATCH_CONFIG_MAIN```).
Nonetheless there are still some occasions where finer control is needed. For these occasions Catch exposes a set of macros for configuring how it is built.
## main()/ implementation
CATCH_CONFIG_MAIN // Designates this as implementation file and defines main()
CATCH_CONFIG_RUNNER // Designates this as implementation file
Although Catch is header only it still, internally, maintains a distinction between interface headers and headers that contain implementation. Only one source file in your test project should compile the implementation headers and this is controlled through the use of one of these macros - one of these identifiers should be defined before including Catch in *exactly one implementation file in your project*.
## Reporter / Listener interfaces
CATCH_CONFIG_EXTERNAL_INTERFACES // Brings in necessary headers for Reporter/Listener implementation
Brings in various parts of Catch that are required for user defined Reporters and Listeners. This means that new Reporters and Listeners can be defined in this file as well as in the main file.
Implied by both `CATCH_CONFIG_MAIN` and `CATCH_CONFIG_RUNNER`.
## Prefixing Catch macros
@@ -32,18 +46,19 @@ To keep test code clean and uncluttered Catch uses short macro names (e.g. ```TE
## Terminal colour
CATCH_CONFIG_COLOUR_WIN32 // Force enables compiling colouring impl based on Win32 console API
CATCH_CONFIG_NO_COLOUR_WIN32 // Force disables ...
CATCH_CONFIG_COLOUR_NONE // completely disables all text colouring
CATCH_CONFIG_COLOUR_WINDOWS // forces the Win32 console API to be used
CATCH_CONFIG_COLOUR_ANSI // forces ANSI colour codes to be used
Yes, Catch2 uses the british spelling of colour.
Yes, I am English, so I will continue to spell "colour" with a 'u'.
Catch2 attempts to autodetect whether the Win32 console colouring API,
`SetConsoleTextAttribute`, is available, and if it is available it compiles
in a console colouring implementation that uses it.
When sending output to the terminal, if it detects that it can, Catch will use colourised text. On Windows the Win32 API, ```SetConsoleTextAttribute```, is used. On POSIX systems ANSI colour escape codes are inserted into the stream.
This option can be used to override Catch2's autodetection and force the
compilation either ON or OFF.
For finer control you can define one of the above identifiers (these are mutually exclusive - but that is not checked so may behave unexpectedly if you mix them):
Note that when ANSI colour codes are used "unistd.h" must be includable - along with a definition of ```isatty()```
Typically you should place the ```#define``` before #including "catch.hpp" in your main source file - but if you prefer you can define it for your whole project by whatever your IDE or build system provides for you to do so.
## Console width
@@ -57,7 +72,7 @@ By default a console width of 80 is assumed but this can be controlled by defini
CATCH_CONFIG_NOSTDOUT
To support platforms that do not provide `std::cout`, `std::cerr` and
`std::clog`, Catch does not use them directly, but rather calls
`std::clog`, Catch does not usem the directly, but rather calls
`Catch::cout`, `Catch::cerr` and `Catch::clog`. You can replace their
implementation by defining `CATCH_CONFIG_NOSTDOUT` and implementing
them yourself, their signatures are:
@@ -97,20 +112,6 @@ This means that defining `CATCH_CONFIG_DEFAULT_REPORTER` to `"console"`
is equivalent with the out-of-the-box experience.
## Bazel support
Compiling Catch2 with `CATCH_CONFIG_BAZEL_SUPPORT` force-enables Catch2's
support for Bazel's environment variables (normally Catch2 looks for
`BAZEL_TEST=1` env var first).
This can be useful if you are using older versions of Bazel, that do not
yet have `BAZEL_TEST` env var support.
> `CATCH_CONFIG_BAZEL_SUPPORT` was [introduced](https://github.com/catchorg/Catch2/pull/2399) in Catch2 3.0.1.
> `CATCH_CONFIG_BAZEL_SUPPORT` was [deprecated](https://github.com/catchorg/Catch2/pull/2459) in Catch2 3.1.0.
## C++11 toggles
CATCH_CONFIG_CPP11_TO_STRING // Use `std::to_string`
@@ -126,13 +127,13 @@ Catch's selection, by defining either `CATCH_CONFIG_CPP11_TO_STRING` or
## C++17 toggles
CATCH_CONFIG_CPP17_UNCAUGHT_EXCEPTIONS // Override std::uncaught_exceptions (instead of std::uncaught_exception) support detection
CATCH_CONFIG_CPP17_STRING_VIEW // Override std::string_view support detection (Catch provides a StringMaker specialization by default)
CATCH_CONFIG_CPP17_UNCAUGHT_EXCEPTIONS // Use std::uncaught_exceptions instead of std::uncaught_exception
CATCH_CONFIG_CPP17_STRING_VIEW // Override std::string_view support detection(Catch provides a StringMaker specialization by default)
CATCH_CONFIG_CPP17_VARIANT // Override std::variant support detection (checked by CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER)
CATCH_CONFIG_CPP17_OPTIONAL // Override std::optional support detection (checked by CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER)
CATCH_CONFIG_CPP17_BYTE // Override std::byte support detection (Catch provides a StringMaker specialization by default)
> `CATCH_CONFIG_CPP17_STRING_VIEW` was [introduced](https://github.com/catchorg/Catch2/issues/1376) in Catch2 2.4.1.
> `CATCH_CONFIG_CPP17_STRING_VIEW` was [introduced](https://github.com/catchorg/Catch2/issues/1376) in Catch 2.4.1.
Catch contains basic compiler/standard detection and attempts to use
some C++17 features whenever appropriate. This automatic detection
@@ -146,33 +147,25 @@ by using `_NO_` in the macro, e.g. `CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS`.
CATCH_CONFIG_COUNTER // Use __COUNTER__ to generate unique names for test cases
CATCH_CONFIG_WINDOWS_SEH // Enable SEH handling on Windows
CATCH_CONFIG_FAST_COMPILE // Sacrifices some (rather minor) features for compilation speed
CATCH_CONFIG_DISABLE_MATCHERS // Do not compile Matchers in this compilation unit
CATCH_CONFIG_POSIX_SIGNALS // Enable handling POSIX signals
CATCH_CONFIG_WINDOWS_CRTDBG // Enable leak checking using Windows's CRT Debug Heap
CATCH_CONFIG_DISABLE_STRINGIFICATION // Disable stringifying the original expression
CATCH_CONFIG_DISABLE // Disables assertions and test case registration
CATCH_CONFIG_WCHAR // Enables use of wchart_t
CATCH_CONFIG_EXPERIMENTAL_REDIRECT // Enables the new (experimental) way of capturing stdout/stderr
CATCH_CONFIG_ENABLE_BENCHMARKING // Enables the integrated benchmarking features (has a significant effect on compilation speed)
CATCH_CONFIG_USE_ASYNC // Force parallel statistical processing of samples during benchmarking
CATCH_CONFIG_ANDROID_LOGWRITE // Use android's logging system for debug output
CATCH_CONFIG_GLOBAL_NEXTAFTER // Use nextafter{,f,l} instead of std::nextafter
CATCH_CONFIG_GETENV // System has a working `getenv`
> [`CATCH_CONFIG_ANDROID_LOGWRITE`](https://github.com/catchorg/Catch2/issues/1743) and [`CATCH_CONFIG_GLOBAL_NEXTAFTER`](https://github.com/catchorg/Catch2/pull/1739) were introduced in Catch2 2.10.0
> `CATCH_CONFIG_GETENV` was [introduced](https://github.com/catchorg/Catch2/pull/2562) in Catch2 3.2.0
> [`CATCH_CONFIG_ANDROID_LOGWRITE`](https://github.com/catchorg/Catch2/issues/1743) and [`CATCH_CONFIG_GLOBAL_NEXTAFTER`](https://github.com/catchorg/Catch2/pull/1739) were introduced in Catch 2.10.0
Currently Catch enables `CATCH_CONFIG_WINDOWS_SEH` only when compiled with MSVC, because some versions of MinGW do not have the necessary Win32 API support.
`CATCH_CONFIG_POSIX_SIGNALS` is on by default, except when Catch is compiled under `Cygwin`, where it is disabled by default (but can be force-enabled by defining `CATCH_CONFIG_POSIX_SIGNALS`).
`CATCH_CONFIG_GETENV` is on by default, except when Catch2 is compiled for
platforms that lacks working `std::getenv` (currently Windows UWP and
Playstation).
`CATCH_CONFIG_WINDOWS_CRTDBG` is off by default. If enabled, Windows's
CRT is used to check for memory leaks, and displays them after the tests
finish running. This option only works when linking against the default
main, and must be defined for the whole library build.
`CATCH_CONFIG_WINDOWS_CRTDBG` is off by default. If enabled, Windows's CRT is used to check for memory leaks, and displays them after the tests finish running.
`CATCH_CONFIG_WCHAR` is on by default, but can be disabled. Currently
it is only used in support for DJGPP cross-compiler.
@@ -191,6 +184,11 @@ should not lead to false negatives.
`CATCH_CONFIG_FAST_COMPILE` has to be either defined, or not defined,
in all translation units that are linked into single test binary.
### `CATCH_CONFIG_DISABLE_MATCHERS`
When `CATCH_CONFIG_DISABLE_MATCHERS` is defined, all mentions of Catch's Matchers are ifdef-ed away from the translation unit. Doing so will speed up compilation of that TU.
_Note: If you define `CATCH_CONFIG_DISABLE_MATCHERS` in the same file as Catch's main is implemented, your test executable will fail to link if you use Matchers anywhere._
### `CATCH_CONFIG_DISABLE_STRINGIFICATION`
This toggle enables a workaround for VS 2017 bug. For details see [known limitations](limitations.md#visual-studio-2017----raw-string-literal-in-assert-fails-to-compile).
@@ -201,6 +199,13 @@ This feature is considered experimental and might change at any point.
_Inspired by Doctest's `DOCTEST_CONFIG_DISABLE`_
## Windows header clutter
On Windows Catch includes `windows.h`. To minimize global namespace clutter in the implementation file, it defines `NOMINMAX` and `WIN32_LEAN_AND_MEAN` before including it. You can control this behaviour via two macros:
CATCH_CONFIG_NO_NOMINMAX // Stops Catch from using NOMINMAX macro
CATCH_CONFIG_NO_WIN32_LEAN_AND_MEAN // Stops Catch from using WIN32_LEAN_AND_MEAN macro
## Enabling stringification
@@ -208,17 +213,18 @@ By default, Catch does not stringify some types from the standard library. This
CATCH_CONFIG_ENABLE_PAIR_STRINGMAKER // Provide StringMaker specialization for std::pair
CATCH_CONFIG_ENABLE_TUPLE_STRINGMAKER // Provide StringMaker specialization for std::tuple
CATCH_CONFIG_ENABLE_CHRONO_STRINGMAKER // Provide StringMaker specialization for std::chrono::duration, std::chrono::timepoint
CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER // Provide StringMaker specialization for std::variant, std::monostate (on C++17)
CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER // Provide StringMaker specialization for std::optional (on C++17)
CATCH_CONFIG_ENABLE_ALL_STRINGMAKERS // Defines all of the above
> `CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1380) in Catch2 2.4.1.
> `CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1380) in Catch 2.4.1.
> `CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1510) in Catch2 2.6.0.
> `CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1510) in Catch 2.6.0.
## Disabling exceptions
> Introduced in Catch2 2.4.0.
> Introduced in Catch 2.4.0.
By default, Catch2 uses exceptions to signal errors and to abort tests
when an assertion from the `REQUIRE` family of assertions fails. We also
@@ -254,7 +260,7 @@ namespace Catch {
## Overriding Catch's debug break (`-b`)
> [Introduced](https://github.com/catchorg/Catch2/pull/1846) in Catch2 2.11.2.
> [Introduced](https://github.com/catchorg/Catch2/pull/1846) in Catch 2.11.2.
You can override Catch2's break-into-debugger code by defining the
`CATCH_BREAK_INTO_DEBUGGER()` macro. This can be used if e.g. Catch2 does

View File

@@ -21,8 +21,8 @@ to the codebase itself.
## Using Git(Hub)
Ongoing development happens in the `devel` branch for Catch2 v3, and in
`v2.x` for maintenance updates to the v2 versions.
Ongoing development happens in the `v2.x` branch for Catch2 v2, and in
`devel` for the next major version, v3.
Commits should be small and atomic. A commit is atomic when, after it is
applied, the codebase, tests and all, still works as expected. Small
@@ -30,8 +30,8 @@ commits are also preferred, as they make later operations with git history,
whether it is bisecting, reverting, or something else, easier.
_When submitting a pull request please do not include changes to the
amalgamated distribution files. This means do not include them in your
git commits!_
single include. This means do not include them in your git commits!_
When addressing review comments in a MR, please do not rebase/squash the
commits immediately. Doing so makes it harder to review the new changes,
@@ -55,15 +55,6 @@ tests from `SelfTest` through a specific reporter and then compare the
generated output with a known good output ("Baseline"). By default, new
tests should be placed here.
To configure a Catch2 build with just the basic tests, use the `basic-tests`
preset, like so:
```
# Assuming you are in Catch2's root folder
cmake -B basic-test-build -S . -DCMAKE_BUILD_TYPE=Debug --preset basic-tests
```
However, not all tests can be written as plain unit tests. For example,
checking that Catch2 orders tests randomly when asked to, and that this
random ordering is subset-invariant, is better done as an integration
@@ -71,61 +62,42 @@ test using an external check script. Catch2 integration tests are written
using CTest, either as a direct command invocation + pass/fail regex,
or by delegating the check to a Python script.
Catch2 is slowly gaining more and more types of tests, currently Catch2
project also has buildable examples, "ExtraTests", and CMake config tests.
Examples present a small and self-contained snippets of code that
use Catch2's facilities for specific purpose. Currently they are assumed
passing if they compile.
There are also two more kinds of tests, examples and "ExtraTests".
Examples serve as a compilation test on the single-header distribution,
and present a small and self-contained snippets of using Catch2 for
writing tests. ExtraTests then are tests that either take a long time
to run, or require separate compilation, e.g. because of testing compile
time configuration options, and take a long time because of that.
ExtraTests then are expensive tests, that we do not want to run all the
time. This can be either because they take a long time to run, or because
they take a long time to compile, e.g. because they test compile time
configuration and require separate compilation.
Both of these are compiled against the single-header distribution of
Catch2, and thus might require you to regenerate it manually. This is
done by calling the `generateSingleHeader.py` script in `scripts`.
Finally, CMake config tests test that you set Catch2's compile-time
configuration options through CMake, using CMake options of the same name.
Examples and ExtraTests are not compiled by default. To compile them,
add `-DCATCH_BUILD_EXAMPLES=ON` and `-DCATCH_BUILD_EXTRA_TESTS=ON` to
the invocation of CMake configuration step.
These test categories can be enabled one by one, by passing
`-DCATCH_BUILD_EXAMPLES=ON`, `-DCATCH_BUILD_EXTRA_TESTS=ON`, and
`-DCATCH_ENABLE_CONFIGURE_TESTS=ON` when configuring the build.
Bringing this all together, the steps below should configure, build,
and run all tests in the `Debug` compilation.
Catch2 also provides a preset that promises to enable _all_ test types,
`all-tests`.
The snippet below will build & run all tests, in `Debug` compilation mode.
<!-- snippet: catch2-build-and-test -->
<a id='snippet-catch2-build-and-test'></a>
```sh
# 1. Regenerate the amalgamated distribution (some tests are built against it)
./tools/scripts/generateAmalgamatedFiles.py
# 2. Configure the full test build
cmake -B debug-build -S . -DCMAKE_BUILD_TYPE=Debug --preset all-tests
# 3. Run the actual build
cmake --build debug-build
# 4. Run the tests using CTest
cd debug-build
ctest -j 4 --output-on-failure -C Debug
1. Regenerate the single header distribution
```
<sup><a href='/tools/scripts/buildAndTest.sh#L6-L19' title='File snippet `catch2-build-and-test` was extracted from'>snippet source</a> | <a href='#snippet-catch2-build-and-test' title='Navigate to start of snippet `catch2-build-and-test`'>anchor</a></sup>
<!-- endSnippet -->
For convenience, the above commands are in the script `tools/scripts/buildAndTest.sh`, and can be run like this:
```bash
cd Catch2
./tools/scripts/buildAndTest.sh
$ cd Catch2
$ ./scripts/generateSingleHeader.py
```
2. Configure the full test build
```
$ cmake -Bdebug-build -H. -DCMAKE_BUILD_TYPE=Debug -DCATCH_BUILD_EXAMPLES=ON -DCATCH_BUILD_EXTRA_TESTS=ON
```
3. Run the actual build
```
$ cmake --build debug-build
```
4. Run the tests using CTest
```
$ cd debug-build
$ ctest -j 4 --output-on-failure -C Debug
```
A Windows version of the script is available at `tools\scripts\buildAndTest.cmd`.
If you added new tests, you will likely see `ApprovalTests` failure.
After you check that the output difference is expected, you should
run `tools/scripts/approve.py` to confirm them, and include these changes
in your commit.
## Writing documentation
@@ -135,8 +107,7 @@ other people can use it as well. This section collects some technical
information that you will need for updating Catch2's documentation, and
possibly some generic advise as well.
### Technicalities
### Technicalities
First, the technicalities:
@@ -147,8 +118,6 @@ should use. It provides you with the top anchor mentioned to link to
<a id="top"></a>
# Cool feature
> [Introduced](https://github.com/catchorg/Catch2/pull/123456) in Catch2 X.Y.Z
Text that explains how to use the cool feature.
@@ -167,9 +136,9 @@ be replaced with the actual version upon release. There are 2 styles
of placeholders used through the documentation, you should pick one that
fits your text better (if in doubt, take a look at the existing version
tags for other features).
* `> [Introduced](link-to-issue-or-PR) in Catch2 X.Y.Z` - this
* `> [Introduced](link-to-issue-or-PR) in Catch X.Y.Z` - this
placeholder is usually used after a section heading
* `> X (Y and Z) was [introduced](link-to-issue-or-PR) in Catch2 X.Y.Z`
* `> X (Y and Z) was [introduced](link-to-issue-or-PR) in Catch X.Y.Z`
- this placeholder is used when you need to tag a subpart of something,
e.g. a list
@@ -210,24 +179,6 @@ If want to contribute code, this section contains some simple rules
and tips on things like code formatting, code constructions to avoid,
and so on.
### C++ standard version
Catch2 currently targets C++14 as the minimum supported C++ version.
Features from higher language versions should be used only sparingly,
when the benefits from using them outweigh the maintenance overhead.
Example of good use of polyfilling features is our use of `conjunction`,
where if available we use `std::conjunction` and otherwise provide our
own implementation. The reason it is good is that the surface area for
maintenance is quite small, and `std::conjunction` can directly use
compiler built-ins, thus providing significant compilation benefits.
Example of bad use of polyfilling features would be to keep around two
sets of metaprogramming in the stringification implementation, once
using C++14 compliant TMP and once using C++17's `if constexpr`. While
the C++17 would provide significant compilation speedups, the maintenance
cost would be too high.
### Formatting
@@ -248,7 +199,7 @@ are problematic and are not always caught by our CI infrastructure.
#### Naked exceptions and exceptions-related function
If you are throwing an exception, it should be done via `CATCH_ERROR`
or `CATCH_RUNTIME_ERROR` in `internal/catch_enforce.hpp`. These macros will handle
or `CATCH_RUNTIME_ERROR` in `catch_enforce.h`. These macros will handle
the differences between compilation with or without exceptions for you.
However, some platforms (IAR) also have problems with exceptions-related
functions, such as `std::current_exceptions`. We do not have IAR in our
@@ -257,18 +208,6 @@ However, if you do, they should be kept behind a
`CATCH_CONFIG_DISABLE_EXCEPTIONS` macro.
#### Avoid `std::move` and `std::forward`
`std::move` and `std::forward` provide nice semantic name for a specific
`static_cast`. However, being function templates they have surprisingly
high cost during compilation, and can also have a negative performance
impact for low-optimization builds.
You should be using `CATCH_MOVE` and `CATCH_FORWARD` macros from
`internal/catch_move_and_forward.hpp` instead. They expand into the proper
`static_cast`, and avoid the overhead of `std::move` and `std::forward`.
#### Unqualified usage of functions from C's stdlib
If you are using a function from C's stdlib, please include the header
@@ -277,57 +216,6 @@ there is no difference is wrong, QNX and VxWorks won't compile if you
include the header as `<cfoo>` and call the function unqualified.
#### User-Defined Literals (UDL) for Catch2' types
Due to messy standardese and ... not great ... implementation of
`-Wreserved-identifier` in Clang, avoid declaring UDLs as
```cpp
Approx operator "" _a(long double);
```
and instead declare them as
```cpp
Approx operator ""_a(long double);
```
Notice that the second version does not have a space between the `""` and
the literal suffix.
### New source file template
If you are adding new source file, there is a template you should use.
Specifically, every source file should start with the licence header:
```cpp
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
```
The include guards for header files should follow the pattern `{FILENAME}_INCLUDED`.
This means that for file `catch_matchers_foo.hpp`, the include guard should
be `CATCH_MATCHERS_FOO_HPP_INCLUDED`, for `catch_generators_bar.hpp`, the include
guard should be `CATCH_GENERATORS_BAR_HPP_INCLUDED`, and so on.
### Adding new `CATCH_CONFIG` option
When adding new `CATCH_CONFIG` option, there are multiple places to edit:
* `CMake/CatchConfigOptions.cmake` - this is used to generate the
configuration options in CMake, so that CMake frontends know about them.
* `docs/configuration.md` - this is where the options are documented
* `src/catch2/catch_user_config.hpp.in` - this is template for generating
`catch_user_config.hpp` which contains the materialized configuration
* `BUILD.bazel` - Bazel does not have configuration support like CMake,
and all expansions need to be done manually
* other files as needed, e.g. `catch2/internal/catch_config_foo.hpp`
for the logic that guards the configuration
## CoC
This project has a [CoC](../CODE_OF_CONDUCT.md). Please adhere to it

View File

@@ -9,6 +9,69 @@ either of these is a breaking change, and thus will not happen until
at least the next major release.
## Deprecations
### `--list-*` return values
The return codes of the `--list-*` family of command line arguments
will no longer be equal to the number of tests/tags/etc found, instead
it will be 0 for success and non-zero for failure.
### `--list-test-names-only`
`--list-test-names-only` command line argument will be removed.
### `ANON_TEST_CASE`
`ANON_TEST_CASE` is scheduled for removal, as it can be fully replaced
by a `TEST_CASE` with no arguments.
### Secondary description amongst tags
Currently, the tags part of `TEST_CASE` (and others) macro can also
contain text that is not part of tags. This text is then separated into
a "description" of the test case, but the description is then never used
apart from writing it out for `--list-tests -v high`.
Because it isn't actually used nor documented, and brings complications
to Catch2's internals, description support will be removed.
### SourceLineInfo::empty()
There should be no reason to ever have an empty `SourceLineInfo`, so the
method will be removed.
### Composing lvalues of already composed matchers
Because a significant bug in this use case has persisted for 2+ years
without a bug report, and to simplify the implementation, code that
composes lvalues of composed matchers will not compile. That is,
this code will no longer work:
```cpp
auto m1 = Contains("string");
auto m2 = Contains("random");
auto composed1 = m1 || m2;
auto m3 = Contains("different");
auto composed2 = composed1 || m3;
REQUIRE_THAT(foo(), !composed1);
REQUIRE_THAT(foo(), composed2);
```
Instead you will have to write this:
```cpp
auto m1 = Contains("string");
auto m2 = Contains("random");
auto m3 = Contains("different");
REQUIRE_THAT(foo(), !(m1 || m2));
REQUIRE_THAT(foo(), m1 || m2 || m3);
```
### `ParseAndAddCatchTests.cmake`
The CMake/CTest integration using `ParseAndAddCatchTests.cmake` is deprecated,
@@ -17,23 +80,64 @@ as it can be replaced by `Catch.cmake` that provides the function
command line interface instead of parsing C++ code with regular expressions.
### `CATCH_CONFIG_BAZEL_SUPPORT`
## Planned changes
Catch2 supports writing the Bazel JUnit XML output file when it is aware
that is within a bazel testing environment. Originally there was no way
to accurately probe the environment for this information so the flag
`CATCH_CONFIG_BAZEL_SUPPORT` was added. This now deprecated. Bazel has now had a change
where it will export `BAZEL_TEST=1` for purposes like the above. Catch2
will now instead inspect the environment instead of relying on build configuration.
### `IEventLister::skipTest( TestCaseInfo const& testInfo )`
### Reporter verbosities
The current implementation of verbosities, where the reporter is checked
up-front whether it supports the requested verbosity, is fundamentally
misguided and will be changed. The new implementation will no longer check
whether the specified reporter supports the requested verbosity, instead
it will be up to the reporters to deal with verbosities as they see fit
(with an expectation that unsupported verbosities will be, at most,
warnings, but not errors).
### Output format of `--list-*` command line parameters
The various list operations will be piped through reporters. This means
that e.g. XML reporter will write the output as machine-parseable XML,
while the Console reporter will keep the current, human-oriented output.
### `CHECKED_IF` and `CHECKED_ELSE`
To make the `CHECKED_IF` and `CHECKED_ELSE` macros more useful, they will
be marked as "OK to fail" (`Catch::ResultDisposition::SuppressFail` flag
will be added), which means that their failure will not fail the test,
making the `else` actually useful.
### Change semantics of `[.]` and tag exclusion
Currently, given these 2 tests
```cpp
TEST_CASE("A", "[.][foo]") {}
TEST_CASE("B", "[.][bar]") {}
```
specifying `[foo]` as the testspec will run test "A" and specifying
`~[foo]` will run test "B", even though it is hidden. Also, specifying
`~[baz]` will run both tests. This behaviour is often surprising and will
be changed so that hidden tests are included in a run only if they
positively match a testspec.
### Console Colour API
The API for Catch2's console colour will be changed to take an extra
argument, the stream to which the colour code should be applied.
### Type erasure in the `PredicateMatcher`
Currently, the `PredicateMatcher` uses `std::function` for type erasure,
so that type of the matcher is always `PredicateMatcher<T>`, regardless
of the type of the predicate. Because of the high compilation overhead
of `std::function`, and the fact that the type erasure is used only rarely,
`PredicateMatcher` will no longer be type erased in the future. Instead,
the predicate type will be made part of the PredicateMatcher's type.
This event (including implementations in derived classes such as `ReporterBase`)
is deprecated and will be removed in the next major release. It is currently
invoked for all test cases that are not going to be executed due to the test run
being aborted (when using `--abort` or `--abortx`). It is however
**NOT** invoked for test cases that are [explicitly skipped using the `SKIP`
macro](skipping-passing-failing.md#top).
---

View File

@@ -1,43 +1,74 @@
<a id="top"></a>
# Event Listeners
An event listener is a bit like a reporter, in that it responds to various
reporter events in Catch2, but it is not expected to write any output.
Instead, an event listener performs actions within the test process, such
as performing global initialization (e.g. of a C library), or cleaning out
in-memory logs if they are not needed (the test case passed).
A `Listener` is a class you can register with Catch that will then be passed events,
such as a test case starting or ending, as they happen during a test run.
`Listeners` are actually types of `Reporters`, with a few small differences:
1. Once registered in code they are automatically used - you don't need to specify them on the command line
2. They are called in addition to (just before) any reporters, and you can register multiple listeners.
3. They derive from `Catch::TestEventListenerBase`, which has default stubs for all the events,
so you are not forced to implement events you're not interested in.
4. You register a listener with `CATCH_REGISTER_LISTENER`
Unlike reporters, each registered event listener is always active. Event
listeners are always notified before reporter(s).
To write your own event listener, you should derive from `Catch::TestEventListenerBase`,
as it provides empty stubs for all reporter events, allowing you to
only override events you care for. Afterwards you have to register it
with Catch2 using `CATCH_REGISTER_LISTENER` macro, so that Catch2 knows
about it and instantiates it before running tests.
## Implementing a Listener
Simply derive a class from `Catch::TestEventListenerBase` and implement the methods you are interested in, either in
the main source file (i.e. the one that defines `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`), or in a
file that defines `CATCH_CONFIG_EXTERNAL_INTERFACES`.
Example event listener:
```cpp
#include <catch2/reporters/catch_reporter_event_listener.hpp>
#include <catch2/reporters/catch_reporter_registrars.hpp>
Then register it using `CATCH_REGISTER_LISTENER`.
class testRunListener : public Catch::EventListenerBase {
public:
using Catch::EventListenerBase::EventListenerBase;
For example ([complete source code](../examples/210-Evt-EventListeners.cpp)):
void testRunStarting(Catch::TestRunInfo const&) override {
lib_foo_init();
```c++
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
struct MyListener : Catch::TestEventListenerBase {
using TestEventListenerBase::TestEventListenerBase; // inherit constructor
void testCaseStarting( Catch::TestCaseInfo const& testInfo ) override {
// Perform some setup before a test case is run
}
void testCaseEnded( Catch::TestCaseStats const& testCaseStats ) override {
// Tear-down after a test case is run
}
};
CATCH_REGISTER_LISTENER(testRunListener)
CATCH_REGISTER_LISTENER( MyListener )
```
_Note that you should not use any assertion macros within a Listener!_
_Note that you should not use any assertion macros within a Listener!_
[You can find the list of events that the listeners can react to on its
own page](reporter-events.md#top).
## Events that can be hooked
The following are the methods that can be overridden in the Listener:
```c++
// The whole test run, starting and ending
virtual void testRunStarting( TestRunInfo const& testRunInfo );
virtual void testRunEnded( TestRunStats const& testRunStats );
// Test cases starting and ending
virtual void testCaseStarting( TestCaseInfo const& testInfo );
virtual void testCaseEnded( TestCaseStats const& testCaseStats );
// Sections starting and ending
virtual void sectionStarting( SectionInfo const& sectionInfo );
virtual void sectionEnded( SectionStats const& sectionStats );
// Assertions before/ after
virtual void assertionStarting( AssertionInfo const& assertionInfo );
virtual bool assertionEnded( AssertionStats const& assertionStats );
// A test is being skipped (because it is "hidden")
virtual void skipTest( TestCaseInfo const& testInfo );
```
More information about the events (e.g. name of the test case) is contained in the structs passed as arguments -
just look in the source code to see what fields are available.
---

View File

@@ -1,94 +0,0 @@
<a id="top"></a>
# Frequently Asked Questions (FAQ)
**Contents**<br>
[How do I run global setup/teardown only if tests will be run?](#how-do-i-run-global-setupteardown-only-if-tests-will-be-run)<br>
[How do I clean up global state between running different tests?](#how-do-i-clean-up-global-state-between-running-different-tests)<br>
[Why cannot I derive from the built-in reporters?](#why-cannot-i-derive-from-the-built-in-reporters)<br>
[What is Catch2's ABI stability policy?](#what-is-catch2s-abi-stability-policy)<br>
[What is Catch2's API stability policy?](#what-is-catch2s-api-stability-policy)<br>
[Does Catch2 support running tests in parallel?](#does-catch2-support-running-tests-in-parallel)<br>
[Can I compile Catch2 into a dynamic library?](#can-i-compile-catch2-into-a-dynamic-library)<br>
[What repeatability guarantees does Catch2 provide?](#what-repeatability-guarantees-does-catch2-provide)<br>
## How do I run global setup/teardown only if tests will be run?
Write a custom [event listener](event-listeners.md#top) and place the
global setup/teardown code into the `testRun*` events.
## How do I clean up global state between running different tests?
Write a custom [event listener](event-listeners.md#top) and place the
cleanup code into either `testCase*` or `testCasePartial*` events,
depending on how often the cleanup needs to happen.
## Why cannot I derive from the built-in reporters?
They are not made to be overridden, in that we do not attempt to maintain
a consistent internal state if a member function is overriden, and by
forbidding users from using them as a base class, we can refactor them
as needed later.
## What is Catch2's ABI stability policy?
Catch2 provides no ABI stability guarantees whatsoever. Catch2 provides
rich C++ interface, and trying to freeze its ABI would take a lot of
pointless work.
Catch2 is not designed to be distributed as dynamic library, and you
should really be able to compile everything with the same compiler binary.
## What is Catch2's API stability policy?
Catch2 follows [semver](https://semver.org/) to the best of our ability.
This means that we will not knowingly make backwards-incompatible changes
without incrementing the major version number.
## Does Catch2 support running tests in parallel?
Not natively, no. We see running tests in parallel as the job of an
external test runner, that can also run them in separate processes,
support test execution timeouts and so on.
However, Catch2 provides some tools that make the job of external test
runners easier. [See the relevant section in our page on best
practices](usage-tips.md#parallel-tests).
## Can I compile Catch2 into a dynamic library?
Yes, Catch2 supports the [standard CMake `BUILD_SHARED_LIBS`
option](https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html).
However, the dynamic library support is provided as-is. Catch2 does not
provide API export annotations, and so you can only use it as a dynamic
library on platforms that default to public visibility, or with tooling
support to force export Catch2's API.
## What repeatability guarantees does Catch2 provide?
There are two places where it is meaningful to talk about Catch2's
repeatability guarantees without taking into account user-provided
code. First one is in the test case shuffling, and the second one is
the output from random generators.
Test case shuffling is repeatable across different platforms since v2.12.0,
and it is also generally repeatable across versions, but we might break
it from time to time. E.g. we broke repeatability with previous versions
in v2.13.4 so that test cases with similar names are shuffled better.
Random generators currently rely on platform's stdlib, specifically
the distributions from `<random>`. We thus provide no extra guarantee
above what your platform does. **Important: `<random>`'s distributions
are not specified to be repeatable across different platforms.**
---
[Home](Readme.md#top)

View File

@@ -1,7 +1,7 @@
<a id="top"></a>
# Data Generators
> Introduced in Catch2 2.6.0.
> Introduced in Catch 2.6.0.
Data generators (also known as _data driven/parametrized test cases_)
let you reuse the same set of assertions across different input values.
@@ -92,7 +92,7 @@ TEST_CASE("Complex mix of sections and generates") {
}
```
> The ability to place `GENERATE` between two `SECTION`s was [introduced](https://github.com/catchorg/Catch2/issues/1938) in Catch2 2.13.0.
> The ability to place `GENERATE` between two `SECTION`s was [introduced](https://github.com/catchorg/Catch2/issues/1938) in Catch 2.13.0.
## Provided generators
@@ -114,12 +114,12 @@ a test case,
* 4 specific purpose generators
* `RandomIntegerGenerator<Integral>` -- generates random Integrals from range
* `RandomFloatGenerator<Float>` -- generates random Floats from range
* `RangeGenerator<T>(first, last)` -- generates all values inside a `[first, last)` arithmetic range
* `RangeGenerator<T>` -- generates all values inside an arithmetic range
* `IteratorGenerator<T>` -- copies and returns values from an iterator range
> `ChunkGenerator<T>`, `RandomIntegerGenerator<Integral>`, `RandomFloatGenerator<Float>` and `RangeGenerator<T>` were introduced in Catch2 2.7.0.
> `ChunkGenerator<T>`, `RandomIntegerGenerator<Integral>`, `RandomFloatGenerator<Float>` and `RangeGenerator<T>` were introduced in Catch 2.7.0.
> `IteratorGenerator<T>` was introduced in Catch2 2.10.0.
> `IteratorGenerator<T>` was introduced in Catch 2.10.0.
The generators also have associated helper functions that infer their
type, making their usage much nicer. These are
@@ -139,11 +139,11 @@ type, making their usage much nicer. These are
* `from_range(InputIterator from, InputIterator to)` for `IteratorGenerator<T>`
* `from_range(Container const&)` for `IteratorGenerator<T>`
> `chunk()`, `random()` and both `range()` functions were introduced in Catch2 2.7.0.
> `chunk()`, `random()` and both `range()` functions were introduced in Catch 2.7.0.
> `from_range` has been introduced in Catch2 2.10.0
> `from_range` has been introduced in Catch 2.10.0
> `range()` for floating point numbers has been introduced in Catch2 2.11.0
> `range()` for floating point numbers has been introduced in Catch 2.11.0
And can be used as shown in the example below to create a generator
that returns 100 odd random number:
@@ -176,7 +176,7 @@ scope and thus capturing references is dangerous. If you need to use
variables inside the generator expression, make sure you thought through
the lifetime implications and use `GENERATE_COPY` or `GENERATE_REF`.**
> `GENERATE_COPY` and `GENERATE_REF` were introduced in Catch2 2.7.1.
> `GENERATE_COPY` and `GENERATE_REF` were introduced in Catch 2.7.1.
You can also override the inferred type by using `as<type>` as the first
argument to the macro. This can be useful when dealing with string literals,
@@ -205,17 +205,13 @@ struct IGenerator : GeneratorUntypedBase {
// Precondition:
// The generator is either freshly constructed or the last call to next() returned true
virtual T const& get() const = 0;
// Returns user-friendly string showing the current generator element
// Does not have to be overridden, IGenerator provides default implementation
virtual std::string stringifyImpl() const;
};
```
However, to be able to use your custom generator inside `GENERATE`, it
will need to be wrapped inside a `GeneratorWrapper<T>`.
`GeneratorWrapper<T>` is a value wrapper around a
`Catch::Detail::unique_ptr<IGenerator<T>>`.
`std::unique_ptr<IGenerator<T>>`.
For full example of implementing your own generator, look into Catch2's
examples, specifically

View File

@@ -88,63 +88,59 @@ because only one thread passes the `REQUIRE` macro and this is not
REQUIRE(cnt == 16);
```
We currently do not plan to support thread-safe assertions.
Because C++11 provides the necessary tools to do this, we are planning
to remove this limitation in the future.
### Process isolation in a test
Catch does not support running tests in isolated (forked) processes. While this might in the future, the fact that Windows does not support forking and only allows full-on process creation and the desire to keep code as similar as possible across platforms, mean that this is likely to take significant development time, that is not currently available.
### Running multiple tests in parallel
Catch's test execution is strictly serial. If you find yourself with a test suite that takes too long to run and you want to make it parallel, there are 2 feasible solutions
* You can split your tests into multiple binaries and then run these binaries in parallel.
* You can have Catch list contained test cases and then run the same test binary multiple times in parallel, passing each instance list of test cases it should run.
Catch2 keeps test execution in one process strictly serial, and there
are no plans to change this. If you find yourself with a test suite
that takes too long to run and you want to make it parallel, you have
to run multiple processes side by side.
There are 2 basic ways to do that,
* you can split your tests into multiple binaries, and run those binaries
in parallel
* you can run the same test binary multiple times, but run a different
subset of the tests in each process
There are multiple ways to achieve the latter, the easiest way is to use
[test sharding](command-line.md#test-sharding).
Both of these solutions have their problems, but should let you wring parallelism out of your test suite.
## 3rd party bugs
This section outlines known bugs in 3rd party components (this means compilers, standard libraries, standard runtimes).
### Visual Studio 2015 -- `GENERATE` does not compile if it would deduce char array
VS 2015 refuses to compile `GENERATE` statements that would deduce to a
char array with known size, e.g. this:
```cpp
TEST_CASE("Deducing string lit") {
auto param = GENERATE("start", "stop");
}
```
A workaround for this is to use the `as` helper and force deduction of
either a `char const*` or a `std::string`.
### Visual Studio 2017 -- raw string literal in assert fails to compile
There is a known bug in Visual Studio 2017 (VC 15), that causes compilation
error when preprocessor attempts to stringize a raw string literal
(`#` preprocessor directive is applied to it). This snippet is sufficient
to trigger the compilation error:
There is a known bug in Visual Studio 2017 (VC 15), that causes compilation error when preprocessor attempts to stringize a raw string literal (`#` preprocessor is applied to it). This snippet is sufficient to trigger the compilation error:
```cpp
#include <catch2/catch_test_macros.hpp>
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
TEST_CASE("test") {
CHECK(std::string(R"("\)") == "\"\\");
}
```
Catch2 provides a workaround, by letting the user disable stringification
of the original expression by defining `CATCH_CONFIG_DISABLE_STRINGIFICATION`,
like so:
Catch provides a workaround, it is possible to disable stringification of original expressions by defining `CATCH_CONFIG_DISABLE_STRINGIFICATION`:
```cpp
#define CATCH_CONFIG_FAST_COMPILE
#define CATCH_CONFIG_DISABLE_STRINGIFICATION
#include <catch2/catch_test_macros.hpp>
#include "catch.hpp"
TEST_CASE("test") {
CHECK(std::string(R"("\)") == "\"\\");
}
```
_Do note that this changes the output:_
_Do note that this changes the output somewhat_
```
catchwork\test1.cpp(6):
PASSED:
@@ -153,11 +149,26 @@ with expansion:
""\" == ""\"
```
### Visual Studio 2015 -- Alignment compilation error (C2718)
VS 2015 has a known bug, where `declval<T>` can cause compilation error
if `T` has alignment requirements that it cannot meet.
A workaround is to explicitly specialize `Catch::is_range` for given
type (this avoids code path that uses `declval<T>` in a SFINAE context).
### Visual Studio 2015 -- Wrong line number reported in debug mode
VS 2015 has a known bug where `__LINE__` macro can be improperly expanded under certain circumstances, while compiling multi-file project in Debug mode.
A workaround is to compile the binary in Release mode.
### Clang/G++ -- skipping leaf sections after an exception
Some versions of `libc++` and `libstdc++` (or their runtimes) have a bug with `std::uncaught_exception()` getting stuck returning `true` after rethrow, even if there are no active exceptions. One such case is this snippet, which skipped the sections "a" and "b", when compiled against `libcxxrt` from the master branch
Some versions of `libc++` and `libstdc++` (or their runtimes) have a bug with `std::uncaught_exception()` getting stuck returning `true` after rethrow, even if there are no active exceptions. One such case is this snippet, which skipped the sections "a" and "b", when compiled against `libcxxrt` from master
```cpp
#include <catch2/catch_test_macros.hpp>
#define CATCH_CONFIG_MAIN
#include <catch.hpp>
TEST_CASE("a") {
CHECK_THROWS(throw 3);
@@ -171,7 +182,12 @@ TEST_CASE("b") {
}
```
If you are seeing a problem like this, i.e. weird test paths that trigger only under Clang with `libc++`, or only under very specific version of `libstdc++`, it is very likely you are seeing this. The only known workaround is to use a fixed version of your standard library.
If you are seeing a problem like this, i.e. a weird test paths that trigger only under Clang with `libc++`, or only under very specific version of `libstdc++`, it is very likely you are seeing this. The only known workaround is to use a fixed version of your standard library.
### Clang/G++ -- `Matches` string matcher always returns false
This is a bug in `libstdc++-4.8`, where all matching methods from `<regex>` return false. Since `Matches` uses `<regex>` internally, if the underlying implementation does not work, it doesn't work either.
Workaround: Use newer version of `libstdc++`.
### libstdc++, `_GLIBCXX_DEBUG` macro and random ordering of tests

View File

@@ -3,12 +3,15 @@
## Already available
- Catch main: [Catch-provided main](../examples/000-CatchMain.cpp)
- Test Case: [Single-file](../examples/010-TestCase.cpp)
- Test Case: [Multiple-file 1](../examples/020-TestCase-1.cpp), [2](../examples/020-TestCase-2.cpp)
- Assertion: [REQUIRE, CHECK](../examples/030-Asn-Require-Check.cpp)
- Fixture: [Sections](../examples/100-Fix-Section.cpp)
- Fixture: [Class-based fixtures](../examples/110-Fix-ClassFixture.cpp)
- BDD: [SCENARIO, GIVEN, WHEN, THEN](../examples/120-Bdd-ScenarioGivenWhenThen.cpp)
- Report: [Catch-provided main](../examples/200-Rpt-CatchMain.cpp)
- Report: [TeamCity reporter](../examples/207-Rpt-TeamCityReporter.cpp)
- Listener: [Listeners](../examples/210-Evt-EventListeners.cpp)
- Configuration: [Provide your own output streams](../examples/231-Cfg-OutputStreams.cpp)
- Generators: [Create your own generator](../examples/300-Gen-OwnGenerator.cpp)

View File

@@ -30,7 +30,7 @@ When the last `CHECK` fails in the "Bar" test case, then only one message will b
## Logging without local scope
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch2 2.7.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch 2.7.0.
`UNSCOPED_INFO` is similar to `INFO` with two key differences:
@@ -106,7 +106,7 @@ This semicolon will be removed with next major version. It is highly advised to
**UNSCOPED_INFO(** _message expression_ **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch2 2.7.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch 2.7.0.
Similar to `INFO`, but messages are not limited to their own scope: They are removed from the buffer after each assertion, section or test case, whichever comes first.

View File

@@ -1,123 +1,77 @@
<a id="top"></a>
# Matchers
**Contents**<br>
[Using Matchers](#using-matchers)<br>
[Built-in matchers](#built-in-matchers)<br>
[Writing custom matchers (old style)](#writing-custom-matchers-old-style)<br>
[Writing custom matchers (new style)](#writing-custom-matchers-new-style)<br>
Matchers are an alternative way to do assertions which are easily extensible and composable.
This makes them well suited to use with more complex types (such as collections) or your own custom types.
Matchers were first popularised by the [Hamcrest](https://en.wikipedia.org/wiki/Hamcrest) family of frameworks.
Matchers, as popularized by the [Hamcrest](https://en.wikipedia.org/wiki/Hamcrest)
framework are an alternative way to write assertions, useful for tests
where you work with complex types or need to assert more complex
properties. Matchers are easily composable and users can write their
own and combine them with the Catch2-provided matchers seamlessly.
## In use
Matchers are introduced with the `REQUIRE_THAT` or `CHECK_THAT` macros, which take two arguments.
The first argument is the thing (object or value) under test. The second part is a match _expression_,
which consists of either a single matcher or one or more matchers combined using `&&`, `||` or `!` operators.
## Using Matchers
For example, to assert that a string ends with a certain substring:
Matchers are most commonly used in tandem with the `REQUIRE_THAT` or
`CHECK_THAT` macros. The `REQUIRE_THAT` macro takes two arguments,
the first one is the input (object/value) to test, the second argument
is the matcher itself.
For example, to assert that a string ends with the "as a service"
substring, you can write the following assertion
```cpp
using Catch::Matchers::EndsWith;
REQUIRE_THAT( getSomeString(), EndsWith("as a service") );
```c++
using Catch::Matchers::EndsWith; // or Catch::EndsWith
std::string str = getStringFromSomewhere();
REQUIRE_THAT( str, EndsWith( "as a service" ) );
```
Individual matchers can also be combined using the C++ logical
operators, that is `&&`, `||`, and `!`, like so:
The matcher objects can take multiple arguments, allowing more fine tuning.
The built-in string matchers, for example, take a second argument specifying whether the comparison is
case sensitive or not:
```cpp
using Catch::Matchers::EndsWith;
using Catch::Matchers::ContainsSubstring;
```c++
REQUIRE_THAT( str, EndsWith( "as a service", Catch::CaseSensitive::No ) );
```
REQUIRE_THAT( getSomeString(),
EndsWith("as a service") && ContainsSubstring("web scale"));
And matchers can be combined:
```c++
REQUIRE_THAT( str,
EndsWith( "as a service" ) ||
(StartsWith( "Big data" ) && !Contains( "web scale" ) ) );
```
The example above asserts that the string returned from `getSomeString`
_both_ ends with the suffix "as a service" _and_ contains the string
"web scale" somewhere.
Both of the string matchers used in the examples above live in the
`catch_matchers_string.hpp` header, so to compile the code above also
requires `#include <catch2/matchers/catch_matchers_string.hpp>`.
**IMPORTANT**: The combining operators do not take ownership of the
matcher objects being combined. This means that if you store combined
matcher object, you have to ensure that the matchers being combined
outlive its last use. What this means is that the following code leads
to a use-after-free (UAF):
_The combining operators do not take ownership of the matcher objects.
This means that if you store the combined object, you have to ensure that
the matcher objects outlive its last use. What this means is that code
like this leads to a use-after-free and (hopefully) a crash:_
```cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_string.hpp>
TEST_CASE("Bugs, bugs, bugs", "[Bug]"){
std::string str = "Bugs as a service";
auto match_expression = Catch::Matchers::EndsWith( "as a service" ) ||
(Catch::Matchers::StartsWith( "Big data" ) && !Catch::Matchers::ContainsSubstring( "web scale" ) );
auto match_expression = Catch::EndsWith( "as a service" ) ||
(Catch::StartsWith( "Big data" ) && !Catch::Contains( "web scale" ) );
REQUIRE_THAT(str, match_expression);
}
```
## Built-in matchers
## Built in matchers
Catch2 provides some matchers by default. They can be found in the
`Catch::Matchers::foo` namespace and are imported into the `Catch`
namespace as well.
Every matcher provided by Catch2 is split into 2 parts, a factory
function that lives in the `Catch::Matchers` namespace, and the actual
matcher type that is in some deeper namespace and should not be used by
the user. In the examples above, we used `Catch::Matchers::Contains`.
This is the factory function for the
`Catch::Matchers::StdString::ContainsMatcher` type that does the actual
matching.
Out of the box, Catch2 provides the following matchers:
There are two parts to each of the built-in matchers, the matcher
type itself and a helper function that provides template argument
deduction when creating templated matchers. As an example, the matcher
for checking that two instances of `std::vector` are identical is
`EqualsMatcher<T>`, but the user is expected to use the `Equals`
helper function instead.
### `std::string` matchers
### String matchers
The string matchers are `StartsWith`, `EndsWith`, `Contains`, `Equals` and `Matches`. The first four match a literal (sub)string against a result, while `Matches` takes and matches an ECMAScript regex. Do note that `Matches` matches the string as a whole, meaning that "abc" will not match against "abcd", but "abc.*" will.
Catch2 provides 5 different matchers that work with `std::string`,
* `StartsWith(std::string str, CaseSensitive)`,
* `EndsWith(std::string str, CaseSensitive)`,
* `ContainsSubstring(std::string str, CaseSensitive)`,
* `Equals(std::string str, CaseSensitive)`, and
* `Matches(std::string str, CaseSensitive)`.
The first three should be fairly self-explanatory, they succeed if
the argument starts with `str`, ends with `str`, or contains `str`
somewhere inside it.
The `Equals` matcher matches a string if (and only if) the argument
string is equal to `str`.
Finally, the `Matches` matcher performs an ECMAScript regex match using
`str` against the argument string. It is important to know that
the match is performed against the string as a whole, meaning that
the regex `"abc"` will not match input string `"abcd"`. To match
`"abcd"`, you need to use e.g. `"abc.*"` as your regex.
The second argument sets whether the matching should be case-sensitive
or not. By default, it is case-sensitive.
> `std::string` matchers live in `catch2/matchers/catch_matchers_string.hpp`
Each of the provided `std::string` matchers also takes an optional second argument, that decides case sensitivity (by-default, they are case sensitive).
### Vector matchers
_Vector matchers have been deprecated in favour of the generic
range matchers with the same functionality._
Catch2 provides 5 built-in matchers that work on `std::vector`.
Catch2 currently provides 5 built-in matchers that work on `std::vector`.
These are
* `Contains` which checks whether a specified vector is present in the result
@@ -125,57 +79,42 @@ These are
* `Equals` which checks whether the result is exactly equal (order matters) to a specific vector
* `UnorderedEquals` which checks whether the result is equal to a specific vector under a permutation
* `Approx` which checks whether the result is "approx-equal" (order matters, but comparison is done via `Approx`) to a specific vector
> Approx matcher was [introduced](https://github.com/catchorg/Catch2/issues/1499) in Catch2 2.7.2.
An example usage:
```cpp
std::vector<int> some_vec{ 1, 2, 3 };
REQUIRE_THAT(some_vec, Catch::Matchers::UnorderedEquals(std::vector<int>{ 3, 2, 1 }));
```
This assertions will pass, because the elements given to the matchers
are a permutation of the ones in `some_vec`.
> vector matchers live in `catch2/matchers/catch_matchers_vector.hpp`
> Approx matcher was [introduced](https://github.com/catchorg/Catch2/issues/1499) in Catch 2.7.2.
### Floating point matchers
Catch2 provides 3 matchers for working with floating point numbers. These
are `WithinAbsMatcher`, `WithinUlpsMatcher` and `WithinRelMatcher`.
Catch2 provides 4 matchers that target floating point numbers. These
are:
The `WithinAbsMatcher` matcher accepts floating point numbers that are
within a certain distance of target. It should be constructed with the
`WithinAbs(double target, double margin)` helper.
* `WithinAbs(double target, double margin)`,
* `WithinULP(FloatingPoint target, uint64_t maxUlpDiff)`, and
* `WithinRel(FloatingPoint target, FloatingPoint eps)`.
* `IsNaN()`
The `WithinUlpsMatcher` matcher accepts floating point numbers that are
within a certain number of [ULPs](https://en.wikipedia.org/wiki/Unit_in_the_last_place)
of the target. Because ULP comparisons need to be done differently for
`float`s and for `double`s, there are two overloads of the helpers for
this matcher, `WithinULP(float target, int64_t ULPs)`, and
`WithinULP(double target, int64_t ULPs)`.
> `WithinRel` matcher was introduced in Catch2 2.10.0
The `WithinRelMatcher` matcher accepts floating point numbers that are
_approximately equal_ with the target number with some specific tolerance.
In other words, it checks that `|lhs - rhs| <= epsilon * max(|lhs|, |rhs|)`,
with special casing for `INFINITY` and `NaN`. There are _4_ overloads of
the helpers for this matcher, `WithinRel(double target, double margin)`,
`WithinRel(float target, float margin)`, `WithinRel(double target)`, and
`WithinRel(float target)`. The latter two provide a default epsilon of
machine epsilon * 100.
> `IsNaN` matcher was introduced in Catch2 3.3.2.
> `WithinRel` matcher was introduced in Catch 2.10.0
The first three serve to compare two floating pointe numbers. For more
details about how they work, read [the docs on comparing floating point
numbers](comparing-floating-point-numbers.md#floating-point-matchers).
`IsNaN` then does exactly what it says on the tin. It matches the input
if it is a NaN (Not a Number). The advantage of using it over just plain
`REQUIRE(std::isnan(x))`, is that if the check fails, with `REQUIRE` you
won't see the value of `x`, but with `REQUIRE_THAT(x, IsNaN())`, you will.
### Miscellaneous matchers
Catch2 also provides some matchers and matcher utilities that do not
quite fit into other categories.
The first one of them is the `Predicate(Callable pred, std::string description)`
matcher. It creates a matcher object that calls `pred` for the provided
argument. The `description` argument allows users to set what the
resulting matcher should self-describe as if required.
Do note that you will need to explicitly specify the type of the
argument, like in this example:
### Generic matchers
Catch also aims to provide a set of generic matchers. Currently this set
contains only a matcher that takes arbitrary callable predicate and applies
it onto the provided object.
Because of type inference limitations, the argument type of the predicate
has to be provided explicitly. Example:
```cpp
REQUIRE_THAT("Hello olleH",
Predicate<std::string>(
@@ -184,256 +123,84 @@ REQUIRE_THAT("Hello olleH",
);
```
> the predicate matcher lives in `catch2/matchers/catch_matchers_predicate.hpp`
The second argument is an optional description of the predicate, and is
used only during reporting of the result.
The other miscellaneous matcher utility is exception matching.
### Exception matchers
Catch2 also provides an exception matcher that can be used to verify
that an exception's message exactly matches desired string. The matcher
is `ExceptionMessageMatcher`, and we also provide a helper function
`Message`.
The matched exception must publicly derive from `std::exception` and
the message matching is done _exactly_, including case.
#### Matching exceptions
Catch2 provides a utility macro for asserting that an expression
throws exception of specific type, and that the exception has desired
properties. The macro is `REQUIRE_THROWS_MATCHES(expr, ExceptionType, Matcher)`.
> `REQUIRE_THROWS_MATCHES` macro lives in `catch2/matchers/catch_matchers.hpp`
Catch2 currently provides two matchers for exceptions.
These are:
* `Message(std::string message)`.
* `MessageMatches(Matcher matcher)`.
> `MessageMatches` was [introduced](https://github.com/catchorg/Catch2/pull/2570) in Catch2 3.3.0
`Message` checks that the exception's
message, as returned from `what` is exactly equal to `message`.
`MessageMatches` applies the provided matcher on the exception's
message, as returned from `what`. This is useful in conjunctions with the `std::string` matchers (e.g. `StartsWith`)
> `ExceptionMessageMatcher` was introduced in Catch 2.10.0
Example use:
```cpp
REQUIRE_THROWS_MATCHES(throwsDerivedException(), DerivedException, Message("DerivedException::what"));
REQUIRE_THROWS_MATCHES(throwsDerivedException(), DerivedException, MessageMatches(StartsWith("DerivedException")));
```
Note that `DerivedException` in the example above has to derive from
`std::exception` for the example to work.
## Custom matchers
It's easy to provide your own matchers to extend Catch or just to work with your own types.
> the exception message matcher lives in `catch2/matchers/catch_matchers_exception.hpp`
You need to provide two things:
1. A matcher class, derived from `Catch::MatcherBase<T>` - where `T` is the type being tested.
The constructor takes and stores any arguments needed (e.g. something to compare against) and you must
override two methods: `match()` and `describe()`.
2. A simple builder function. This is what is actually called from the test code and allows overloading.
### Generic range Matchers
> Generic range matchers were introduced in Catch2 3.0.1
Catch2 also provides some matchers that use the new style matchers
definitions to handle generic range-like types. These are:
* `IsEmpty()`
* `SizeIs(size_t target_size)`
* `SizeIs(Matcher size_matcher)`
* `Contains(T&& target_element, Comparator = std::equal_to<>{})`
* `Contains(Matcher element_matcher)`
* `AllMatch(Matcher element_matcher)`
* `AnyMatch(Matcher element_matcher)`
* `NoneMatch(Matcher element_matcher)`
* `AllTrue()`, `AnyTrue()`, `NoneTrue()`
* `RangeEquals(TargetRangeLike&&, Comparator = std::equal_to<>{})`
* `UnorderedRangeEquals(TargetRangeLike&&, Comparator = std::equal_to<>{})`
> `IsEmpty`, `SizeIs`, `Contains` were introduced in Catch2 3.0.1
> `All/Any/NoneMatch` were introduced in Catch2 3.0.1
> `All/Any/NoneTrue` were introduced in Catch2 3.1.0
> `RangeEquals` and `UnorderedRangeEquals` matchers were [introduced](https://github.com/catchorg/Catch2/pull/2377) in Catch2 3.3.0
`IsEmpty` should be self-explanatory. It successfully matches objects
that are empty according to either `std::empty`, or ADL-found `empty`
free function.
`SizeIs` checks range's size. If constructed with `size_t` arg, the
matchers accepts ranges whose size is exactly equal to the arg. If
constructed from another matcher, then the resulting matcher accepts
ranges whose size is accepted by the provided matcher.
`Contains` accepts ranges that contain specific element. There are
again two variants, one that accepts the desired element directly,
in which case a range is accepted if any of its elements is equal to
the target element. The other variant is constructed from a matcher,
in which case a range is accepted if any of its elements is accepted
by the provided matcher.
`AllMatch`, `NoneMatch`, and `AnyMatch` match ranges for which either
all, none, or any of the contained elements matches the given matcher,
respectively.
`AllTrue`, `NoneTrue`, and `AnyTrue` match ranges for which either
all, none, or any of the contained elements are `true`, respectively.
It works for ranges of `bool`s and ranges of elements (explicitly)
convertible to `bool`.
`RangeEquals` compares the range that the matcher is constructed with
(the "target range") against the range to be tested, element-wise. The
match succeeds if all elements from the two ranges compare equal (using
`operator==` by default). The ranges do not need to be the same type,
and the element types do not need to be the same, as long as they are
comparable. (e.g. you may compare `std::vector<int>` to `std::array<char>`).
`UnorderedRangeEquals` is similar to `RangeEquals`, but the order
does not matter. For example "1, 2, 3" would match "3, 2, 1", but not
"1, 1, 2, 3" As with `RangeEquals`, `UnorderedRangeEquals` compares
the individual elements using using `operator==` by default.
Both `RangeEquals` and `UnorderedRangeEquals` optionally accept a
predicate which can be used to compare the containers element-wise.
To check a container elementwise against a given matcher, use
`AllMatch`.
## Writing custom matchers (old style)
The old style of writing matchers has been introduced back in Catch
Classic. To create an old-style matcher, you have to create your own
type that derives from `Catch::Matchers::MatcherBase<ArgT>`, where
`ArgT` is the type your matcher works for. Your type has to override
two methods, `bool match(ArgT const&) const`,
and `std::string describe() const`.
As the name suggests, `match` decides whether the provided argument
is matched (accepted) by the matcher. `describe` then provides a
human-oriented description of what the matcher does.
We also recommend that you create factory function, just like Catch2
does, but that is mostly useful for template argument deduction for
templated matchers (assuming you do not have CTAD available).
To combine these into an example, let's say that you want to write
a matcher that decides whether the provided argument is a number
within certain range. We will call it `IsBetweenMatcher<T>`:
Here's an example for asserting that an integer falls within a given range
(note that it is all inline for the sake of keeping the example short):
```c++
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers.hpp>
// ...
template <typename T>
class IsBetweenMatcher : public Catch::Matchers::MatcherBase<T> {
T m_begin, m_end;
// The matcher class
class IntRange : public Catch::MatcherBase<int> {
int m_begin, m_end;
public:
IsBetweenMatcher(T begin, T end) : m_begin(begin), m_end(end) {}
IntRange( int begin, int end ) : m_begin( begin ), m_end( end ) {}
bool match(T const& in) const override {
return in >= m_begin && in <= m_end;
// Performs the test for this matcher
bool match( int const& i ) const override {
return i >= m_begin && i <= m_end;
}
std::string describe() const override {
// Produces a string describing what this matcher does. It should
// include any provided data (the begin/ end in this case) and
// be written as if it were stating a fact (in the output it will be
// preceded by the value under test).
virtual std::string describe() const override {
std::ostringstream ss;
ss << "is between " << m_begin << " and " << m_end;
return ss.str();
}
};
template <typename T>
IsBetweenMatcher<T> IsBetween(T begin, T end) {
return { begin, end };
// The builder function
inline IntRange IsBetween( int begin, int end ) {
return IntRange( begin, end );
}
// ...
TEST_CASE("Numbers are within range") {
// infers `double` for the argument type of the matcher
CHECK_THAT(3., IsBetween(1., 10.));
// infers `int` for the argument type of the matcher
CHECK_THAT(100, IsBetween(1, 10));
// Usage
TEST_CASE("Integers are within a range")
{
CHECK_THAT( 3, IsBetween( 1, 10 ) );
CHECK_THAT( 100, IsBetween( 1, 10 ) );
}
```
Obviously, the code above can be improved somewhat, for example you
might want to `static_assert` over the fact that `T` is an arithmetic
type... or generalize the matcher to cover any type for which the user
can provide a comparison function object.
Running this test gives the following in the console:
Note that while any matcher written using the old style can also be
written using the new style, combining old style matchers should
generally compile faster. Also note that you can combine old and new
style matchers arbitrarily.
> `MatcherBase` lives in `catch2/matchers/catch_matchers.hpp`
## Writing custom matchers (new style)
> New style matchers were introduced in Catch2 3.0.1
To create a new-style matcher, you have to create your own type that
derives from `Catch::Matchers::MatcherGenericBase`. Your type has to
also provide two methods, `bool match( ... ) const` and overridden
`std::string describe() const`.
Unlike with old-style matchers, there are no requirements on how
the `match` member function takes its argument. This means that the
argument can be taken by value or by mutating reference, but also that
the matcher's `match` member function can be templated.
This allows you to write more complex matcher, such as a matcher that
can compare one range-like (something that responds to `begin` and
`end`) object to another, like in the following example:
```cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_templated.hpp>
// ...
template<typename Range>
struct EqualsRangeMatcher : Catch::Matchers::MatcherGenericBase {
EqualsRangeMatcher(Range const& range):
range{ range }
{}
template<typename OtherRange>
bool match(OtherRange const& other) const {
using std::begin; using std::end;
return std::equal(begin(range), end(range), begin(other), end(other));
}
std::string describe() const override {
return "Equals: " + Catch::rangeToString(range);
}
private:
Range const& range;
};
template<typename Range>
auto EqualsRange(const Range& range) -> EqualsRangeMatcher<Range> {
return EqualsRangeMatcher<Range>{range};
}
TEST_CASE("Combining templated matchers", "[matchers][templated]") {
std::array<int, 3> container{{ 1,2,3 }};
std::array<int, 3> a{{ 1,2,3 }};
std::vector<int> b{ 0,1,2 };
std::list<int> c{ 4,5,6 };
REQUIRE_THAT(container, EqualsRange(a) || EqualsRange(b) || EqualsRange(c));
}
```
Do note that while you can rewrite any matcher from the old style to
a new style matcher, combining new style matchers is more expensive
in terms of compilation time. Also note that you can combine old style
and new style matchers arbitrarily.
> `MatcherGenericBase` lives in `catch2/matchers/catch_matchers_templated.hpp`
/**/TestFile.cpp:123: FAILED:
CHECK_THAT( 100, IsBetween( 1, 10 ) )
with expansion:
100 is between 1 and 10
```
---

View File

@@ -1,98 +0,0 @@
<a id="top"></a>
# Migrating from v2 to v3
v3 is the next major version of Catch2 and brings three significant changes:
* Catch2 is now split into multiple headers
* Catch2 is now compiled as a static library
* C++14 is the minimum required C++ version
There are many reasons why we decided to go from the old single-header
distribution model to a more standard library distribution model. The
big one is compile-time performance, but moving over to a split header
distribution model also improves the future maintainability and
extendability of the codebase. For example v3 adds a new kind of matchers
without impacting the compilation times of users that do not use matchers
in their tests. The new model is also more friendly towards package
managers, such as vcpkg and Conan.
The result of this move is a significant improvement in compilation
times, e.g. the inclusion overhead of Catch2 in the common case has been
reduced by roughly 80%. The improved ease of maintenance also led to
various runtime performance improvements and the introduction of new features.
For details, look at [the release notes of 3.0.1](release-notes.md#301).
_Note that we still provide one header + one translation unit (TU)
distribution but do not consider it the primarily supported option. You
should also expect that the compilation times will be worse if you use
this option._
## How to migrate projects from v2 to v3
To migrate to v3, there are two basic approaches to do so.
1. Use `catch_amalgamated.hpp` and `catch_amalgamated.cpp`.
2. Build Catch2 as a proper (static) library, and move to piecewise headers
Doing 1 means downloading the [amalgamated header](/extras/catch_amalgamated.hpp)
and the [amalgamated sources](/extras/catch_amalgamated.cpp) from `extras`,
dropping them into your test project, and rewriting your includes from
`<catch2/catch.hpp>` to `"catch_amalgamated.hpp"` (or something similar,
based on how you set up your paths).
The disadvantage of using this approach are increased compilation times,
at least compared to the second approach, but it does let you avoid
dealing with consuming libraries in your build system of choice.
However, we recommend doing 2, and taking extra time to migrate to v3
properly. This lets you reap the benefits of significantly improved
compilation times in the v3 version. The basic steps to do so are:
1. Change your CMakeLists.txt to link against `Catch2WithMain` target if
you use Catch2's default main. (If you do not, keep linking against
the `Catch2` target.). If you use pkg-config, change `pkg-config catch2` to
`pkg-config catch2-with-main`.
2. Delete TU with `CATCH_CONFIG_RUNNER` or `CATCH_CONFIG_MAIN` defined,
as it is no longer needed.
3. Change `#include <catch2/catch.hpp>` to `#include <catch2/catch_all.hpp>`
4. Check that everything compiles. You might have to modify namespaces,
or perform some other changes (see the
[Things that can break during porting](#things-that-can-break-during-porting)
section for the most common things).
5. Start migrating your test TUs from including `<catch2/catch_all.hpp>`
to piecemeal includes. You will likely want to start by including
`<catch2/catch_test_macros.hpp>`, and then go from there. (see
[other notes](#other-notes) for further ideas)
## Other notes
* The main test include is now `<catch2/catch_test_macros.hpp>`
* Big "subparts" like Matchers, or Generators, have their own folder, and
also their own "big header", so if you just want to include all matchers,
you can include `<catch2/matchers/catch_matchers_all.hpp>`,
or `<catch2/generators/catch_generators_all.hpp>`
## Things that can break during porting
* The namespaces of Matchers were flattened and cleaned up.
Matchers are no longer declared deep within an internal namespace and
then brought up into `Catch` namespace. All Matchers now live in the
`Catch::Matchers` namespace.
* The `Contains` string matcher was renamed to `ContainsSubstring`.
* The reporter interfaces changed in a breaking manner.
If you are using a custom reporter or listener, you will likely need to
modify them to conform to the new interfaces. Unlike before in v2,
the [interfaces](reporters.md#top) and the [events](reporter-events.md#top)
are now documented.
---
[Home](Readme.md#top)

View File

@@ -1,28 +1,22 @@
<a id="top"></a>
# Open Source projects using Catch2
# Open Source projects using Catch
Catch2 is great for open source. It is licensed under the [Boost Software
License (BSL)](../LICENSE.txt), has no further dependencies and supports
two file distribution.
Catch is great for open source. With its [liberal license](../LICENSE.txt) and single-header, dependency-free, distribution
it's easy to just drop the header into your project and start writing tests - what's not to like?
As a result, Catch2 is used for testing in many different Open Source
projects. This page lists at least some of them, even though it will
obviously never be complete (and does not have the ambition to be
complete). Note that the list below is intended to be in alphabetical
order, to avoid implications of relative importance of the projects.
_Please only add projects here if you are their maintainer, or have the
maintainer's explicit consent._
As a result Catch is now being used in many Open Source projects, including some quite well known ones.
This page is an attempt to track those projects. Obviously it can never be complete.
This effort largely relies on the maintainers of the projects themselves updating this page and submitting a PR
(or, if you prefer contact one of the maintainers of Catch directly, use the
[forums](https://groups.google.com/forum/?fromgroups#!forum/catch-forum)), or raise an [issue](https://github.com/philsquared/Catch/issues) to let us know).
Of course users of those projects might want to update this page too. That's fine - as long you're confident the project maintainers won't mind.
If you're an Open Source project maintainer and see your project listed here but would rather it wasn't -
just let us know via any of the previously mentioned means - although I'm sure there won't be many who feel that way.
Listing a project here does not imply endorsement and the plan is to keep these ordered alphabetically to avoid an implication of relative importance.
## Libraries & Frameworks
### [accessorpp](https://github.com/wqking/accessorpp)
C++ library for implementing property and data binding.
### [alpaka](https://github.com/alpaka-group/alpaka)
A header-only C++14 abstraction library for accelerator development.
### [ApprovalTests.cpp](https://github.com/approvals/ApprovalTests.cpp)
C++11 implementation of Approval Tests, for quick, convenient testing of legacy code.
@@ -53,26 +47,17 @@ Header-only C++11 library to encode/decode base64, base64url, base32, base32hex
### [DtCraft](https://github.com/twhuang-uiuc/DtCraft)
A High-performance Cluster Computing Engine.
### [eventpp](https://github.com/wqking/eventpp)
C++ event library for callbacks, event dispatcher, and event queue. With eventpp you can easily implement signal and slot mechanism, publisher and subscriber pattern, or observer pattern.
### [forest](https://github.com/xorz57/forest)
Template Library of Tree Data Structures.
### [Fuxedo](https://github.com/fuxedo/fuxedo)
Open source Oracle Tuxedo-like XATMI middleware for C and C++.
### [HIP CPU Runtime](https://github.com/ROCm-Developer-Tools/HIP-CPU)
A header-only library that allows CPUs to execute unmodified HIP code. It is generic and does not assume a particular CPU vendor or architecture.
### [Inja](https://github.com/pantor/inja)
A header-only template engine for modern C++.
### [LLAMA](https://github.com/alpaka-group/llama)
A C++17 template header-only library for the abstraction of memory access patterns.
### [libcluon](https://github.com/chrberger/libcluon)
A single-header-only library written in C++14 to glue distributed software components (UDP, TCP, shared memory) supporting natively Protobuf, LCM/ZCM, MsgPack, and JSON for dynamic message transformations in-between.
A single-header-only library written in C++14 to glue distributed software components (UDP, TCP, shared memory) supporting natively Protobuf, LCM/ZCM, MsgPack, and JSON for dynamic message transformations in-between.
### [MNMLSTC Core](https://github.com/mnmlstc/core)
A small and easy to use C++11 library that adds a functionality set that will be available in C++14 and later, as well as some useful additions.
@@ -83,9 +68,6 @@ A small C++ library wrapper for the native C ODBC API.
### [Nonius](https://github.com/libnonius/nonius)
A header-only framework for benchmarking small snippets of C++ code.
### [OpenALpp](https://github.com/Laguna1989/OpenALpp)
A modern OOP C++14 audio library built on OpenAL for Windows, Linux and web (emscripten).
### [polymorphic_value](https://github.com/jbcoe/polymorphic_value)
A polymorphic value-type for C++.
@@ -118,12 +100,6 @@ A high available cloud native micro-service application management platform impl
### [ArangoDB](https://github.com/arangodb/arangodb)
ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values.
### [Cytopia](https://github.com/CytopiaTeam/Cytopia)
Cytopia is a free, open source retro pixel-art city building game with a big focus on mods. It utilizes a custom isometric rendering engine based on SDL2.
### [d-SEAMS](https://github.com/d-SEAMS/seams-core)
Open source molecular dynamics simulation structure analysis suite of tools in modern C++.
### [Giada - Your Hardcore Loop Machine](https://github.com/monocasual/giada)
Minimal, open-source and cross-platform audio tool for live music production.

View File

@@ -15,8 +15,6 @@ stringification machinery to the _expr_ and records the result. As with
evaluates to `true`. `CHECKED_ELSE( expr )` work similarly, but the block
is entered only if the _expr_ evaluated to `false`.
> `CHECKED_X` macros were changed to not count as failure in Catch2 3.0.1.
Example:
```cpp
int a = ...;
@@ -59,9 +57,9 @@ TEST_CASE( "SUCCEED showcase" ) {
}
```
* `STATIC_REQUIRE` and `STATIC_CHECK`
* `STATIC_REQUIRE`
> `STATIC_REQUIRE` was [introduced](https://github.com/catchorg/Catch2/issues/1362) in Catch2 2.4.2.
> [Introduced](https://github.com/catchorg/Catch2/issues/1362) in Catch 2.4.2.
`STATIC_REQUIRE( expr )` is a macro that can be used the same way as a
`static_assert`, but also registers the success with Catch2, so it is
@@ -77,20 +75,6 @@ TEST_CASE("STATIC_REQUIRE showcase", "[traits]") {
}
```
> `STATIC_CHECK` was [introduced](https://github.com/catchorg/Catch2/pull/2318) in Catch2 3.0.1.
`STATIC_CHECK( expr )` is equivalent to `STATIC_REQUIRE( expr )`, with the
difference that when `CATCH_CONFIG_RUNTIME_STATIC_REQUIRE` is defined, it
becomes equivalent to `CHECK` instead of `REQUIRE`.
Example:
```cpp
TEST_CASE("STATIC_CHECK showcase", "[traits]") {
STATIC_CHECK( std::is_void<void>::value );
STATIC_CHECK_FALSE( std::is_void<int>::value );
}
```
## Test case related macros
* `METHOD_AS_TEST_CASE`
@@ -133,9 +117,24 @@ is initiated. This means that it either needs to be done in a global
constructor, or before Catch2's session is created in user's own main._
* `ANON_TEST_CASE`
`ANON_TEST_CASE` is a `TEST_CASE` replacement that will autogenerate
unique name. The advantage of this is that you do not have to think
of a name for the test case, the disadvantage is that the name doesn't
necessarily remain stable across different links, and thus it might be
hard to run directly.
Example:
```cpp
ANON_TEST_CASE() {
SUCCEED("Hello from anonymous test case");
}
```
* `DYNAMIC_SECTION`
> Introduced in Catch2 2.3.0.
> Introduced in Catch 2.3.0.
`DYNAMIC_SECTION` is a `SECTION` where the user can use `operator<<` to
create the final name for that section. This can be useful with e.g.

View File

@@ -2,69 +2,63 @@
# Supplying main() yourself
**Contents**<br>
[Let Catch2 take full control of args and config](#let-catch2-take-full-control-of-args-and-config)<br>
[Amending the Catch2 config](#amending-the-catch2-config)<br>
[Let Catch take full control of args and config](#let-catch-take-full-control-of-args-and-config)<br>
[Amending the config](#amending-the-config)<br>
[Adding your own command line options](#adding-your-own-command-line-options)<br>
[Version detection](#version-detection)<br>
The easiest way to use Catch2 is to use its own `main` function, and let
it handle the command line arguments. This is done by linking against
Catch2Main library, e.g. through the [CMake target](cmake-integration.md#cmake-targets),
or pkg-config files.
The easiest way to use Catch is to let it supply ```main()``` for you and handle configuring itself from the command line.
If you want to provide your own `main`, then you should link against
the static library (target) only, without the main part. You will then
have to write your own `main` and call into Catch2 test runner manually.
This is achieved by writing ```#define CATCH_CONFIG_MAIN``` before the ```#include "catch.hpp"``` in *exactly one* source file.
Below are some basic recipes on what you can do supplying your own main.
Sometimes, though, you need to write your own version of main(). You can do this by writing ```#define CATCH_CONFIG_RUNNER``` instead. Now you are free to write ```main()``` as normal and call into Catch yourself manually. You now have a lot of flexibility - but here are three recipes to get your started:
**Important note: you can only provide `main` in the same file you defined `CATCH_CONFIG_RUNNER`.**
## Let Catch2 take full control of args and config
## Let Catch take full control of args and config
This is useful if you just need to have code that executes before/after
Catch2 runs tests.
If you just need to have code that executes before and/ or after Catch this is the simplest option.
```cpp
#include <catch2/catch_session.hpp>
```c++
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
int main( int argc, char* argv[] ) {
// your setup ...
// global setup...
int result = Catch::Session().run( argc, argv );
// your clean-up...
// global clean-up...
return result;
}
```
_Note that if you only want to run some set up before tests are run, it
might be simpler to use [event listeners](event-listeners.md#top) instead._
## Amending the config
## Amending the Catch2 config
If you want Catch2 to process command line arguments, but also want to
programmatically change the resulting configuration of Catch2 run,
you can do it in two ways:
If you still want Catch to process the command line, but you want to programmatically tweak the config, you can do so in one of two ways:
```c++
int main( int argc, char* argv[] ) {
Catch::Session session; // There must be exactly one instance
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
int main( int argc, char* argv[] )
{
Catch::Session session; // There must be exactly one instance
// writing to session.configData() here sets defaults
// this is the preferred way to set them
int returnCode = session.applyCommandLine( argc, argv );
if( returnCode != 0 ) // Indicates a command line error
return returnCode;
// writing to session.configData() or session.Config() here
// writing to session.configData() or session.Config() here
// overrides command line args
// only do this if you know you need to
int numFailed = session.run();
// numFailed is clamped to 255 as some unices only use the lower 8 bits.
// This clamping has already been applied, so just return it here
// You can also do any post run clean-up here
@@ -72,32 +66,38 @@ int main( int argc, char* argv[] ) {
}
```
If you want full control of the configuration, don't call `applyCommandLine`.
Take a look at the definitions of Config and ConfigData to see what you can do with them.
To take full control of the config simply omit the call to ```applyCommandLine()```.
## Adding your own command line options
You can add new command line options to Catch2, by composing the premade
CLI parser (called Clara), and add your own options.
Catch embeds a powerful command line parser called [Clara](https://github.com/philsquared/Clara).
As of Catch2 (and Clara 1.0) Clara allows you to write _composable_ option and argument parsers,
so extending Catch's own command line options is now easy.
```cpp
int main( int argc, char* argv[] ) {
```c++
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
int main( int argc, char* argv[] )
{
Catch::Session session; // There must be exactly one instance
int height = 0; // Some user variable you want to be able to set
// Build a new parser on top of Catch2's
// Build a new parser on top of Catch's
using namespace Catch::Clara;
auto cli
= session.cli() // Get Catch2's command line parser
auto cli
= session.cli() // Get Catch's composite command line parser
| Opt( height, "height" ) // bind variable to a new option, with a hint string
["-g"]["--height"] // the option names it will respond to
("how high?"); // description string for the help output
// Now pass the new composite back to Catch2 so it uses that
session.cli( cli );
// Let Catch2 (using Clara) parse the command line
// Now pass the new composite back to Catch so it uses that
session.cli( cli );
// Let Catch (using Clara) parse the command line
int returnCode = session.applyCommandLine( argc, argv );
if( returnCode != 0 ) // Indicates a command line error
return returnCode;
@@ -110,13 +110,12 @@ int main( int argc, char* argv[] ) {
}
```
See the [Clara documentation](https://github.com/catchorg/Clara/blob/master/README.md)
for more details on how to use the Clara parser.
See the [Clara documentation](https://github.com/philsquared/Clara/blob/master/README.md) for more details.
## Version detection
Catch2 provides a triplet of macros providing the header's version,
Catch provides a triplet of macros providing the header's version,
* `CATCH_VERSION_MAJOR`
* `CATCH_VERSION_MINOR`

View File

@@ -2,14 +2,9 @@
# Release notes
**Contents**<br>
[3.3.2](#332)<br>
[3.3.1](#331)<br>
[3.3.0](#330)<br>
[3.2.1](#321)<br>
[3.2.0](#320)<br>
[3.1.1](#311)<br>
[3.1.0](#310)<br>
[3.0.1](#301)<br>
[2.13.10](#21310)<br>
[2.13.9](#2139)<br>
[2.13.8](#2138)<br>
[2.13.7](#2137)<br>
[2.13.6](#2136)<br>
[2.13.5](#2135)<br>
@@ -56,363 +51,32 @@
[Even Older versions](#even-older-versions)<br>
## 3.3.2
### Improvements
* Further reduced allocations
* The compact, console, TAP and XML reporters perform less allocations in various cases
* Removed 1 allocation per entered `SECTION`/`TEST_CASE`.
* Removed 2 allocations per test case exit, if stdout/stderr is captured
* Improved performance
* Section tracking is 10%-25% faster than in v3.3.0
* Assertion handling is 5%-10% faster than in v3.3.0
* Test case registration is 1%-2% faster than in v3.3.0
* Tiny speedup for registering listeners
* Tiny speedup for `CAPTURE`, `TEST_CASE_METHOD`, `METHOD_AS_TEST_CASE`, and `TEMPLATE_LIST_TEST_*` macros.
* `Contains`, `RangeEquals` and `UnorderedRangeEquals` matchers now support ranges with iterator + sentinel pair
* Added `IsNaN` matcher
* Unlike `REQUIRE(isnan(x))`, `REQUIRE_THAT(x, IsNaN())` shows you the value of `x`.
* Suppressed `declared_but_not_referenced` warning for NVHPC (#2637)
## 2.13.10
### Fixes
* Fixed performance regression in section tracking introduced in v3.3.1
* Extreme cases would cause the tracking to run about 4x slower than in 3.3.0
* Fixed issue with `catch_discover_tests` when there is multiple of 256 tests (#2401, #2503)
* Catch2-provided `main` and `wmain` are explicitly marked as `__cdecl` when compiled with MSVC (#2486, #2487)
* Improved break-into-debugger behaviour for ARM Macs. It should now be possible to step execution after the break (#2422)
* Replaced deprecated `std::aligned_storage` (#2419, #2420)
## 3.3.1
### Improvements
* Reduced allocations and improved performance
* The exact improvements are dependent on your usage of Catch2.
* For example running Catch2's SelfTest binary performs 8k less allocations.
* The main improvement comes from smarter handling of `SECTION`s, especially sibling `SECTION`s
## 3.3.0
### Improvements
* Added `MessageMatches` exception matcher (#2570)
* Added `RangeEquals` and `UnorderedRangeEquals` generic range matchers (#2377)
* Added `SKIP` macro for skipping tests from within the test body (#2360)
* All built-in reporters have been extended to handle it properly, whether your custom reporter needs changes depends on how it was written
* `skipTest` reporter event **is unrelated** to this, and has been deprecated since it has practically no uses
* Restored support for PPC Macs in the break-into-debugger functionality (#2619)
* Made our warning suppression compatible with CUDA toolkit pre 11.5 (#2626)
* Cleaned out some static analysis complaints
## 2.13.9
### Fixes
* Fixed macro redefinition warning when NVCC was reporting as MSVC (#2603)
* Fixed throws in generator constructor causing the whole binary to abort (#2615)
* Now it just fails the test
* Fixed missing transitive include with libstdc++13 (#2611)
* Fixed issue with `-#` (filename-as-tag) flag when `__FILE__` expands into filename without directories (#2328, #2393)
* Fixed `CAPTURE` macro not being variadic when disabled through `CATCH_CONFIG_DISABLE` (#2316, #2378)
### Miscellaneous
* Improved support for dynamic library build with non-MSVC compilers on Windows (#2630)
* When used as a subproject, Catch2 keeps its generated header in a separate directory from the main project (#2604)
## 3.2.1
### Improvements
* Fix the reworked decomposer to work with older (pre 9) GCC versions (#2571)
* **This required more significant changes to properly support C++20, there might be bugs.**
## 3.2.0
### Improvements
* Catch2 now compiles on PlayStation (#2562)
* Added `CATCH_CONFIG_GETENV` compile-time toggle (#2562)
* This toggle guards whether Catch2 calls `std::getenv` when reading env variables
* Added support for more Bazel test environment variables
* `TESTBRIDGE_TEST_ONLY` is now supported (#2490)
* Sharding variables, `TEST_SHARD_INDEX`, `TEST_TOTAL_SHARDS`, `TEST_SHARD_STATUS_FILE`, are now all supported (#2491)
* Bunch of small tweaks and improvements in reporters
* The TAP and SonarQube reporters output the used test filters
* The XML reporter now also reports the version of its output format
* The compact reporter now uses the same summary output as the console reporter (#878, #2554)
* Added support for asserting on types that can only be compared with literal 0 (#2555)
* A canonical example is C++20's `std::*_ordering` types, which cannot be compared with an `int` variable, only `0`
* The support extends to any type with this property, not just the ones in stdlib
* This change imposes 2-3% slowdown on compiling files that are heavy on `REQUIRE` and friends
* **This required significant rewrite of decomposition, there might be bugs**
* Simplified internals of matcher related macros
* This provides about ~2% speed up compiling files that are heavy on `REQUIRE_THAT` and friends
## 2.13.8
### Fixes
* Cleaned out some warnings and static analysis issues
* Suppressed `-Wcomma` warning rarely occuring in templated test cases (#2543)
* Constified implementation details in `INFO` (#2564)
* Made `MatcherGenericBase` copy constructor const (#2566)
* Fixed serialization of test filters so the output roundtrips
* This means that e.g. `./tests/SelfTest "aaa bbb", [approx]` outputs `Filters: "aaa bbb",[approx]`
### Miscellaneous
* Catch2's build no longer leaks `-ffile-prefix-map` setting to dependees (#2533)
## 3.1.1
* Made `Approx::operator()` const (#2288)
* Improved pkg-config files (#2284)
* Fixed warning suppression leaking out of Catch2 when compiled with clang.exe (#2280)
* The macro-generated names for things like `TEST_CASE` no longer create reserved identifiers (#2336)
### Improvements
* Added `Catch::getSeed` function that user code can call to retrieve current rng-seed
* Better detection of compiler support for `-ffile-prefix-map` (#2517)
* Catch2's shared libraries now have `SOVERSION` set (#2516)
* `catch2/catch_all.hpp` convenience header no longer transitively includes `windows.h` (#2432, #2526)
### Fixes
* Fixed compilation on Universal Windows Platform
* Fixed compilation on VxWorks (#2515)
* Fixed compilation on Cygwin (#2540)
* Remove unused variable in reporter registration (#2538)
* Fixed some symbol visibility issues with dynamic library on Windows (#2527)
* Suppressed `-Wuseless-cast` warnings in `REQUIRE_THROWS*` macros (#2520, #2521)
* This was triggered when the potentially throwing expression evaluates to `void`
* Fixed "warning: storage class is not first" with `nvc++` (#2533)
* Fixed handling of `DL_PATHS` argument to `catch_discover_tests` on MacOS (#2483)
* Suppressed `*-avoid-c-arrays` clang-tidy warning in `TEMPLATE_TEST_CASE` (#2095, #2536)
### Miscellaneous
* Fixed CMake install step for Catch2 build as dynamic library (#2485)
* Raised minimum CMake version to 3.10 (#2523)
* Expect the minimum CMake version to increase once more in next few releases.
* Whole bunch of doc updates and fixes
* #1444, #2497, #2547, #2549, and more
* Added support for building Catch2 with Meson (#2530, #2539)
## 3.1.0
### Improvements
* Improved suppression of `-Wparentheses` for older GCCs
* Turns out that even GCC 9 does not properly handle `_Pragma`s in the C++ frontend.
* Added type constraints onto `random` generator (#2433)
* These constraints copy what the standard says for the underlying `std::uniform_int_distribution`
* Suppressed -Wunused-variable from nvcc (#2306, #2427)
* Suppressed -Wunused-variable from MinGW (#2132)
* Added All/Any/NoneTrue range matchers (#2319)
* These check that all/any/none of boolean values in a range are true.
* The JUnit reporter now normalizes classnames from C++ namespaces to Java-like namespaces (#2468)
* This provides better support for other JUnit based tools.
* The Bazel support now understands `BAZEL_TEST` environment variable (#2459)
* The `CATCH_CONFIG_BAZEL_SUPPORT` configuration option is also still supported.
* Returned support for compiling Catch2 with GCC 5 (#2448)
* This required removing inherited constructors from Catch2's internals.
* I recommend updating to a newer GCC anyway.
* `catch_discover_tests` now has a new options for setting library load path(s) when running the Catch2 binary (#2467)
### Fixes
* Fixed crash when listing listeners without any registered listeners (#2442)
* Fixed nvcc compilation error in constructor benchmarking helper (#2477)
* Catch2's CMakeList supports pre-3.12 CMake again (#2428)
* The gain from requiring CMake 3.12 was very minor, but y'all should really update to newer CMake
### Miscellaneous
* Fixed SelfTest build on MinGW (#2447)
* The in-repo conan recipe exports the CMake helper (#2460)
* Added experimental CMake script to showcase using test case sharding together with CTest
* Compared to `catch_discover_tests`, it supports very limited number of options and customization
* Added documentation page on best practices when running Catch2 tests
* Catch2 can be built as a dynamic library (#2397, #2398)
* Note that Catch2 does not have visibility annotations, and you are responsible for ensuring correct visibility built into the resulting library.
## 3.0.1
**Catch2 now uses statically compiled library as its distribution model.
This also means that to get all of Catch2's functionality in a test file,
you have to include multiple headers.**
You probably want to look into the [migration docs](migrate-v2-to-v3.md#top),
which were written to help people coming from v2.x.x versions to the
v3 releases.
### FAQ
* Why is Catch2 moving to separate headers?
* The short answer is future extensibility and scalability. The long answer is complex and can be found on my blog, but at the most basic level, it is that providing single-header distribution is at odds with providing variety of useful features. When Catch2 was distributed in a single header, adding a new Matcher would cause overhead for everyone, but was useful only to a subset of users. This meant that the barrier to entry for new Matchers/Generators/etc is high in single header model, but much smaller in the new model.
* Will Catch2 again distribute single-header version in the future?
* No. But we do provide sqlite-style amalgamated distribution option. This means that you can download just 1 .cpp file and 1 header and place them next to your own sources. However, doing this has downsides similar to using the `catch_all.hpp` header.
* Why the big breaking change caused by replacing `catch.hpp` with `catch_all.hpp`?
* The convenience header `catch_all.hpp` exists for two reasons. One of them is to provide a way for quick migration from Catch2, the second one is to provide a simple way to test things with Catch2. Using it for migration has one drawback in that it is **big**. This means that including it _will_ cause significant compile time drag, and so using it to migrate should be a conscious decision by the user, not something they can just stumble into unknowingly.
### (Potentially) Breaking changes
* **Catch2 now uses statically compiled library as its distribution model**
* **Including `catch.hpp` no longer works**
* **Catch2 now uses C++14 as the minimum support language version**
* `ANON_TEST_CASE` has been removed, use `TEST_CASE` with no arguments instead (#1220)
* `--list*` commands no longer have non-zero return code (#1410)
* `--list-test-names-only` has been removed (#1190)
* You should use verbosity-modifiers for `--list-tests` instead
* `--list*` commands are now piped through the reporters
* The top-level reporter interface provides default implementation that works just as the old one
* XmlReporter outputs a machine-parseable XML
* `TEST_CASE` description support has been removed
* If the second argument has text outside tags, the text will be ignored.
* Hidden test cases are no longer included just because they don't match an exclusion tag
* Previously, a `TEST_CASE("A", "[.foo]")` would be included by asking for `~[bar]`.
* `PredicateMatcher` is no longer type erased.
* This means that the type of the provided predicate is part of the `PredicateMatcher`'s type
* `SectionInfo` no longer contains section description as a member (#1319)
* You can still write `SECTION("ShortName", "Long and wordy description")`, but the description is thrown away
* The description type now must be a `const char*` or be implicitly convertible to it
* The `[!hide]` tag has been removed.
* Use `[.]` or `[.foo]` instead.
* Lvalues of composed matchers cannot be composed further
* Uses of `REGISTER_TEST_CASE` macro need to be followed by a semicolon
* This does not change `TEST_CASE` and friends in any way
* `IStreamingReporter::IsMulti` member function was removed
* This is _very_ unlikely to actually affect anyone, as it was default-implemented in the interface, and only used internally
* Various classes not designed for user-extension have been made final
* `ListeningReporter` is now `final`
* Concrete Matchers (e.g. `UnorderedEquals` vector matcher) are now `final`
* All Generators are now `final`
* Matcher namespacing has been redone
* Matcher types are no longer in deeply nested namespaces
* Matcher factory functions are no longer brought into `Catch` namespace
* This means that all public-facing matcher-related functionality is now in `Catch::Matchers` namespace
* Defining `CATCH_CONFIG_MAIN` will no longer create main in that TU.
* Link with `libCatch2Main.a`, or the proper CMake/pkg-config target
* If you want to write custom main, include `catch2/catch_session.hpp`
* `CATCH_CONFIG_EXTERNAL_INTERFACES` has been removed.
* You should instead include the appropriate headers as needed.
* `CATCH_CONFIG_IMPL` has been removed.
* The implementation is now compiled into a static library.
* Event Listener interface has changed
* `TestEventListenerBase` was renamed to `EventListenerBase`
* `EventListenerBase` now directly derives from `IStreamingReporter`, instead of deriving from `StreamingReporterBase`
* `GENERATE` decays its arguments (#2012, #2040)
* This means that `str` in `auto str = GENERATE("aa", "bb", "cc");` is inferred to `char const*` rather than `const char[2]`.
* `--list-*` flags write their output to file specified by the `-o` flag
* Many changes to reporter interfaces
* With the exception of the XmlReporter, the outputs of first party reporters should remain the same
* New pair of events were added
* One obsolete event was removed
* The base class has been renamed
* The built-in reporter class hierarchy has been redone
* Catch2 generates a random seed if one hasn't been specified by the user
* The short flag for `--list-tests`, `-l`, has been removed.
* This is not a commonly used flag and does not need to use up valuable single-letter space.
* The short flag for `--list-tags`, `-t`, has been removed.
* This is not a commonly used flag and does not need to use up valuable single-letter space.
* The `--colour` option has been replaced with `--colour-mode` option
### Improvements
* Matchers have been extended with the ability to use different signatures of `match` (#1307, #1553, #1554, #1843)
* This includes having templated `match` member function
* See the [rewritten Matchers documentation](matchers.md#top) for details
* Catch2 currently provides _some_ generic matchers, but there should be more before final release of v3
* `IsEmpty`, `SizeIs` which check that the range has specific properties
* `Contains`, which checks whether a range contains a specific element
* `AllMatch`, `AnyMatch`, `NoneMatch` range matchers, which apply matchers over a range of elements
* Significant compilation time improvements
* including `catch_test_macros.hpp` is 80% cheaper than including `catch.hpp`
* Some runtime performance optimizations
* In all tested cases the v3 branch was faster, so the table below shows the speedup of v3 to v2 at the same task
<a id="v3-runtime-optimization-table"></a>
| task | debug build | release build |
|:------------------------------------------- | ------------:| -------------:|
| Run 1M `REQUIRE(true)` | 1.10 ± 0.01 | 1.02 ± 0.06 |
| Run 100 tests, 3^3 sections, 1 REQUIRE each | 1.27 ± 0.01 | 1.04 ± 0.01 |
| Run 3k tests, no names, no tags | 1.29 ± 0.01 | 1.05 ± 0.01 |
| Run 3k tests, names, tags | 1.49 ± 0.01 | 1.22 ± 0.01 |
| Run 1 out of 3k tests no names, no tags | 1.68 ± 0.02 | 1.19 ± 0.22 |
| Run 1 out of 3k tests, names, tags | 1.79 ± 0.02 | 2.06 ± 0.23 |
* POSIX platforms use `gmtime_r`, rather than `gmtime` when constructing a date string (#2008, #2165)
* `--list-*` flags write their output to file specified by the `-o` flag (#2061, #2163)
* `Approx::operator()` is now properly `const`
* Catch2's internal helper variables no longer use reserved identifiers (#578)
* `--rng-seed` now accepts string `"random-device"` to generate random seed using `std::random_device`
* Catch2 now supports test sharding (#2257)
* You can ask for the tests to be split into N groups and only run one of them.
* This greatly simplifies parallelization of tests in a binary through external runner.
* The embedded CLI parser now supports repeatedly callable lambdas
* A lambda-based option parser can opt into being repeatedly specifiable.
* Added `STATIC_CHECK` macro, similar to `STATIC_REQUIRE` (#2318)
* When deferred tu runtime, it behaves like `CHECK`, and not like `REQUIRE`.
* You can have multiple tests with the same name, as long as other parts of the test identity differ (#1915, #1999, #2175)
* Test identity includes test's name, test's tags and and test's class name if applicable.
* Added new warning, `UnmatchedTestSpec`, to error on test specs with no matching tests
* The `-w`, `--warn` warning flags can now be provided multiple times to enable multiple warnings
* The case-insensitive handling of tags is now more reliable and takes up less memory
* Test case and assertion counting can no longer reasonably overflow on 32 bit systems
* The count is now kept in `uint64_t` on all platforms, instead of using `size_t` type.
* The `-o`, `--out` output destination specifiers recognize `-` as stdout
* You have to provide it as `--out=-` to avoid CLI error about missing option
* The new reporter specification also recognizes `-` as stdout
* Multiple reporters can now run at the same time and write to different files (#1712, #2183)
* To support this, the `-r`, `--reporter` flag now also accepts optional output destination
* For full overview of the semantics of using multiple reporters, look into the reporter documentation
* To enable the new syntax, reporter names can no longer contain `::`.
* Console colour support has been rewritten and significantly improved
* The colour implementation based on ANSI colour codes is always available
* Colour implementations respect their associated stream
* previously e.g. Win32 impl would change console colour even if Catch2 was writing to a file
* The colour API is resilient against changing evaluation order of expressions
* The associated CLI flag and compile-time configuration options have changed
* For details see the docs for command-line and compile-time Catch2 configuration
* Added a support for Bazel integration with `XML_OUTPUT_FILE` env var (#2399)
* This has to be enabled during compilation.
* Added `--skip-benchmarks` flag to run tests without any `BENCHMARK`s (#2392, #2408)
* Added option to list all listeners in the binary via `--list-listeners`
### Fixes
* The `INFO` macro no longer contains superfluous semicolon (#1456)
* The `--list*` family of command line flags now return 0 on success (#1410, #1146)
* Various ways of failing a benchmark are now counted and reporter properly
* The ULP matcher now handles comparing numbers with different signs properly (#2152)
* Universal ADL-found operators should no longer break decomposition (#2121)
* Reporter selection is properly case-insensitive
* Previously it forced lower cased name, which would fail for reporters with upper case characters in name
* The cumulative reporter base stores benchmark results alongside assertion results
* Catch2's SE handling should no longer interferes with ASan on Windows (#2334)
* Fixed Windows console colour handling for tests that redirect stdout (#2345)
* Fixed issue with the `random` generators returning the same value over and over again
### Other changes
* `CATCH_CONFIG_DISABLE_MATCHERS` no longer exists.
* If you do not want to use Matchers in a TU, do not include their header.
* `CATCH_CONFIG_ENABLE_CHRONO_STRINGMAKER` no longer exists.
* `StringMaker` specializations for `<chrono>` are always provided
* Catch2's CMake now provides 2 targets, `Catch2` and `Catch2WithMain`.
* `Catch2` is the statically compiled implementation by itself
* `Catch2WithMain` also links in the default main
* Catch2's pkg-config integration also provides 2 packages
* `catch2` is the statically compiled implementation by itself
* `catch2-with-main` also links in the default main
* Passing invalid test specifications passed to Catch2 are now reported before tests are run, and are a hard error.
* Running 0 tests (e.g. due to empty binary, or test spec not matching anything) returns non-0 exit code
* Flag `--allow-running-no-tests` overrides this behaviour.
* `NoTests` warning has been removed because it is fully subsumed by this change.
* Catch2's compile-time configuration options (`CATCH_CONFIG_FOO`) can be set through CMake options of the same name
* They use the same semantics as C++ defines, including the `CATCH_CONFIG_NO_FOO` overrides,
* `-DCATCH_CONFIG_DEFAULT_REPORTER=compact` changes default reporter to "compact"
* `-DCATCH_CONFIG_NO_ANDROID_LOGWRITE=ON` forces android logwrite to off
* `-DCATCH_CONFIG_ANDROID_LOGWRITE=OFF` does nothing (the define will not exist)
* Clang-tidy should no longer warn about missing virtual dispatch in `FilterGenerator`'s constructor (#2314)
## 2.13.7
@@ -448,6 +112,9 @@ v3 releases.
* Including `windows.h` without `NOMINMAX` remains a really bad idea, don't do it
### Miscellaneous
* `Catch2WithMain` target (static library) is no longer built by default (#2142)
* Building it by default was at best unnecessary overhead for people not using it, and at worst it caused trouble with install paths
* To have it built, set CMake option `CATCH_BUILD_STATIC_LIBRARY` to `ON`
* The check whether Catch2 is being built as a subproject is now more reliable (#2202, #2204)
* The problem was that if the variable name used internally was defined the project including Catch2 as subproject, it would not be properly overwritten for Catch2's CMake.
@@ -558,7 +225,7 @@ v3 releases.
### Improvements
* `std::result_of` is not used if `std::invoke_result` is available (#1934)
* JUnit reporter writes out `status` attribute for tests (#1899)
* Suppressed clang-tidy's `hicpp-vararg` warning (#1921)
* Suppresed clang-tidy's `hicpp-vararg` warning (#1921)
* Catch2 was already suppressing the `cppcoreguidelines-pro-type-vararg` alias of the warning
@@ -571,7 +238,6 @@ v3 releases.
* Added support for `^` (bitwise xor) to `CHECK` and `REQUIRE`
## 2.12.0
### Improvements
@@ -594,7 +260,6 @@ v3 releases.
* Improved detection of stdlib's support for `std::uncaught_exceptions` (#1911)
## 2.11.3
### Fixes
@@ -626,7 +291,6 @@ v3 releases.
* This bug has been present for the last ~2 years and nobody reported it
## 2.11.1
### Improvements
@@ -640,7 +304,6 @@ v3 releases.
* Some more cleanups in the benchmarking support
## 2.11.0
### Improvements
@@ -774,7 +437,6 @@ v3 releases.
### Fixes
* Fix benchmarking compilation failure in files without `CATCH_CONFIG_EXTERNAL_INTERFACES` (or implementation)
## 2.9.0
### Improvements
@@ -1107,7 +769,7 @@ than `single_include/catch.hpp`.**
* CLR objects (`T^`) can now be stringified (#1216)
* This affects code compiled as C++/CLI
* Added `PredicateMatcher`, a matcher that takes an arbitrary predicate function (#1236)
* See [documentation for details](https://github.com/catchorg/Catch2/blob/devel/docs/matchers.md)
* See [documentation for details](https://github.com/catchorg/Catch2/blob/v2.x/docs/matchers.md)
### Others
* Modified CMake-installed pkg-config to allow `#include <catch.hpp>`(#1239)
@@ -1135,7 +797,7 @@ than `single_include/catch.hpp`.**
* Added an option to warn (+ exit with error) when no tests were ran (#1158)
* Use as `-w NoTests`
* Added provisional support for Emscripten (#1114)
* [Added a way to override the fallback stringifier](https://github.com/catchorg/Catch2/blob/devel/docs/configuration.md#fallback-stringifier) (#1024)
* [Added a way to override the fallback stringifier](https://github.com/catchorg/Catch2/blob/v2.x/docs/configuration.md#fallback-stringifier) (#1024)
* This allows project's own stringification machinery to be easily reused for Catch
* `Catch::Session::run()` now accepts `char const * const *`, allowing it to accept array of string literals (#1031, #1178)
* The embedded version of Clara was bumped to v1.1.3

View File

@@ -1,7 +1,7 @@
<a id="top"></a>
# How to release
When enough changes have accumulated, it is time to release new version of Catch. This document describes the process in doing so, that no steps are forgotten. Note that all referenced scripts can be found in the `tools/scripts/` directory.
When enough changes have accumulated, it is time to release new version of Catch. This document describes the process in doing so, that no steps are forgotten. Note that all referenced scripts can be found in the `scripts/` directory.
## Necessary steps
@@ -40,10 +40,14 @@ After version number is incremented, single-include header is regenerated and re
After pushing changes to GitHub, GitHub release *needs* to be created.
Tag version and release title should be same as the new version,
description should contain the release notes for the current release.
We also attach the two amalgamated files as "binaries".
Single header version of `catch.hpp` *needs* to be attached as a binary,
as that is where the official download link links to. Preferably
it should use linux line endings. All non-bundled reporters (Automake, TAP,
TeamCity, SonarQube) should also be attached as binaries, as they might be
dependent on a specific version of the single-include header.
Since 2.5.0, the release tag and the "binaries" (amalgamated files) should
be PGP signed.
Since 2.5.0, the release tag and the "binaries" (headers) should be PGP
signed.
#### Signing a tag
@@ -53,14 +57,17 @@ is the version being released, e.g. `git tag -s v2.6.0`.
Use the version name as the short message and the release notes as
the body (long) message.
#### Signing the amalgamated files
#### Signing the headers
This will create ASCII-armored signatures for the two amalgamated files
that are uploaded to the GitHub release:
This will create ASCII-armored signatures for the headers that are
uploaded to the GitHub release:
```
gpg --armor --output extras/catch_amalgamated.hpp.asc --detach-sig extras/catch_amalgamated.hpp
gpg --armor --output extras/catch_amalgamated.cpp.asc --detach-sig extras/catch_amalgamated.cpp
$ gpg2 --armor --output catch.hpp.asc --detach-sig catch.hpp
$ gpg2 --armor --output catch_reporter_automake.hpp.asc --detach-sig catch_reporter_automake.hpp
$ gpg2 --armor --output catch_reporter_teamcity.hpp.asc --detach-sig catch_reporter_teamcity.hpp
$ gpg2 --armor --output catch_reporter_tap.hpp.asc --detach-sig catch_reporter_tap.hpp
$ gpg2 --armor --output catch_reporter_sonarqube.hpp.asc --detach-sig catch_reporter_sonarqube.hpp
```
_GPG does not support signing multiple files in single invocation._

View File

@@ -1,175 +0,0 @@
<a id="top"></a>
# Reporter events
**Contents**<br>
[Test running events](#test-running-events)<br>
[Benchmarking events](#benchmarking-events)<br>
[Listings events](#listings-events)<br>
[Miscellaneous events](#miscellaneous-events)<br>
Reporter events are one of the customization points for user code. They
are used by [reporters](reporters.md#top) to customize Catch2's output,
and by [event listeners](event-listeners.md#top) to perform in-process
actions under some conditions.
There are currently 21 reporter events in Catch2, split between 4 distinct
event groups:
* test running events (10 events)
* benchmarking (4 events)
* listings (3 events)
* miscellaneous (4 events)
## Test running events
Test running events are always paired so that for each `fooStarting` event,
there is a `fooEnded` event. This means that the 10 test running events
consist of 5 pairs of events:
* `testRunStarting` and `testRunEnded`,
* `testCaseStarting` and `testCaseEnded`,
* `testCasePartialStarting` and `testCasePartialEnded`,
* `sectionStarting` and `sectionEnded`,
* `assertionStarting` and `assertionEnded`
### `testRun` events
```cpp
void testRunStarting( TestRunInfo const& testRunInfo );
void testRunEnded( TestRunStats const& testRunStats );
```
The `testRun` events bookend the entire test run. `testRunStarting` is
emitted before the first test case is executed, and `testRunEnded` is
emitted after all the test cases have been executed.
### `testCase` events
```cpp
void testCaseStarting( TestCaseInfo const& testInfo );
void testCaseEnded( TestCaseStats const& testCaseStats );
```
The `testCase` events bookend one _full_ run of a specific test case.
Individual runs through a test case, e.g. due to `SECTION`s or `GENERATE`s,
are handled by a different event.
### `testCasePartial` events
> Introduced in Catch2 3.0.1
```cpp
void testCasePartialStarting( TestCaseInfo const& testInfo, uint64_t partNumber );
void testCasePartialEnded(TestCaseStats const& testCaseStats, uint64_t partNumber );
```
`testCasePartial` events bookend one _partial_ run of a specific test case.
This means that for any given test case, these events can be emitted
multiple times, e.g. due to multiple leaf sections.
In regards to nesting with `testCase` events, `testCasePartialStarting`
will never be emitted before the corresponding `testCaseStarting`, and
`testCasePartialEnded` will always be emitted before the corresponding
`testCaseEnded`.
### `section` events
```cpp
void sectionStarting( SectionInfo const& sectionInfo );
void sectionEnded( SectionStats const& sectionStats );
```
`section` events are emitted only for active `SECTION`s, that is, sections
that are entered. Sections that are skipped in this test case run-through
do not cause events to be emitted.
_Note that test cases always contain one implicit section. The event for
this section is emitted after the corresponding `testCasePartialStarting`
event._
### `assertion` events
```cpp
void assertionStarting( AssertionInfo const& assertionInfo );
void assertionEnded( AssertionStats const& assertionStats );
```
`assertionStarting` is called after the expression is captured, but before
the assertion expression is evaluated. This might seem like a minor
distinction, but what it means is that if you have assertion like
`REQUIRE( a + b == c + d )`, then what happens is that `a + b` and `c + d`
are evaluated before `assertionStarting` is emitted, while the `==` is
evaluated after the event.
## Benchmarking events
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
```cpp
void benchmarkPreparing( StringRef name ) override;
void benchmarkStarting( BenchmarkInfo const& benchmarkInfo ) override;
void benchmarkEnded( BenchmarkStats<> const& benchmarkStats ) override;
void benchmarkFailed( StringRef error ) override;
```
Due to the benchmark lifecycle being bit more complicated, the benchmarking
events have their own category, even though they could be seen as parallel
to the `assertion*` events. You should expect running a benchmark to
generate at least 2 of the events above.
To understand the explanation below, you should read the [benchmarking
documentation](benchmarks.md#top) first.
* `benchmarkPreparing` event is sent after the environmental probe
finishes, but before the user code is first estimated.
* `benchmarkStarting` event is sent after the user code is estimated,
but has not been benchmarked yet.
* `benchmarkEnded` event is sent after the user code has been benchmarked,
and contains the benchmarking results.
* `benchmarkFailed` event is sent if either the estimation or the
benchmarking itself fails.
## Listings events
> Introduced in Catch2 3.0.1.
Listings events are events that correspond to the test binary being
invoked with `--list-foo` flag.
There are currently 3 listing events, one for reporters, one for tests,
and one for tags. Note that they are not exclusive to each other.
```cpp
void listReporters( std::vector<ReporterDescription> const& descriptions );
void listTests( std::vector<TestCaseHandle> const& tests );
void listTags( std::vector<TagInfo> const& tagInfos );
```
## Miscellaneous events
```cpp
void reportInvalidTestSpec( StringRef unmatchedSpec );
void fatalErrorEncountered( StringRef error );
void noMatchingTestCases( StringRef unmatchedSpec );
```
These are one-off events that do not neatly fit into other categories.
`reportInvalidTestSpec` is sent for each [test specification command line
argument](command-line.md#specifying-which-tests-to-run) that wasn't
parsed into a valid spec.
`fatalErrorEncountered` is sent when Catch2's POSIX signal handling
or Windows SE handler is called into with a fatal signal/exception.
`noMatchingTestCases` is sent for each user provided test specification
that did not match any registered tests.
---
[Home](Readme.md#top)

View File

@@ -1,212 +1,46 @@
<a id="top"></a>
# Reporters
Reporters are a customization point for most of Catch2's output, e.g.
formatting and writing out [assertions (whether passing or failing),
sections, test cases, benchmarks, and so on](reporter-events.md#top).
Catch2 comes with a bunch of reporters by default (currently 8), and
you can also write your own reporter. Because multiple reporters can
be active at the same time, your own reporters do not even have to handle
all reporter event, just the ones you are interested in, e.g. benchmarks.
Catch has a modular reporting system and comes bundled with a handful of useful reporters built in.
You can also write your own reporters.
## Using different reporters
You can see which reporters are available by running the test binary
with `--list-reporters`. You can then pick one of them with the [`-r`,
`--reporter` option](command-line.md#choosing-a-reporter-to-use), followed
by the name of the desired reporter, like so:
The reporter to use can easily be controlled from the command line.
To specify a reporter use [`-r` or `--reporter`](command-line.md#choosing-a-reporter-to-use), followed by the name of the reporter, e.g.:
```
--reporter xml
-r xml
```
You can also select multiple reporters to be used at the same time.
In that case you should read the [section on using multiple
reporters](#multiple-reporters) to avoid any surprises from doing so.
If you don't specify a reporter then the console reporter is used by default.
There are four reporters built in to the single include:
* `console` writes as lines of text, formatted to a typical terminal width, with colours if a capable terminal is detected.
* `compact` similar to `console` but optimised for minimal output - each entry on one line
* `junit` writes xml that corresponds to Ant's [junitreport](http://help.catchsoftware.com/display/ET/JUnit+Format) target. Useful for build systems that understand Junit.
Because of the way the junit format is structured the run must complete before anything is written.
* `xml` writes an xml format tailored to Catch. Unlike `junit` this is a streaming format so results are delivered progressively.
<a id="multiple-reporters"></a>
## Using multiple reporters
There are a few additional reporters, for specific build systems, in the Catch repository (in `include\reporters`) which you can `#include` in your project if you would like to make use of them.
Do this in one source file - the same one you have `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`.
> Support for having multiple parallel reporters was [introduced](https://github.com/catchorg/Catch2/pull/2183) in Catch2 3.0.1
Catch2 supports using multiple reporters at the same time while having
them write into different destinations. The two main uses of this are
* having both human-friendly and machine-parseable (e.g. in JUnit format)
output from one run of binary
* having "partial" reporters that are highly specialized, e.g. having one
reporter that writes out benchmark results as markdown tables and does
nothing else, while also having standard testing output separately
Specifying multiple reporter looks like this:
```
--reporter JUnit::out=result-junit.xml --reporter console::out=-::colour-mode=ansi
```
This tells Catch2 to use two reporters, `JUnit` reporter that writes
its machine-readable XML output to file `result-junit.xml`, and the
`console` reporter that writes its user-friendly output to stdout and
uses ANSI colour codes for colouring the output.
Using multiple reporters (or one reporter and one-or-more [event
listeners](event-listener.md#top)) can have surprisingly complex semantics
when using customization points provided to reporters by Catch2, namely
capturing stdout/stderr from test cases.
As long as at least one reporter (or listener) asks Catch2 to capture
stdout/stderr, captured stdout and stderr will be available to all
reporters and listeners.
Because this might be surprising to the users, if at least one active
_reporter_ is non-capturing, then Catch2 tries to roughly emulate
non-capturing behaviour by printing out the captured stdout/stderr
just before `testCasePartialEnded` event is sent out to the active
reporters and listeners. This means that stdout/stderr is no longer
printed out from tests as it is being written, but instead it is written
out in batch after each runthrough of a test case is finished.
* `teamcity` writes the native, streaming, format that [TeamCity](https://www.jetbrains.com/teamcity/) understands.
Use this when building as part of a TeamCity build to see results as they happen ([code example](../examples/207-Rpt-TeamCityReporter.cpp)).
* `tap` writes in the TAP ([Test Anything Protocol](https://en.wikipedia.org/wiki/Test_Anything_Protocol)) format.
* `automake` writes in a format that correspond to [automake .trs](https://www.gnu.org/software/automake/manual/html_node/Log-files-generation-and-test-results-recording.html) files
* `sonarqube` writes the [SonarQube Generic Test Data](https://docs.sonarqube.org/latest/analysis/generic-test/) XML format.
You see what reporters are available from the command line by running with `--list-reporters`.
By default all these reports are written to stdout, but can be redirected to a file with [`-o` or `--out`](command-line.md#sending-output-to-a-file)
## Writing your own reporter
You can also write your own custom reporter and tell Catch2 to use it.
When writing your reporter, you have two options:
* Derive from `Catch::ReporterBase`. When doing this, you will have
to provide handling for all [reporter events](reporter-events.md#top).
* Derive from one of the provided [utility reporter bases in
Catch2](#utility-reporter-bases).
Generally we recommend doing the latter, as it is less work.
Apart from overriding handling of the individual reporter events, reporters
have access to some extra customization points, described below.
### Utility reporter bases
Catch2 currently provides two utility reporter bases:
* `Catch::StreamingReporterBase`
* `Catch::CumulativeReporterBase`
`StreamingReporterBase` is useful for reporters that can format and write
out the events as they come in. It provides (usually empty) implementation
for all reporter events, and if you let it handle the relevant events,
it also handles storing information about active test run and test case.
`CumulativeReporterBase` is a base for reporters that need to see the whole
test run, before they can start writing the output, such as the JUnit
and SonarQube reporters. This post-facto approach requires the assertions
to be stringified when it is finished, so that the assertion can be written
out later. Because the stringification can be expensive, and not all
cumulative reporters need the assertions, this base provides customization
point to change whether the assertions are saved or not, separate for
passing and failing assertions.
_Generally we recommend that if you override a member function from either
of the bases, you call into the base's implementation first. This is not
necessarily in all cases, but it is safer and easier._
Writing your own reporter then looks like this:
```cpp
#include <catch2/reporters/catch_reporter_streaming_base.hpp>
#include <catch2/catch_test_case_info.hpp>
#include <catch2/reporters/catch_reporter_registrars.hpp>
#include <iostream>
class PartialReporter : public Catch::StreamingReporterBase {
public:
using StreamingReporterBase::StreamingReporterBase;
static std::string getDescription() {
return "Reporter for testing TestCasePartialStarting/Ended events";
}
void testCasePartialStarting(Catch::TestCaseInfo const& testInfo,
uint64_t partNumber) override {
std::cout << "TestCaseStartingPartial: " << testInfo.name << '#' << partNumber << '\n';
}
void testCasePartialEnded(Catch::TestCaseStats const& testCaseStats,
uint64_t partNumber) override {
std::cout << "TestCasePartialEnded: " << testCaseStats.testInfo->name << '#' << partNumber << '\n';
}
};
CATCH_REGISTER_REPORTER("partial", PartialReporter)
```
This create a simple reporter that responds to `testCasePartial*` events,
and calls itself "partial" reporter, so it can be invoked with
`--reporter partial` command line flag.
### `ReporterPreferences`
Each reporter instance contains instance of `ReporterPreferences`, a type
that holds flags for the behaviour of Catch2 when this reporter run.
Currently there are two customization options:
* `shouldRedirectStdOut` - whether the reporter wants to handle
writes to stdout/stderr from user code, or not. This is useful for
reporters that output machine-parseable output, e.g. the JUnit
reporter, or the XML reporter.
* `shouldReportAllAssertions` - whether the reporter wants to handle
`assertionEnded` events for passing assertions as well as failing
assertions. Usually reporters do not report successful assertions
and don't need them for their output, but sometimes the desired output
format includes passing assertions even without the `-s` flag.
### Per-reporter configuration
> Per-reporter configuration was introduced in Catch2 3.0.1
Catch2 supports some configuration to happen per reporter. The configuration
options fall into one of two categories:
* Catch2-recognized options
* Reporter-specific options
The former is a small set of universal options that Catch2 handles for
the reporters, e.g. output file or console colour mode. The latter are
options that the reporters have to handle themselves, but the keys and
values can be arbitrary strings, as long as they don't contain `::`. This
allows writing reporters that can be significantly customized at runtime.
Reporter-specific options always have to be prefixed with "X" (large
letter X).
### Other expected functionality of a reporter
When writing a custom reporter, there are few more things that you should
keep in mind. These are not important for correctness, but they are
important for the reporter to work _nicely_.
* Catch2 provides a simple verbosity option for users. There are three
verbosity levels, "quiet", "normal", and "high", and if it makes sense
for reporter's output format, it should respond to these by changing
what, and how much, it writes out.
* Catch2 operates with an rng-seed. Knowing what seed a test run had
is important if you want to replicate it, so your reporter should
report the rng-seed, if at all possible given the target output format.
* Catch2 also operates with test filters, or test specs. If a filter
is present, you should also report the filter, if at all possible given
the target output format.
You can write your own custom reporter and register it with Catch.
At time of writing the interface is subject to some changes so is not, yet, documented here.
If you are determined you shouldn't have too much trouble working it out from the existing implementations -
but do keep in mind upcoming changes (these will be minor, simplifying, changes such as not needing to forward calls to the base class).
---

View File

@@ -1,129 +0,0 @@
<a id="top"></a>
# Explicitly skipping, passing, and failing tests at runtime
## Skipping Test Cases at Runtime
> [Introduced](https://github.com/catchorg/Catch2/pull/2360) in Catch2 3.3.0.
In some situations it may not be possible to meaningfully execute a test case,
for example when the system under test is missing certain hardware capabilities.
If the required conditions can only be determined at runtime, it often
doesn't make sense to consider such a test case as either passed or failed,
because it simply can not run at all.
To properly express such scenarios, Catch2 provides a way to explicitly
_skip_ test cases, using the `SKIP` macro:
```
SKIP( [streamable expression] )
```
Example usage:
```c++
TEST_CASE("copy files between drives") {
if(getNumberOfHardDrives() < 2) {
SKIP("at least two hard drives required");
}
// ...
}
```
This test case is then reported as _skipped_ instead of _passed_ or _failed_.
The `SKIP` macro behaves similarly to an explicit [`FAIL`](#passing-and-failing-test-cases),
in that it is the last expression that will be executed:
```c++
TEST_CASE("my test") {
printf("foo");
SKIP();
printf("bar"); // not printed
}
```
However a failed assertion _before_ a `SKIP` still causes the entire
test case to fail:
```c++
TEST_CASE("failing test") {
CHECK(1 == 2);
SKIP();
}
```
### Interaction with Sections and Generators
Sections, nested sections as well as specific outputs from [generators](generators.md#top)
can all be individually skipped, with the rest executing as usual:
```c++
TEST_CASE("complex test case") {
int value = GENERATE(2, 4, 6);
SECTION("a") {
SECTION("a1") { CHECK(value < 8); }
SECTION("a2") {
if (value == 4) {
SKIP();
}
CHECK(value % 2 == 0);
}
}
}
```
This test case will report 5 passing assertions; one for each of the three
values in section `a1`, and then two in section `a2`, from values 2 and 4.
Note that as soon as one section is skipped, the entire test case will
be reported as _skipped_ (unless there is a failing assertion, in which
case the test is handled as _failed_ instead).
Note that if all test cases in a run are skipped, Catch2 returns a non-zero
exit code, same as it does if no test cases have run. This behaviour can
be overridden using the [--allow-running-no-tests](command-line.md#no-tests-override)
flag.
## Passing and failing test cases
Test cases can also be explicitly passed or failed, without the use of
assertions, and with a specific message. This can be useful to handle
complex preconditions/postconditions and give useful error messages
when they fail.
* `SUCCEED( [streamable expression] )`
`SUCCEED` is morally equivalent with `INFO( [streamable expression] ); REQUIRE( true );`.
Note that it does not stop further test execution, so it cannot be used
to guard failing assertions from being executed.
_In practice, `SUCCEED` is usually used as a test placeholder, to avoid
[failing a test case due to missing assertions](command-line.md#warnings)._
```cpp
TEST_CASE( "SUCCEED showcase" ) {
int I = 1;
SUCCEED( "I is " << I );
// ... execution continues here ...
}
```
* `FAIL( [streamable expression] )`
`FAIL` is morally equivalent with `INFO( [streamable expression] ); REQUIRE( false );`.
_In practice, `FAIL` is usually used to stop executing test that is currently
known to be broken, but has to be fixed later._
```cpp
TEST_CASE( "FAIL showcase" ) {
FAIL( "This test case causes segfault, which breaks CI." );
// ... this will not be executed ...
}
```
---
[Home](Readme.md#top)

106
docs/slow-compiles.md Normal file
View File

@@ -0,0 +1,106 @@
<a id="top"></a>
# Why do my tests take so long to compile?
**Contents**<br>
[Short answer](#short-answer)<br>
[Long answer](#long-answer)<br>
[Practical example](#practical-example)<br>
[Using the static library Catch2WithMain](#using-the-static-library-catch2withmain)<br>
[Other possible solutions](#other-possible-solutions)<br>
Several people have reported that test code written with Catch takes much longer to compile than they would expect. Why is that?
Catch is implemented entirely in headers. There is a little overhead due to this - but not as much as you might think - and you can minimise it simply by organising your test code as follows:
## Short answer
Exactly one source file must ```#define``` either ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER``` before ```#include```-ing Catch. In this file *do not write any test cases*! In most cases that means this file will just contain two lines (the ```#define``` and the ```#include```).
## Long answer
Usually C++ code is split between a header file, containing declarations and prototypes, and an implementation file (.cpp) containing the definition, or implementation, code. Each implementation file, along with all the headers that it includes (and which those headers include, etc), is expanded into a single entity called a translation unit - which is then passed to the compiler and compiled down to an object file.
But functions and methods can also be written inline in header files. The downside to this is that these definitions will then be compiled in *every* translation unit that includes the header.
Because Catch is implemented *entirely* in headers you might think that the whole of Catch must be compiled into every translation unit that uses it! Actually it's not quite as bad as that. Catch mitigates this situation by effectively maintaining the traditional separation between the implementation code and declarations. Internally the implementation code is protected by ```#ifdef```s and is conditionally compiled into only one translation unit. This translation unit is that one that ```#define```s ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER```. Let's call this the main source file.
As a result the main source file *does* compile the whole of Catch every time! So it makes sense to dedicate this file to *only* ```#define```-ing the identifier and ```#include```-ing Catch (and implementing the runner code, if you're doing that). Keep all your test cases in other files. This way you won't pay the recompilation cost for the whole of Catch.
## Practical example
Assume you have the `Factorial` function from the [tutorial](tutorial.md#top) in `factorial.cpp` (with forward declaration in `factorial.h`) and want to test it and keep the compile times down when adding new tests. Then you should have 2 files, `tests-main.cpp` and `tests-factorial.cpp`:
```cpp
// tests-main.cpp
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
```
```cpp
// tests-factorial.cpp
#include "catch.hpp"
#include "factorial.h"
TEST_CASE( "Factorials are computed", "[factorial]" ) {
REQUIRE( Factorial(1) == 1 );
REQUIRE( Factorial(2) == 2 );
REQUIRE( Factorial(3) == 6 );
REQUIRE( Factorial(10) == 3628800 );
}
```
After compiling `tests-main.cpp` once, it is enough to link it with separately compiled `tests-factorial.cpp`. This means that adding more tests to `tests-factorial.cpp`, will not result in recompiling Catch's main and the resulting compilation times will decrease substantially.
```
$ g++ tests-main.cpp -c
$ g++ factorial.cpp -c
$ g++ tests-main.o factorial.o tests-factorial.cpp -o tests && ./tests -r compact
Passed 1 test case with 4 assertions.
```
Now, the next time we change the file `tests-factorial.cpp` (say we add `REQUIRE( Factorial(0) == 1)`), it is enough to recompile the tests instead of recompiling main as well:
```
$ g++ tests-main.o factorial.o tests-factorial.cpp -o tests && ./tests -r compact
tests-factorial.cpp:11: failed: Factorial(0) == 1 for: 0 == 1
Failed 1 test case, failed 1 assertion.
```
## Using the static library Catch2WithMain
Catch2 also provides a static library that implements the runner. Note
that this support is experimental, due to interactions between Catch2 v2
implementation and C++ linking limitations.
As with the `Catch2` target, the `Catch2WithMain` CMake target can be used
either from a subdirectory, or from installed build.
### CMake
```cmake
add_executable(tests-factorial tests-factorial.cpp)
target_link_libraries(tests-factorial Catch2::Catch2WithMain)
```
### bazel
```python
cc_test(
name = "hello_world_test",
srcs = [
"test/hello_world_test.cpp",
],
deps = [
"lib_hello_world",
"@catch2//:catch2_with_main",
],
)
```
## Other possible solutions
You can also opt to sacrifice some features in order to speed-up Catch's compilation times. For details see the [documentation on Catch's compile-time configuration](configuration.md#other-toggles).
---
[Home](Readme.md#top)

View File

@@ -15,18 +15,9 @@ Instead Catch provides a powerful mechanism for nesting test case sections withi
Test cases and sections are very easy to use in practice:
* **TEST_CASE(** _test name_ \[, _tags_ \] **)**
* **SECTION(** _section name_, \[, _section description_ \] **)**
* **SECTION(** _section name_ **)**
_test name_ and _section name_ are free form, quoted, strings.
The optional _tags_ argument is a quoted string containing one or more
tags enclosed in square brackets, and are discussed below.
_section description_ can be used to provide long form description
of a section while keeping the _section name_ short for use with the
[`-c` command line parameter](command-line.md#specify-the-section-to-run).
**The combination of test names and tags must be unique within the Catch2
executable.**
_test name_ and _section name_ are free form, quoted, strings. The optional _tags_ argument is a quoted string containing one or more tags enclosed in square brackets. Tags are discussed below. Test names must be unique within the Catch executable.
For examples see the [Tutorial](tutorial.md#top)
@@ -45,21 +36,13 @@ The tag expression, ```"[widget]"``` selects A, B & D. ```"[gadget]"``` selects
For more detail on command line selection see [the command line docs](command-line.md#specifying-which-tests-to-run)
Tag names are not case sensitive and can contain any ASCII characters.
This means that tags `[tag with spaces]` and `[I said "good day"]`
are both allowed tags and can be filtered on. However, escapes are not
supported however and `[\]]` is not a valid tag.
The same tag can be specified multiple times for a single test case,
but only one of the instances of identical tags will be kept. Which one
is kept is functionally random.
Tag names are not case sensitive and can contain any ASCII characters. This means that tags `[tag with spaces]` and `[I said "good day"]` are both allowed tags and can be filtered on. Escapes are not supported however and `[\]]` is not a valid tag.
### Special Tags
All tag names beginning with non-alphanumeric characters are reserved by Catch. Catch defines a number of "special" tags, which have meaning to the test runner itself. These special tags all begin with a symbol character. Following is a list of currently defined special tags and their meanings.
* `[.]` - causes test cases to be skipped from the default list (i.e. when no test cases have been explicitly selected through tag expressions or name wildcards). The hide tag is often combined with another, user, tag (for example `[.][integration]` - so all integration tests are excluded from the default run but can be run by passing `[integration]` on the command line). As a short-cut you can combine these by simply prefixing your user tag with a `.` - e.g. `[.integration]`.
* `[!hide]` or `[.]` - causes test cases to be skipped from the default list (i.e. when no test cases have been explicitly selected through tag expressions or name wildcards). The hide tag is often combined with another, user, tag (for example `[.][integration]` - so all integration tests are excluded from the default run but can be run by passing `[integration]` on the command line). As a short-cut you can combine these by simply prefixing your user tag with a `.` - e.g. `[.integration]`. Because the hide tag has evolved to have several forms, all forms are added as tags if you use one of them.
* `[!throws]` - lets Catch know that this test is likely to throw an exception even if successful. This causes the test to be excluded when running with `-e` or `--nothrow`.
@@ -69,13 +52,11 @@ All tag names beginning with non-alphanumeric characters are reserved by Catch.
* `[!nonportable]` - Indicates that behaviour may vary between platforms or compilers.
* `[#<filename>]` - these tags are added to test cases when you run Catch2
with [`-#` or `--filenames-as-tags`](command-line.md#filenames-as-tags).
* `[#<filename>]` - running with `-#` or `--filenames-as-tags` causes Catch to add the filename, prefixed with `#` (and with any extension stripped), as a tag to all contained tests, e.g. tests in testfile.cpp would all be tagged `[#testfile]`.
* `[@<alias>]` - tag aliases all begin with `@` (see below).
* `[!benchmark]` - this test case is actually a benchmark. Currently this only serves to hide the test case by default, to avoid the execution time costs.
* `[!benchmark]` - this test case is actually a benchmark. This is an experimental feature, and currently has no documentation. If you want to try it out, look at `projects/SelfTest/Benchmark.tests.cpp` for details.
## Tag aliases
@@ -101,65 +82,15 @@ This macro maps onto ```TEST_CASE``` and works in the same way, except that the
* **WHEN(** _something_ **)**
* **THEN(** _something_ **)**
These macros map onto ```SECTION```s except that the section names are the _something_ texts prefixed by
"given: ", "when: " or "then: " respectively. These macros also map onto the AAA or A<sup>3</sup> test pattern
(standing either for [Assemble-Activate-Assert](http://wiki.c2.com/?AssembleActivateAssert) or
[Arrange-Act-Assert](http://wiki.c2.com/?ArrangeActAssert)), and in this context, the macros provide both code
documentation and reporting of these parts of a test case without the need for extra comments or code to do so.
Semantically, a `GIVEN` clause may have multiple _independent_ `WHEN` clauses within it. This allows a test
to have, e.g., one set of "given" objects and multiple subtests using those objects in various ways in each
of the `WHEN` clauses without repeating the initialisation from the `GIVEN` clause. When there are _dependent_
clauses -- such as a second `WHEN` clause that should only happen _after_ the previous `WHEN` clause has been
executed and validated -- there are additional macros starting with `AND_`:
These macros map onto ```SECTION```s except that the section names are the _something_s prefixed by "given: ", "when: " or "then: " respectively.
* **AND_GIVEN(** _something_ **)**
* **AND_WHEN(** _something_ **)**
* **AND_THEN(** _something_ **)**
These are used to chain ```GIVEN```s, ```WHEN```s and ```THEN```s together. The `AND_*` clause is placed
_inside_ the clause on which it depends. There can be multiple _independent_ clauses that are all _dependent_
on a single outer clause.
```cpp
SCENARIO( "vector can be sized and resized" ) {
GIVEN( "An empty vector" ) {
auto v = std::vector<std::string>{};
Similar to ```GIVEN```, ```WHEN``` and ```THEN``` except that the prefixes start with "and ". These are used to chain ```GIVEN```s, ```WHEN```s and ```THEN```s together.
// Validate assumption of the GIVEN clause
THEN( "The size and capacity start at 0" ) {
REQUIRE( v.size() == 0 );
REQUIRE( v.capacity() == 0 );
}
// Validate one use case for the GIVEN object
WHEN( "push_back() is called" ) {
v.push_back("hullo");
THEN( "The size changes" ) {
REQUIRE( v.size() == 1 );
REQUIRE( v.capacity() >= 1 );
}
}
}
}
```
This code will result in two runs through the scenario:
```
Scenario : vector can be sized and resized
Given : An empty vector
Then : The size and capacity start at 0
Scenario : vector can be sized and resized
Given : An empty vector
When : push_back() is called
Then : The size changes
```
See also [runnable example on godbolt](https://godbolt.org/z/eY5a64r99),
with a more complicated (and failing) example.
> `AND_GIVEN` was [introduced](https://github.com/catchorg/Catch2/issues/1360) in Catch2 2.4.0.
> `AND_GIVEN` was [introduced](https://github.com/catchorg/Catch2/issues/1360) in Catch 2.4.0.
When any of these macros are used the console reporter recognises them and formats the test case header such that the Givens, Whens and Thens are aligned to aid readability.
@@ -169,15 +100,11 @@ Other than the additional prefixes and the formatting in the console reporter th
In addition to `TEST_CASE`s, Catch2 also supports test cases parametrised
by types, in the form of `TEMPLATE_TEST_CASE`,
`TEMPLATE_PRODUCT_TEST_CASE` and `TEMPLATE_LIST_TEST_CASE`. These macros
are defined in the `catch_template_test_macros.hpp` header, so compiling
the code examples below also requires
`#include <catch2/catch_template_test_macros.hpp>`.
`TEMPLATE_PRODUCT_TEST_CASE` and `TEMPLATE_LIST_TEST_CASE`.
* **TEMPLATE_TEST_CASE(** _test name_ , _tags_, _type1_, _type2_, ..., _typen_ **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1437) in Catch2 2.5.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1437) in Catch 2.5.0.
_test name_ and _tag_ are exactly the same as they are in `TEST_CASE`,
with the difference that the tag string must be provided (however, it
@@ -229,7 +156,7 @@ TEMPLATE_TEST_CASE( "vectors can be sized and resized", "[vector][template]", in
* **TEMPLATE_PRODUCT_TEST_CASE(** _test name_ , _tags_, (_template-type1_, _template-type2_, ..., _template-typen_), (_template-arg1_, _template-arg2_, ..., _template-argm_) **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1468) in Catch2 2.6.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1468) in Catch 2.6.0.
_template-type1_ through _template-typen_ is list of template template
types which should be combined with each of _template-arg1_ through
@@ -268,9 +195,13 @@ TEMPLATE_PRODUCT_TEST_CASE("Product with differing arities", "[template][product
}
```
_While there is an upper limit on the number of types you can specify
in single `TEMPLATE_TEST_CASE` or `TEMPLATE_PRODUCT_TEST_CASE`, the limit
is very high and should not be encountered in practice._
* **TEMPLATE_LIST_TEST_CASE(** _test name_, _tags_, _type list_ **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1627) in Catch2 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1627) in Catch 2.9.0.
_type list_ is a generic list of types on which test case should be instantiated.
List can be `std::tuple`, `boost::mpl::list`, `boost::mp11::mp_list` or anything with
@@ -290,14 +221,12 @@ TEMPLATE_LIST_TEST_CASE("Template test case with test types specified inside std
## Signature based parametrised test cases
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch 2.8.0.
In addition to [type parametrised test cases](#type-parametrised-test-cases) Catch2 also supports
signature base parametrised test cases, in form of `TEMPLATE_TEST_CASE_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_SIG`.
These test cases have similar syntax like [type parametrised test cases](#type-parametrised-test-cases), with one
additional positional argument which specifies the signature. These macros are defined in the
`catch_template_test_macros.hpp` header, so compiling the code examples below also requires
`#include <catch2/catch_template_test_macros.hpp>`.
additional positional argument which specifies the signature.
### Signature
Signature has some strict rules for these tests cases to work properly:
@@ -314,7 +243,7 @@ Currently Catch2 support up to 11 template parameters in signature
* **TEMPLATE_TEST_CASE_SIG(** _test name_ , _tags_, _signature_, _type1_, _type2_, ..., _typen_ **)**
Inside `TEMPLATE_TEST_CASE_SIG` test case you can use the names of template parameters as defined in _signature_.
Inside `TEMPLATE_TEST_CASE_SIG` test case you can use the names of template parameters as defined in _signature_.
```cpp
TEMPLATE_TEST_CASE_SIG("TemplateTestSig: arrays can be created from NTTP arguments", "[vector][template][nttp]",

View File

@@ -3,7 +3,7 @@
## Defining test fixtures
Although Catch allows you to group tests together as [sections within a test case](test-cases-and-sections.md), it can still be convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too. You define the test fixture as a simple structure:
Although Catch allows you to group tests together as sections within a test case, it can still be convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too. You define the test fixture as a simple structure:
```c++
class UniqueTestsFixture {
@@ -59,10 +59,7 @@ struct Template_Fixture {
T m_a;
};
TEMPLATE_TEST_CASE_METHOD(Template_Fixture,
"A TEMPLATE_TEST_CASE_METHOD based test run that succeeds",
"[class][template]",
int, float, double) {
TEMPLATE_TEST_CASE_METHOD(Template_Fixture,"A TEMPLATE_TEST_CASE_METHOD based test run that succeeds", "[class][template]", int, float, double) {
REQUIRE( Template_Fixture<TestType>::m_a == 1 );
}
@@ -80,11 +77,7 @@ struct Foo_class {
}
};
TEMPLATE_PRODUCT_TEST_CASE_METHOD(Template_Template_Fixture,
"A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test succeeds",
"[class][template]",
(Foo_class, std::vector),
int) {
TEMPLATE_PRODUCT_TEST_CASE_METHOD(Template_Template_Fixture, "A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test succeeds", "[class][template]", (Foo_class, std::vector), int) {
REQUIRE( Template_Template_Fixture<TestType>::m_a.size() == 0 );
}
```
@@ -95,7 +88,7 @@ the limit is very high and should not be encountered in practice._
## Signature-based parametrised test fixtures
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch 2.8.0.
Catch2 also provides `TEMPLATE_TEST_CASE_METHOD_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG` to support
fixtures using non-type template parameters. These test cases work similar to `TEMPLATE_TEST_CASE_METHOD` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD`,
@@ -108,12 +101,7 @@ struct Nttp_Fixture{
int value = V;
};
TEMPLATE_TEST_CASE_METHOD_SIG(
Nttp_Fixture,
"A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds",
"[class][template][nttp]",
((int V), V),
1, 3, 6) {
TEMPLATE_TEST_CASE_METHOD_SIG(Nttp_Fixture, "A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds", "[class][template][nttp]",((int V), V), 1, 3, 6) {
REQUIRE(Nttp_Fixture<V>::value > 0);
}
@@ -129,13 +117,8 @@ struct Template_Foo_2 {
size_t size() { return V; }
};
TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG(
Template_Fixture_2,
"A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds",
"[class][template][product][nttp]",
((typename T, size_t S), T, S),
(std::array, Template_Foo_2),
((int,2), (float,6))) {
TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG(Template_Fixture_2, "A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds", "[class][template][product][nttp]", ((typename T, size_t S), T, S),(std::array, Template_Foo_2), ((int,2), (float,6)))
{
REQUIRE(Template_Fixture_2<TestType>{}.m_a.size() >= 2);
}
```
@@ -149,10 +132,8 @@ only difference is the source of types. This allows you to reuse the template ty
Example:
```cpp
using MyTypes = std::tuple<int, char, double>;
TEMPLATE_LIST_TEST_CASE_METHOD(Template_Fixture,
"Template test case method with test types specified inside std::tuple",
"[class][template][list]",
MyTypes) {
TEMPLATE_LIST_TEST_CASE_METHOD(Template_Fixture, "Template test case method with test types specified inside std::tuple", "[class][template][list]", MyTypes)
{
REQUIRE( Template_Fixture<TestType>::m_a == 1 );
}
```

View File

@@ -64,14 +64,14 @@ namespace Catch {
By default all exceptions deriving from `std::exception` will be translated to strings by calling the `what()` method. For exception types that do not derive from `std::exception` - or if `what()` does not return a suitable string - use `CATCH_TRANSLATE_EXCEPTION`. This defines a function that takes your exception type, by reference, and returns a string. It can appear anywhere in the code - it doesn't have to be in the same translation unit. For example:
```cpp
CATCH_TRANSLATE_EXCEPTION( MyType const& ex ) {
CATCH_TRANSLATE_EXCEPTION( MyType& ex ) {
return ex.message();
}
```
## Enums
> Introduced in Catch2 2.8.0.
> Introduced in Catch 2.8.0.
Enums that already have a `<<` overload for `std::ostream` will convert to strings as expected.
If you only need to convert enums to strings for test reporting purposes you can provide a `StringMaker` specialisations as any other type.
@@ -110,7 +110,7 @@ TEST_CASE() {
## Floating point precision
> [Introduced](https://github.com/catchorg/Catch2/issues/1614) in Catch2 2.8.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1614) in Catch 2.8.0.
Catch provides a built-in `StringMaker` specialization for both `float`
and `double`. By default, it uses what we think is a reasonable precision,

View File

@@ -3,21 +3,32 @@
**Contents**<br>
[Getting Catch2](#getting-catch2)<br>
[Where to put it?](#where-to-put-it)<br>
[Writing tests](#writing-tests)<br>
[Test cases and sections](#test-cases-and-sections)<br>
[BDD style testing](#bdd-style-testing)<br>
[Data and Type driven tests](#data-and-type-driven-tests)<br>
[BDD-Style](#bdd-style)<br>
[Scaling up](#scaling-up)<br>
[Type parametrised test cases](#type-parametrised-test-cases)<br>
[Next steps](#next-steps)<br>
## Getting Catch2
Ideally you should be using Catch2 through its [CMake integration](cmake-integration.md#top).
Catch2 also provides pkg-config files and two file (header + cpp)
distribution, but this documentation will assume you are using CMake. If
you are using the two file distribution instead, remember to replace
the included header with `catch_amalgamated.hpp`.
The simplest way to get Catch2 is to download the latest [single header version](https://raw.githubusercontent.com/catchorg/Catch2/v2.x/single_include/catch2/catch.hpp). The single header is generated by merging a set of individual headers but it is still just normal source code in a header file.
Alternative ways of getting Catch2 include using your system package
manager, or installing it using [its CMake package](cmake-integration.md#installing-catch2-from-git-repository).
The full source for Catch2, including test projects, documentation, and other things, is hosted on GitHub. [http://catch-lib.net](http://catch-lib.net) will redirect you there.
## Where to put it?
Catch2 is header only. All you need to do is drop the file somewhere reachable from your project - either in some central location you can set your header search path to find, or directly into your project tree itself! This is a particularly good option for other Open-Source projects that want to use Catch for their test suite. See [this blog entry for more on that](https://levelofindirection.com/blog/unit-testing-in-cpp-and-objective-c-just-got-ridiculously-easier-still.html).
The rest of this tutorial will assume that the Catch2 single-include header (or the include folder) is available unqualified - but you may need to prefix it with a folder name if necessary.
_If you have installed Catch2 from system package manager, or CMake
package, you need to include the header as `#include <catch2/catch.hpp>`_
## Writing tests
@@ -29,8 +40,11 @@ unsigned int Factorial( unsigned int number ) {
}
```
To keep things simple we'll put everything in a single file (<a href="#scaling-up">see later for more on how to structure your test files</a>).
```c++
#include <catch2/catch_test_macros.hpp>
#define CATCH_CONFIG_MAIN // This tells Catch to provide a main() - only do this in one cpp file
#include "catch.hpp"
unsigned int Factorial( unsigned int number ) {
return number <= 1 ? number : Factorial(number-1)*number;
@@ -46,10 +60,13 @@ TEST_CASE( "Factorials are computed", "[factorial]" ) {
This will compile to a complete executable which responds to [command line arguments](command-line.md#top). If you just run it with no arguments it will execute all test cases (in this case there is just one), report any failures, report a summary of how many tests passed and failed and return the number of failed tests (useful for if you just want a yes/ no answer to: "did it work").
Anyway, as the tests above as written will pass, but there is a bug.
The problem is that `Factorial(0)` should return 1 (due to [its
definition](https://en.wikipedia.org/wiki/Factorial#Factorial_of_zero)).
Let's add that as an assertion to the test case:
If you run this as written it will pass. Everything is good. Right?
Well, there is still a bug here. In fact the first version of this tutorial I posted here genuinely had the bug in! So it's not completely contrived (thanks to Daryle Walker (```@CTMacUser```) for pointing this out).
What is the bug? Well what is the factorial of zero?
[The factorial of zero is one](http://mathforum.org/library/drmath/view/57128.html) - which is just one of those things you have to know (and remember!).
Let's add that to the test case:
```c++
TEST_CASE( "Factorials are computed", "[factorial]" ) {
@@ -61,8 +78,7 @@ TEST_CASE( "Factorials are computed", "[factorial]" ) {
}
```
After another compile & run cycle, we will see a test failure. The output
will look something like:
Now we get a failure - something like:
```
Example.cpp:9: FAILED:
@@ -71,51 +87,37 @@ with expansion:
0 == 1
```
Note that the output contains both the original expression,
`REQUIRE( Factorial(0) == 1 )` and the actual value returned by the call
to the `Factorial` function: `0`.
Note that we get the actual return value of Factorial(0) printed for us (0) - even though we used a natural expression with the == operator. That lets us immediately see what the problem is.
Let's change the factorial function to:
We can fix this bug by slightly modifying the `Factorial` function to:
```c++
unsigned int Factorial( unsigned int number ) {
return number > 1 ? Factorial(number-1)*number : 1;
}
```
Now all the tests pass.
Of course there are still more issues to deal with. For example we'll hit problems when the return value starts to exceed the range of an unsigned int. With factorials that can happen quite quickly. You might want to add tests for such cases and decide how to handle them. We'll stop short of doing that here.
### What did we do here?
Although this was a simple test it's been enough to demonstrate a few
things about how Catch2 is used. Let's take a moment to consider those
before we move on.
* We introduce test cases with the `TEST_CASE` macro. This macro takes
one or two string arguments - a free form test name and, optionally,
one or more tags (for more see [Test cases and Sections](#test-cases-and-sections)).
* The test automatically self-registers with the test runner, and user
does not have do anything more to ensure that it is picked up by the test
framework. _Note that you can run specific test, or set of tests,
through the [command line](command-line.md#top)._
* The individual test assertions are written using the `REQUIRE` macro.
It accepts a boolean expression, and uses expression templates to
internally decompose it, so that it can be individually stringified
on test failure.
On the last point, note that there are more testing macros available,
because not all useful checks can be expressed as a simple boolean
expression. As an example, checking that an expression throws an exception
is done with the `REQUIRE_THROWS` macro. More on that later.
Although this was a simple test it's been enough to demonstrate a few things about how Catch is used. Let's take a moment to consider those before we move on.
1. All we did was ```#define``` one identifier and ```#include``` one header and we got everything - even an implementation of ```main()``` that will [respond to command line arguments](command-line.md#top). You can only use that ```#define``` in one implementation file, for (hopefully) obvious reasons. Once you have more than one file with unit tests in you'll just ```#include "catch.hpp"``` and go. Usually it's a good idea to have a dedicated implementation file that just has ```#define CATCH_CONFIG_MAIN``` and ```#include "catch.hpp"```. You can also provide your own implementation of main and drive Catch yourself (see [Supplying-your-own-main()](own-main.md#top)).
2. We introduce test cases with the ```TEST_CASE``` macro. This macro takes one or two arguments - a free form test name and, optionally, one or more tags (for more see <a href="#test-cases-and-sections">Test cases and Sections</a>). The test name must be unique. You can run sets of tests by specifying a wildcarded test name or a tag expression. See the [command line docs](command-line.md#top) for more information on running tests.
3. The name and tags arguments are just strings. We haven't had to declare a function or method - or explicitly register the test case anywhere. Behind the scenes a function with a generated name is defined for you, and automatically registered using static registry classes. By abstracting the function name away we can name our tests without the constraints of identifier names.
4. We write our individual test assertions using the ```REQUIRE``` macro. Rather than a separate macro for each type of condition we express the condition naturally using C/C++ syntax. Behind the scenes a simple set of expression templates captures the left-hand-side and right-hand-side of the expression so we can display the values in our test report. As we'll see later there _are_ other assertion macros - but because of this technique the number of them is drastically reduced.
<a id="test-cases-and-sections"></a>
## Test cases and sections
Like most test frameworks, Catch2 supports a class-based fixture mechanism,
where individual tests are methods on class and setup/teardown can be
done in constructor/destructor of the type.
Most test frameworks have a class-based fixture mechanism. That is, test cases map to methods on a class and common setup and teardown can be performed in ```setup()``` and ```teardown()``` methods (or constructor/ destructor in languages, like C++, that support deterministic destruction).
However, their use in Catch2 is rare, because idiomatic Catch2 tests
instead use _sections_ to share setup and teardown code between test code.
This is best explained through an example ([code](../examples/100-Fix-Section.cpp)):
While Catch fully supports this way of working there are a few problems with the approach. In particular the way your code must be split up, and the blunt granularity of it, may cause problems. You can only have one setup/ teardown pair across a set of methods, but sometimes you want slightly different setup in each method, or you may even want several levels of setup (a concept which we will clarify later on in this tutorial). It was <a href="http://jamesnewkirk.typepad.com/posts/2007/09/why-you-should-.html">problems like these</a> that led James Newkirk, who led the team that built NUnit, to start again from scratch and <a href="http://jamesnewkirk.typepad.com/posts/2007/09/announcing-xuni.html">build xUnit</a>).
Catch takes a different approach (to both NUnit and xUnit) that is a more natural fit for C++ and the C family of languages. This is best explained through an example ([code](../examples/100-Fix-Section.cpp)):
```c++
TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
@@ -152,75 +154,125 @@ TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
}
```
For each `SECTION` the `TEST_CASE` is executed from the start. This means
that each section is entered with a freshly constructed vector `v`, that
we know has size 5 and capacity at least 5, because the two assertions
are also checked before the section is entered. Each run through a test
case will execute one, and only one, leaf section.
For each ```SECTION``` the ```TEST_CASE``` is executed from the start - so as we enter each section we know that size is 5 and capacity is at least 5. We enforced those requirements with the ```REQUIRE```s at the top level so we can be confident in them.
This works because the ```SECTION``` macro contains an if statement that calls back into Catch to see if the section should be executed. One leaf section is executed on each run through a ```TEST_CASE```. The other sections are skipped. Next time through the next section is executed, and so on until no new sections are encountered.
Section can also be nested, in which case the parent section can be
entered multiple times, once for each leaf section. Nested sections are
most useful when you have multiple tests that share part of the set up.
To continue on the vector example above, you could add a check that
`std::vector::reserve` does not remove unused excess capacity, like this:
So far so good - this is already an improvement on the setup/teardown approach because now we see our setup code inline and use the stack.
```cpp
The power of sections really shows, however, when we need to execute a sequence of checked operations. Continuing the vector example, we might want to verify that attempting to reserve a capacity smaller than the current capacity of the vector changes nothing. We can do that, naturally, like so:
```c++
SECTION( "reserving bigger changes capacity but not size" ) {
v.reserve( 10 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );
SECTION( "reserving down unused capacity does not change capacity" ) {
SECTION( "reserving smaller again does not change capacity" ) {
v.reserve( 7 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );
}
}
```
Another way to look at sections is that they are a way to define a tree
of paths through the test. Each section represents a node, and the final
tree is walked in depth-first manner, with each path only visiting only
one leaf node.
Sections can be nested to an arbitrary depth (limited only by your stack size). Each leaf section (i.e. a section that contains no nested sections) will be executed exactly once, on a separate path of execution from any other leaf section (so no leaf section can interfere with another). A failure in a parent section will prevent nested sections from running - but then that's the idea.
There is no practical limit on nesting sections, as long as your compiler
can handle them, but keep in mind that overly nested sections can become
unreadable. From experience, having section nest more than 3 levels is
usually very hard to follow and not worth the removed duplication.
## BDD-Style
If you name your test cases and sections appropriately you can achieve a BDD-style specification structure. This became such a useful way of working that first class support has been added to Catch. Scenarios can be specified using ```SCENARIO```, ```GIVEN```, ```WHEN``` and ```THEN``` macros, which map on to ```TEST_CASE```s and ```SECTION```s, respectively. For more details see [Test cases and sections](test-cases-and-sections.md#top).
The vector example can be adjusted to use these macros like so ([example code](../examples/120-Bdd-ScenarioGivenWhenThen.cpp)):
```c++
SCENARIO( "vectors can be sized and resized", "[vector]" ) {
GIVEN( "A vector with some items" ) {
std::vector<int> v( 5 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 5 );
WHEN( "the size is increased" ) {
v.resize( 10 );
THEN( "the size and capacity change" ) {
REQUIRE( v.size() == 10 );
REQUIRE( v.capacity() >= 10 );
}
}
WHEN( "the size is reduced" ) {
v.resize( 0 );
THEN( "the size changes but not capacity" ) {
REQUIRE( v.size() == 0 );
REQUIRE( v.capacity() >= 5 );
}
}
WHEN( "more capacity is reserved" ) {
v.reserve( 10 );
THEN( "the capacity changes but not the size" ) {
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );
}
}
WHEN( "less capacity is reserved" ) {
v.reserve( 0 );
THEN( "neither size nor capacity are changed" ) {
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 5 );
}
}
}
}
```
Conveniently, these tests will be reported as follows when run:
```
Scenario: vectors can be sized and resized
Given: A vector with some items
When: more capacity is reserved
Then: the capacity changes but not the size
```
<a id="scaling-up"></a>
## Scaling up
To keep the tutorial simple we put all our code in a single file. This is fine to get started - and makes jumping into Catch even quicker and easier. As you write more real-world tests, though, this is not really the best approach.
The requirement is that the following block of code ([or equivalent](own-main.md#top)):
```c++
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
```
appears in _exactly one_ source file. Use as many additional cpp files (or whatever you call your implementation files) as you need for your tests, partitioned however makes most sense for your way of working. Each additional file need only ```#include "catch.hpp"``` - do not repeat the ```#define```!
In fact it is usually a good idea to put the block with the ```#define``` [in its own source file](slow-compiles.md#top) (code example [main](../examples/020-TestCase-1.cpp), [tests](../examples/020-TestCase-2.cpp)).
Do not write your tests in header files!
## BDD style testing
## Type parametrised test cases
Catch2 also provides some basic support for BDD-style testing. There are
macro aliases for `TEST_CASE` and `SECTIONS` that you can use so that
the resulting tests read as BDD spec. `SCENARIO` acts as a `TEST_CASE`
with "Scenario: " name prefix. Then there are `GIVEN`, `WHEN`, `THEN`
(and their variants with `AND_` prefix), which act as a `SECTION`,
similarly prefixed with the macro name.
Test cases in Catch2 can be also parametrised by type, via the
`TEMPLATE_TEST_CASE` and `TEMPLATE_PRODUCT_TEST_CASE` macros,
which behave in the same way the `TEST_CASE` macro, but are run for
every type or type combination.
For more details on the macros look at the [test cases and
sections](test-cases-and-sections.md#top) part of the reference docs,
or at the [vector example done with BDD macros](../examples/120-Bdd-ScenarioGivenWhenThen.cpp).
## Data and Type driven tests
Test cases in Catch2 can also be driven by types, input data, or both
at the same time.
For more details look into the Catch2 reference, either at the
[type parametrized test cases](test-cases-and-sections.md#type-parametrised-test-cases),
or [data generators](generators.md#top).
For more details, see our documentation on [test cases and
sections](test-cases-and-sections.md#type-parametrised-test-cases).
## Next steps
This page is a brief introduction to get you up and running with Catch2,
and to show the basic features of Catch2. The features mentioned here
can get you quite far, but there are many more. However, you can read
about these as you go, in the ever-growing [reference section](Readme.md#top)
of the documentation.
This has been a brief introduction to get you up and running with Catch, and to point out some of the key differences between Catch and other frameworks you may already be familiar with. This will get you going quite far already and you are now in a position to dive in and write some tests.
Of course there is more to learn - most of which you should be able to page-fault in as you go. Please see the ever-growing [Reference section](Readme.md#top) for what's available.
---

View File

@@ -1,100 +0,0 @@
<a id="top"></a>
# Best practices and other tips on using Catch2
## Running tests
Your tests should be run in a manner roughly equivalent with:
```
./tests --order rand --warn NoAssertions
```
Notice that all the tests are run in a large batch, their relative order
is randomized, and that you ask Catch2 to fail test whose leaf-path
does not contain an assertion.
The reason I recommend running all your tests in the same process is that
this exposes your tests to interference from their runs. This can be both
positive interference, where the changes in global state from previous
test allow later tests to pass, but also negative interference, where
changes in global state from previous test causes later tests to fail.
In my experience, interference, especially destructive interference,
usually comes from errors in the code under test, rather than the tests
themselves. This means that by allowing interference to happen, our tests
can find these issues. Obviously, to shake out interference coming from
different orderings of tests, the test order also need to be shuffled
between runs.
However, running all tests in a single batch eventually becomes impractical
as they will take too long to run, and you will want to run your tests
in parallel.
<a id="parallel-tests"></a>
## Running tests in parallel
There are multiple ways of running tests in parallel, with various level
of structure. If you are using CMake and CTest, then we provide a helper
function [`catch_discover_tests`](cmake-integration.md#automatic-test-registration)
that registers each Catch2 `TEST_CASE` as a single CTest test, which
is then run in a separate process. This is an easy way to set up parallel
tests if you are already using CMake & CTest to run your tests, but you
will lose the advantage of running tests in batches.
Catch2 also supports [splitting tests in a binary into multiple
shards](command-line.md#test-sharding). This can be used by any test
runner to run batches of tests in parallel. Do note that when selecting
on the number of shards, you should have more shards than there are cores,
to avoid issues with long-running tests getting accidentally grouped in
the same shard, and causing long-tailed execution time.
**Note that naively composing sharding and random ordering of tests will break.**
Invoking Catch2 test executable like this
```text
./tests --order rand --shard-index 0 --shard-count 3
./tests --order rand --shard-index 1 --shard-count 3
./tests --order rand --shard-index 2 --shard-count 3
```
does not guarantee covering all tests inside the executable, because
each invocation will have its own random seed, thus it will have its own
random order of tests and thus the partitioning of tests into shards will
be different as well.
To do this properly, you need the individual shards to share the random
seed, e.g.
```text
./tests --order rand --shard-index 0 --shard-count 3 --rng-seed 0xBEEF
./tests --order rand --shard-index 1 --shard-count 3 --rng-seed 0xBEEF
./tests --order rand --shard-index 2 --shard-count 3 --rng-seed 0xBEEF
```
Catch2 actually provides a helper to automatically register multiple shards
as CTest tests, with shared random seed that changes each CTest invocation.
For details look at the documentation of
[`CatchShardTests.cmake` CMake script](cmake-integration.md#catchshardtestscmake).
## Organizing tests into binaries
Both overly large and overly small test binaries can cause issues. Overly
large test binaries have to be recompiled and relinked often, and the
link times are usually also long. Overly small test binaries in turn pay
significant overhead from linking against Catch2 more often per compiled
test case, and also make it hard/impossible to run tests in batches.
Because there is no hard and fast rule for the right size of a test binary,
I recommend having 1:1 correspondence between libraries in project and test
binaries. (At least if it is possible, in some cases it is not.) Having
a test binary for each library in project keeps related tests together,
and makes tests easy to navigate by reflecting the project's organizational
structure.
---
[Home](Readme.md#top)

View File

@@ -6,53 +6,40 @@ including (but not limited to),
[Google Test](http://code.google.com/p/googletest/),
[Boost.Test](http://www.boost.org/doc/libs/1_49_0/libs/test/doc/html/index.html),
[CppUnit](http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=Main_Page),
[Cute](http://www.cute-test.com), and
[Cute](http://www.cute-test.com),
[many, many more](http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C.2B.2B).
So what does Catch2 bring to the party that differentiates it from these? Apart from the catchy name, of course.
So what does Catch bring to the party that differentiates it from these? Apart from a Catchy name, of course.
## Key Features
* Quick and easy to get started. Just download two files, add them into your project and you're away.
* No external dependencies. As long as you can compile C++14 and have the C++ standard library available.
* Quick and Really easy to get started. Just download catch.hpp, `#include` it and you're away.
* No external dependencies. As long as you can compile C++11 and have a C++ standard library available.
* Write test cases as, self-registering, functions (or methods, if you prefer).
* Divide test cases into sections, each of which is run in isolation (eliminates the need for fixtures).
* Use BDD-style Given-When-Then sections as well as traditional unit test cases.
* Only one core assertion macro for comparisons. Standard C/C++ operators are used for the comparison - yet the full expression is decomposed and lhs and rhs values are logged.
* Tests are named using free-form strings - no more couching names in legal identifiers.
## Other core features
* Tests can be tagged for easily running ad-hoc groups of tests.
* Failures can (optionally) break into the debugger on common platforms.
* Failures can (optionally) break into the debugger on Windows and Mac.
* Output is through modular reporter objects. Basic textual and XML reporters are included. Custom reporters can easily be added.
* JUnit xml output is supported for integration with third-party tools, such as CI servers.
* A default main() function is provided, but you can supply your own for complete control (e.g. integration into your own test runner GUI).
* A command line parser is provided and can still be used if you choose to provided your own main() function.
* Catch can test itself.
* Alternative assertion macro(s) report failures but don't abort the test case
* Good set of facilities for floating point comparisons (`Catch::Approx` and full set of matchers)
* Floating point tolerance comparisons are built in using an expressive Approx() syntax.
* Internal and friendly macros are isolated so name clashes can be managed
* Data generators (data driven test support)
* Hamcrest-style Matchers for testing complex properties
* Microbenchmarking support
* Matchers
## Who else is using Catch?
## Who else is using Catch2?
See the list of [open source projects using Catch](opensource-users.md#top).
A whole lot of people. According to the 2021 JetBrains C++ ecosystem survey,
about 11% of C++ programmers use Catch2 for unit testing, making it the
second most popular unit testing framework.
You can also take a look at the (incomplete) list of [open source projects](opensource-users.md#top)
or the (very incomplete) list of [commercial users of Catch2](commercial-users.md#top)
for some idea on who else also uses Catch2.
---
See the [tutorial](tutorial.md#top) to get more of a taste of using
Catch2 in practice.
See the [tutorial](tutorial.md#top) to get more of a taste of using Catch in practice
---

View File

@@ -0,0 +1,15 @@
// 000-CatchMain.cpp
// In a Catch project with multiple files, dedicate one file to compile the
// source code of Catch itself and reuse the resulting object file for linking.
// Let Catch provide main():
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
// That's it
// Compile implementation of Catch for use with files that do contain tests:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -c 000-CatchMain.cpp
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% -c 000-CatchMain.cpp

View File

@@ -1,8 +1,11 @@
// 010-TestCase.cpp
// And write tests in the same file:
#include <catch2/catch_test_macros.hpp>
static int Factorial( int number ) {
// Let Catch provide main():
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
int Factorial( int number ) {
return number <= 1 ? number : Factorial( number - 1 ) * number; // fail
// return number <= 1 ? 1 : Factorial( number - 1 ) * number; // pass
}
@@ -19,7 +22,7 @@ TEST_CASE( "Factorials of 1 and higher are computed (pass)", "[single-file]" ) {
}
// Compile & run:
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 010-TestCase 010-TestCase.cpp && 010-TestCase --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 010-TestCase 010-TestCase.cpp && 010-TestCase --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 010-TestCase.cpp && 010-TestCase --success
// Expected compact output (all assertions):

View File

@@ -1,6 +1,12 @@
// 020-TestCase-1.cpp
#include <catch2/catch_test_macros.hpp>
// In a Catch project with multiple files, dedicate one file to compile the
// source code of Catch itself and reuse the resulting object file for linking.
// Let Catch provide main():
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
TEST_CASE( "1: All test cases reside in other .cpp files (empty)", "[multi-file:1]" ) {
}
@@ -10,8 +16,8 @@ TEST_CASE( "1: All test cases reside in other .cpp files (empty)", "[multi-file:
// Here just to show there are two source files via option --list-tests.
// Compile & run:
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -c 020-TestCase-1.cpp
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 020-TestCase TestCase-1.o 020-TestCase-2.cpp && 020-TestCase --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -c 020-TestCase-1.cpp
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 020-TestCase TestCase-1.o 020-TestCase-2.cpp && 020-TestCase --success
//
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% -c 020-TestCase-1.cpp
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% -Fe020-TestCase.exe 020-TestCase-1.obj 020-TestCase-2.cpp && 020-TestCase --success

View File

@@ -2,9 +2,9 @@
// main() provided by Catch in file 020-TestCase-1.cpp.
#include <catch2/catch_test_macros.hpp>
#include <catch2/catch.hpp>
static int Factorial( int number ) {
int Factorial( int number ) {
return number <= 1 ? number : Factorial( number - 1 ) * number; // fail
// return number <= 1 ? 1 : Factorial( number - 1 ) * number; // pass
}

View File

@@ -8,11 +8,11 @@
// - REQUIRE_FALSE() stops at first failure.
// - CHECK_FALSE() continues after failure.
// main() provided by linkage to Catch2WithMain
// main() provided in 000-CatchMain.cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/catch.hpp>
static std::string one() {
std::string one() {
return "1";
}
@@ -53,8 +53,8 @@ TEST_CASE( "Assert that something is false (continue after failure)", "[check-fa
}
// Compile & run:
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 030-Asn-Require-Check 030-Asn-Require-Check.cpp && 030-Asn-Require-Check --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 030-Asn-Require-Check.cpp && 030-Asn-Require-Check --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 030-Asn-Require-Check 030-Asn-Require-Check.cpp 000-CatchMain.o && 030-Asn-Require-Check --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 030-Asn-Require-Check.cpp 000-CatchMain.obj && 030-Asn-Require-Check --success
// Expected compact output (all assertions):
//

View File

@@ -4,10 +4,9 @@
// - Sections (this file)
// - Traditional class-based fixtures
// main() provided by linkage to Catch2WithMain
// main() provided in 000-CatchMain.cpp
#include <catch2/catch_test_macros.hpp>
#include <vector>
#include <catch2/catch.hpp>
TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
@@ -45,8 +44,8 @@ TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
}
// Compile & run:
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 100-Fix-Section 100-Fix-Section.cpp && 100-Fix-Section --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 100-Fix-Section.cpp && 100-Fix-Section --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 100-Fix-Section 100-Fix-Section.cpp 000-CatchMain.o && 100-Fix-Section --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 100-Fix-Section.cpp 000-CatchMain.obj && 100-Fix-Section --success
// Expected compact output (all assertions):
//

View File

@@ -4,9 +4,9 @@
// - Sections
// - Traditional class-based fixtures (this file)
// main() provided by linkage to Catch2WithMain
// main() provided in 000-CatchMain.cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/catch.hpp>
class DBConnection
{
@@ -52,11 +52,8 @@ TEST_CASE_METHOD( UniqueTestsFixture, "Create Employee/Normal", "[create]" ) {
}
// Compile & run:
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 110-Fix-ClassFixture 110-Fix-ClassFixture.cpp && 110-Fix-ClassFixture --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 110-Fix-ClassFixture.cpp && 110-Fix-ClassFixture --success
//
// Compile with pkg-config:
// - g++ -std=c++14 -Wall $(pkg-config catch2-with-main --cflags) -o 110-Fix-ClassFixture 110-Fix-ClassFixture.cpp $(pkg-config catch2-with-main --libs)
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 110-Fix-ClassFixture 110-Fix-ClassFixture.cpp 000-CatchMain.o && 110-Fix-ClassFixture --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 110-Fix-ClassFixture.cpp 000-CatchMain.obj && 110-Fix-ClassFixture --success
// Expected compact output (all assertions):
//

View File

@@ -1,8 +1,8 @@
// 120-Bdd-ScenarioGivenWhenThen.cpp
// main() provided by linkage with Catch2WithMain
// main() provided in 000-CatchMain.cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/catch.hpp>
SCENARIO( "vectors can be sized and resized", "[vector]" ) {
@@ -48,8 +48,8 @@ SCENARIO( "vectors can be sized and resized", "[vector]" ) {
}
// Compile & run:
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 120-Bdd-ScenarioGivenWhenThen 120-Bdd-ScenarioGivenWhenThen.cpp && 120-Bdd-ScenarioGivenWhenThen --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 120-Bdd-ScenarioGivenWhenThen.cpp && 120-Bdd-ScenarioGivenWhenThen --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 120-Bdd-ScenarioGivenWhenThen 120-Bdd-ScenarioGivenWhenThen.cpp 000-CatchMain.o && 120-Bdd-ScenarioGivenWhenThen --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 120-Bdd-ScenarioGivenWhenThen.cpp 000-CatchMain.obj && 120-Bdd-ScenarioGivenWhenThen --success
// Expected compact output (all assertions):
//

View File

@@ -0,0 +1,27 @@
// 200-Rpt-CatchMain.cpp
// In a Catch project with multiple files, dedicate one file to compile the
// source code of Catch itself and reuse the resulting object file for linking.
// Let Catch provide main():
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
#ifdef CATCH_EXAMPLE_RPT_1
#include CATCH_EXAMPLE_RPT_1
#endif
#ifdef CATCH_EXAMPLE_RPT_2
#include CATCH_EXAMPLE_RPT_2
#endif
#ifdef CATCH_EXAMPLE_RPT_3
#include CATCH_EXAMPLE_RPT_3
#endif
// That's it
// Compile implementation of Catch for use with files that do contain tests:
// - g++ -std=c++11 -Wall -I$(CATCH_ROOT) -DCATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" -o 200-Rpt-CatchMainTeamCity.o -c 200-Rpt-CatchMain.cpp
// cl -EHsc -I%CATCH_ROOT% -DCATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" -Fo200-Rpt-CatchMainTeamCity.obj -c 200-Rpt-CatchMain.cpp

View File

@@ -0,0 +1,171 @@
// 207-Rpt-TeamCityReporter.cpp
// Catch has built-in and external reporters:
// Built-in:
// - compact
// - console
// - junit
// - xml
// External:
// - automake
// - tap
// - teamcity (this example)
// main() and reporter code provided in 200-Rpt-CatchMain.cpp
#include <catch2/catch.hpp>
#ifdef _MSC_VER
# pragma warning (disable : 4702) // Disable warning: unreachable code
#endif
TEST_CASE( "TeamCity passes unconditionally succeeding assertion", "[teamcity]" ) {
SUCCEED();
}
TEST_CASE( "TeamCity reports unconditionally failing assertion", "[teamcity]" ) {
FAIL();
}
TEST_CASE( "TeamCity reports failing check", "[teamcity]" ) {
REQUIRE( 3 == 7 );
}
TEST_CASE( "TeamCity reports failing check-false", "[teamcity]" ) {
REQUIRE_FALSE( 3 == 3 );
}
TEST_CASE( "TeamCity reports failing check-that", "[teamcity]" ) {
using namespace Catch;
REQUIRE_THAT( "hello", Contains( "world" ) );
}
TEST_CASE( "TeamCity reports unexpected exception", "[teamcity]" ) {
REQUIRE( (throw std::runtime_error("surprise!"), true) );
}
TEST_CASE( "TeamCity reports undesired exception", "[teamcity]" ) {
REQUIRE_NOTHROW( (throw std::runtime_error("surprise!"), true) );
}
TEST_CASE( "TeamCity reports missing expected exception", "[teamcity]" ) {
REQUIRE_THROWS( true );
}
TEST_CASE( "TeamCity reports missing specific expected exception", "[teamcity]" ) {
REQUIRE_THROWS_AS( throw std::bad_alloc(), std::runtime_error );
}
TEST_CASE( "TeamCity reports unexpected message in expected exception", "[teamcity]" ) {
using namespace Catch;
CHECK_THROWS_WITH( throw std::runtime_error("hello"), "world" );
CHECK_THROWS_WITH( throw std::runtime_error("hello"), Contains("world") );
}
struct MyException: public std::runtime_error
{
MyException( char const * text )
: std::runtime_error( text ) {}
~MyException() override;
};
// prevent -Wweak-vtables:
MyException::~MyException() = default;
struct MyExceptionMatcher : Catch::MatcherBase< std::runtime_error >
{
std::string m_text;
MyExceptionMatcher( char const * text )
: m_text( text )
{}
~MyExceptionMatcher() override;
bool match( std::runtime_error const & arg ) const override
{
return m_text == arg.what() ;
}
std::string describe() const override
{
return "it's me";
}
};
// prevent -Wweak-vtables:
MyExceptionMatcher::~MyExceptionMatcher() = default;
TEST_CASE( "TeamCity failing check-throws-matches", "[teamcity]" ) {
CHECK_THROWS_MATCHES( throw MyException("hello"), MyException, MyExceptionMatcher("world") );
}
// [!throws] - lets Catch know that this test is likely to throw an exception even if successful.
// This causes the test to be excluded when running with -e or --nothrow.
// No special effects for the reporter.
TEST_CASE( "TeamCity throwing exception with tag [!throws]", "[teamcity][!throws]" ) {
REQUIRE_THROWS( throw std::runtime_error("unsurprisingly") );
}
// [!mayfail] - doesn't fail the test if any given assertion fails (but still reports it). This can be useful to flag a work-in-progress, or a known issue that you don't want to immediately fix but still want to track in your tests.
TEST_CASE( "TeamCity failing assertion with tag [!mayfail]", "[teamcity][!mayfail] " ) {
REQUIRE( 3 == 7 ); // doesn't fail test case this time, reports: testIgnored
REQUIRE( 3 == 3 );
}
// [!shouldfail] - like [!mayfail] but fails the test if it passes.
// This can be useful if you want to be notified of accidental, or third-party, fixes.
TEST_CASE( "TeamCity succeeding assertion with tag [!shouldfail]", "[teamcity][!shouldfail]" ) {
SUCCEED( "Marked [!shouldfail]" );
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_ROOT) -DCATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" -o 200-Rpt-CatchMainTeamCity.o -c 200-Rpt-CatchMain.cpp
// - g++ -std=c++11 -Wall -I$(CATCH_ROOT) -o 207-Rpt-TeamCityReporter 207-Rpt-TeamCityReporter.cpp 200-Rpt-CatchMainTeamCity.o && 207-Rpt-TeamCityReporter --list-reporters
//
// - cl -EHsc -I%CATCH_ROOT% -DCATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" -Fo200-Rpt-CatchMainTeamCity.obj -c 200-Rpt-CatchMain.cpp
// - cl -EHsc -I%CATCH_ROOT% 207-Rpt-TeamCityReporter.cpp 200-Rpt-CatchMainTeamCity.o && 207-Rpt-TeamCityReporter --list-reporters
// Compilation output (--list-reporters):
// Available reporters:
// compact: Reports test results on a single line, suitable for IDEs
// console: Reports test results as plain lines of text
// junit: Reports test results in an XML format that looks like Ant's
// junitreport target
// teamcity: Reports test results as TeamCity service messages
// xml: Reports test results as an XML document
// Expected output (abbreviated and broken into shorter lines):
//
// prompt> 207-Rpt-TeamCityReporter.exe --reporter teamcity
// ##teamcity[testSuiteStarted name='207-Rpt-TeamCityReporter.exe']
// ##teamcity[testStarted name='TeamCity passes unconditionally succeeding assertion']
// ##teamcity[testFinished name='TeamCity passes unconditionally succeeding assertion' duration='1']
// ##teamcity[testStarted name='TeamCity reports unconditionally failing assertion']
// ##teamcity[testFailed name='TeamCity reports unconditionally failing assertion' /
// message='.../examples/207-Rpt-TeamCityReporter.cpp:23|n/
// ...............................................................................|n|n/
// .../examples/207-Rpt-TeamCityReporter.cpp:25|nexplicit failure']
// ##teamcity[testFinished name='TeamCity reports unconditionally failing assertion' duration='3']
// ...

View File

@@ -5,26 +5,22 @@
// 2. My listener and registration
// 3. Test cases
#include <catch2/catch_test_macros.hpp>
#include <catch2/reporters/catch_reporter_event_listener.hpp>
#include <catch2/reporters/catch_reporter_registrars.hpp>
#include <catch2/catch_test_case_info.hpp>
// main() provided in 000-CatchMain.cpp
// Let Catch provide the required interfaces:
#define CATCH_CONFIG_EXTERNAL_INTERFACES
#include <catch2/catch.hpp>
#include <iostream>
// -----------------------------------------------------------------------
// 1. Printing of listener data:
//
namespace {
std::string ws(int const level) {
return std::string( 2 * level, ' ' );
}
std::ostream& operator<<(std::ostream& out, Catch::Tag t) {
return out << "original: " << t.original;
}
template< typename T >
std::ostream& operator<<( std::ostream& os, std::vector<T> const& v ) {
os << "{ ";
@@ -32,6 +28,7 @@ std::ostream& operator<<( std::ostream& os, std::vector<T> const& v ) {
os << x << ", ";
return os << "}";
}
// struct SourceLineInfo {
// char const* file;
// std::size_t line;
@@ -122,36 +119,32 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
os << ws(level+1) << "- aborting: " << info.aborting << "\n";
}
// struct Tag {
// StringRef original, lowerCased;
// };
// struct TestCaseInfo {
// enum SpecialProperties{
// None = 0,
// IsHidden = 1 << 1,
// ShouldFail = 1 << 2,
// MayFail = 1 << 3,
// Throws = 1 << 4,
// NonPortable = 1 << 5,
// Benchmark = 1 << 6
// };
//
// bool isHidden() const;
// bool throws() const;
// bool okToFail() const;
// bool expectedToFail() const;
//
// enum class TestCaseProperties : uint8_t {
// None = 0,
// IsHidden = 1 << 1,
// ShouldFail = 1 << 2,
// MayFail = 1 << 3,
// Throws = 1 << 4,
// NonPortable = 1 << 5,
// Benchmark = 1 << 6
// };
// std::string tagsAsString() const;
//
//
// struct TestCaseInfo : NonCopyable {
//
// bool isHidden() const;
// bool throws() const;
// bool okToFail() const;
// bool expectedToFail() const;
//
//
// std::string name;
// std::string className;
// std::vector<Tag> tags;
// SourceLineInfo lineInfo;
// TestCaseProperties properties = TestCaseProperties::None;
// };
// std::string name;
// std::string className;
// std::string description;
// std::vector<std::string> tags;
// std::vector<std::string> lcaseTags;
// SourceLineInfo lineInfo;
// SpecialProperties properties;
// };
void print( std::ostream& os, int const level, std::string const& title, Catch::TestCaseInfo const& info ) {
os << ws(level ) << title << ":\n"
@@ -162,9 +155,11 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
<< ws(level+1) << "- tagsAsString(): '" << info.tagsAsString() << "'\n"
<< ws(level+1) << "- name: '" << info.name << "'\n"
<< ws(level+1) << "- className: '" << info.className << "'\n"
<< ws(level+1) << "- tags: " << info.tags << "\n";
<< ws(level+1) << "- description: '" << info.description << "'\n"
<< ws(level+1) << "- tags: " << info.tags << "\n"
<< ws(level+1) << "- lcaseTags: " << info.lcaseTags << "\n";
print( os, level+1 , "- lineInfo", info.lineInfo );
os << ws(level+1) << "- properties (flags): 0x" << std::hex << static_cast<uint32_t>(info.properties) << std::dec << "\n";
os << ws(level+1) << "- properties (flags): 0x" << std::hex << info.properties << std::dec << "\n";
}
// struct TestCaseStats {
@@ -177,7 +172,7 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
void print( std::ostream& os, int const level, std::string const& title, Catch::TestCaseStats const& info ) {
os << ws(level ) << title << ":\n";
print( os, level+1 , "- testInfo", *info.testInfo );
print( os, level+1 , "- testInfo", info.testInfo );
print( os, level+1 , "- totals" , info.totals );
os << ws(level+1) << "- stdOut: " << info.stdOut << "\n"
<< ws(level+1) << "- stdErr: " << info.stdErr << "\n"
@@ -278,8 +273,8 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
print( os, level+1 , "- getSourceInfo(): ", info.getSourceInfo() );
os << ws(level+1) << "- getTestMacroName(): '" << info.getTestMacroName() << "'\n";
print( os, level+1 , "- *** m_info (AssertionInfo)", info.m_info );
print( os, level+1 , "- *** m_resultData (AssertionResultData)", info.m_resultData );
// print( os, level+1 , "- *** m_info (AssertionInfo)", info.m_info );
// print( os, level+1 , "- *** m_resultData (AssertionResultData)", info.m_resultData );
}
// struct AssertionStats {
@@ -302,13 +297,12 @@ void print( std::ostream& os, int const level, std::string const& title, Catch::
char const * dashed_line =
"--------------------------------------------------------------------------";
struct MyListener : Catch::TestEventListenerBase {
struct MyListener : Catch::EventListenerBase {
using EventListenerBase::EventListenerBase; // inherit constructor
using TestEventListenerBase::TestEventListenerBase; // inherit constructor
// Get rid of Wweak-tables
~MyListener() override;
~MyListener();
// The whole test run starting
void testRunStarting( Catch::TestRunInfo const& testRunInfo ) override {
@@ -366,14 +360,13 @@ struct MyListener : Catch::EventListenerBase {
print( std::cout, 1, "- assertionInfo", assertionInfo );
}
void assertionEnded( Catch::AssertionStats const& assertionStats ) override {
bool assertionEnded( Catch::AssertionStats const& assertionStats ) override {
std::cout << "\nEvent: assertionEnded:\n";
print( std::cout, 1, "- assertionStats", assertionStats );
return true;
}
};
} // end anonymous namespace
CATCH_REGISTER_LISTENER( MyListener )
// Get rid of Wweak-tables
@@ -420,8 +413,8 @@ TEST_CASE_METHOD( Fixture, "3: Testcase with class-based fixture", "[tag-C][tag-
}
// Compile & run:
// - g++ -std=c++14 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 210-Evt-EventListeners 210-Evt-EventListeners.cpp && 210-Evt-EventListeners --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 210-Evt-EventListeners.cpp && 210-Evt-EventListeners --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 210-Evt-EventListeners 210-Evt-EventListeners.cpp 000-CatchMain.o && 210-Evt-EventListeners --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 210-Evt-EventListeners.cpp 000-CatchMain.obj && 210-Evt-EventListeners --success
// Expected compact output (all assertions):
//

View File

@@ -5,10 +5,11 @@
// semantic, because it buffers the output. For most uses however,
// there is no important difference between having `std::cerr` buffered
// or unbuffered.
#include <catch2/catch_test_macros.hpp>
#include <sstream>
#include <cstdio>
#define CATCH_CONFIG_NOSTDOUT
#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
class out_buff : public std::stringbuf {
std::FILE* m_stream;

View File

@@ -4,14 +4,10 @@
// Specifically we will implement a random number generator for integers
// It will have infinite capacity and settable lower/upper bound
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators.hpp>
#include <catch2/generators/catch_generators_adapters.hpp>
#include <catch2/catch.hpp>
#include <random>
namespace {
// This class shows how to implement a simple generator for Catch tests
class RandomIntGenerator : public Catch::Generators::IGenerator<int> {
std::minstd_rand m_rand;
@@ -42,15 +38,9 @@ int const& RandomIntGenerator::get() const {
// Notice that it returns an instance of GeneratorWrapper<int>, which
// is a value-wrapper around std::unique_ptr<IGenerator<int>>.
Catch::Generators::GeneratorWrapper<int> random(int low, int high) {
return Catch::Generators::GeneratorWrapper<int>(
new RandomIntGenerator(low, high)
// Another possibility:
// Catch::Detail::make_unique<RandomIntGenerator>(low, high)
);
return Catch::Generators::GeneratorWrapper<int>(std::unique_ptr<Catch::Generators::IGenerator<int>>(new RandomIntGenerator(low, high)));
}
} // end anonymous namespaces
// The two sections in this test case are equivalent, but the first one
// is much more readable/nicer to use
TEST_CASE("Generating random ints", "[example][generator]") {
@@ -60,7 +50,7 @@ TEST_CASE("Generating random ints", "[example][generator]") {
REQUIRE(i <= 100);
}
SECTION("Creating the random generator directly") {
auto i = GENERATE(take(100, GeneratorWrapper<int>(Catch::Detail::make_unique<RandomIntGenerator>(-100, 100))));
auto i = GENERATE(take(100, GeneratorWrapper<int>(std::unique_ptr<IGenerator<int>>(new RandomIntGenerator(-100, 100)))));
REQUIRE(i >= -100);
REQUIRE(i <= 100);
}

View File

@@ -5,14 +5,11 @@
// that converts the strings using stoi, so the returned type is actually
// an int.
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators_adapters.hpp>
#include <catch2/catch.hpp>
#include <string>
#include <sstream>
namespace {
// Returns a line from a stream. You could have it e.g. read lines from
// a file, but to avoid problems with paths in examples, we will use
// a fixed stringstream.
@@ -23,12 +20,12 @@ public:
LineGenerator() {
m_stream.str("1\n2\n3\n4\n");
if (!next()) {
Catch::Generators::Detail::throw_generator_exception("Couldn't read a single line");
throw Catch::GeneratorException("Couldn't read a single line");
}
}
std::string const& get() const override;
bool next() override {
return !!std::getline(m_stream, m_line);
}
@@ -43,17 +40,18 @@ std::string const& LineGenerator::get() const {
// is a value-wrapper around std::unique_ptr<IGenerator<std::string>>.
Catch::Generators::GeneratorWrapper<std::string> lines(std::string /* ignored for example */) {
return Catch::Generators::GeneratorWrapper<std::string>(
new LineGenerator()
std::unique_ptr<Catch::Generators::IGenerator<std::string>>(
new LineGenerator()
)
);
}
} // end anonymous namespace
TEST_CASE("filter can convert types inside the generator expression", "[example][generator]") {
auto num = GENERATE(map<int>([](std::string const& line) { return std::stoi(line); },
lines("fake-file")));
REQUIRE(num > 0);
}

View File

@@ -2,8 +2,7 @@
// Shows how to use table to run a test many times with different inputs. Lifted from examples on
// issue #850.
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators.hpp>
#include <catch2/catch.hpp>
#include <string>
struct TestSubject {
@@ -44,11 +43,11 @@ TEST_CASE("Table allows pre-computed test inputs and outputs", "[example][genera
/* Possible simplifications where less legacy toolchain support is needed:
*
* - With libstdc++6 or newer, the make_tuple() calls can be ommitted
* - With libstdc++6 or newer, the make_tuple() calls can be omitted
* (technically C++17 but does not require -std in GCC/Clang). See
* https://stackoverflow.com/questions/12436586/tuple-vector-and-initializer-list
*
* - In C++17 mode std::tie() and the preceding variable delcarations can be
* - In C++17 mode std::tie() and the preceeding variable delcarations can be
* replaced by structured bindings: auto [test_input, expected] = GENERATE(
* table<std::string, size_t>({ ...
*/

View File

@@ -6,9 +6,7 @@
// _WILL_ outlive the variables -- thus they should be either captured
// by value directly, or copied by the generators during construction.
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators_adapters.hpp>
#include <catch2/generators/catch_generators_random.hpp>
#include <catch2/catch.hpp>
TEST_CASE("Generate random doubles across different ranges",
"[generator][example][advanced]") {

View File

@@ -9,9 +9,7 @@
// per-variable custom capture list, this example shows how to achieve
// that.
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators_adapters.hpp>
#include <catch2/generators/catch_generators_random.hpp>
#include <catch2/catch.hpp>
TEST_CASE("Generate random doubles across different ranges",
"[generator][example][advanced]") {

View File

@@ -1,30 +1,44 @@
cmake_minimum_required( VERSION 3.10 )
#
# Build examples.
#
# Requires CATCH_BUILD_EXAMPLES to be defined 'true', see ../CMakeLists.txt.
#
project( Catch2Examples LANGUAGES CXX )
cmake_minimum_required( VERSION 3.0 )
project( CatchExamples CXX )
message( STATUS "Examples included" )
# define folders used:
# Some one-offs first:
# 1) Tests and main in one file
add_executable( 010-TestCase
010-TestCase.cpp
)
set( EXAMPLES_DIR ${CATCH_DIR}/examples )
set( HEADER_DIR ${CATCH_DIR}/single_include )
set( REPORTER_HEADER_DIR ${CATCH_DIR}/include/reporters )
# 2) Tests and main across two files
add_executable( 020-MultiFile
020-TestCase-1.cpp
020-TestCase-2.cpp
)
# single-file sources:
add_executable(231-Cfg_OutputStreams
set( SOURCES_SINGLE_FILE
010-TestCase.cpp
231-Cfg-OutputStreams.cpp
)
target_link_libraries(231-Cfg_OutputStreams Catch2_buildall_interface)
target_compile_definitions(231-Cfg_OutputStreams PUBLIC CATCH_CONFIG_NOSTDOUT)
# These examples use the standard separate compilation
set( SOURCES_IDIOMATIC_EXAMPLES
# multiple-file modules:
set( SOURCES_020
020-TestCase-1.cpp
020-TestCase-2.cpp
)
# main for idiomatic test sources:
set( SOURCES_IDIOMATIC_MAIN
000-CatchMain.cpp
)
# sources to combine with 000-CatchMain.cpp:
set( SOURCES_IDIOMATIC_TESTS
030-Asn-Require-Check.cpp
100-Fix-Section.cpp
110-Fix-ClassFixture.cpp
@@ -37,25 +51,108 @@ set( SOURCES_IDIOMATIC_EXAMPLES
311-Gen-CustomCapture.cpp
)
string( REPLACE ".cpp" "" BASENAMES_IDIOMATIC_EXAMPLES "${SOURCES_IDIOMATIC_EXAMPLES}" )
set( TARGETS_IDIOMATIC_EXAMPLES ${BASENAMES_IDIOMATIC_EXAMPLES} )
# main-s for reporter-specific test sources:
foreach( name ${TARGETS_IDIOMATIC_EXAMPLES} )
add_executable( ${name}
${EXAMPLES_DIR}/${name}.cpp )
endforeach()
set(ALL_EXAMPLE_TARGETS
${TARGETS_IDIOMATIC_EXAMPLES}
010-TestCase
020-MultiFile
set( SOURCES_REPORTERS_MAIN
200-Rpt-CatchMain.cpp
)
foreach( name ${ALL_EXAMPLE_TARGETS} )
target_link_libraries( ${name} Catch2 Catch2WithMain )
string( REPLACE ".cpp" "" BASENAMES_REPORTERS_MAIN 200-Rpt-CatchMain.cpp )
set( NAMES_REPORTERS TeamCity )
foreach( reporter ${NAMES_REPORTERS} )
list( APPEND SOURCES_SPECIFIC_REPORTERS_MAIN ${BASENAMES_REPORTERS_MAIN}${reporter}.cpp )
endforeach()
# sources to combine with 200-Rpt-CatchMain{Reporter}.cpp:
set( SOURCES_REPORTERS_TESTS
207-Rpt-TeamCityReporter.cpp
)
# check if all sources are listed, warn if not:
set( SOURCES_ALL
${SOURCES_020}
${SOURCES_SINGLE_FILE}
${SOURCES_IDIOMATIC_MAIN}
${SOURCES_IDIOMATIC_TESTS}
${SOURCES_REPORTERS_MAIN}
${SOURCES_REPORTERS_TESTS}
)
foreach( name ${SOURCES_ALL} )
list( APPEND SOURCES_ALL_PATH ${EXAMPLES_DIR}/${name} )
endforeach()
CheckFileList( SOURCES_ALL_PATH ${EXAMPLES_DIR} )
# create target names:
string( REPLACE ".cpp" "" BASENAMES_SINGLE_FILE "${SOURCES_SINGLE_FILE}" )
string( REPLACE ".cpp" "" BASENAMES_IDIOMATIC_TESTS "${SOURCES_IDIOMATIC_TESTS}" )
string( REPLACE ".cpp" "" BASENAMES_REPORTERS_TESTS "${SOURCES_REPORTERS_TESTS}" )
string( REPLACE ".cpp" "" BASENAMES_REPORTERS_MAIN "${SOURCES_REPORTERS_MAIN}" )
set( TARGETS_SINGLE_FILE ${BASENAMES_SINGLE_FILE} )
set( TARGETS_IDIOMATIC_TESTS ${BASENAMES_IDIOMATIC_TESTS} )
set( TARGETS_REPORTERS_TESTS ${BASENAMES_REPORTERS_TESTS} )
set( TARGETS_REPORTERS_MAIN ${BASENAMES_REPORTERS_MAIN} )
set( TARGETS_ALL
${TARGETS_SINGLE_FILE}
020-TestCase
${TARGETS_IDIOMATIC_TESTS} CatchMain
${TARGETS_REPORTERS_TESTS} CatchMainTeamCity
)
# define program targets:
add_library( CatchMain OBJECT ${EXAMPLES_DIR}/${SOURCES_IDIOMATIC_MAIN} ${HEADER_DIR}/catch2/catch.hpp )
#add_library( CatchMainAutomake OBJECT ${EXAMPLES_DIR}/200-Rpt-CatchMain.cpp ${HEADER_DIR}/catch2/catch.hpp )
#add_library( CatchMainTap OBJECT ${EXAMPLES_DIR}/200-Rpt-CatchMain.cpp ${HEADER_DIR}/catch2/catch.hpp )
add_library( CatchMainTeamCity OBJECT ${EXAMPLES_DIR}/200-Rpt-CatchMain.cpp ${HEADER_DIR}/catch2/catch.hpp )
#target_compile_definitions( CatchMainAutomake PRIVATE CATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_automake.hpp\" )
#target_compile_definitions( CatchMainTap PRIVATE CATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_tap.hpp\" )
target_compile_definitions( CatchMainTeamCity PRIVATE CATCH_EXAMPLE_RPT_1=\"include/reporters/catch_reporter_teamcity.hpp\" )
foreach( name ${TARGETS_SINGLE_FILE} )
add_executable( ${name} ${EXAMPLES_DIR}/${name}.cpp ${HEADER_DIR}/catch2/catch.hpp )
endforeach()
foreach( name ${TARGETS_IDIOMATIC_TESTS} )
add_executable( ${name} ${EXAMPLES_DIR}/${name}.cpp $<TARGET_OBJECTS:CatchMain> ${HEADER_DIR}/catch2/catch.hpp )
endforeach()
add_executable( 020-TestCase ${EXAMPLES_DIR}/020-TestCase-1.cpp ${EXAMPLES_DIR}/020-TestCase-2.cpp ${HEADER_DIR}/catch2/catch.hpp )
#add_executable( 207-Rpt-AutomakeReporter ${EXAMPLES_DIR}/207-Rpt-AutomakeReporter.cpp $<TARGET_OBJECTS:CatchMainAutomake> ${HEADER_DIR}/catch2/catch.hpp )
#add_executable( 207-Rpt-TapReporter ${EXAMPLES_DIR}/207-Rpt-TapReporter.cpp $<TARGET_OBJECTS:CatchMainTap> ${HEADER_DIR}/catch2/catch.hpp )
add_executable( 207-Rpt-TeamCityReporter ${EXAMPLES_DIR}/207-Rpt-TeamCityReporter.cpp $<TARGET_OBJECTS:CatchMainTeamCity> ${HEADER_DIR}/catch2/catch.hpp )
#foreach( name ${TARGETS_REPORTERS_TESTS} )
# add_executable( ${name} ${EXAMPLES_DIR}/${name}.cpp $<TARGET_OBJECTS:CatchMain> ${HEADER_DIR}/catch2/catch.hpp )
#endforeach()
foreach( name ${TARGETS_ALL} )
target_include_directories( ${name} PRIVATE ${HEADER_DIR} ${CATCH_DIR} )
set_property(TARGET ${name} PROPERTY CXX_STANDARD 11)
set_property(TARGET ${name} PROPERTY CXX_EXTENSIONS OFF)
# Add desired warnings
if ( CMAKE_CXX_COMPILER_ID MATCHES "Clang|AppleClang|GNU" )
target_compile_options( ${name} PRIVATE -Wall -Wextra -Wunreachable-code )
endif()
# Clang specific warning go here
if ( CMAKE_CXX_COMPILER_ID MATCHES "Clang" )
# Actually keep these
target_compile_options( ${name} PRIVATE -Wweak-vtables -Wexit-time-destructors -Wglobal-constructors -Wmissing-noreturn )
endif()
if ( CMAKE_CXX_COMPILER_ID MATCHES "MSVC" )
target_compile_options( ${name} PRIVATE /W4 /w44265 /WX )
endif()
endforeach()
list(APPEND CATCH_WARNING_TARGETS ${ALL_EXAMPLE_TARGETS})
set(CATCH_WARNING_TARGETS ${CATCH_WARNING_TARGETS} PARENT_SCOPE)

View File

@@ -1,66 +0,0 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
# Supported optional args:
# * SHARD_COUNT - number of shards to split target's tests into
# * REPORTER - reporter spec to use for tests
# * TEST_SPEC - test spec used for filtering tests
function(catch_add_sharded_tests TARGET)
if (${CMAKE_VERSION} VERSION_LESS "3.10.0")
message(FATAL_ERROR "add_sharded_catch_tests only supports CMake versions 3.10.0 and up")
endif()
cmake_parse_arguments(
""
""
"SHARD_COUNT;REPORTER;TEST_SPEC"
""
${ARGN}
)
if (NOT DEFINED _SHARD_COUNT)
set(_SHARD_COUNT 2)
endif()
# Generate a unique name based on the extra arguments
string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS} ${_REPORTER} ${_OUTPUT_DIR} ${_OUTPUT_PREFIX} ${_OUTPUT_SUFFIX} ${_SHARD_COUNT}")
string(SUBSTRING ${args_hash} 0 7 args_hash)
set(ctest_include_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}-sharded-tests-include-${args_hash}.cmake")
set(ctest_tests_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}-sharded-tests-impl-${args_hash}.cmake")
file(WRITE "${ctest_include_file}"
"if(EXISTS \"${ctest_tests_file}\")\n"
" include(\"${ctest_tests_file}\")\n"
"else()\n"
" add_test(${TARGET}_NOT_BUILT-${args_hash} ${TARGET}_NOT_BUILT-${args_hash})\n"
"endif()\n"
)
set_property(DIRECTORY
APPEND PROPERTY TEST_INCLUDE_FILES "${ctest_include_file}"
)
set(shard_impl_script_file "${CMAKE_CURRENT_LIST_DIR}/CatchShardTestsImpl.cmake")
add_custom_command(
TARGET ${TARGET} POST_BUILD
BYPRODUCTS "${ctest_tests_file}"
COMMAND "${CMAKE_COMMAND}"
-D "TARGET_NAME=${TARGET}"
-D "TEST_BINARY=$<TARGET_FILE:${TARGET}>"
-D "CTEST_FILE=${ctest_tests_file}"
-D "SHARD_COUNT=${_SHARD_COUNT}"
-D "REPORTER_SPEC=${_REPORTER}"
-D "TEST_SPEC=${_TEST_SPEC}"
-P "${shard_impl_script_file}"
VERBATIM
)
endfunction()

View File

@@ -1,52 +0,0 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
# Indirection for CatchShardTests that allows us to delay the script
# file generation until build time.
# Expected args:
# * TEST_BINARY - full path to the test binary to run sharded
# * CTEST_FILE - full path to ctest script file to write to
# * TARGET_NAME - name of the target to shard (used for test names)
# * SHARD_COUNT - number of shards to split the binary into
# Optional args:
# * REPORTER_SPEC - reporter specs to be passed down to the binary
# * TEST_SPEC - test spec to pass down to the test binary
if(NOT EXISTS "${TEST_BINARY}")
message(FATAL_ERROR
"Specified test binary '${TEST_BINARY}' does not exist"
)
endif()
set(other_args "")
if (TEST_SPEC)
set(other_args "${other_args} ${TEST_SPEC}")
endif()
if (REPORTER_SPEC)
set(other_args "${other_args} --reporter ${REPORTER_SPEC}")
endif()
# foreach RANGE in cmake is inclusive of the end, so we have to adjust it
math(EXPR adjusted_shard_count "${SHARD_COUNT} - 1")
file(WRITE "${CTEST_FILE}"
"string(RANDOM LENGTH 8 ALPHABET \"0123456789abcdef\" rng_seed)\n"
"\n"
"foreach(shard_idx RANGE ${adjusted_shard_count})\n"
" add_test(${TARGET_NAME}-shard-" [[${shard_idx}]] "/${adjusted_shard_count}\n"
" ${TEST_BINARY}"
" --shard-index " [[${shard_idx}]]
" --shard-count ${SHARD_COUNT}"
" --rng-seed " [[0x${rng_seed}]]
" --order rand"
"${other_args}"
"\n"
" )\n"
"endforeach()\n"
)

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,20 +0,0 @@
# License: Boost 1.0
# By Paul Dreik 2020
# add a library that brings in the main() function from libfuzzer
# and has all the dependencies, so the individual fuzzers can be
# added one line each.
add_library(fuzzhelper NullOStream.h NullOStream.cpp)
target_link_libraries(fuzzhelper PUBLIC Catch2::Catch2)
# use C++17 so we can get string_view
target_compile_features(fuzzhelper PUBLIC cxx_std_17)
# This should be possible to set from the outside to be oss-fuzz compatible,
# fix later. For now, target libFuzzer only.
target_link_options(fuzzhelper PUBLIC "-fsanitize=fuzzer")
foreach(fuzzer TestSpecParser XmlWriter textflow)
add_executable(fuzz_${fuzzer} fuzz_${fuzzer}.cpp)
target_link_libraries(fuzz_${fuzzer} PRIVATE fuzzhelper)
endforeach()

View File

@@ -1,10 +0,0 @@
#include "NullOStream.h"
void NullOStream::avoidOutOfLineVirtualCompilerWarning()
{
}
int NullStreambuf::overflow(int c){
setp(dummyBuffer, dummyBuffer + sizeof(dummyBuffer));
return (c == traits_type::eof()) ? '\0' : c;
}

View File

@@ -1,20 +0,0 @@
#pragma once
#include <ostream>
#include <streambuf>
// from https://stackoverflow.com/a/8244052
class NullStreambuf : public std::streambuf {
char dummyBuffer[64];
protected:
virtual int overflow(int c) override final;
};
class NullOStream final : private NullStreambuf, public std::ostream {
public:
NullOStream() : std::ostream(this) {}
NullStreambuf *rdbuf() { return this; }
virtual void avoidOutOfLineVirtualCompilerWarning();
};

Some files were not shown because too many files have changed in this diff Show More