Compare commits

..

633 Commits

Author SHA1 Message Date
Martin Hořeňovský
f526ff0fc3 Bump version to v3.0.0 preview 5 2022-04-20 23:46:07 +02:00
Martin Hořeňovský
17a04f88d9 Always report rng seed from builtin reporters
Not all reporters use a format that supports this, so TeamCity
and Automake reporters still do not report it. The console
reporter now reports it even on successful runs, where before
it only reported the rng seed in the header, which was showed
either for failed run, or for run with `-s`.

CLoses #2065
2022-04-19 10:32:13 +02:00
Martin Hořeňovský
90e6905050 Remove no longer used CATCH_USE_VALGRIND cmake option 2022-04-18 12:14:21 +02:00
Martin Hořeňovský
6bdc7e1a65 Remove unused function in Clara 2022-04-17 23:30:40 +02:00
Martin Hořeňovský
7b93a2014c Rename catch_stream -> catch_reusable_string_stream
After everything else was split out, this name much reflects the
actual contents of the file(s).
2022-04-16 16:34:08 +02:00
Martin Hořeňovský
98bb638fb2 Split out IStream out of catch_stream.hpp 2022-04-16 16:33:57 +02:00
Martin Hořeňovský
05e85c5652 Split out Catch::cout/cerr/clog into their own file 2022-04-16 16:33:50 +02:00
Martin Hořeňovský
b520257676 Split REMOVE_PARENS from catch_preprocessor.hpp
This way, some headers can include just the new header, which
transitively decreases the total include size of test_macros.hpp.
2022-04-15 17:14:28 +02:00
Martin Hořeňovský
574d042821 Push catch_preprocessor include down from test_macros header 2022-04-15 16:40:06 +02:00
Martin Hořeňovský
c742ea9ad9 Some IWYU include cleanups 2022-04-14 17:16:01 +02:00
Martin Hořeňovský
a243cbae52 Introduce type to handle reporter specs with defaults filled-in 2022-04-13 15:02:25 +02:00
Martin Hořeňovský
79d1e82381 Store IStream instance owning ptrs in reporter instances
This ended up being a surprisingly large refactoring, motivated
by removing a `const_cast` from `Config`'s handling of reporter
streams, forced by previous commit.
2022-04-13 15:00:50 +02:00
Martin Hořeňovský
4f09f1120b Make IStream::stream non-const
This way it makes much more sense from logically-const point
of view, and also means that concrete implementations don't
have to always have a `mutable` keyword on the stream member.
2022-04-13 14:59:39 +02:00
Martin Hořeňovský
9934b7de13 Remove pointless mutable from ScopedElement's XmlWriter ptr 2022-04-12 22:31:51 +02:00
Martin Hořeňovský
7a89916198 Remove superfluous default argument from XMLWriter's constructor 2022-04-12 22:28:40 +02:00
Martin Hořeňovský
61f803126d Use parent constructor where applicable in reporters 2022-04-12 22:27:12 +02:00
Martin Hořeňovský
d698776ec5 Remove unused RDBufStream type 2022-04-12 22:22:04 +02:00
Martin Hořeňovský
f25236ff43 Small cleanup for pluralise 2022-04-11 20:37:54 +02:00
Martin Hořeňovský
8cdaebe964 struct -> class normalization for various interface types 2022-04-11 18:32:35 +02:00
Martin Hořeňovský
1a56ba851b struct -> class normalization for matchers 2022-04-11 13:06:54 +02:00
Martin Hořeňovský
9abe49ec53 struct -> class normalization for reporters 2022-04-11 12:58:58 +02:00
Martin Hořeňovský
195a6ac941 Use StringRefs for operation name in StringMatcherBase 2022-04-11 10:58:45 +02:00
Martin Hořeňovský
be948f1fd0 Add COST property to the CMakeConfig tests 2022-04-10 21:55:52 +02:00
Martin Hořeňovský
4e006a93ff Rename IStreamingReporterPtr => IEventListenerPtr
This reflects the renaming of the reporter/listener base class.
2022-04-10 21:50:51 +02:00
Martin Hořeňovský
73d8fb5bca Update release-notes.md 2022-04-10 21:43:28 +02:00
Martin Hořeňovský
0a33405983 Mention Bazel support in release notes 2022-04-09 20:55:13 +02:00
Lukasz Okraszewski
cb551b4f6d Add default reporter for Bazel integration (#2399)
When the added Bazel configuration flag is enabled,
a default JUnit reporter will be added if the XML
envrioment variable is defined.
Fix include paths for generated config header.
Enable Bazel config by default when building with
Bazel.


Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-04-08 18:01:59 +02:00
Martin Hořeňovský
4b78157981 Remove Config::openStream function
It has become completely vestigial, as it only ever passed-through
the argument down to a different function, and being private
member function, it didn't even introduce a useful compilation
firewall.
2022-04-07 12:50:20 +02:00
Martin Hořeňovský
46b3f7ee5f Document magic %-prefixed streams 2022-04-07 12:50:18 +02:00
Martin Hořeňovský
f9f4e58dfb Add missing version placeholder to --colour-mode cli option docs 2022-04-07 12:50:14 +02:00
Martin Hořeňovský
d5bfce4d84 Rename config::colourMode to config::defaultConfigMode
This describes the actual intention much better.
2022-04-07 12:50:13 +02:00
Martin Hořeňovský
c43947eb47 Fix CLI hints for colour and reporter options 2022-04-07 12:50:10 +02:00
Martin Hořeňovský
423e1d2ebb Integrate the new reporter specs into Catch2
This means that the CLI interface now uses the new key-value oriented
reporter spec, the common reporter base creates the colour implementation
based on the reporter-specific configuration, and it also stores the
custom configuration options for each reporter instance.

Closes #339 as it allows per-reporter forcing of ansi colour codes.
2022-04-07 12:50:08 +02:00
Martin Hořeňovský
3c06bcb374 Fix reporter parser spec to use the right key for colour-mode 2022-04-06 21:41:45 +02:00
Martin Hořeňovský
a51fd07bd0 Add helper for parsing the new reporter specs
The new reporter spec generalizes key-value options that can be
passed to the reporter, looking like this
`reporterName[::key=value]*`. A key can be either Catch2-recognized,
which currently means either `out` or `colour`, or reporter-specific
which is anything prefixed with `X`, e.g. `Xfoo`.
2022-04-05 15:45:54 +02:00
Martin Hořeňovský
8ac86495de Cleanup ColourMode parsing in CLI 2022-04-05 13:42:21 +02:00
Martin Hořeňovský
d750da13a8 Split out colour mode string parsing into its own function 2022-04-05 13:42:06 +02:00
Martin Hořeňovský
c045733d05 Add splitReporterSpec helper 2022-04-05 12:57:33 +02:00
Martin Hořeňovský
9fea3d253f Add missing include to catch_optional.hpp 2022-04-05 12:55:30 +02:00
Martin Hořeňovský
797c3e7318 Refactor tests 2022-03-31 22:30:38 +02:00
Martin Hořeňovský
6206db5a73 Fix missing licence headers 2022-03-31 20:04:08 +02:00
Daniel Feist
78e33ce51f Added TestCaseInfoHasher and tests. (#2394)
Test case hashing includes tags and class name

As the hasher involves more code now, it was split out into its own file
and it got its own set of tests.

Closes #2304
2022-03-31 16:46:41 +02:00
Martin Hořeňovský
1a8a793178 Refactor colour handling to prepare for per-reporter colour modes
This includes always compiling the ANSI and None colour
implementations, as they don't need to touch any platform
specific APIs, and removing their respective compile-time
configuration options.

Because the Win32 colour implementation requires Win32-specific
APIs, it is still hidden behind a compile-time toggle,
`CATCH_CONFIG_COLOUR_WIN32` (renamed from `..._COLOUR_WINDOWS`).

The commandline options for colours were also changed. The
option now uses different name, and allows to select between
different implementations, rather than changing whether
the compiled-in colour implementation is used through
"yes/no/default" options.
2022-03-28 13:10:13 +02:00
Martin Hořeňovský
a4e4e82474 Always indent test names at 2 spaces with default listing
This makes the output slightly more parsable in cases where
using the xml output from xml reporter is impractical, e.g. from
CMake scripts.
2022-03-24 16:03:46 +01:00
Martin Hořeňovský
6227ca317e Add tests for ColourGuard 2022-03-24 16:02:56 +01:00
Martin Hořeňovský
081a1e9aba ColourGuard is no longer constructed engaged
Forcing it to be engaged explicitly, either via `op<<`, or by
`ColourGuard::engage`, fixes an issue with multiple `ColourGuard`s
being constructed in a single expression. Because the construction
of the `ColourGuard` instances can happen in arbitrary order,
colours would be applied in arbitrary order too. However, a chain
of `op<<`s has strict call orders, fixing this issue.
2022-03-24 16:02:54 +01:00
Martin Hořeňovský
4d8acafecb Rename isStdout => isConsole and add doccomment 2022-03-24 16:02:51 +01:00
Kochetkov, Yuriy
cf6dd937ab Add test for FatalConditionHandler and stack unwinding
Signed-off-by: Kochetkov, Yuriy <yuriyx.kochetkov@intel.com>
2022-03-24 15:49:02 +01:00
Kochetkov, Yuriy
2ce64d1d8f Fix disengage failure logs for FatalConditionHandlerGuard
FatalConditionHandlerGuard is used within RunContext::invokeActiveTestCase().
The intent of this guard is to avoid binary crash without failed test being
reported.
Still in case FatalConditionHandlerGuard destructor being called during stack
unwinding AND finds unexpected top-level filter for SEH unhandled exception,
the binary may still crash. As result of such crash the original exception
details are being hidden.

As the Catch2 provides only `CATCH_CATCH_ANON` macro, with no access to
exception details by design, looks like the best way to handle issue is to:
 - state requirements explicitly by `noexcept` specifier
 - use `Catch::cerr()` to print out possible issue notification

Signed-off-by: Kochetkov, Yuriy <yuriyx.kochetkov@intel.com>
2022-03-24 15:49:02 +01:00
Kochetkov, Yuriy
7882f7359e Remove unnecessary casts
May break the logic in case of WinAPI changes with no warnings

Signed-off-by: Kochetkov, Yuriy <yuriyx.kochetkov@intel.com>
2022-03-24 15:49:02 +01:00
Martin Hořeňovský
0e176c318b Refactored colour handling
* POSIX colour impl is now compiled for all platforms.
* Deciding whether a colour impl should be picked is now stream
  dependent, and thus incompatible implementations can be removed
  immediately, rather than checking when the colour is being used.
2022-03-18 14:04:07 +01:00
Martin Hořeňovský
c1c72c7e05 Better doccomment for IEventListener 2022-03-18 10:30:48 +01:00
Martin Hořeňovský
06092f727d Remove special handling of default stream for reporters
This fixes an issue where reporter with default-output to stdout
would think that it was given a stream _not_ backed by console,
thus not using colour.
2022-03-18 00:41:47 +01:00
Martin Hořeňovský
4acc520f76 Event listeners no longer take reporter config in constructor
This also required splitting out Listener factory from
the reporter factory hierarchy. In return, the listener
factories only need to take in `IConfig const*`, which
opens up further refactorings down the road in the colour
selection and implementation.
2022-03-18 00:36:18 +01:00
Alan Jowett
18c58667d7 Generate Visual Studio project files with constant project guids
By default, CMake derives a Visual Studio project GUID from the
file path but the GUID can be overridden via a property
(see https://gitlab.kitware.com/cmake/cmake/-/commit/c85367f4).

Using a non-constant GUID can cause problems if other projects/repos
want to reference the catch2 vcxproj files, so we force a constant GUID here.

Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Resolves: #2388
2022-03-18 00:21:12 +01:00
Vertexwahn
634cdb4efe Fix borken Bazel build 2022-03-16 14:01:18 +01:00
wqking
38879296a7 Add eventpp and accessorpp to Catch2 users 2022-03-13 16:56:07 +01:00
schallerr
81f612c96c Refactor MatchersRanges test utils (#2386)
- do not hardcode content of containers
- prefix members with m_
- add const and non-const iterators to `with_mocked_iterator_access`
- remove `m_touched` as it wasn't filled properly and isn't used anyway
2022-03-09 23:13:37 +01:00
Martin Hořeňovský
913f79a661 Each reporter keeps its own colour implementation
This opens path to per-reporter colour output customization,
and fixes multiple issues with the old colour implementation.

Under the old implementation, using Win32-backed colouring
would always change the colour used by the console, even if the
actual output was written elsewhere, such as a file passed by
the `--out` flag. This will no longer happen, as the reporter's
colour impl will check that the reporter's stream is pointed
to console before trying to change the colours.

POSIX/ANSI colour implementation suffered a similar-ish issue,
in that it only wrote the colour escape codes into the default
output stream, even if the reporter asking for colouring was
actually writing to a completely different output stream.
2022-03-08 12:51:13 +01:00
Martin Hořeňovský
06f74a0f8e Add %stderr and %stdout as recognized magic stream names 2022-03-07 10:59:27 +01:00
Martin Hořeňovský
61d0f7a9af Pass the whole IStream wrapper into reporter
This will become useful when reworking colour support, because
Win32 colour support requires checking whether the output is
stdout, which is done through the `IStream` wrapper.
2022-03-06 16:34:13 +01:00
Martin Hořeňovský
9b01c404f5 Fix cli help tooltip for --use-colour 2022-03-06 16:28:30 +01:00
Martin Hořeňovský
f206162b2d Fix some messy formatting in catch_stats 2022-03-06 16:28:28 +01:00
Martin Hořeňovský
05d4ec62c8 Add common ReporterBase as parent of the helper bases
This is useful as a centralized place for handling common reporter
problems like handling output streams, and soon also colour impl
handling.
2022-03-06 16:28:22 +01:00
Martin Hořeňovský
4dd5e2eece Rename base reporter iface IStreamingReporter -> IEventListener 2022-03-06 16:28:21 +01:00
John Beard
f9facc1881 Add variadic CAPTURE tests
This makes sure that CAPTURE works when called with variadic arguments,
and also works that way when disabled.

The underlying fix to #2316 is not applicable (CAPTURE is already
variadic when disabled).

This is a port of 5e94498ed0 to the
devel branch.
2022-03-05 16:06:54 +01:00
Joel Uckelman
2ebc041903 Fix name of with-main library. 2022-03-04 00:40:55 +01:00
Martin Hořeňovský
529eec97bb Mention CMake configure options in release notes 2022-02-23 14:16:36 +01:00
Martin Hořeňovský
ff5b311898 Remove superfluous warning suppression in tests 2022-02-22 20:13:17 +01:00
Martin Hořeňovský
4a2eb90302 Add basic CMake presets 2022-02-22 15:47:11 +01:00
Martin Hořeňovský
715cd25081 Add IStream::isStdOut 2022-02-22 15:45:51 +01:00
GravisZro
1d4b42ad7b Minor whitespace fix 2022-02-21 20:04:43 +01:00
Martin Hořeňovský
72f0372664 Add tests for CMake configure toggles passing to Catch2 build 2022-02-20 19:32:30 +01:00
Martin Hořeňovský
4396a9119f Add some missing configuration includes 2022-02-20 19:32:25 +01:00
Martin Hořeňovský
bda4b7df84 Fix fallback stringifier config 2022-02-20 19:32:22 +01:00
John Beard
0c722564c3 Examples: remove references to catch_default_main
catch_default_main.hpp was removed in db32550898

The example 000-CatchMain.cpp is no longer compiled, but is still
present in the examples and is still references by other example
files. Remove the file and references to it, as they are confusing.
2022-02-20 13:10:42 +01:00
Martin Hořeňovský
33ffc3b6fc Allow passing compile-time configuration options through CMake 2022-02-18 11:15:08 +01:00
Martin Hořeňovský
fc5552d27b Push down handling of default reporter to Config's constructor
This simplifies the handling of default reporter in console
parsing, at the cost of making `Config`'s constructor responsible
for more things.
2022-02-17 23:02:30 +01:00
Martin Hořeňovský
7cf2f88e50 Add small missing things to release notes 2022-02-14 18:22:11 +01:00
Martin Hořeňovský
a9ed2c235d Rename MiscFunctions.cmake -> CatchMiscFunctions.cmake 2022-02-07 00:05:11 +01:00
Martin Hořeňovský
a1e5934aa9 Add licence header to MiscFunctions.cmake 2022-02-06 23:59:15 +01:00
Guillaume Egles
190f71792a proper support for lib suffix in conan 2022-02-05 20:53:43 +01:00
Martin Hořeňovský
c912f62fc4 Remove Windows.h related defines 2022-02-05 16:21:06 +01:00
Martin Hořeňovský
aa3c7be434 Fix ToC in CLI documentation 2022-02-05 16:14:47 +01:00
Martin Hořeňovský
b0279e0c14 Remove mention of CATCH_CONFIG_MAIN from docs/configuration.md 2022-02-05 14:55:01 +01:00
Certseeds
9afb6ce138 feature: double -> const auto when static_cast in right
</subject>

Branch: devel

<type>:
- [ ] Bug fix
- [ ] Bug fix (Test)
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] This change requires a documentation update

<body>

<footer>

Signed-off-by: Certseeds <51754303+Certseeds@users.noreply.github.com>
2022-02-04 00:09:01 +01:00
Martin Hořeňovský
efb54926ee Don't cache stdout handle in Windows console colour impl
The cached handle would become invalid if some other code, say
a user-provided test code, redirects stdout through `freopen`
or `_dup2`, which would then cause AppVerifier to complain.

Fixes #2345
2022-01-29 23:12:55 +01:00
Martin Hořeňovský
7a2f9f4633 Extend checkLicense.py to also check tests 2022-01-29 13:10:54 +01:00
Martin Hořeňovský
79e4cd1af4 Cleanup summary comments in ExtraTests 2022-01-29 00:40:03 +01:00
Martin Hořeňovský
635db2785f Use the new licence header in ExtraTests's cpp files 2022-01-29 00:07:44 +01:00
Martin Hořeňovský
51888d360a Use the new licence header in TestScripts 2022-01-29 00:04:49 +01:00
Martin Hořeňovský
f83332d89b Use the new licence header in SelfTest's cpp files 2022-01-29 00:03:43 +01:00
Martin Hořeňovský
b5dbdc858d Link to the v2-v3 migration docs from release notes 2022-01-28 23:55:27 +01:00
Martin Hořeňovský
e53a75b425 Add integration test for listener-reporter event ordering 2022-01-28 00:00:52 +01:00
Martin Hořeňovský
4ff8b27bb6 Cleanups output normalization in ApprovalTests
Removed:
 * NaN normalization
 * INFINITY normalization
 * errno normalization
 * Completely unused duration regex

Tests using these macros should be tagged `[approvals]`
so they are not run as part of approval tests.

Also simplified regex for the test's executable filename,
and hidden some tests relying on nullptr normalization.
2022-01-27 21:11:51 +01:00
Martin Hořeňovský
d861e73f86 Handle full unique test identifiers in testRandomOrder.py 2022-01-27 21:11:37 +01:00
Martin Hořeňovský
dc86d51af2 Update version string to mention Catch2 2022-01-27 20:15:05 +01:00
Simon Weis
5121660e7f Add OpenALpp as an open source library that uses Catch2 2022-01-22 21:33:19 +01:00
Rodrigo Severo
ce556fd646 Ignoring VSCode settings dir 2022-01-15 15:45:56 +01:00
Martin Hořeňovský
b6ff2c3dda Error out reporter registration for reporter names with '::' 2022-01-09 19:01:14 +01:00
Martin Hořeňovský
875299cff0 Remove unused member from Totals
At one point it was inserted there as the simplest way to smuggle
around an extra return value for specific errors in executing
tests. Since then, the error has been changed to be handled
differently, and the member became unused.
2022-01-07 10:34:25 +01:00
Martin Hořeňovský
39d3de17f3 Fix mentions of CATCH_INSTALL_HELPERS
Fixes #2347
2022-01-06 23:37:44 +01:00
Martin Hořeňovský
fff494c10a Tear out trivial_abi on unique_ptr
Turns out people are bad at not combining code compiled with GCC
and Clang, and the improvement from `trivial_abi` on `unique_ptr`
is not worth the maintenance cost of having this be an opt-in
change.

Closes #2344
2022-01-05 00:04:25 +01:00
Martin Hořeňovský
103cb16696 Rename listening reporter to multi reporter 2022-01-05 00:00:28 +01:00
Martin Hořeňovský
244680d512 Update release-process.md for v3 2022-01-03 23:30:39 +01:00
Martin Hořeňovský
f4af9f6926 Bump version to v3.0.0 preview 4 2022-01-03 23:16:39 +01:00
Martin Hořeňovský
57c9c935ee Update release notes 2022-01-03 23:11:55 +01:00
Alan Jowett
98a6c69e1e Switch from AddVectoredExceptionHandler to SetUnhandledExceptionFilter
This avoids issues with Catch2's handler firing too early, on
structured exceptions that would be handled later. This issue
meant that the old attempts at structured exception handling
were incompatible with Windows's ASan, because it throws
continuable `C0000005` exception, which it then handles.

With the new handling, Catch2 is only notified if nothing else,
including the debugger, has handled the exception.

Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Closes #2332
Closes #2286
Closes #898
2022-01-03 22:50:32 +01:00
Martin Hořeňovský
d3199c42c2 Redo reporter CLI and reporter-specific docs
Closes #2259
2022-01-03 20:38:13 +01:00
Martin Hořeňovský
eeee4a49af Remove short flags from --list-tests and --list-tags 2022-01-02 21:23:29 +01:00
Martin Hořeňovský
0d1bdea69f Remove all mentions of the removed --list-test-names-only flag 2022-01-02 21:23:26 +01:00
Martin Hořeňovský
3ab981fa21 Add documentation for the --verbosity option 2022-01-02 21:02:43 +01:00
Martin Hořeňovský
54e89e8364 Update release notes to account for recent changes 2022-01-01 23:50:12 +01:00
Martin Hořeňovský
88b28ab592 Extra compilation checks for compiling without exceptions 2022-01-01 23:21:37 +01:00
Martin Hořeňovský
ef3374ed81 Add more macros to the DisabledMacros test 2022-01-01 16:45:28 +01:00
Martin Hořeňovský
f2f585b478 Merge pull request #2183 from mjerabek/dev-multiple-reporters
Add support for multiple reporters
2022-01-01 16:19:31 +01:00
Martin Hořeňovský
b5547f2ef6 Passthrough stdout/err in multireporter's partial test end
This should provide the same overall stdout/err, but the new
output should feel "faster" for test cases that are entered
and exited multiple times (e.g. due to generators).
2022-01-01 14:20:23 +01:00
Martin Hořeňovský
93882f7fab Support '-' as output path for stdout 2022-01-01 14:16:12 +01:00
Martin Hořeňovský
4752545a69 Add version placeholders to the reporter docs 2022-01-01 14:16:12 +01:00
Martin Hořeňovský
fae0be25b3 Improve error handling in parsing reporter specs 2022-01-01 14:16:11 +01:00
Martin Hořeňovský
899554bff2 Add test for handling of reporter's assertion preferences 2022-01-01 14:15:44 +01:00
Martin Hořeňovský
b4efa4751a Add test for handling of Listeners requesting stdout capture 2022-01-01 14:13:48 +01:00
Martin Hořeňovský
22547a3c5f Add test for stdout pass-through behaviour in MultiReporter 2022-01-01 14:11:49 +01:00
Martin Hořeňovský
8baf9c05a3 Add more unit tests
* Call order of listeners/reporters in multireporter
* Adding listeners/reporters properly updates reporter preferences
2022-01-01 14:10:52 +01:00
Martin Jeřábek
ccd67b293d Add support for multiple parallel reporters
This requires a bunch of different changes across the reporter
subsystem.

* We need to handle multiple reporters and their differing
  preferences in `ListeningReporter`, e.g. what to do when
  we mix reporters that capture and don't capture stdout.
* We need to change how the reporter is given output and
  how we parse reporter's output destination from CLI.
* Approval tests need to handle multireporter option
2022-01-01 14:02:23 +01:00
Martin Hořeňovský
6b55f5d780 Don't send full stdout so far to testCasePartialEnded event 2022-01-01 12:21:41 +01:00
Martin Hořeňovský
c9c3b74805 Remove some obsolete tests 2021-12-31 15:12:53 +01:00
Martin Jeřábek
8711b63a0a Add tests for StringRef's copy operations being shallow 2021-12-29 20:20:29 +01:00
Martin Hořeňovský
72a09de236 Add tests for Optional's op== and != 2021-12-29 16:58:22 +01:00
Martin Jeřábek
f0a89b7345 Add ops == and != for Optional 2021-12-29 16:58:16 +01:00
Martin Hořeňovský
f00b6e2019 Add more tests for various less common tag-related cases 2021-12-28 00:22:53 +01:00
Martin Hořeňovský
45577a1f4c Refactor implementation of case-insensitivity in tags
By not materializing the lower cased tags ahead of time, we
save allocations at the cost of worsened performance when comparing
two tags.

Since there are rarely many tags, and commonly they are not
compared even if present, this is almost always a win. The new
implementation also improves the robustness of the code
responsible for handling tags in a case-insensitive manner.
2021-12-28 00:20:55 +01:00
Martin Hořeňovský
cbb6764fb1 Add CaseInsensitiveEqualTo comparison type 2021-12-26 18:54:47 +01:00
Martin Hořeňovský
156e6fdfa9 CaseInsensitiveLess takes args as StringRefs 2021-12-26 00:52:52 +01:00
Martin Hořeňovský
187bf6db2b Split out CaseInsensitiveCmp into its own file 2021-12-26 00:35:46 +01:00
Alan Jowett
cde26de803 Add buildAndTest.cmd for Windows users (#2338)
* Add buildAndTest.cmd for Windows users

Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-12-18 23:52:35 +01:00
Martin Hořeňovský
3cc0c033e4 Allow enabling multiple warnings in one invocation 2021-12-18 20:50:05 +01:00
Martin Hořeňovský
840acedf62 Implement warning for unmatched test specs 2021-12-18 20:50:03 +01:00
Martin Hořeňovský
9f2dca5384 Exit with non-0 return code if no tests were run
A new flag, `--allow-running-no-tests` was added to override this
behaviour if exit code of 0 was desired.

This change also made `-w NoTests` obsolete, and so it has been
removed.
2021-12-18 20:50:00 +01:00
Martin Hořeňovský
602e484f02 Rename reportInvalidArguments -> reportInvalidTestSpec
This is has been what it actually does for a long time, but it
has not been renamed earlier due to API compatibility.
2021-12-18 20:49:50 +01:00
Martin Hořeňovský
08939cc8bb Error out early if invalid test specs are provided 2021-12-18 20:49:47 +01:00
Martin Hořeňovský
3bfe900bbc Move shard logic check after help flag check 2021-12-18 20:49:44 +01:00
Alan Jowett
d30d0c01a7 Disable JIT debug for ctest.ext
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
2021-12-16 12:17:52 +01:00
Martin Hořeňovský
dcf9479c85 Counts internally use uint64_t instead of size_t
This ensures that even for 32 bit platforms, the assertion count
should not plausibly overflow.
2021-12-06 20:42:51 +01:00
Martin Hořeňovský
c49faa62dd Let users of amalgamated distribution override the default main
Closes #2324
2021-12-02 13:49:16 +01:00
Martin Hořeňovský
c097609115 Cleanup release scripts a bit 2021-12-02 11:48:50 +01:00
Martin Hořeňovský
9d6fffb922 Remove no longer used Python scripts 2021-12-02 11:48:28 +01:00
Martin Hořeňovský
153965a655 Delete useless MatcherMethod base class
All of its functionality can be moved into the `MatcherBase` class,
simplifying the code a bit and removing a warning about class with
virtual member functions but no virtual destructor.

Closes #2182 as it is no longer relevant.
2021-11-26 00:38:13 +01:00
Martin Hořeňovský
0ac9f44985 Add SECURITY.md 2021-11-26 00:10:01 +01:00
Martin Hořeňovský
b9baae6d93 Cleanup StringRef impl a bit 2021-11-23 23:16:46 +01:00
Martin Hořeňovský
c95072408f Add catch_sharding.hpp to CML 2021-11-23 23:16:17 +01:00
Martin Hořeňovský
8cb8f0b08b Sweep out some Wsign-conversion warnings 2021-11-23 23:16:08 +01:00
Tristan Stenner
9952f29f01 Fix link in event-listeners.md 2021-11-22 11:44:36 +01:00
SketchyLizard
2db1cf3404 Fix template args to avoid conversion warnings
Fix avoids a warning about sign conversion when included from a file compiled with -Werror=sign-conversion.
2021-11-20 21:38:01 +01:00
Martin Hořeňovský
fabe614ba8 Rewrite own-main.md for v3 2021-11-19 15:52:32 +01:00
Martin Hořeňovský
acdb85c398 Add FAQ page to the documentation 2021-11-19 15:23:50 +01:00
Martin Hořeňovský
726fdd7f8e Update known-limitations.md
Removed 3rd party bugs for no-longer supported compiler versions,
reworded section on parallel test execution to take into account
sharding.
2021-11-19 00:32:59 +01:00
Martin Hořeňovský
0ccb1c30c6 Rewrite event listener docs and add event documentation 2021-11-18 23:05:45 +01:00
Martin Hořeňovský
dd12ce8141 Rewrite tutorial to shorten BDD section and mention generators 2021-11-16 23:46:23 +01:00
Martin Hořeňovský
d32e89eb84 Reorganize the main readme 2021-11-16 23:46:22 +01:00
Martin Hořeňovský
ce6aca81ad Remove Google Groups link from README.md 2021-11-16 23:46:21 +01:00
Martin Hořeňovský
61489e863e Fixup ToC link to 3.0.1 release notes 2021-11-16 23:46:19 +01:00
Martin Hořeňovský
2287d225e5 Mention that RangeGenerator works on half open ranges
Closes #2145
2021-11-15 23:52:26 +01:00
Martin Hořeňovský
4eb00afe69 Add allowing tests with same name to release notes 2021-11-15 23:18:55 +01:00
Martin Hořeňovský
e86f84b8ef Mention that run == iteration in benchmarking docs 2021-11-15 23:09:18 +01:00
Martin Hořeňovský
d012735c6e Add test for runtime behaviour of STATIC_CHECK 2021-11-15 20:06:29 +01:00
Martin Hořeňovský
67caef6f45 Mention STATIC_CHECK in release notes 2021-11-15 13:04:08 +01:00
Morwenn
f41d761674 Add STATIC_CHECK and STATIC_CHECK_FALSE (#2318) 2021-11-15 00:28:27 +01:00
Martin Hořeňovský
edc2f6e8a3 Further refactoring of StreamingReporterBase 2021-11-14 11:41:29 +01:00
Martin Hořeňovský
b2ac27423a Cleanup visibility in StreamingReporterBase 2021-11-14 11:41:27 +01:00
Martin Hořeňovský
a754cb9062 Make TestRunInfo constexpr 2021-11-14 11:41:26 +01:00
Martin Hořeňovský
5f38cc39fa JUnit/SonarQube reporters use base's stream member 2021-11-14 11:41:25 +01:00
Martin Hořeňovský
b892ab133c Cleanup visibility in CumulativeReporterBase 2021-11-14 11:41:24 +01:00
Martin Hořeňovský
0c9fe16537 Opt out JUnit/SonarQube reporter from expanding passing assertions
Closes #1966
2021-11-14 11:27:45 +01:00
Martin Hořeňovský
d02ea5adee Cumulative reporter base can be customized to not expand assertions 2021-11-14 11:27:44 +01:00
Martin Hořeňovský
9b4e69333f Small cleanup in tests 2021-11-14 11:27:43 +01:00
Martin Hořeňovský
4d9bfb2951 Avoid static analysis warning in Approx 2021-11-14 11:27:42 +01:00
Martin Hořeňovský
c4df47c246 Approvals normalize paths with both backward and forward slash 2021-11-13 22:10:25 +01:00
Martin Hořeňovský
9200b4078b Move reporter_registrars.hpp to reporters/ 2021-11-10 23:52:59 +01:00
Martin Hořeňovský
6603f1d972 Use case in names of default reporters 2021-11-10 23:32:01 +01:00
Martin Hořeňovský
62d8913d67 Cumulative reporter base records benchmark results 2021-11-09 11:52:50 +01:00
Martin Hořeňovský
8780425385 Make reporter lookup case insensitive, registration case preserving
Previously registration was case preserving, but lookup used
lowercased reporter name, so a reporter whose name contained
upper case character could not be requested by the user.
2021-11-09 11:50:03 +01:00
Martin Hořeňovský
7800fe9708 Lift toLower(char) to header 2021-11-09 11:44:54 +01:00
Martin Hořeňovský
141e384c60 Fix missing include in reporter_registrars.hpp 2021-11-08 11:32:27 +01:00
Martin Hořeňovský
f1239b2045 Add doxygen doccomments to IStreamingReporter 2021-11-05 14:24:22 +01:00
Martin Hořeňovský
912df7df35 Fix quadratic runtime when linebreaking strings without newlines
The problem was that every line would iterate from current line
start position to the end of the string, looking for a newline
to break on, leading to accidentally quadratic runtime. With this
change, the code only ever searches up to the current line's
length and not more.

Credit to @jorgenpt for the fix suggestion.

Closes #2315
2021-11-04 00:23:56 +01:00
Martin Hořeňovský
931f41b4d6 Add some tests for TextFlow::Column 2021-11-01 22:51:17 +01:00
Martin Hořeňovský
70c4ec78fb Improve comments and names in TextFlow::Column 2021-11-01 19:14:37 +01:00
Martin Hořeňovský
455ae0c561 Typedef Column::iterator as Column::const_iterator not vice versa 2021-10-31 13:01:41 +01:00
Martin Hořeňovský
2520ad4b6e Return const_iterator from Column::begin/end const
This is what should normally happen, even if it does not change
anything given that `Column::const_iterator` is currently a typedef
for `Column::iterator`.
2021-10-31 12:59:00 +01:00
Martin Hořeňovský
e539e1cb52 Move strings in Clara's result type 2021-10-29 23:04:24 +02:00
Martin Hořeňovský
3c5c86a4e4 Add test for filtering out multiple initial values in filter gen 2021-10-28 11:26:53 +02:00
Martin Hořeňovský
514206df36 Add accept-many Clara lambdas to release notes 2021-10-27 20:54:49 +02:00
Martin Hořeňovský
becab0cf74 Add test sharding to the release notes 2021-10-27 20:49:29 +02:00
Martin Hořeňovský
12d14a3c63 Add support for multiply calling lambda parsers in Clara
Previously a lambda parser in Clara could only be invoked once,
even if it internally was ok with being invoked multiple times.

With this change, a lambda parser can mark itself as `accept_many`,
in which case it will be invoked multiple times if the appropriate
flag was supplied multiple times by the user.
2021-10-27 20:15:28 +02:00
Martin Hořeňovský
f17725a186 Split void_type into its own header and rename it to void_t 2021-10-27 20:01:13 +02:00
Martin Hořeňovský
ec2d5013fb Make testSharding.py test script executable 2021-10-27 17:24:30 +02:00
Martin Hořeňovský
342ef5ca7e Cleanup the shard integration test script 2021-10-27 17:24:30 +02:00
Martin Hořeňovský
5ac1ffe9ee Improve shardIndex/Count cli argument parsing 2021-10-27 17:24:30 +02:00
Ben Dunkin
3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský
6456ee8b01 Return Clara parsing error message by const-ref 2021-10-27 15:10:52 +02:00
Martin Hořeňovský
905bf438ae Fix bad indentation calculation in the console reporter
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".

The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).

We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
           a
           n
           c
           y
           .
           .
           .
```

Fixes #2309
2021-10-25 15:21:28 +02:00
Martin Hořeňovský
0fdee1c273 Stop declaring compiled Catch2 artifacts as arch independent 2021-10-25 14:46:54 +02:00
Anders Schau Knatten
22750cde0e Disable false positive from clang-tidy
Clang-tidy is smart enough to understand that the conditional is never
updated in the loop body. It will let you get away with it if it can
prove that the conditional is always false, but that is not always
possible.

Here is an example where it's not able to prove it, and thus gives a
false positive. This is a minimal reproduction of an actual case I hit
in production, where `function` is picking the function based on some
`constexpr` logic related to which type argument is currently being
tested.

```
int f();

TEMPLATE_TEST_CASE("reproduction", "", int) {
    const auto function = []() {
        return f;
    }();
    const int error = function();
    REQUIRE(error == 0); // clang-tidy complains: bugprone-infinite-loop
}
```

I did not choose to add this test to the test suite, since we're not
running `clang-tidy` in CI afaik. To run it manually, simply add the
snippet above somewhere and run clang-tidy with
`--checks=bugprone-infinite-loop`. Or see an example at
https://godbolt.org/z/4v8b8WexP.

The reason we get the infinite loop warning in the first place is the
conditional at the end of this `do`-loop. Ideally, this conditional
would just be `while(false)`, but the actual content of the
`REQUIRE`-statement has been included here too in order to not loose
warnings from signed/unsigned comparisons. In short, if you do
`REQUIRE(i < j)`, where `i` is a negative signed integer and `j` is an
unsigned integer, you're supposed to get a warning from
`-Wsign-compare`. Due to the decomposition in Catch2, you lose this
warning, which is why the content of the `REQUIRE` statement has been
added to the conditional to force the compiler to evaluate the actual
comparison as well.

This was discussed on Discord today, and an alternative approach (which
I don't have time to implement) would be to in the decomposition replace
the comparison operators with `cmp_less` and friends. These are C++20
though, and would have to be implemented manually. I am also not sure
it's a good idea to "magically" change the semantics of `<` when it's
used inside a `REQUIRE` macro.

Another alternative approach would be to trigger this warning in a
different way, by including the content of the `REQUIRE` macro in a
different way which doesn't affect the for loop. But I don't have enough
of an overview here to know where would be a good place and how to test
that I didn't break anything.
2021-10-21 22:45:19 +02:00
Martin Hořeňovský
bf5c58adf6 The limit on TEMPLATE* test cases is actually reachable 2021-10-21 15:52:07 +02:00
Alecto Irene Perez
06cf2a4724 Apply PR #2297 to devel branch (#2300)
* Apply PR #2297 to devel branch

It turns out that Issue #2272 partially affected the devel branch. When
building tests with C++20, the compiler emits a warning that top-level
comma expressions in array subscripts are depricated. Warnings are
treated as errors, so this caused the build to fail.

This commit adds localized warning suppression
in accordance with this recommendation here:
https://github.com/catchorg/Catch2/pull/2297#discussion_r720848392

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>

* Fixed unknown pragma warning on old versions of gcc & clang

This commit fixes an unkwown pragma warning on older versions of GCC
and Clang. These older versions don't have a warning for depricated use
of the comma subscript. Because warning suppression is localized, and
only refers to the comma subscript warning, it doesn't affect compiler
warnings in other parts of the code.

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>

* More #warning backwards compatibility fixes

Signed-off-by: Alecto Irene Perez <perez.cs@pm.me>
2021-10-21 15:47:21 +02:00
Martin Hořeňovský
4436a60456 Add myself to FUNDING 2021-10-11 10:44:29 +02:00
Martin Hořeňovský
36b4a71ff0 Pick release notes for 2.13.7 2021-10-10 22:23:14 +02:00
Martin Hořeňovský
b406ad52a7 Mark !mayfail tests as skipped in the JUnit reporter
Should fix #2116
2021-10-10 22:21:39 +02:00
Martin Hořeňovský
de67278e14 JUnit reporter uses only 3 decimal places when reporting durations
We used to use whatever precision we ended up having from C++'s
stdlib. However, some relatively popular tools, like Jenkins,
use Maven SureFire XML schema to validate JUnit test reports, and
Maven SureFire schema requires the duration to have at most 3
decimal places.

For compatibility, the JUnit reporter will now respect this
limitation.

Closes #2221
2021-10-10 22:10:48 +02:00
Martin Hořeňovský
1d9696d22d Cleanup reporting of rng seed in existing reporters 2021-10-09 00:02:30 +02:00
Martin Hořeňovský
ed1f343a41 Provide random-device option for --rng-seed and make it default 2021-10-08 21:35:45 +02:00
Martin Hořeňovský
200a487cf2 Add generateRandomSeed utility to generate randomness seed 2021-10-08 21:35:41 +02:00
Martin Hořeňovský
fce42b62ad Generate random rng seed if user did not specify one
Closes #2161
2021-10-08 21:35:29 +02:00
Martin Hořeňovský
4e6d306742 Rename Catch::Option to Optional 2021-10-05 20:39:28 +02:00
Martin Hořeňovský
c6c46a168f Add non-empty assertion to Option's deref op 2021-10-05 19:22:22 +02:00
Martin Hořeňovský
48a889859b Simplify how ratio symbol is returned when stringifying ratios 2021-10-03 20:23:10 +02:00
Martin Hořeňovský
d65ee04b74 Remove double serialization when stringifying std::optional 2021-10-03 20:19:36 +02:00
Martin Hořeňovský
928e198ef2 Demote some single character strings into plain chars 2021-10-03 10:37:30 +02:00
Martin Hořeňovský
3e9c6fec22 Remove XmlWriter::writeBlankLine
There is no good reason to provide a "add empty line" primitive
for writing XML documents, and the fact that it remains unused
after all the time it was provided only confirms this further.
2021-10-02 14:39:54 +02:00
Martin Hořeňovský
13670f535f Add more tests for XmlWrite::write* members 2021-10-02 14:39:52 +02:00
Martin Hořeňovský
c6640e4f47 Improve documentation of CATCH_MOVE and CATCH_FORWARD 2021-09-30 20:40:24 +02:00
Martin Hořeňovský
23f0d94b4f Use CATCH_FORWARD in Detail::make_unique 2021-09-30 20:27:38 +02:00
Martin Hořeňovský
77c7e9803e Improve documentation on Detail::unique_ptr 2021-09-30 20:17:59 +02:00
Martin Hořeňovský
1d79683ea8 Add [[trivial_abi]] to Detail::unique_ptr when compiled with Clang
This decreases code size and improves performance of passing around
`unique_ptr` instances by value somewhat. It virtually guarantees
problems when combining code compiled with Clang and GCC, but that
was never supported anyway.
2021-09-30 20:11:22 +02:00
Anders Schau Knatten
eb452e9b35 Bring back useful comment
In b7b346c3e5 this conditional was simplified to just
`while( false)` rather than the current one. Then in 3a33315ff8, that
change was reverted. I found it hard to understand this
complicated conditional, but after some digging in history I found this
comment which used to be here. It was removed in b7b346c3e5, but not
restored together with the revert in 3a33315ff8. Let's revive it.
2021-09-29 18:19:31 +02:00
Martin Hořeňovský
2deafc33e9 Demote getCurrentNanosecondsSinceEpoch to internal linkage 2021-09-29 11:26:38 +02:00
Martin Hořeňovský
5250cf6d58 Remove getEstimatedClockResolution from Timer
It used to be part of the experimental benchmarking support, but
since that was replaced with proper benchmarking support with its
own timer facilities, it is now a dead code and useless.
2021-09-29 11:24:18 +02:00
Martin Hořeňovský
426954032f Rename Contains string matcher builder to ContainsSubstring
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.

Closes #2131
2021-09-27 21:28:33 +02:00
Martin Hořeňovský
f02c2678a1 Use StringRef for className in TestCaseInfo 2021-09-27 19:07:45 +02:00
Martin Hořeňovský
21b99d6f58 Add StringRef::compare for three way comparison 2021-09-27 18:41:17 +02:00
Martin Hořeňovský
d42e7a23a0 Change startsWith(char) to take StringRef as argument 2021-09-27 14:55:28 +02:00
Martin Hořeňovský
7bb00a42be Remove stub of obsolete test 2021-09-27 14:45:39 +02:00
Martin Hořeňovský
fb4153e05e Multiple tests can have same name as long as their tags differ
This change also changes it so that test case macros using a
class name can have same name **and** tags as long as the
used class name differs.

Closes #1915
Closes #1999
2021-09-25 21:37:03 +02:00
Martin Hořeňovský
e8e28ba401 Remove obsolete documentation page 2021-09-23 23:42:16 +02:00
Bernhard Manfred Gruber
ee1435793e add alpaka and LLAMA to Catch2 users 2021-09-23 12:39:56 +02:00
Martin Hořeňovský
3f8cae8025 Add consistency-checking event listener to SelfTest
This event listener performs basic consistency checks (akin to
matching braces) on events that are passed to the listeners
when the `SelfTest` test binary is run.

The current checks are about nesting events (e.g. `testCaseStarting`
cannot be received before `testRunStarting`, `sectionStarting`
can only be received when a test case is active, etc), and matching
up counts of starting/ended events.

The simplicity means that it could be confused by starting/ended
events matching up but being out of order, e.g.
```
* test case A starting
* test case B ended
* test case B starting
* test case A ended
```
would be accepted, even though it is wrong. However, doing full
order checking would be much more implementation work, for relatively
little benefit, so it is left out for now.
2021-09-15 23:38:43 +02:00
Martin Hořeňovský
2c82f82ee2 Mark first party reporters final
They are not intended to be derived from, so forbidding it
explicitly improves our ability to refactor them later.
2021-09-15 21:29:18 +02:00
Martin Hořeňovský
785436cd74 Make benchmark* and fatalError reporter events pure virtual 2021-09-15 21:17:39 +02:00
Martin Hořeňovský
f314fa1f8c assertionEnded event in reporters no longer returns bool 2021-09-15 18:57:56 +02:00
Martin Hořeňovský
013edc208b Refactor noMatchingTestCases and skipTests reporter methods
They now take `StringRef` as the argument, and are virtual only
in the basic interface.

Also cleaned out the various reporters and their overrides
of these members which were often empty or delegating up.
2021-09-15 18:08:31 +02:00
Martin Hořeňovský
10fb93cce8 Add new reporter event called for each test case enter/exit
This means that e.g. for `TEST_CASE` with two sibling `SECTION`s
the event will fire twice, because the `TEST_CASE` will be entered
twice.

Closes #2107 (the event mentioned there already exists, but this
is its counterpart that we also want to provide to users)
2021-09-14 13:19:45 +02:00
Martin Hořeňovský
4dcf8382c7 Default-implement assertion* events in StreamingReporterBase
This means that a reporter deriving from `StreamingReporterBase`
does not have to override them if it isn't interested in the individual
assertions.
2021-09-14 12:41:24 +02:00
Biswapriyo Nath
efd8cc8777 pkgconfig: Add missing entries
This adds prefix and exec_prefix fields
2021-09-11 19:02:21 +02:00
Martin Hořeňovský
12bca890b7 Use StringRefs in SectionTracker's filter bookkeeping 2021-09-09 11:04:34 +02:00
Martin Hořeňovský
317db82396 Use StringRef in more places in reporters 2021-09-09 10:41:09 +02:00
Martin Hořeňovský
cf5ccaa9df AssertionResult::getMessage returns StringRef to internal string 2021-09-08 10:08:07 +02:00
Martin Hořeňovský
b3a84c7983 Use StringRef in pluralise implementation
This means that it can no longer be safely made ahead of time,
but nothing in our existing code used it like that. Normally it
is constructed and used in the same expression, which is now
more efficient.
2021-09-08 00:24:40 +02:00
Martin Hořeňovský
c0f866c7cf Make CumulativeReporterBase's members protected 2021-09-08 00:23:37 +02:00
Martin Hořeňovský
29caae5ce5 Pass around the test suite name by StringRef 2021-09-08 00:01:31 +02:00
Martin Hořeňovský
ea49210eae Remove Config::getProcessName because it was unused 2021-09-08 00:00:16 +02:00
Martin Hořeňovský
e4719fb51c Update release notes with new fixes, improvements and breaking changes 2021-09-07 22:00:57 +02:00
Martin Hořeňovský
290c1b60e6 Store singular TestRunNode in CumulativeReporterBase
There can never be more than one, so there is no reason to provide
genericity over multiple nodes.
2021-09-07 21:18:48 +02:00
Martin Hořeňovský
e5938007f7 Completely remove the testGroup events
This was done because they were 1:1 mapping to testRun events, and
I could not think of a reasonable way to make them their own thing.
2021-09-07 20:00:22 +02:00
Martin Hořeňovský
ab3fe0053d Remove superfluous warning suppression from console reporter 2021-09-07 19:57:55 +02:00
Martin Hořeňovský
432d03d1aa Use -Wextra-semi instead of -Wextra-semi-stmt for stricter warnings 2021-09-02 11:04:55 +02:00
Roc
9ac9fb164e Suffix the library name with 'd' for debug builds 2021-08-28 00:47:54 +02:00
Martin Hořeňovský
07018e2fba Force __cdecl calling convention for wmain
Fixes #2219
2021-08-26 23:13:27 +02:00
Martin Hořeňovský
ff0a5227ca Make Approx::operator() const
Closes #2273
2021-08-26 21:27:43 +02:00
Martin Hořeňovský
54edab53bf Add missing include to benchmark/catch_execution_plan.hpp
Fixes #2275
2021-08-26 20:19:25 +02:00
Martin Hořeňovský
0a8516aeea Use make_unique when registering an exception translator 2021-08-21 21:32:15 +02:00
Martin Hořeňovský
928ecbaccf Change callable/model::clone to return a unique_ptr 2021-08-21 21:00:00 +02:00
Martin Hořeňovský
1cbbc5d2cb Add test for !shouldfail and failed benchmarks 2021-08-21 00:06:31 +02:00
Martin Hořeňovský
7f3297f7e8 Sweep out unused <utility> includes 2021-08-19 21:44:17 +02:00
Martin Hořeňovský
7ff54ebc06 Add _t suffix typedef for internal traits 2021-08-19 21:29:41 +02:00
Martin Hořeňovský
ca8546efc6 Mention C++14 baseline in contributing documentation 2021-08-19 19:16:12 +02:00
Martin Hořeňovský
4113a12c69 Fix Wreserved-identifier for UDLs in Catch2
See #578
2021-08-19 19:15:46 +02:00
Martin Hořeňovský
edad4d0af7 Prefer _t over ::type form for std type traits 2021-08-19 00:21:13 +02:00
Martin Hořeňovský
88c27ffaf2 Mention CATCH_MOVE and CATCH_FORWARD in contributing docs 2021-08-16 16:22:21 +02:00
Martin Hořeňovský
d2ee7100d2 Replace uses of std::move and std::forward with macros
This improves the SelfTest build times by about 3% (measured
with Clang 10 on random Linux box I had lying around).
2021-08-16 16:22:19 +02:00
Martin Hořeňovský
03ce304102 Add std::move/std::forward replacement macros
Using the `CATCH_MOVE` and `CATCH_FORWARD` macros instead of the
`std::move` and `std::forward<T>` utility functions can improve
compilation times and debug build's performance, and thus will
be preferred going forward.
2021-08-16 16:22:16 +02:00
Geoffrey M Gunter
7040f03b54 Fix typo in docs/matchers.md
There were a few instances in matchers.md where header files erroneously
had a .h file extension instead of .hpp
2021-08-16 15:19:49 +02:00
Martin Hořeňovský
1554251f97 Fix typo in docs/cmake-integration.md 2021-08-11 22:57:36 +02:00
Martin Hořeňovský
2b54f1e7a6 Remove superfluous catch_enforce include from complete_invoke
Also added missing transitive include to `catch_benchmark.hpp`.
2021-08-11 22:52:04 +02:00
Martin Hořeňovský
2c84854b90 Use unique_ptr to hold Environment measurements in benchmarking 2021-08-11 22:52:01 +02:00
Martin Hořeňovský
3579c055c8 Use Detail::make_unique<T> in more places 2021-08-11 22:51:59 +02:00
Martin Hořeňovský
3ec63324a8 Replace owning naked IStream pointers with unique_ptrs 2021-08-11 22:51:57 +02:00
Martin Hořeňovský
7d0770adf2 Merge pull request #2269 from numaru/feature/rename-contrib-in-docs
Rename contrib/ to extras/ in the docs
2021-08-11 00:26:38 +02:00
Kin Numaru
74db06199b Rename contrib/ to extras/ in the docs
The directory has been renamed in 918aa32 but the doc was not updated
yet.
2021-08-10 10:41:16 +02:00
Martin Hořeňovský
52a3144145 Update users docs 2021-08-08 23:42:24 +02:00
Martin Hořeňovský
a62974eb6a Update why-catch.md 2021-08-08 23:42:17 +02:00
Martin Hořeňovský
a0d84654dd Add GitHub Actions badges to README 2021-08-07 21:18:00 +02:00
Martin Hořeňovský
557e5118f1 Remove TravisCI badge 2021-08-07 16:25:22 +02:00
Martin Hořeňovský
0a3f511cfe Fix inconsistencies in reporting benchmarking failures
With these changes, all these benchmarks
```cpp
BENCHMARK("Empty benchmark") {};
BENCHMARK("Throwing benchmark") {
    throw "just a plain literal, bleh";
};
BENCHMARK("Asserting benchmark") {
    REQUIRE(1 == 2);
};
BENCHMARK("FAIL'd benchmark") {
    FAIL("This benchmark only fails, nothing else");
};
```

report the respective failure and mark the outer `TEST_CASE` as
failed. Previously, the first two would not fail the `TEST_CASE`,
and the latter two would break xml reporter's formatting, because
`benchmarkFailed`, `benchmarkEnded` etc would not be be called
properly in failure cases.
2021-08-07 15:16:11 +02:00
Martin Hořeňovský
9ef510b769 Remove unused travis yaml 2021-08-07 15:09:50 +02:00
Martin Hořeňovský
1b1f3a88bc Don't debug log empty strings 2021-08-06 11:38:54 +02:00
Martin Hořeňovský
02ab64da2e Cleanup ExceptionTranslatorRegistry::translateActiveException 2021-08-06 00:56:09 +02:00
Martin Hořeňovský
77df08b44d Better arg name for run_for_at_least 2021-08-06 00:55:31 +02:00
Martin Hořeňovský
79c2daa4a0 Update tutorial docs 2021-08-03 13:53:44 +02:00
Martin Hořeňovský
1e0dc61d16 Update CMake integration documentation 2021-07-31 21:46:11 +02:00
Martin Hořeňovský
02e5951f11 Add major changes and their motivation to v2 -> v3 migration docs
Closes #2135
2021-07-29 23:19:48 +02:00
Martin Hořeňovský
1ecc79bb56 Fix superfluous uppercase C in matcher docs 2021-07-28 23:38:27 +02:00
Martin Hořeňovský
73cae40a90 Fix WithinULP matcher factory name in docs 2021-07-28 23:37:23 +02:00
Robert Andrzejuk
6c4c961207 Update migrate-v2-to-v3.md
#2135 Adding "Changes overview" to "Migrating from v2 to v3" doc
2021-07-28 23:18:50 +02:00
Martin Hořeňovský
340a61af50 Always use 'Detail' for the detail namespace 2021-07-27 21:45:27 +02:00
Martin Hořeňovský
3d1cf95b32 Fix ulp distance calculation for numbers with different signs
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.

Closes #2152
2021-07-27 21:04:41 +02:00
Vishesh Yadav
6f21a3609c Update CMake doc for automatic test registration
`FetchContent` doesn't include `contrib` directory as part of `CMAKE_MODULE_PATH`. This results into `include(Catch)` to fail. This patch just updates the documentation describing how to do include the path, so the new users don't have to figure this out themselves.

Source: https://github.com/catchorg/Catch2/issues/2103#issuecomment-730626324
2021-06-30 00:12:36 +02:00
Martin Hořeňovský
bf61a418cb Remove the ill-conceived compilation perf tests using real tests 2021-06-20 19:15:02 +02:00
Martin Hořeňovský
849002aec0 Move specialization for global op<< to the proper place
In v2 it was placed in a very central header due to the way it was
stitched together. Now that we don't do that, we can move it to the
proper place, removing the potential for confusion given that the
original header was split apart and renamed.
2021-06-20 19:14:56 +02:00
Martin Hořeňovský
4eb9af51af Use non-reserved identifiers for test autoregistration objects 2021-06-19 23:15:11 +02:00
Martin Hořeňovský
78e4fbdb12 Use non-reserved identifiers for BENCHMARK macro implementation 2021-06-19 23:14:20 +02:00
Martin Hořeňovský
a7533707ff Use unique_ptr for passing around exception translators 2021-06-19 20:57:06 +02:00
Martin Hořeňovský
28a33497be Move extractClassName to the single calling TU and make it static 2021-06-18 23:39:07 +02:00
Martin Hořeňovský
70f5392210 Devirtualize more things in trackers 2021-06-18 10:35:02 +02:00
Martin Hořeňovský
61461dfd1d Move default impls of isFooTracker to the interface 2021-06-18 10:35:01 +02:00
Martin Hořeňovský
c064322a9d Devirtualize ITracker::parent 2021-06-18 10:34:58 +02:00
Martin Hořeňovský
a14d67cace ITracker::parent returns pointer not reference
We were already using it mostly to get pointers rather than
references, so this makes it make more sense.
2021-06-18 10:34:55 +02:00
Martin Hořeňovský
4ce8a23edd Tiny cleanups in tracking 2021-06-18 10:34:43 +02:00
Martin Hořeňovský
c1b59b7071 Mark JUnit reporter's internal members as private 2021-06-18 00:00:24 +02:00
AlCash07
c77ba5314a Fix decomposing in presence of universal ADL-found operators
Closes #2121
2021-06-08 23:36:06 +02:00
Martin Hořeňovský
65c9a1d31a Add test for comparing immovable types 2021-06-07 20:05:03 +02:00
Martin Hořeňovský
fa31d58934 Don't use removed-in-C++20 allocator parts in CustomAllocator tests 2021-06-07 17:44:39 +02:00
Martin Hořeňovský
9ac8cad2d1 Merge pull request #2244 from jbadwaik/devel
Disable CATCH_INTERNAL_IGNORE_BUT_WARN for NVHPC Compilers
2021-06-07 15:22:53 +02:00
Jayesh Badwaik (FZ Juelich)
c71f42cc29 Disable CATCH_INTERNAL_IGNORE_BUT_WARN for NVHPC Compilers
- NVHPC's implementation of `__builtin_constant_p` has a bug which
    results in calls to the immediately evaluated lambda expressions to be
    reported as unevaluated lambdas.

    https://developer.nvidia.com/nvidia_bug/3321845.

  - Hence, we disable CATCH_INTERNAL_IGNORE_BUT_WARN for NVHPC Compilers
2021-06-07 12:28:16 +02:00
Martin Hořeňovský
e6da4e10ae Use make_unique instead of unique_ptr(new T) 2021-06-01 00:17:22 +02:00
Martin Hořeňovský
816f69416b Use TestCaseInfo in tests directly 2021-06-01 00:17:19 +02:00
Martin Hořeňovský
aee31d0620 Disallow empty tags in test case specification 2021-06-01 00:17:13 +02:00
Martin Hořeňovský
c9371865d4 Use StringRef as the argument to benchmark{Failed,Preparing} 2021-05-31 08:42:03 +02:00
Martin Hořeňovský
5741de9ccd Add explicit test for serialization of boolean attributes in XML 2021-05-31 08:41:28 +02:00
Martin Hořeňovský
0e2895934c Use precomputed string sizes when constructing std::strings 2021-05-31 08:41:26 +02:00
Martin Hořeňovský
a01073d871 Write single characters directly instead of as C-strings in Sonarqube 2021-05-31 08:41:20 +02:00
Martin Hořeňovský
02839ba934 Remove superfluous usings in the Contains matcher 2021-05-31 08:40:09 +02:00
Martin Hořeňovský
8d6a1c27ef Use StringRef literals where possible in XML using reporters
This let's us avoid running `strlen` at runtime to convert the
plain string literals to `StringRef`s, by guaranteeing that we
instead have the size available after compilation.

In optimized builds the performance improvement should be even
greater, as the `StringRef` UDL and the related constructor
are both `constexpr`, and thus can be baked completely during
compilation.
2021-05-30 13:59:47 +02:00
Martin Hořeňovský
41ad0fda11 Improve XmlWriter::writeAttribute overload set for string-like types
Previously, string literals and `std::string`s would match the
template variant, which would serialize them into a stream and then
call the `StringRef` overload for resulting string. This caused
bunch of codebloat and unnecessary pessimization for common usage.
2021-05-30 13:06:03 +02:00
Martin Hořeňovský
d9f72868b2 Use StringRef instead of std::string in XmlWriter::ScopedElement 2021-05-29 21:47:12 +02:00
Martin Hořeňovský
c7241bb76e Use StringRef in XmLWriter when the text is not stored 2021-05-29 21:23:24 +02:00
Martin Hořeňovský
1d04427fcd Use StringRef through XmlEncode
This introduces a potential lifetime risk when using the API, but
the intended way to use the `XmlEncode` class is to use it directly,
e.g. `out << XmlEncode(some-text-argument)`, not to store it around.

The benefit is that we avoid allocations for strings that do not fit
into SSO for given platform.
2021-05-29 16:41:57 +02:00
Daniel Edwards
aba114d6fe Don't include CTest in non-development builds.
When Catch2 is used as a CMake subproject (via add_subdirectory) and is not built in development mode (CATCH_DEVELOPMENT_BUILD) CTest is not required for the build, as testing is off. When CTest is included it creates various targets (around 20) which are shown in IDEs such as Clion and pollute the list of configurations.

This patch conditionally includes CTest only if Catch2 is built in development mode.
2021-05-29 16:24:44 +02:00
Martin Hořeňovský
0221148ac3 Cleanup string escaping in XmlWriter with raw string literals 2021-05-29 14:11:50 +02:00
Martin Hořeňovský
9f091cbe9d Remove some code duplication from default test listing impl 2021-05-29 14:06:40 +02:00
Martin Hořeňovský
4c1e896d47 Flush less in default test listing impl 2021-05-29 13:52:23 +02:00
Martin Hořeňovský
2c04850f88 Add some constexpr in compact reporter impl 2021-05-29 13:49:33 +02:00
Martin Hořeňovský
c9027375a3 Replace std::endl with \n and flush
In some places the `std::flush` was not added, as it was sufficiently
obvious that the flush semantics are not intended. There are likely
other places where the flush semantics aren't intended, but that
is a cleanup for later.
2021-05-29 13:15:46 +02:00
Martin Hořeňovský
2ae28fc852 Use make_unique instead of unique_ptr(new T) in more places 2021-05-28 23:43:15 +02:00
Martin Hořeňovský
f9ec34ce01 Make operator BenchmarkStats<Duration2> explicit 2021-05-28 23:19:07 +02:00
Martin Hořeňovský
96790b1d23 Always install the used compiler in Github Actions
This avoids the CI breaking again when the base images remove
an older compiler again.
2021-05-28 11:30:06 +02:00
Dimitrij Mijoski
86f86c4c23 Fix compiling with C++17 + Clang 5 + libstdc++ v5
This basically tests the combination where the compiler supports most
of C++17 but the library does not.
2021-05-26 00:34:23 +02:00
Martin Hořeňovský
023b5306b4 Add deep const to unique_ptr::operator-> 2021-05-19 11:00:53 +02:00
Martin Hořeňovský
9137e591fa Cleanup and optimize stringifying of string-like types
More specifically, made the actual implementation of string-like
type handling take argument as `Catch::StringRef`, instead of
taking `std::string const&`.

This means that string-like types that are not `std::string` no
longer need to pay for an extra construction of `std::string`
(including the potential allocation), before they can be stringified.

The actual string stringification routine is now also better about
reserving sufficient space.
2021-05-18 23:15:22 +02:00
Martin Hořeňovský
f50a06affa Small optimization for stringifying chrono::time_point 2021-05-18 00:17:34 +02:00
Martin Hořeňovský
4cc247cc70 Use constexpr StringRef instance for unprintable strings 2021-05-18 00:17:32 +02:00
Martin Hořeňovský
8ee422d6bf Remove pointless branch from stringifying matcher assertions 2021-05-15 11:47:02 +02:00
Martin Hořeňovský
0c0f73a48d Constexprify NameAndTags constructor 2021-05-15 00:02:09 +02:00
Martin Hořeňovský
61e16416a9 Pass other StringRef arguments by value instead of by-ref
Apart from being clearer, it also improves the overall codesize
of the implementation library, and should improve the performance
as well, by removing one level of indirection.
2021-05-14 23:45:59 +02:00
Martin Hořeňovský
074017f5ad Inline MessageBuilder's constructor 2021-05-14 16:00:15 +02:00
Martin Hořeňovský
0a89e7f0c4 Pass StringRef by value instead of by const-ref 2021-05-14 15:59:42 +02:00
Martin Hořeňovský
28f6698ec8 Remove unneeded interface IRunner 2021-05-13 09:14:26 +02:00
Martin Hořeňovský
b36f8daaad Clarify bunch of default, out of line destructors 2021-05-12 23:55:46 +02:00
Martin Hořeňovský
d86cb5f95d Add override to virtual destructor of derived class 2021-05-12 23:23:02 +02:00
Martin Hořeňovský
5eb7aa4f90 Add test for dots in tag names 2021-05-12 21:43:14 +02:00
Martin Hořeňovský
35cba5486d Remove mention of ANON_TEST_CASE from documentation
It was removed from v3, because the standard `TEST_CASE` can now be
anonymous as well.
2021-05-10 21:49:45 +02:00
Jozef Grajciar
eb911aa995 Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187)
Resolves #1390

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
Martin Hořeňovský
313071e8fe Pick release notes for v2.13.6 2021-05-09 23:45:00 +02:00
Martin Hořeňovský
f9bb2668e4 Pick release notes for v2.13.5 2021-05-09 23:43:05 +02:00
Matteo Beniamino
baf0cd0be4 Fixed [dis]engage_platform declarations mismatch 2021-05-09 23:34:17 +02:00
Martin Hořeňovský
c0d0a50bdb Significantly refactor fatal error handling
Because new glibc has changed `MINSIGSTKSZ` to be a syscall instead
of being constant, the signal posix handling needed changes, as it
used the value in constexpr context, for deciding size of an array.
It would be simple to fix it by having the handler determine the
signal handling stack size and allocate the memory every time the
handler is being installed, but that would add another allocation
and a syscall every time a test case is entered.

Instead, I split apart the idea of preparing fatal error handlers,
and engaging them, so that the memory can be allocated only once
and still be guarded by RAII.

Also turns out that Catch2's use of `MINSIGSTKSZ` was wrong, and
we should've been using `SIGSTKSZ` the whole time, which we use now.

Closes #2178
2021-05-09 23:34:15 +02:00
Pavel Kamenov
cbcab2dbcd Add lcc to the list of unwanted compilers that mimic gcc 2021-05-09 18:12:58 +02:00
Scott Hutchinson
ea44e73961 Wrap all std::min and std::max calls in parentheses 2021-05-09 18:10:50 +02:00
Rob Boehne
d61fe3ecc3 [Issue 2154] Correct error when building with IBM's latest XLC (#2155)
* [Issue 2154] Correct error when building with IBM's latest XLC compiler with xlclang++ front-end.

On AIX, the XLC 16.1.0.1 compiler considers the call to `std::abs` ambigious, so it needs help with a static_cast to the type of the template argument.

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-09 18:05:39 +02:00
Tom de Geus
b325c6d81e Making target detection on Mac more robust 2021-05-09 18:05:37 +02:00
Martin Hořeňovský
d4a3cd9992 Make the static library build reproducible with supported compilers
This is based on bed285af07 from
`v2.x` branch, but done properly for the build in v3 branch,
effectively porting #2141 to devel.
2021-05-09 18:05:34 +02:00
Martin Hořeňovský
342dd3445c Pick documentation changes for 2.13.4 2021-05-09 18:05:31 +02:00
Martin Hořeňovský
23760327ae Fix updateVersionPlaceholder when the placeholder starts the line 2021-05-09 18:05:27 +02:00
Reinhold Gschweicher
48f220b68a Add deprecation warning in ParseAndCatchTests
Parsing C++ with regex in CMake is error prone and regularly leads to silently
dropped (not run) test cases.

Going forward the function `catch_discover_tests` from `contrib/CMake.cmake`
should be used.

For more information see https://github.com/catchorg/Catch2/issues/2092#issuecomment-747342765
2021-05-09 18:05:25 +02:00
Reinhold Gschweicher
031a57e7b7 Fix Catch.cmake helper by setting variable globally
Set `_CATCH_DISCOVER_TESTS_SCRIPT` helper variable globally. Otherwise in a
scoped call (like `add_subdirectory()`) the variable gets lost. This lost
variable results in a post build error with not much information to lead to the
root of the problem.

This enables the usage of the helper script with the following example structure

- CMakeLists.txt (project root with `add_subdirectory(external/catch2)`
- external/catch2
  - CMakeLists.txt (contents listed below)
  - contrib/Catch.cmake
  - contrib/CatchAddTests.cmake
  - catch2/catch.hpp
- tests
  - CMakeLists.txt (add tests with `catch_discover_tests(${PROJECT_NAME})`)

contents of project specific helper `external/catch2/CMakeLists.txt`
```cmake
cmake_minimum_required (VERSION 3.1...${CMAKE_VERSION})
project(Catch2 LANGUAGES CXX VERSION 2.13.3)
add_library(Catch2 INTERFACE)
target_include_directories(Catch2
  INTERFACE
    $<BUILD_INTERFACE:${CMAKE_CURRENT_LIST_DIR}>
)
 # provide a namespaced alias for clients to 'link' against if catch is included as a sub-project
add_library(Catch2::Catch2 ALIAS Catch2)
include(contrib/Catch.cmake)
```
2021-05-09 18:05:18 +02:00
Sergio Losilla
cdf4748d1c Modified hash to make it more independent of the choice of test names. 2021-05-09 18:05:14 +02:00
Martin Hořeňovský
520b6dace9 Remove conan to bintray packaging task from travis
As bintray is dead, it is no longer relevant and in fact, it will
fail during build.
2021-05-09 17:45:29 +02:00
Martin Hořeňovský
2cb5210caf Fix and improve GitHub Actions CI
* Fixed problem with older compilers by explicitly installing the
  right compiler version.
* Split apart simple builds that can be well described by a build
  matrix, and builds that need special CMake defines or similar
  special snowflake handling.
* Ported some extras + examples builds from TravisCI to GitHub
  Actions.
2021-04-30 23:26:28 +02:00
Clare Macrae
2dc5a5f402 Prepare to prevent Catch2 v3's tests from being run, and CTest targets from being added, if Catch2 is configured as a sub-project. (#2205)
This commit fixes issue that happens if the project above us uses the same variable name, thus confusing our script which see the variable scoped from the project including Catch2, rather than ours

See #2202
2021-04-16 21:19:58 +02:00
Georg Schwab
04166514fe fixed inconsistent semicolon expansion in catch_discover_tests (Bug #2214) 2021-04-16 17:22:14 +02:00
Julien Brianceau
e8cdfdca87 Fix typos in the code base (#2206)
Note that only documentation and comments are impacted by this change.
2021-04-07 20:43:50 +02:00
Martin Hořeňovský
a5abec9cb5 Remove superfluous virtual from ExceptionTranslatorRegistry 2021-03-26 21:11:49 +01:00
Martin Hořeňovský
f1d7a10e06 Remove StringRef::isNullTerminated and StringRef::c_str
Both of them were fundamentally unsafe to use and shouldn't be used
at all.
2021-03-26 21:11:44 +01:00
Martin Hořeňovský
e50e10ef8f Use std::string instead of StringRef as argument to the makeStream
This is done because `makeStream` was the only place using
`StringRef::c_str()`, which is an error-prone and unsafe API that
I want to remove.
2021-03-26 21:11:42 +01:00
Alexander Shaduri
2c269eb633 Fixed compilation under MinGW - use gmtime_s() instead of gmtime_r(). 2021-03-23 23:46:04 +01:00
Martin Hořeňovský
4b5812e932 Remove trailing whitespace from test-cases-and-sections.md 2021-03-23 13:02:24 +01:00
Martin Hořeňovský
9f44bd57f1 Add more documentation to BDD macros 2021-03-23 13:02:07 +01:00
matthew-limbinar
6734c0aa64 Expand documentation for BDD test macros (#2179) 2021-03-23 12:20:33 +01:00
Jozef Grajciar
037ddbc75c Fix introduced in version placeholders
Resolves #2082
2021-03-13 10:26:12 +01:00
Martin Hořeňovský
6d803cba5d Add example of negated tag selection to the CLI test spec docs
Close #2180
2021-02-25 15:54:27 +01:00
Martin Hořeňovský
551946c45b Split out INTERNAL_CATCH_UNIQUE_NAME into its own header
Also small cleanup of includes of related headers.
2021-02-24 23:14:18 +01:00
Martin Hořeňovský
653764d53b Simplify workaround for MSVC issue with XmlWriter 2021-02-21 11:19:02 +01:00
Martin Hořeňovský
3afea8128a Increase timing window for min duration tests to 1s
This test tends to be brittle on Mac CI machines, which are
heavily loaded and bursty. Since the tests are only run as part
of the "extra tests" test set, this increase should not have
a significant impact on the total duration of CI runs.
2021-02-20 23:09:02 +01:00
Martin Hořeňovský
749d953712 Add Clang 10 + C++17 build to linux CI 2021-02-20 22:26:22 +01:00
Martin Hořeňovský
4b50b13970 Disable C++ extensions for linux CI builds 2021-02-20 20:39:23 +01:00
Martin Hořeňovský
1ee0940427 Merge pull request #2177 from catchorg/devel-generating-surrogates
Add surrogate TUs option to CMake
2021-02-20 10:29:01 +01:00
Martin Hořeňovský
29050daec0 Add Linux and Windows CI builds with surrogate TUs 2021-02-19 23:23:00 +01:00
Martin Hořeňovský
e5e9afad16 Fix headers whose surrogate TUs did not compile 2021-02-19 23:22:43 +01:00
Martin Hořeňovský
8b27041fbe Add CMake option to generate surrogate TUs in development build
A surrogate TU is TU that includes 1 specific header, and does
nothing else. This is useful to verify that the header is
self-sufficient and does not require other headers to be included
for compilation to succeed.

Closes #2106
Closes #2166 (this is a better solution)
2021-02-19 23:22:39 +01:00
Martin Hořeňovský
c12170ff69 Extend release notes
Included some changes that weren't updated at the time of merging,
and also results of a new runtime benchmark.
2021-02-13 15:34:38 +01:00
Rohit Goswami
3eade52fc0 users: Add d-SEAMS 2021-02-04 22:53:00 +01:00
Martin Hořeňovský
2dbe63a6ba Merge pull request #2165 from catchorg/devel-gmtime-fixup
Use gmtime_r instead of gmtime when compiling for posixy platforms
2021-01-27 19:55:41 +01:00
Martin Hořeňovský
477540760a Use gmtime_r instead of gmtime when compiling for posixy platforms 2021-01-27 11:49:52 +01:00
Martin Hořeňovský
b435e391c4 Merge pull request #2163 from catchorg/devel-list-outputs-to-out-flag
The output of --list-* flags obeys -o flag
2021-01-27 09:07:15 +01:00
Martin Hořeňovský
971b1fc32a --list-* flags write to target specified by the -o flag
Also added tests for the default implementations of list* reporter
helpers.

Closes #2061
2021-01-26 18:36:54 +01:00
Martin Hořeňovský
6798c139a6 Extract the default listing implementation out of reporter interface
Also generalized the implementations to write to the provided
output stream, which will be required for the follow up changes,
where the listings should happen to the location user asked for
by specifying the `-o` flag.
2021-01-26 11:23:37 +01:00
Martin Hořeňovský
7111b2a8e2 Cleanup IStreamingReporter::listTests 2021-01-16 15:50:52 +01:00
Martin Hořeňovský
5509ceff60 Remove superfluous IConfig argument from IStreamingReporter::list*
The previous commit made it so that the `IConfig` is stored in
`IStreamingReporter` and thus always available.
2021-01-16 15:48:29 +01:00
Martin Hořeňovský
74f2f4ba5e Hold reporter's IConfig instance in the interface
Previously, every base derived from the IStreamingReporter had
its own `IConfig const* m_config` member, so this just centralizes
the handling thereof.

Part of #2061
2021-01-16 15:27:56 +01:00
John Zimmermann
ba81505168 Do not match exact amount of spaces for errno macro expansion in approvalTests.py
E.g. musl libc expands errno() to __errno_location() without a space between, glibc has 1 space.
2021-01-08 11:31:28 +01:00
Martin Hořeňovský
f5b413aa58 Merge pull request #2138 from catchorg/devel-config-splitting
Split some configuration options out of catch_compiler_capabilities
2020-12-29 12:37:27 +01:00
Martin Hořeňovský
4e8832fc88 Split CATCH_CONFIG_COUNTER into its own header
Part of #2041
2020-12-29 11:07:45 +01:00
Martin Hořeňovský
bdd1e5c613 Remove some superfluous includes 2020-12-28 21:11:54 +01:00
Martin Hořeňovský
1d269211bd Split CATCH_CONFIG_WCHAR into its own header
Part of #2041
2020-12-28 20:51:49 +01:00
Roman Proskuryakov
0acb371b92 Fix Wold-style-cast error (#2125)
* Add Wold-style-cast to cmake flags
* Fix old style cast in catch_stats.hpp
* Fix old style cast in catch_stats.cpp
2020-12-28 14:00:19 +01:00
Florian Berchtold
045feff834 Update cmake-integration.md (#2115)
* Update cmake-integration.md

CMake related, mainly more modern and provide an executable to be correct

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-12-28 13:41:55 +01:00
Martin Hořeňovský
965afc4b2e Split CATCH_CONFIG_ANDROID_LOGWRITE into its own header
Part of #2041
2020-12-28 12:53:52 +01:00
Martin Hořeňovský
77643ce2e5 Add more comprehensive tests for the quantifier matchers
This includes
* Testing both positive and negative path through the matchers
* Testing them with types whose `begin` and `end` member functions
require ADL
* Testing them with types that return different types from `begin`
and `end`
2020-12-27 20:20:55 +01:00
Uriel García Rivas
552af8920d Added AnyMatch, AllMatch and NoneMatch 2020-12-27 20:20:55 +01:00
Alex Voicu
ce54ec185f Add the HIP CPU Runtime as a Catch2 user (#2123) 2020-12-18 13:23:17 +01:00
Martin Hořeňovský
c787b1edc9 Make Linux builds print stdout on ctest test failure 2020-12-15 09:58:19 +01:00
Martin Hořeňovský
a091853f4a Rename file catch_matchers_floating -> catch_matchers_floating_point
The old name was a legacy of v2 era, where all headers were stitched
into one. With v3 using separate headers, it is better when they have
proper name.
2020-12-13 18:12:20 +01:00
Norbert Wenzel
be813faaa0 Fix typo 2020-12-12 15:31:20 +01:00
Martin Hořeňovský
4b51d0dd3b Remove rest of std::shared_ptr usage in Trackers
Closes #2088
2020-12-05 15:19:23 +01:00
Martin Hořeňovský
6350851f9a Return plain ptr from ITracker::findChild instead of shared_ptr
This is part of wider rework of trackers to avoid `std::shared_ptr`
everywhere.

Related to #2088
2020-12-04 21:37:24 +01:00
cat_mint
21c97f2fad Use snprintf instead of sprintf (#2111) 2020-12-01 18:59:27 +01:00
Martin Hořeňovský
5b1a6ae00a Merge pull request #2112 from Emil-Jarosz/patch-1
Update cmake-integration.md
2020-11-30 21:18:25 +01:00
Emil Jarosz
b9fe8a208f Update cmake-integration.md
The "contrib" folder was renamed to "extras", but the name wasn't updated in the documentation.
2020-11-30 13:10:26 +00:00
Martin Hořeňovský
c19b8ec5d7 Remove TravisCI builds subsumed by GitHub Actions builds 2020-11-29 23:18:57 +01:00
Martin Hořeňovský
230f23e6ee Add more Clang builds to github actions 2020-11-29 10:22:21 +01:00
Martin Hořeňovský
88504e5332 Use Ubuntu-20.04 runner for running header guard validation
Using `ubuntu-latest` is triggering warning that it will soon migrate
from 18.04 to 20.04, so I am migrating preventively.
2020-11-28 23:09:10 +01:00
Martin Hořeňovský
4da0c18526 C++17 Mac builds 2020-11-28 21:13:00 +01:00
Martin Hořeňovský
1d746a15ac Move Examples and ExtraTests Mac builds to Github Actions 2020-11-28 21:03:38 +01:00
Martin Hořeňovský
19cbdebb0e Lenghten timing threshold in --min-duration integration test 2020-11-28 21:00:09 +01:00
Martin Hořeňovský
f30a9e3feb Move some Mac builds from Travis to GitHub Actions 2020-11-28 17:47:26 +01:00
Martin Hořeňovský
e7740316e3 Split TestFailureException into its own header 2020-11-28 11:18:54 +01:00
Martin Hořeňovský
72525a3053 Rename catch_common.hpp -> catch_source_line_info.hpp
Previous splitting of catch_common.hpp left it containing only one
actual thing, which is the `SourceLineInfo` type. Given that, there
is no reason to keep the old name.

Also found out that it was included in some places for no reason
(primarily Matchers).
2020-11-28 11:18:51 +01:00
Martin Hořeňovský
1982c0d5ee Split out the StreamEndStop helper into its own header 2020-11-28 11:18:46 +01:00
Martin Hořeňovský
0442229dc9 Start using GitHub actions for Linux builds
For now we switched only some of the builds, those that do not
need manually installed compiler packages to function.
2020-11-28 11:13:16 +01:00
Martin Hořeňovský
130bf835b5 Simple Doxygen file 2020-11-26 18:43:31 +01:00
Martin Hořeňovský
c3e8ae642f Remove pointless virtual in WildcardPattern 2020-11-26 18:35:20 +01:00
Martin Hořeňovský
3bd5fd6bc5 Mention GENERATE decaying inputs in release notes docs 2020-11-26 18:35:02 +01:00
Martin Hořeňovský
f36e059453 Remove superfluous <memory> include from catch_reporter_cumulative_base.hpp 2020-11-26 18:06:21 +01:00
Martin Hořeňovský
677adf8ade Replace shared_ptrs with unique_ptrs in CumulativeReporterBase nodes
Closes #2089
2020-11-26 14:45:27 +01:00
Martin Hořeňovský
bfe5553416 CumulativeReporterBase doesn't use shared_ptrs for sectionStack 2020-11-26 10:21:12 +01:00
Martin Hořeňovský
c673db7a4e Add explanatory comment to CumulativeReporterBase 2020-11-26 09:45:28 +01:00
Martin Hořeňovský
b10a19545b CumulativeReporterBase stores TestRunNodes directly 2020-11-26 09:43:23 +01:00
Martin Hořeňovský
e5ccb79bf8 Remove pointless virtual on StreamingReporterBase::Node destructor 2020-11-25 21:57:30 +01:00
Martin Hořeňovský
3610eb81b1 Explicitly inline trivial function in CumulativeReporterBase impl 2020-11-17 21:02:11 +01:00
Martin Hořeňovský
bd1e76cc3a Untypedef some types for clarity when refactoring 2020-11-17 21:01:35 +01:00
Martin Hořeňovský
166c520598 Remove another unused member variable from CumulativeReporterBase 2020-11-14 16:17:48 +01:00
Martin Hořeňovský
a29deeb129 Remove unused member in StreamingReporterBase
Part of #2089
2020-11-14 16:17:39 +01:00
Martin Hořeňovský
1cef51b69b Default StreamingReporterBase::Node destructor 2020-11-14 16:17:35 +01:00
Martin Hořeňovský
79c1bf9301 Use plain pointer to point to deepest section in CumulativeReporterBase
Part of #2089
2020-11-14 16:17:27 +01:00
Martin Hořeňovský
4f14922aa3 Don't use shared_ptr to store test run info in CumulativeReporterBase
Part of #2089
2020-11-14 16:16:28 +01:00
Martin Hořeňovský
0fa133a0c5 Run checking scripts directly 2020-11-10 15:25:52 +01:00
Martin Hořeňovský
447b39cae0 Better names for misc-checks workflow steps 2020-11-10 15:25:24 +01:00
Martin Hořeňovský
851a0e907e Make scripts executable 2020-11-10 15:22:15 +01:00
Martin Hořeňovský
93312b369e Merge pull request #2099 from innerout/check-license
Adds license check in CI
2020-11-10 11:10:03 +01:00
George Xanthakis
d913837a5d Adds license check in CI 2020-11-10 00:51:24 +02:00
Martin Hořeňovský
a9941d4231 Merge pull request #2098 from innerout/check-duplicates
Adds duplication check for source files in CI
2020-11-07 20:59:37 +01:00
George Xanthakis
39e13bf530 Adds duplication check for source files in CI 2020-11-07 21:35:41 +02:00
Martin Hořeňovský
fefa001bb6 Fixup approvals 2020-11-07 20:16:53 +01:00
Martin Hořeňovský
135103bacf Arg::parse accepts plain Args as argument 2020-11-07 18:00:27 +01:00
Martin Hořeňovský
2baa472bcc Add GitHub action to check that include guards use standard pattern
Based on work by @innerout, but finished and merged by me, because
the GitHub action refused to run for him.

Closes #2075
Closes #2091
2020-11-06 21:36:31 +01:00
Martin Hořeňovský
f97436a1f7 Normalize include guard in catch_config_ucnaught_exceptions.hpp 2020-11-06 18:01:27 +01:00
Morwenn
dd5652933a Silence -Wuseless-cast warning with GCC10
With GCC 10, the `static_cast<bool>` triggers the -Wuseless-cast warning. This commit changes the cast into `static_cast<const bool&>`: it achieves the same thing but doesn't trigger the warning thanks to the "gratuitous" type conversion to `const bool&`. As per references rules, `const bool&` should bind to anything, be it `const` or not, an rvalue or an lvalue, so I doubt that this change is breaking anything.
2020-11-02 16:01:17 +01:00
Alexandr Timofeev
3a15433d52 Add support for Components 2020-11-02 15:37:35 +01:00
Alexandr Timofeev
67a9561fb5 Fix slashes for Windows builds 2020-11-02 15:37:35 +01:00
Alexandr Timofeev
2f31f9037d Inject conan_basic_setup to properly set MD/MT flags 2020-11-02 15:37:35 +01:00
Martin Hořeňovský
33bcdc6bf5 Picked release notes for v2.13.3 2020-11-02 14:42:24 +01:00
Reinhold Gschweicher
74b397e6b8 Consider CMP0110 add_test() policy
CMake 3.19 introduces new add_test() behavior guarded with the policy
CMP0110.

See: https://cmake.org/cmake/help/latest/policy/CMP0110.html

Update the helper script ParseAndAddCatchTests to consider the policy and
handle the test case name accordingly.
2020-11-02 14:42:22 +01:00
Reinhold Gschweicher
730ec39a74 Fix indentation in ParseAndAddCatchTests
Consistently use 4 spaces instead of tabs
2020-11-02 14:42:19 +01:00
Reinhold Gschweicher
71328bae90 Fix CMake regex to add tests
Fix regex that requires two string arguments in the form of
TEST_CASE("a", "b") resulting in not finding TEST_CASE("a") entries.

See https://regex101.com/r/JygOND/1

Fixes: https://github.com/catchorg/Catch2/issues/2055
2020-11-02 14:42:17 +01:00
laoshanxi
ed9ef85a34 Add AppMesh to Open Source projects using Catch 2020-11-02 14:42:13 +01:00
Martin Hořeňovský
e4474021ff Merge pull request #2085 from AugustasV/patch-1
[DOCS] update from Hudson to Jenkins
2020-11-01 15:12:58 +01:00
Martin Hořeňovský
79a5cd795c Merge pull request #2086 from AMS21/patch-1
Enable IPO for CMake versions that support it
2020-11-01 15:11:17 +01:00
AMS21
b8ae2878b4 Enable IPO for CMake versions that support it 2020-11-01 09:36:42 +00:00
Augustas
dc3c22f9ec update from Hudson to Jenkins
Hudson is discontinued in 2016. Changed to Jenkins
2020-10-31 21:48:28 +02:00
Martin Hořeňovský
735f46ed6d Link to the migration docs from main readme and docs/readme 2020-10-31 09:08:58 +01:00
Martin Hořeňovský
39aabede62 Add a short documentation for migrating from v2 to v3 2020-10-31 08:14:50 +01:00
Martin Hořeňovský
d7ced69db2 Update README.md to point to v2 branch 2020-10-30 15:31:41 +01:00
Hannes Kaeufler
f797ae7a8f Bump OpenCppCoverage to 0.9.9.0
Also modified the AppVeyor script to account for changes in what
happens to PATH with the new installer.

Closes #2059
2020-10-29 13:39:35 +01:00
Clare Macrae
40b9df567f Add missing cmake option to fix buildAndTest.sh
Note that the documentation was updated automatically, by running:
 ./tools/scripts/updateDocumentSnippets.py
2020-10-28 11:38:06 +01:00
Clare Macrae
c6352c3e1f Embed the script buildAndTest.sh inside the docs 2020-10-28 11:38:06 +01:00
Clare Macrae
4035beb988 Add a script - buildAndTest.sh
This contains the currently-documented steps for a developer build.
Also, make git ignore its output directory.
2020-10-28 11:38:06 +01:00
Clare Macrae
8c3970465d Add infrastructure for embedding code samples in docs. 2020-10-28 11:38:06 +01:00
Clare Macrae
f57689f888 Fix minor typo in docs. 2020-10-28 11:38:06 +01:00
Martin Hořeňovský
967b82231c Sweep out two leftover references to master
This is basically cherry-picking the usable parts from #2067
2020-10-23 23:02:44 +02:00
Martin Hořeňovský
7b9bf633be Tighten the test for matching filenames-as-tags 2020-10-23 23:02:09 +02:00
Martin Hořeňovský
4c8454b5ec Fix potential infinite loops in generators combined with section filter
The problem was that under specific circumstances, namely that none
of their children progressed, `GeneratorTracker` will not progress.
This was changed recently, to allow for code like this, where a
`SECTION` follows a `GENERATE` at the same level:

```cpp
SECTION("A") {}
auto a = GENERATE(1, 2);
SECTION("B") {}
```

However, this interacted badly with `SECTION` filters (`-c foo`),
as they could deactivate all `SECTION`s below a generator, and thus
stop it from progressing forever. This commit makes GeneratorTracker
check whether there are any filters active, and if they are, it checks
whether its section-children can ever run.

Fixes #2025
2020-10-23 21:21:15 +02:00
Martin Hořeňovský
8878f90323 Fix matching of non-lowercase filename tags
The bug was caused by forgetting to lower-case the filename tag
for matching against test spec.

Fixes #2064
2020-10-22 16:20:41 +02:00
Martin Hořeňovský
0c7f96ba63 Update documentation not to reference master branch 2020-10-21 19:59:59 +02:00
Martin Hořeňovský
923bcc5d6f Special case --list-tests --verbosity quiet
The new output (mostly) follows the old `--list-test-names-only`
format, with the exception of no longer supporting line output
for `--verbosity high`.

Fixes #2051
2020-10-20 15:09:48 +02:00
Martin Hořeňovský
b6a3e2e26b Merge pull request #2058 from catchorg/devel-generators-cache
Fix types in generators: decay types, properly constrain forwarding
2020-10-20 12:46:31 +02:00
Martin Hořeňovský
6ffac61719 Fix types in generators: decay types, properly constrain forwarding
Fixes #2040
Closes #2012
2020-10-20 10:57:37 +02:00
Michael Vlach
4b2bc8757c Update catch_compiler_capabilities.hpp 2020-10-10 17:56:43 +02:00
Martin Hořeňovský
faffc29253 Enable "Try online" badge again 2020-10-10 15:13:24 +02:00
Martin Hořeňovský
4ea18d6d17 Merge pull request #2044 from lkeegan/fix_cmake_catch_add_tests_warning
fix warning in extras/CatchAddTests.cmake
2020-10-08 17:15:05 +02:00
Liam Keegan
c44d9cc718 fix warning in extras/CatchAddTests.cmake
- don't warn on zero return code of --list-reporters
  - previously return code was the number of reporters (#1410, #1146)
  - as of 2c06ee9 return code is zero on success
2020-10-08 16:44:20 +02:00
Martin Hořeňovský
b9853b4b35 Bump version to v3.0.0 preview 3 2020-10-08 15:26:30 +02:00
Martin Hořeňovský
853565bfb8 Pick release notes for v2.13.2 2020-10-08 12:51:07 +02:00
Martin Hořeňovský
3f9e779542 Improve detection of std::uncaught_exceptions support
The problem was that Catch2 did not reliably include `<exception>`
before it checked for the feature test macro for
`std::uncaught_exceptions`. To avoid overhead of including
`<exception>` everywhere, the configuration check was split out
into a separate header.

Closes #2021
2020-10-08 12:39:50 +02:00
Rémy Salim
863cc6a155 Add WORKING_DIRECTORY to CatchAddTests.cmake commands 2020-10-08 11:41:35 +02:00
Martin Hořeňovský
b601b7faca Workaround AppleClang bug by renaming TestHasher constructor argument
As far as I understand the standard, if there is a function called
`rng` in the global namespace, and a function argument called `rng`,
then the argument should shadow the function. This then means that
uses of `rng` inside the function should refer to the argument.

This is not the case for AppleClang 12.0.0. Luckily the workaround
is simple enough; just rename the argument. Given that the function
is 3 lines and uncomplicated, the change of the name doesn't really
affect readability.

Still, WTF AppleClang?

Closes #2030
2020-10-08 11:41:05 +02:00
Matt Godbolt
b841650253 Add missing syntax highlighting tag 2020-10-08 11:37:31 +02:00
Ansel Sermersheim
1d01464730 Support template test cases in ParseAndAddCatchTests
* Change regex to allow parentheses inside the test macro for a type list
* Append a wildcard to the CTestName if the test case is a template
* Also change the regular expression so parentheses are allowed in names
  (fixes #1848)
2020-10-08 11:37:11 +02:00
Martin Stump
c522e88afa Add REPORTER and OUTPUT_* args 2020-10-08 11:35:42 +02:00
Florian Berchtold
b1df96e7e4 Docu/Show how to use CMake FetchContent (#2028) 2020-10-08 11:35:18 +02:00
Will Pazner
a4dfcf9042 Disable __builtin_constant_p when compiling with nvcc 2020-10-08 11:34:57 +02:00
kotaiadam
9e172c707e fixes bug in example - undeclared identifier
j was not declared in `SECTION("two")`
2020-10-08 11:34:25 +02:00
Travis Wilson
b0214ff862 Make experimental capture work on Windows with read-write temp file behavior 2020-10-08 11:28:50 +02:00
Paul Dreik
2454cfffb7 add fuzzer for columns 2020-10-07 17:38:27 +02:00
Paul Dreik
0098a76fef first take basing fuzz on v3 2020-10-07 17:38:27 +02:00
Martin Hořeňovský
340ff00058 Rewrite contributing docs for v3 2020-10-06 12:36:20 +02:00
Martin Hořeňovský
60dfec559f Provide 1 .hpp + 1 .cpp distribution of Catch2
This commits also adds a script that does the amalgamation of headers
and .cpp files into the distributable version, removes the old
`generateSingleHeader` script, and also adds a very simple compilation
test for the amalgamated distribution.
2020-09-09 13:02:50 +02:00
Martin Hořeňovský
8b89a60bf6 Reenable VS2017+examples+extra-tests build on AppVeyor 2020-09-08 22:20:12 +02:00
Martin Hořeňovský
99d70c0c9d Remove superfluous whitespace before #include 2020-09-07 20:01:04 +02:00
Martin Hořeňovský
d1625f30b1 Pick release notes from v2.13.1 2020-09-07 14:34:36 +02:00
Dawid Kurek
8f44e09a72 Remove superfluous values
The `0`s were needed for the expansion of parameter pack for the sake of
expander. Now when we have `index++` it is no more needed.
2020-09-07 14:30:58 +02:00
Sean Middleditch
31d4831245 Support sentinel-based ranges in default stringify (#2004) 2020-09-07 14:23:47 +02:00
mattkurz
08fb5cbab2 Fix typo in generators docs 2020-09-07 13:02:38 +02:00
Gregory Bond
5ad1a4fe61 Issue 1992: Better c++17 std::byte detection
This fixed buld errors for ubuntu-16 + clang and similar situations
where C++17 is supported in the compiler but not the default
C++ standard library.
2020-09-07 13:01:51 +02:00
Karthik Nishanth
2c1c02f7e7 Update ParseAndAddCatchTests.cmake 2020-09-07 13:00:55 +02:00
Martin Jeřábek
8851e779cf console colour: fix unintended colouring of user's stderr on POSIX
At some places, the colour reset code is printed after a newline.
Since the default output buffering to console is line-based, the reset
code is not actually written out. If messages from user code are printed
to stderr (different stream, same console), they are printed before
the colour reset code, and thus they are coloured.

Explicitly flushing the stream after writing the colour escape code solves
this.
2020-09-07 11:53:45 +02:00
Martin Hořeňovský
2d4f8ac8e6 Disable problematic test on Windows 2020-09-06 20:41:58 +02:00
Martin Hořeňovský
9155a9ff20 Fix -Wparentheses issue on g++9 2020-09-06 16:02:01 +02:00
Martin Hořeňovský
cc18bd719d Sweep out Wshadow
Most of the changes are completely pointless renaming of constructor
arguments so that they do not use the same name as the type members,
but 🤷

Closes #2015
2020-09-06 13:11:42 +02:00
Martin Hořeňovský
90aeffb97d Add standardized copyright notice + SPDX identifier to source files
This should also be done for test files, but that has lower priority.
2020-08-30 15:43:45 +02:00
Martin Hořeňovský
c2453c2bf8 Remove the rest of old-style copyright headers 2020-08-30 14:45:05 +02:00
Martin Hořeňovský
a822cb9717 Standardize include guard patterns to FILE_NAME_EXTENSION_INCLUDED
This commit also strips the old copyright comment header in touched
files, as those will also be replaced with a more standardized and
machine-friendly version.
2020-08-30 14:09:27 +02:00
Martin Hořeňovský
c26693df23 Fix doccomment on shouldShowDuration reporter helper 2020-08-29 19:55:17 +02:00
Martin Hořeňovský
33ad1ee2ac Split EventListener base from streaming_base.hpp
The base was also renamed from `TestEventListenerBase` to
`EventListenerBase`, and modified to derive directly from the
reporter interface, rather than deriving from `StreamingReporterBase`.
2020-08-29 19:09:54 +02:00
Martin Hořeňovský
f9fdc96cbf Push LazyStat down into catch_reporter_streaming_base.hpp 2020-08-24 11:27:21 +02:00
Martin Hořeňovský
360b82620e Refactor serializeFilters implementation 2020-08-24 10:19:26 +02:00
Martin Hořeňovský
2a8e317cfb Split various reporter helpers out from streaming_base.hpp
Due to also adding a new TU, there is no improvement to the
compilation times of the static library, but it improves the
compilation times of consumer's reporter TUs.
2020-08-24 10:19:24 +02:00
Martin Hořeňovský
6a08d401aa Split out ReporterFactory out of catch_interfaces_reporter.hpp 2020-08-23 22:35:01 +02:00
Martin Hořeňovský
9677df6d8b Split IReporterRegistry into its own header file
Doing this removes `<map>` from the include set of the base reporter
interface, and thus from bunch more TUs. This provides about 1.5%
improvements in the debug build of the static library, and 1% in
release build.
2020-08-23 21:01:04 +02:00
Martin Hořeňovský
ed7eaf2df3 Split catch_reporter_bases.hpp into two separate headers
Each of the two reporter bases now has its own header file, and
cpp file. Even though this adds another TU to the compilation,
the total CPU time taken by compilation is reduced by about 1%
for debug build and ~0.5% for optimized build of the main library.
(The improvement would be roughly doubles without splitting the TUs,
but the maintainability hit is not worth it.)

The code size of the static library build has also somewhat decreased.

Follow up: Introduce combined TU for reporters, and further split
apart the catch_reporter_streaming_base.hpp header into its
constituent parts, as it still contains a whole bunch of other stuff.
2020-08-23 07:30:26 +02:00
Martin Hořeňovský
24559493bf Anchor some Clara vtables into the cpp file
Anchoring the vtables does 2 things
1) Fixes some instances of `-Wweak-vtables`
2) Decreases code size and linker pressure

However, there are still some unanchored ones, and thus we have
to keep suppressing `-Wweak-vtables` warning for Clang.
2020-08-20 23:16:11 +02:00
Martin Hořeňovský
7500ad1ffd Turn Clara::Detail::ResultBase::Type into its own enum class 2020-08-20 22:59:06 +02:00
Martin Hořeňovský
1a97af45f1 Cleanup some stuff found by MSVC's /analyze
* Added some missing `noexcept`s on custom destructors.
* Fixed `std::move` being called on a const-reference.
* Initialized `ScopedMessage::m_moved` in class definition, instead
of doing so in constructors explicitly.
* Turned some `enum`s into `enum class`es.
* Initialized `StreamingReporterBase::currentTestCaseInfo` in class
definition.
* Some cleanups in SelfTest code.
2020-08-20 20:42:21 +02:00
Martin Hořeňovský
2e480b6e56 Use StringRef for CasedString::caseSensitivitySuffix return type 2020-08-18 21:02:34 +02:00
Martin Hořeňovský
f16be402f7 Make XmlEncoding tests slightly more efficient 2020-08-18 21:02:31 +02:00
Martin Hořeňovský
e418e75c74 Turn CaseSensitive into proper enum class 2020-08-18 21:02:30 +02:00
Martin Hořeňovský
6a46b344c0 Split out CaseSensitive enum out of catch_common.hpp
Only very few places actually use it, so there is no need to have
it be central to everywhere.
2020-08-18 21:02:25 +02:00
Martin Hořeňovský
e7eb749815 Split the NonCopyable helper out of catch_common.hpp 2020-08-18 21:02:23 +02:00
Martin Hořeňovský
c1bb699d45 Remove superfluous catch_reporter_registrars.hpp includes from reporters
The includes were an artifact of the old design where the built-in
reporter TUs used autoregistration to add the reporters to the
factory. Because the current design is to add them explicitly in
the central reporter factory TU, the includes are useless.
2020-08-14 09:33:53 +02:00
Martin Hořeňovský
bf1e902ca1 Remove <algorithm> include from catch_commandline.cpp 2020-08-14 09:33:42 +02:00
Martin Hořeňovský
49b55d53e4 Prune includes in catch_run_context.cpp 2020-08-14 09:33:28 +02:00
Martin Hořeňovský
07fb96d42c Don't include catch_random_number_generator.hpp in catch_session.cpp 2020-08-14 09:33:16 +02:00
Martin Hořeňovský
34d9a588bb Cleanup includes in catch_list.cpp 2020-08-14 09:32:59 +02:00
Martin Hořeňovský
05d7014e75 ApproxMacher uses Approx's overloads for double directly
This saves a tiny little bit of compilation times when the
`ApproxMatcher` is used and `epsilon`, `margin`, or `scale` are
used to customize its behaviour.
2020-08-11 19:07:37 +02:00
Martin Hořeňovský
aa28a917cb Turn UseColour into enum class 2020-08-11 17:47:54 +02:00
Martin Hořeňovský
b0531404e4 Turn RunTests into TestRunOrder enum class 2020-08-11 17:47:00 +02:00
Martin Hořeňovský
60cc4c293d Turn ShowDurations into an enum class 2020-08-11 15:03:37 +02:00
Martin Hořeňovský
6dc8345261 IStreamingReporter::list* only uses IConfig instead of full Config
As the full `Config` is not needed, the TUs implementing the `list*`
functions can require the less heavy header `catch_interfaces_config.hpp`
instead of the much heavier `catch_config.hpp`.

This commit also fixes up some other TUs that include `Config`,
while using just `IConfig`, to cleanup the includes further.
2020-08-11 09:39:00 +02:00
Martin Hořeňovský
24b83edf8a Clara cleanups
* Clara is now split between a header and a cpp file.
* Removed the deprecated `+` and `+=` operators for composing
  a parser.
* Renamed `clara` and `detail` namespaces to be inline with the
rest of Catch2 (they are now `Clara` and `Detail` respectively).
* Taken most of user-exposed types out of the `Detail` namespace
completely (instead of using `using` directives to bring them into
the outer namespace).
2020-08-10 20:30:40 +02:00
Martin Hořeňovský
b824d06844 Fixup missing transitive includes
Found after updating to newest MSVC...
2020-08-10 20:28:14 +02:00
Martin Hořeňovský
e7aa432850 Split TextFlow out from Clara
Now that it has its own header, various reporter TUs that want to
format text do not have to also include Clara. Together with
outlining implementations from a header into a separate TU, this
has noticeably improved the compilation times of the testing impl.

As part of this split, I also implemented some improvements to the
TextFlow code in comparison to the upstream code. These are:

* Replaced the `Spacer` type with a free function that constructs
special `Column` that does the same thing.
* Generic performance improvements, such as eliminating needless
allocations, reserving space in needed allocations, and using smarter
algorithms in some places.
* Because `Column` only ever stored 1 string in its vector, it now
holds the string directly instead.
2020-08-03 23:40:14 +02:00
Martin Hořeňovský
81aa2d5582 Replace ostringstream with ReusableStringStream in ConsoleReporter 2020-08-03 22:59:43 +02:00
Martin Hořeňovský
9d591f19ff Allow explicitly setting ReusableStringStream's serialized data 2020-08-03 22:59:09 +02:00
Martin Hořeňovský
a4ac07d104 Split out default CATCH_CONFIG_CONSOLE_WIDTH into its own header
This means that code that uses it no longer has to include all of
catch_config.hpp, which seems to provide significant reduction in
size of unoptimized compilation of the final static library.
2020-08-03 22:57:43 +02:00
Martin Hořeňovský
8b0845b1a2 Split catch_config.hpp out of reporter bases 2020-07-30 08:46:07 +02:00
Martin Hořeňovský
c5037743e6 Avoid copying SectionInfo while construction Section 2020-07-29 21:51:10 +02:00
Martin Hořeňovský
9d6ac62aff Remove unused member from Section 2020-07-29 21:51:08 +02:00
Martin Hořeňovský
ed0ea30149 Apply some IWYU suggestions
This is not nearly all of them, because IWYU does not support the
way Catch2 manages includes -- it expects that non-system includes
are done using `#include "foo/bar/baz.hpp"`, while Catch2 uses
`<foo/bar/baz.hpp>`. This causes trouble, because IWYU suggests
removing every single internal header, and then adding them again,
but using `""` in the include directive... the resulting suggestions
cannot be used without a lot of manual work, as they are largely
bogus.

For bonus points, IWYU also _loves_ to suggest kinda-random stdlib
headers for `size_t` and similar. Still, the resulting inclusion
graph is somewhat better than it was before.
2020-07-29 21:51:05 +02:00
Martin Hořeňovský
35098a62d8 Sanitize includes in random_number_generator.cpp
I have no idea how those got there, but now its dependency graph
is much saner.
2020-07-29 14:58:14 +02:00
Martin Hořeňovský
ba57c17310 Remove some extraneous includes from run_context.hpp 2020-07-29 14:54:55 +02:00
Martin Hořeňovský
4e0af77e29 Push includes for TestCaseInfo down into reporter TUs 2020-07-29 14:43:05 +02:00
Martin Hořeňovský
eaf7113fd3 Cleanup stdlib includes in reporter_bases.hpp 2020-07-29 12:50:51 +02:00
Martin Hořeňovský
d090074da7 Devirtualize handling of ReporterPreferences
The new scheme is that there is one protected member instance of
`ReporterPreferences` in the `IStreamingReporter` base class,
and derived classes can modify it to express their own preferences.

Retrieving the preferences is now a non-virtual operation, which
makes it much cheaper to read them frequently. Previously, we
avoided doing so by caching the preferences in another variable,
but we still read them at least once per test case run.
2020-07-28 09:24:57 +02:00
Martin Hořeňovský
d218d6f9e2 Avoid recalculating string-literal size on root tracker construction
This is a tiiiiiiny performance optimization, but it's free.
2020-07-27 20:08:44 +02:00
Martin Hořeňovský
ef92178058 Devirtualize nested tracker handling 2020-07-27 19:53:54 +02:00
Martin Hořeňovský
125d4b4164 Devirtualize test case registration 2020-07-27 08:49:49 +02:00
Martin Hořeňovský
c9b4867441 Move some impls of StreamingReporterBase members to .cpp file 2020-07-27 08:46:35 +02:00
Martin Hořeňovský
258cac63f8 Move impls of CumulativeReporterBase member functions to .cpp file
`catch_reporter_bases.hpp` turned out fairly expensive for parsing
when building the main library, and the significant amount of code
in headers likely doesn't help. Since the reason it is in the header
is legacy from CRTP reporter bases, moving as much of the
implementations to the .cpp file is free compilation perf.
2020-07-27 07:32:21 +02:00
Martin Hořeňovský
5f6990d746 Only start Section's timer if the duration will be used
This is a small potential runtime optimization on systems with
virtual syscalls, and a significant runtime optimization on systems
without.
2020-07-26 21:33:49 +02:00
Martin Hořeňovský
5ca68829e1 Refactor how shortcircuiting of old style matchers is tested 2020-07-26 21:31:29 +02:00
Martin Hořeňovský
ac54ba7e12 Add test for shortcircuiting behaviour of generic matcher combinators 2020-07-26 21:24:05 +02:00
Reinhold Gschweicher
95c0c88d84 Fix CMake add test helper for CMake 3.18.0
With CMake 3.18.0 the `add_test(NAME)` handling changed. The escaped
double quotes confuse the new call. Work around this upstream change.

fixes: https://github.com/catchorg/Catch2/issues/1984
2020-07-26 15:15:50 +02:00
Martin Hořeňovský
6efeecc179 Cherry-pick doc updates for v2.13.0 2020-07-26 15:14:38 +02:00
Martin Hořeňovský
6b3c563c38 Remove pointless CompactReporter::getPreferences override 2020-07-26 14:07:19 +02:00
Martin Hořeňovský
a004423c7f Improve documentation for --min-duration 2020-07-26 14:06:52 +02:00
Martin Hořeňovský
4b344f11ea Document GENERATE's new usage between SECTIONs 2020-07-26 14:06:34 +02:00
Gavin S
87d0197cbd Update catch_reporter_tap.hpp
TAP format requires all results to be reported.
Removed extraneous preferences function (handled by parent)
Incorporated fix from 3d9e7db2e0
Simplified total printing
2020-07-26 14:00:10 +02:00
Martin Hořeňovský
4565b826cf Modify generator tracking to allow GENERATEs between SECTIONs
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-26 11:35:06 +02:00
Martin Hořeňovský
250d9b9c72 Fix how testRandomOrder.py builds tag arguments 2020-07-26 10:51:55 +02:00
Martin Hořeňovský
90d6fd849e Increase tolerances in --min-duration tests
The underpowered and oversubscribed CI servers are hell.
2020-07-26 10:48:03 +02:00
Martin Hořeňovský
13917c44b4 --min-duration is overriden by -d no 2020-07-26 10:48:01 +02:00
Martin Hořeňovský
e6d947f6d4 Refactor tests for duration reporting threshold 2020-07-26 10:47:58 +02:00
John Bytheway
80b0d6975c Add --min-duration option
A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.

When there are many tests, this produces output spam which makes it hard
to find the test failure output.  Nevertheless, it is helpful to be
informed of tests which are unusually slow.

Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed.  This allows slow
tests to be visible without mentioning every test.
2020-07-26 10:47:53 +02:00
Eric Huber
36131f7ffa Escaping literal "*" ("times") to fix markdown 2020-07-24 22:36:58 +02:00
Martin Hořeňovský
f52018205d Cherry pick release notes for v2.12.4 2020-07-24 22:36:13 +02:00
Martin Hořeňovský
b32d2fa016 Update FUNDING file (-Patreon, +PayPal) 2020-07-24 22:35:17 +02:00
George Rhoten
a25c1a24af Fix for macOS on ARM 2020-07-24 22:34:22 +02:00
Martin Hořeňovský
e28018c659 Cherry pick release notes for v2.12.3 2020-07-24 22:33:59 +02:00
Martin Hořeňovský
2a25a267ea Cherry pick release notes for v2.12.2 2020-07-24 22:33:26 +02:00
Ryan Pavlik
7f58840163 Add OverallResultsCases element to XML reporter 2020-07-24 22:30:28 +02:00
Martin Hořeňovský
3b0f8c7ff0 Replace a TODO comment in examples 2020-07-22 21:49:58 +02:00
Richard Ash
2840ce1e70 Add an example of using GENERATE(table())
There are some examples on issue #850 of using this feature, but they
are not easily found from the documentation. Adding them here as an
example makes them more findable and ensures they keep working if the
API changes.
2020-07-22 21:49:54 +02:00
Richard Ash
ed9be5a00b Add notes on compiling the examples.
This took me some time to figure out so document for others.
2020-07-22 21:43:53 +02:00
offa
273111d1a6 Clang-format configuration added.
Some notes on the configuration options chosen:

* We want `AllowShortEnumsOnASingleLine` set to `false`, but that
option is clang-format-11 and up, which is not out yet.
* `IndentPPDirectives` is currently inconsistent, but `AfterHash`
is the preferred style in new code.
* `NamespaceIndentation` is a mess, but `All` is closer to the effect
we want than `Inner`.
* `SpacesInParentheses` set to `true` is not ideal due to it also
introducing extra spaces in preprocessor expressions, but using it
is much closer to the current style than not.

All in all, using this setting globally would reformat pretty much
every line of code in the codebase, but it is as close as possible
to the bespoke style currently used. Still, it should only be used
on the diffs.

Closes #1182
2020-07-22 21:43:30 +02:00
Richard Ash
a862924601 Make scripts/updateDocumentToC.py executable.
On systems where the file system has excute permissions, this script was
not marked as executable in a clean git checkout and so could be run
without first changing the permissions. Fixed by setting the relevant
git flag.
2020-07-22 21:43:16 +02:00
Martin Hořeňovský
0e77adee05 Add explicit test for shortcircuiting behaviour of combined matchers 2020-07-22 21:36:14 +02:00
Martin Hořeňovský
b74996a29c Devirtualize NameAndLocation query on trackers 2020-07-22 21:23:12 +02:00
Martin Hořeňovský
de53773e46 Fix copy paste error in 7-arg TEMPLATE_TEST_CASE_SIG implementation
Closes #1954
2020-07-22 21:18:52 +02:00
Martin Hořeňovský
314bb7e632 Clarify documentation about nested generators
Closes #1947
2020-07-22 21:18:49 +02:00
Matthias Blankertz
9221a6ff65 Hide std::exception_ptr and friends if exceptions disabled
Some compilers, e.g. the Green Hills C++ compiler, react badly to the
appearance of std::exception_ptr, std::current_exception,
std::rethrow_exception and std::uncaught_exception(s). To allow usage of
Catch2 with these compilers when exceptions are disabled, hide the usage
of std::exception_ptr etc. when compiling with
CATCH_CONFIG_DISABLE_EXCEPTIONS.
2020-07-22 21:18:47 +02:00
Martin Hořeňovský
657ebf5db2 Replace stray tabs with spaces 2020-07-22 20:46:58 +02:00
Martin Hořeňovský
480f3f418b Improved generator tracking
* Successive executions of the same `GENERATE` macro (e.g. because
of a for loop) no longer lead to multiple nested generators.
* The same line can now contain multiple `GENERATE` macros without
issues.

Fixes #1913
2020-07-22 20:44:22 +02:00
bogdasar1985
3ceaad7d66 fixing UB 2020-07-22 17:17:33 +02:00
Martin Hořeňovský
5c502320e8 Remove obsolete comment in UnorderedEquals vector matcher 2020-07-12 21:28:50 +02:00
Martin Hořeňovský
f3fe2dcb11 Add status attribute to JUnit's section reporting
This brings our output inline with GTest's. We do not handle skipped
tests properly, but that should be currently less important than
having the attribute exist with proper value for non-skipped tests.

Thanks @joda-01.

Closes #1899
2020-07-12 21:22:15 +02:00
Natsu
8b5f6e26d3 Fix compilation failure when using libstdc++10 (#1929)
The issue is caused by deleted `std::__detail::begin` declared in `bits/iterator_concepts.h`. This would be found by ADL, and because it is deleted, compilation would fail. This change makes it so that we SFINAE on `begin(std::declval<T>())` and `end(std::declval<T>())` being well-formed.
2020-07-12 21:01:49 +02:00
Billy Robert O'Neal III
c24f7e5b34 Fix invalid isspace call detected by PREfast
D:\vcpkg\toolsrc\include\catch2\catch.hpp(11285): warning C6330: 'char' passed as _Param_(1) when 'unsigned char' is required in call to 'isspace'.
D:\vcpkg\toolsrc\include\catch2\catch.hpp(11288): warning C6330: 'char' passed as _Param_(1) when 'unsigned char' is required in call to 'isspace'.

ISO/IEC 9899:2011:
"7.4 Character handling <ctype.h>"/1
[...] In all cases the argument is an int, the value of which shall be
representable as an unsigned char or shall equal the value of the macro
EOF. If the argument has any other value, the behavior is undefined.

This means if isspace was passed a character like ñ it could corrupt
memory without the static_cast to treat it as a positive value after
integral promotion (and C libraries commonly use the int index supplied
as a key into a table which result in out of bounds access if the
resulting int is negative).
2020-07-12 21:00:25 +02:00
Martin Hořeňovský
7dae3efad2 Silence clang-tidy's hicpp-vararg (alias of coreguidelines vararg)
Ideally, clang-tidy would be smart that if one alias of a warning
is suppressed, then the other one is suppressed as well, but as of
right now, it isn't. This means that for now we have to suppress
both aliases of this warning. Opened upstream issue to fix this:
https://bugs.llvm.org/show_bug.cgi?id=45859

Obviously, ideally clang-tidy would also not warn that we are calling
a vararg function when it is an unevaluated magic builtin, but that
also is not happening right now and I opened an issue for it:
https://bugs.llvm.org/show_bug.cgi?id=45860

Closes #1921
2020-07-12 20:59:29 +02:00
Corentin Jabot
a71721801e Remove extra semi-colons 2020-07-10 13:40:42 +02:00
Corentin Jabot
2cd5e70012 Silence a -Wsign-conversio warning in Clara under GCC 2020-07-09 12:24:25 +02:00
Martin Hořeňovský
392e44ec21 Minor refactoring of CompactReporter 2020-05-31 22:34:37 +02:00
Martin Hořeňovský
317145514f Add op+(StringRef, StringRef) -> std::string 2020-05-31 22:30:41 +02:00
Martin Hořeňovský
f2b9508081 Remove last vestiges of shared ptr use with IConfig 2020-05-31 15:25:14 +02:00
Martin Hořeňovský
1d1ccf8f3c Replace all uses of std::unique_ptr with Catch::Detail::unique_ptr
Doing some benchmarking with ClangBuildAnalyzer suggests that
compiling Catch2's `SelfTest` spends 10% of the time instantiating
`std::unique_ptr` for some interface types required for registering
and running tests.

The lesser compilation overhead of `Catch::Detail::unique_ptr` should
significantly reduce that time.

The compiled implementation was also changed to use the custom impl,
to avoid having to convert between using `std::unique_ptr` and
`Catch::Detail::unique_ptr`. This will likely also improve the compile
times of the implementation, but that is less important than improving
compilation times of the user's TUs with tests.
2020-05-31 15:20:24 +02:00
Martin Hořeňovský
41bbaa6d57 Implement a simplified variant of std::unique_ptr<T>
This simplified variant supports only a subset of the functionality
in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports
single element pointer (no array support) with default deleter.

By removing the support for custom deleters, we also avoid requiring
significant machinery to support EBO, speeding up instantiations of
`unique_ptr<T>` significantly. Catch2 also currently does not need
to support `unique_ptr<T[]>`, so that is not supported either.
2020-05-31 15:08:47 +02:00
Martin Hořeňovský
66ab942903 Refactor StreamEndStop slightly 2020-05-30 15:44:46 +02:00
Valentin Tolmer
d05a8e2e24 Add Bazel support for the v3 branch 2020-05-26 14:49:49 +02:00
Martin Hořeňovský
1356788ea8 Avoid using std::result_of when std::invoke_result is available 2020-05-21 22:58:33 +02:00
Martin Hořeňovský
21d284df34 Session now holds Config in unique_ptr instead of shared_ptr 2020-05-20 20:25:51 +02:00
Martin Hořeňovský
668454b36b Do not share ownership of Config in RunContext 2020-05-20 20:25:49 +02:00
Martin Hořeňovský
458241cc90 Do not use shared_ptr<Config> when listing things 2020-05-20 20:25:48 +02:00
Martin Hořeňovský
fa160cf3f2 Keep reporter pointer around in TestGroup 2020-05-20 20:25:45 +02:00
Martin Hořeňovský
a17b9f754a Remove unused function 2020-05-20 20:25:43 +02:00
Martin Hořeňovský
c2852c9944 Remove some uses of std::shared_ptr 2020-05-20 20:25:40 +02:00
Martin Hořeňovský
4394d3ae65 Translate exceptions by const reference instead of plain ref 2020-05-20 08:15:27 +02:00
Martin Hořeňovský
4b2f1da02a Split CATCH_TRANSLATE_EXCEPTION into its own header
As far as I know, only a few users actually use it, but these changes
allow us to avoid including a surprising amount of code in the main
compilation path.
2020-05-20 08:15:11 +02:00
Martin Hořeňovský
0c6fda6e7d Cleanup benchmark headers a tiny bit 2020-05-20 08:00:24 +02:00
Martin Hořeňovský
bad8b7c866 Fix make_shared used instead of make_unique 2020-05-20 07:15:21 +02:00
Martin Hořeňovský
964303706a Fixup catch_discover_tests
Fixed its use of removed `--list-test-names-only` switch, and also
updated the return code check to reflect changes in v3.

Closes #1933
2020-05-20 07:14:06 +02:00
Martin Hořeňovský
54882dbb11 Don't include <tuple> everywhere in benchmarks 2020-05-20 07:09:15 +02:00
Martin Hořeňovský
b4a61cfd29 Remove superfluous overload of operator== 2020-05-18 21:32:08 +02:00
Martin Hořeňovský
d86834e5b5 Push down #include <ostream> to .cpp files 2020-05-18 21:31:41 +02:00
Martin Hořeňovský
39e093021c Remove some superfluous includes 2020-05-18 20:55:21 +02:00
Martin Hořeňovský
e867ce7769 Fix missing backticks on release notes
Closes #1932
2020-05-15 10:54:20 +02:00
452 changed files with 121450 additions and 27409 deletions

4
.bazelrc Normal file
View File

@@ -0,0 +1,4 @@
build:gcc9 --cxxopt=-std=c++2a
build:clang13 --cxxopt=-std=c++17
build:vs2019 --cxxopt=/std:c++17
build:vs2022 --cxxopt=/std:c++17

25
.clang-format Normal file
View File

@@ -0,0 +1,25 @@
---
AccessModifierOffset: '-4'
AlignEscapedNewlines: Left
AllowAllConstructorInitializersOnNextLine: 'true'
BinPackArguments: 'false'
BinPackParameters: 'false'
BreakConstructorInitializers: AfterColon
ConstructorInitializerAllOnOneLineOrOnePerLine: 'true'
DerivePointerAlignment: 'false'
FixNamespaceComments: 'true'
IncludeBlocks: Regroup
IndentCaseLabels: 'false'
IndentPPDirectives: AfterHash
IndentWidth: '4'
Language: Cpp
NamespaceIndentation: All
PointerAlignment: Left
SpaceBeforeCtorInitializerColon: 'false'
SpaceInEmptyParentheses: 'false'
SpacesInParentheses: 'true'
Standard: Cpp11
TabWidth: '4'
UseTab: Never
...

View File

@@ -26,8 +26,8 @@ class BuilderSettings(object):
""" Set Catch2 repository to be used on upload.
The upload server address could be customized by env var
CONAN_UPLOAD. If not defined, the method will check the branch name.
Only master or CONAN_STABLE_BRANCH_PATTERN will be accepted.
The master branch will be pushed to testing channel, because it does
Only devel or CONAN_STABLE_BRANCH_PATTERN will be accepted.
The devel branch will be pushed to testing channel, because it does
not match the stable pattern. Otherwise it will upload to stable
channel.
"""

View File

@@ -1,19 +1,12 @@
cmake_minimum_required(VERSION 3.2.0)
project(test_package CXX)
# We set it only for the convenience of calling the executable
# in the package test function
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/bin)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELWITHDEBINFO ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_MINSIZEREL ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
include("${CMAKE_BINARY_DIR}/conanbuildinfo.cmake")
conan_basic_setup()
find_package(Catch2 REQUIRED CONFIG)
add_executable(${PROJECT_NAME} test_package.cpp)
# Note: Conan 1.21 doesn't support granular target generation yet.
# The Main library would be included into the unified target.
# It's controlled by the `with_main` option in the recipe.
target_link_libraries(${PROJECT_NAME} Catch2::Catch2)
target_link_libraries(${PROJECT_NAME} Catch2::Catch2WithMain)
set_target_properties(${PROJECT_NAME} PROPERTIES CXX_STANDARD 14)

View File

@@ -6,7 +6,7 @@ import os
class TestPackageConan(ConanFile):
settings = "os", "compiler", "build_type", "arch"
generators = "cmake_find_package_multi"
generators = "cmake_find_package_multi", "cmake"
def build(self):
cmake = CMake(self)
@@ -14,6 +14,7 @@ class TestPackageConan(ConanFile):
cmake.build()
def test(self):
assert os.path.isfile(os.path.join(self.deps_cpp_info["catch2"].rootpath, "licenses", "LICENSE.txt"))
assert os.path.isfile(os.path.join(
self.deps_cpp_info["catch2"].rootpath, "licenses", "LICENSE.txt"))
bin_path = os.path.join("bin", "test_package")
self.run("%s -s" % bin_path, run_environment=True)

3
.github/FUNDING.yml vendored
View File

@@ -1 +1,2 @@
patreon: horenmar
github: "horenmar"
custom: "https://www.paypal.me/horenmar"

View File

@@ -0,0 +1,94 @@
# The builds in this file are more complex (e.g. they need custom CMake
# configuration) and thus are unsuitable to the simple build matrix
# approach used in simple-builds
name: Linux builds (complex)
on: [push, pull_request]
jobs:
build:
name: ${{matrix.build_description}}, ${{matrix.cxx}}, C++${{matrix.std}} ${{matrix.build_type}}
runs-on: ubuntu-20.04
strategy:
matrix:
# We add builds one by one in this case, because there are no
# dimensions that are shared across the builds
include:
# Single surrogate header build
- cxx: clang++-10
build_description: Surrogates build
build_type: Debug
std: 14
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_SURROGATES=ON
# Extras and examples with gcc-7
- cxx: g++-7
build_description: Extras + Examples
build_type: Debug
std: 14
other_pkgs: g++-7
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
- cxx: g++-7
build_description: Extras + Examples
build_type: Release
std: 14
other_pkgs: g++-7
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
# Extras and examples with Clang-10
- cxx: clang++-10
build_description: Extras + Examples
build_type: Debug
std: 17
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
- cxx: clang++-10
build_description: Extras + Examples
build_type: Release
std: 17
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
# Configure tests with Clang-10
- cxx: clang++-10
build_description: CMake configuration tests
build_type: Debug
std: 14
other_pks: clang-10
cmake_configurations: -DCATCH_ENABLE_CONFIGURE_TESTS=ON
steps:
- uses: actions/checkout@v2
- name: Prepare environment
run: sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
- name: Configure build
working-directory: ${{runner.workspace}}
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
cmake -Bbuild -H$GITHUB_WORKSPACE \
-DCMAKE_BUILD_TYPE=${{matrix.build_type}} \
-DCMAKE_CXX_STANDARD=${{matrix.std}} \
-DCMAKE_CXX_EXTENSIONS=OFF \
-DCATCH_DEVELOPMENT_BUILD=ON \
${{matrix.cmake_configurations}} \
-G Ninja
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: ninja
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2

View File

@@ -0,0 +1,97 @@
name: Linux builds (basic)
on: [push, pull_request]
jobs:
build:
name: ${{matrix.cxx}}, C++${{matrix.std}}, ${{matrix.build_type}}
runs-on: ubuntu-20.04
strategy:
matrix:
cxx:
# - g++-6
- g++-7
- g++-8
- g++-9
- g++-10
- clang++-6.0
- clang++-7
- clang++-8
- clang++-9
- clang++-10
build_type: [Debug, Release]
std: [14]
include:
# cannot be installed on ubuntu-20.04 be default?
# - cxx: g++-6
# other_pkgs: g++-6
- cxx: g++-7
other_pkgs: g++-7
- cxx: g++-8
other_pkgs: g++-8
- cxx: g++-9
other_pkgs: g++-9
- cxx: g++-10
other_pkgs: g++-10
- cxx: clang++-6.0
other_pkgs: clang-6.0
- cxx: clang++-7
other_pkgs: clang-7
- cxx: clang++-8
other_pkgs: clang-8
- cxx: clang++-9
other_pkgs: clang-9
- cxx: clang++-10
other_pkgs: clang-10
# Clang 6 + C++17
# does not work with the default libstdc++ version thanks
# to a disagreement on variant implementation.
# - cxx: clang++-6.0
# build_type: Debug
# std: 17
# other_pkgs: clang-6.0
# - cxx: clang++-6.0
# build_type: Release
# std: 17
# other_pkgs: clang-6.0
# Clang 10 + C++17
- cxx: clang++-10
build_type: Debug
std: 17
other_pkgs: clang-10
- cxx: clang++-10
build_type: Release
std: 17
other_pkgs: clang-10
steps:
- uses: actions/checkout@v2
- name: Prepare environment
run: sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
- name: Configure build
working-directory: ${{runner.workspace}}
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
cmake -Bbuild -H$GITHUB_WORKSPACE \
-DCMAKE_BUILD_TYPE=${{matrix.build_type}} \
-DCMAKE_CXX_STANDARD=${{matrix.std}} \
-DCMAKE_CXX_EXTENSIONS=OFF \
-DCATCH_DEVELOPMENT_BUILD=ON \
-G Ninja
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: ninja
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2

45
.github/workflows/mac-builds.yml vendored Normal file
View File

@@ -0,0 +1,45 @@
name: Mac builds
on: [push, pull_request]
jobs:
build:
runs-on: macos-10.15
strategy:
matrix:
cxx:
- g++-9
- clang++
build_type: [Debug, Release]
std: [14, 17]
include:
- build_type: Debug
examples: ON
extra_tests: ON
steps:
- uses: actions/checkout@v2
- name: Configure build
working-directory: ${{runner.workspace}}
env:
CXX: ${{matrix.cxx}}
CXXFLAGS: ${{matrix.cxxflags}}
# Note: $GITHUB_WORKSPACE is distinct from ${{runner.workspace}}.
# This is important
run: |
cmake -Bbuild -H$GITHUB_WORKSPACE \
-DCMAKE_BUILD_TYPE=${{matrix.build_type}} -DCMAKE_CXX_STANDARD=${{matrix.std}} \
-DCATCH_DEVELOPMENT_BUILD=ON -DCATCH_BUILD_EXAMPLES=${{matrix.examples}} \
-DCATCH_BUILD_EXTRA_TESTS=${{matrix.examples}}
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: make -j 2
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2

View File

@@ -0,0 +1,36 @@
name: Check header guards
on: [push, pull_request]
jobs:
build:
# Set the type of machine to run on
runs-on: ubuntu-20.04
steps:
- name: Checkout source code
uses: actions/checkout@v2
- name: Setup Dependencies
uses: actions/setup-python@v2
with:
python-version: '3.7'
- name: Install checkguard
run: pip install guardonce
- name: Check that include guards are properly named
run: |
wrong_files=$(checkguard -r src/catch2/ -p "name | append _INCLUDED | upper")
if [[ $wrong_files ]]; then
echo "Files with wrong header guard:"
echo $wrong_files
exit 1
fi
- name: Check that there are no duplicated filenames
run: |
./tools/scripts/checkDuplicateFilenames.py
- name: Check that all source files have the correct license header
run: |
./tools/scripts/checkLicense.py

4
.gitignore vendored
View File

@@ -28,3 +28,7 @@ Build
cmake-build-*
benchmark-dir
.conan/test_package/build
bazel-*
build-fuzzers
debug-build
.vscode

View File

@@ -1,207 +0,0 @@
language: cpp
dist: xenial
branches:
except:
- /dev-appveyor.*/
common_sources: &all_sources
- ubuntu-toolchain-r-test
- llvm-toolchain-xenial
- llvm-toolchain-xenial-3.8
- llvm-toolchain-xenial-3.9
- llvm-toolchain-xenial-4.0
- llvm-toolchain-xenial-5.0
- llvm-toolchain-xenial-6.0
- llvm-toolchain-xenial-7
- llvm-toolchain-xenial-8
matrix:
include:
# Clang builds
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.8']
env: COMPILER='clang++-3.8' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.8', 'lcov']
env: COMPILER='clang++-3.8' CPP14=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-3.9']
env: COMPILER='clang++-3.9' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-4.0']
env: COMPILER='clang++-4.0' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-5.0']
env: COMPILER='clang++-5.0' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0']
env: COMPILER='clang++-6.0' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-6.0', 'libstdc++-8-dev']
env: COMPILER='clang++-6.0' CPP17=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-7']
env: COMPILER='clang++-7' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-8']
env: COMPILER='clang++-8' CPP14=1
- os: linux
compiler: clang
addons:
apt:
sources: *all_sources
packages: ['clang-8', 'libstdc++-8-dev']
env: COMPILER='clang++-8' CPP17=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
# GCC builds
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-5']
env: COMPILER='g++-5' CPP14=1
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-6']
env: COMPILER='g++-6' CPP14=1
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-7', 'lcov']
env: COMPILER='g++-7' CPP14=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
- os: linux
compiler: gcc
addons:
apt:
sources: *all_sources
packages: ['g++-8']
env: COMPILER='g++-8' CPP17=1
# OSX Clang Builds
- os: osx
osx_image: xcode9.4
compiler: clang
env: COMPILER='clang++' CPP14=1
- os: osx
osx_image: xcode10.3
compiler: clang
env: COMPILER='clang++' CPP14=1
- os: osx
osx_image: xcode11.2
compiler: clang
env: COMPILER='clang++' CPP14=1
- os: osx
osx_image: xcode11.2
compiler: clang
env: COMPILER='clang++' CPP14=1 EXAMPLES=1 COVERAGE=1 EXTRAS=1
# Special builds, e.g. conan
- language: python
python:
- "3.7"
install:
- pip install conan-package-tools
env:
- CONAN_GCC_VERSIONS=8
- CONAN_DOCKER_IMAGE=conanio/gcc8
- CPP14=1
script:
- python .conan/build.py
before_script:
- export CXX=${COMPILER}
- cd ${TRAVIS_BUILD_DIR}
- |
if [[ ${CPP17} -eq 1 ]]; then
export CPP_STANDARD=17
elif [[ ${CPP14} -eq 1 ]]; then
export CPP_STANDARD=14
else
travis_terminate 4;
fi
# Use Debug builds for running Valgrind and building examples
- cmake -H. -BBuild-Debug -DCMAKE_BUILD_TYPE=Debug -Wdev -DCATCH_USE_VALGRIND=${VALGRIND} -DCATCH_BUILD_EXAMPLES=${EXAMPLES} -DCATCH_ENABLE_COVERAGE=${COVERAGE} -DCATCH_BUILD_EXTRA_TESTS=${EXTRAS} -DCMAKE_CXX_STANDARD=${CPP_STANDARD} -DCMAKE_CXX_STANDARD_REQUIRED=On -DCMAKE_CXX_EXTENSIONS=OFF -DCATCH_DEVELOPMENT_BUILD=ON
# Don't bother with release build for coverage build
- cmake -H. -BBuild-Release -DCMAKE_BUILD_TYPE=Release -Wdev -DCMAKE_CXX_STANDARD=${CPP_STANDARD} -DCMAKE_CXX_STANDARD_REQUIRED=On -DCMAKE_CXX_EXTENSIONS=OFF -DCATCH_DEVELOPMENT_BUILD=ON
script:
- cd Build-Debug
- make -j 2
- CTEST_OUTPUT_ON_FAILURE=1 ctest -j 2
# Coverage collection does not work for OS X atm
- |
if [[ "${TRAVIS_OS_NAME}" == "linux" ]] && [[ "${COVERAGE}" == "1" ]]; then
make gcov
make lcov
bash <(curl -s https://codecov.io/bash) -X gcov || echo "Codecov did not collect coverage reports"
fi
- # Go to release build
- cd ../Build-Release
- make -j 2
- CTEST_OUTPUT_ON_FAILURE=1 ctest -j 2

91
BUILD.bazel Normal file
View File

@@ -0,0 +1,91 @@
# Load the cc_library rule.
load("@rules_cc//cc:defs.bzl", "cc_library")
load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
expand_template(
name = "catch_user_config",
out = "catch2/catch_user_config.hpp",
substitutions = {
"#cmakedefine CATCH_CONFIG_ANDROID_LOGWRITE": "",
"#cmakedefine CATCH_CONFIG_BAZEL_SUPPORT": "#define CATCH_CONFIG_BAZEL_SUPPORT",
"#cmakedefine CATCH_CONFIG_NO_COLOUR_WIN32": "",
"#cmakedefine CATCH_CONFIG_COLOUR_WIN32": "",
"#cmakedefine CATCH_CONFIG_COUNTER": "",
"#cmakedefine CATCH_CONFIG_CPP11_TO_STRING": "",
"#cmakedefine CATCH_CONFIG_CPP17_BYTE": "",
"#cmakedefine CATCH_CONFIG_CPP17_OPTIONAL": "",
"#cmakedefine CATCH_CONFIG_CPP17_STRING_VIEW": "",
"#cmakedefine CATCH_CONFIG_CPP17_UNCAUGHT_EXCEPTIONS": "",
"#cmakedefine CATCH_CONFIG_CPP17_VARIANT": "",
"#cmakedefine CATCH_CONFIG_DISABLE_EXCEPTIONS_CUSTOM_HANDLER": "",
"#cmakedefine CATCH_CONFIG_DISABLE_EXCEPTIONS": "",
"#cmakedefine CATCH_CONFIG_DISABLE_STRINGIFICATION": "",
"#cmakedefine CATCH_CONFIG_DISABLE": "",
"#cmakedefine CATCH_CONFIG_ENABLE_ALL_STRINGMAKERS": "",
"#cmakedefine CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_ENABLE_PAIR_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_ENABLE_TUPLE_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER": "",
"#cmakedefine CATCH_CONFIG_EXPERIMENTAL_REDIRECT": "",
"#cmakedefine CATCH_CONFIG_FALLBACK_STRINGIFIER @CATCH_CONFIG_FALLBACK_STRINGIFIER@": "",
"#cmakedefine CATCH_CONFIG_FAST_COMPILE": "",
"#cmakedefine CATCH_CONFIG_GLOBAL_NEXTAFTER": "",
"#cmakedefine CATCH_CONFIG_NO_COUNTER": "",
"#cmakedefine CATCH_CONFIG_NO_CPP11_TO_STRING": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_BYTE": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_OPTIONAL": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_STRING_VIEW": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS": "",
"#cmakedefine CATCH_CONFIG_NO_CPP17_VARIANT": "",
"#cmakedefine CATCH_CONFIG_NO_GLOBAL_NEXTAFTER": "",
"#cmakedefine CATCH_CONFIG_NO_POSIX_SIGNALS": "",
"#cmakedefine CATCH_CONFIG_NO_USE_ASYNC": "",
"#cmakedefine CATCH_CONFIG_NO_WCHAR": "",
"#cmakedefine CATCH_CONFIG_NO_WINDOWS_SEH": "",
"#cmakedefine CATCH_CONFIG_NOSTDOUT": "",
"#cmakedefine CATCH_CONFIG_POSIX_SIGNALS": "",
"#cmakedefine CATCH_CONFIG_PREFIX_ALL": "",
"#cmakedefine CATCH_CONFIG_USE_ASYNC": "",
"#cmakedefine CATCH_CONFIG_WCHAR": "",
"#cmakedefine CATCH_CONFIG_WINDOWS_CRTDBG": "",
"#cmakedefine CATCH_CONFIG_WINDOWS_SEH": "",
"#cmakedefine CATCH_CONFIG_NO_ANDROID_LOGWRITE": "",
"@CATCH_CONFIG_DEFAULT_REPORTER@": "console",
"@CATCH_CONFIG_CONSOLE_WIDTH@": "80",
},
template = "src/catch2/catch_user_config.hpp.in",
)
# Generated header library, modifies the include prefix to account for
# generation path so that we can include <catch2/catch_user_config.hpp>
# correctly.
cc_library(
name = "catch2_generated",
hdrs = ["catch2/catch_user_config.hpp"],
include_prefix = ".", # to manipulate -I of dependenices
visibility = ["//visibility:public"],
)
# Static library, without main.
cc_library(
name = "catch2",
srcs = glob(
["src/catch2/**/*.cpp"],
exclude = ["src/catch2/internal/catch_main.cpp"],
),
hdrs = glob(["src/catch2/**/*.hpp"]),
includes = ["src/"],
linkstatic = True,
visibility = ["//visibility:public"],
deps = [":catch2_generated"],
)
# Static library, with main.
cc_library(
name = "catch2_main",
srcs = ["src/catch2/internal/catch_main.cpp"],
includes = ["src/"],
linkstatic = True,
visibility = ["//visibility:public"],
deps = [":catch2"],
)

View File

@@ -0,0 +1,77 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE_1_0.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
##
# This file contains options that are materialized into the Catch2
# compiled library. All of them default to OFF, as even the positive
# forms correspond to the user _forcing_ them to ON, while being OFF
# means that Catch2 can use its own autodetection.
#
# For detailed docs look into docs/configuration.md
macro(AddOverridableConfigOption OptionBaseName)
option(CATCH_CONFIG_${OptionBaseName} "Read docs/configuration.md for details" OFF)
option(CATCH_CONFIG_NO_${OptionBaseName} "Read docs/configuration.md for details" OFF)
endmacro()
macro(AddConfigOption OptionBaseName)
option(CATCH_CONFIG_${OptionBaseName} "Read docs/configuration.md for details" OFF)
endmacro()
set(_OverridableOptions
"ANDROID_LOGWRITE"
"BAZEL_SUPPORT"
"COLOUR_WIN32"
"COUNTER"
"CPP11_TO_STRING"
"CPP17_BYTE"
"CPP17_OPTIONAL"
"CPP17_STRING_VIEW"
"CPP17_UNCAUGHT_EXCEPTIONS"
"CPP17_VARIANT"
"GLOBAL_NEXTAFTER"
"POSIX_SIGNALS"
"USE_ASYNC"
"WCHAR"
"WINDOWS_SEH"
)
foreach(OptionName ${_OverridableOptions})
AddOverridableConfigOption(${OptionName})
endforeach()
set(_OtherConfigOptions
"DISABLE_EXCEPTIONS"
"DISABLE_EXCEPTIONS_CUSTOM_HANDLER"
"DISABLE"
"DISABLE_STRINGIFICATION"
"ENABLE_ALL_STRINGMAKERS"
"ENABLE_OPTIONAL_STRINGMAKER"
"ENABLE_PAIR_STRINGMAKER"
"ENABLE_TUPLE_STRINGMAKER"
"ENABLE_VARIANT_STRINGMAKER"
"EXPERIMENTAL_REDIRECT"
"FAST_COMPILE"
"NOSTDOUT"
"PREFIX_ALL"
"WINDOWS_CRTDBG"
)
foreach(OptionName ${_OtherConfigOptions})
AddConfigOption(${OptionName})
endforeach()
set(CATCH_CONFIG_DEFAULT_REPORTER "console" CACHE STRING "Read docs/configuration.md for details. The name of the reporter should be without quotes.")
set(CATCH_CONFIG_CONSOLE_WIDTH "80" CACHE STRING "Read docs/configuration.md for details. Must form a valid integer literal.")
# There is no good way to both turn this into a CMake cache variable,
# and keep reasonable default semantics inside the project. Thus we do
# not define it and users have to provide it as an outside variable.
#set(CATCH_CONFIG_FALLBACK_STRINGIFIER "" CACHE STRING "Read docs/configuration.md for details.")

View File

@@ -1,3 +1,11 @@
# Copyright Catch2 Authors
# Distributed under the Boost Software License, Version 1.0.
# (See accompanying file LICENSE_1_0.txt or copy at
# https://www.boost.org/LICENSE_1_0.txt)
# SPDX-License-Identifier: BSL-1.0
include(CheckCXXCompilerFlag)
function(add_cxx_flag_if_supported_to_targets flagname targets)
check_cxx_compiler_flag("${flagname}" HAVE_FLAG_${flagname})
@@ -45,7 +53,7 @@ function(add_warnings_to_targets targets)
"-Wglobal-constructors"
"-Wmissing-noreturn"
"-Wparentheses"
"-Wextra-semi-stmt"
"-Wextra-semi"
"-Wunreachable-code"
"-Wstrict-aliasing"
"-Wreturn-std-move"
@@ -62,6 +70,8 @@ function(add_warnings_to_targets targets)
"-Wcall-to-pure-virtual-from-ctor-dtor"
"-Wdeprecated-register"
"-Wsuggest-override"
"-Wshadow"
"-Wold-style-cast"
)
foreach(warning ${CHECKED_WARNING_FLAGS})
add_cxx_flag_if_supported_to_targets(${warning} "${targets}")
@@ -75,3 +85,13 @@ function(add_warnings_to_targets targets)
endif()
endif()
endfunction()
# Adds flags required for reproducible build to the target
# Currently only supports GCC and Clang
function(add_build_reproducibility_settings target)
# Make the build reproducible on versions of g++ and clang that supports -ffile-prefix-map
if(("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND NOT ${CMAKE_CXX_COMPILER_VERSION} VERSION_LESS 8) OR
("${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang" AND NOT ${CMAKE_CXX_COMPILER_VERSION} VERSION_LESS 10))
target_compile_options(${target} PRIVATE "-ffile-prefix-map=${CATCH_DIR}=.")
endif()
endfunction()

View File

@@ -7,4 +7,4 @@ Description: A modern, C++-native test framework for C++14 and above (links in d
Version: ${pkg_version}
Requires: catch2 = ${pkg_version}
Cflags: -I${includedir}
Libs: -L${libdir} -lCatch2WithMain
Libs: -L${libdir} -lCatch2Main

View File

@@ -1,3 +1,5 @@
prefix=@CMAKE_INSTALL_PREFIX@
exec_prefix=${prefix}
includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@
libdir=@CMAKE_INSTALL_FULL_LIBDIR@

View File

@@ -4,18 +4,24 @@ cmake_minimum_required(VERSION 3.5)
# disable testsuite in that case
if(NOT DEFINED PROJECT_NAME)
set(NOT_SUBPROJECT ON)
else()
set(NOT_SUBPROJECT OFF)
endif()
option(CATCH_INSTALL_DOCS "Install documentation alongside library" ON)
option(CATCH_INSTALL_EXTRAS "Install extras alongside library" ON)
option(CATCH_INSTALL_EXTRAS "Install extras (CMake scripts, debugger helpers) alongside library" ON)
option(CATCH_DEVELOPMENT_BUILD "Build tests, enable warnings, enable Werror, etc" OFF)
include(CMakeDependentOption)
cmake_dependent_option(CATCH_BUILD_TESTING "Build the SelfTest project" ON "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_EXAMPLES "Build code examples" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_EXTRA_TESTS "Build extra tests" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_FUZZERS "Build fuzzers" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_COVERAGE "Generate coverage for codecov.io" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_WERROR "Enables Werror during build" ON "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_SURROGATES "Enable generating and building surrogate TUs for the main headers" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_CONFIGURE_TESTS "Enable CMake configuration tests. WARNING: VERY EXPENSIVE" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
# Catch2's build breaks if done in-tree. You probably should not build
# things in tree anyway, but we can allow projects that include Catch2
@@ -24,6 +30,11 @@ if (CMAKE_BINARY_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
message(FATAL_ERROR "Building in-source is not supported! Create a build dir and remove ${CMAKE_SOURCE_DIR}/CMakeCache.txt")
endif()
if(CMAKE_VERSION VERSION_GREATER 3.8)
# Enable IPO for CMake versions that support it
cmake_policy(SET CMP0069 NEW)
endif()
project(Catch2 LANGUAGES CXX VERSION 3.0.0)
@@ -31,7 +42,10 @@ project(Catch2 LANGUAGES CXX VERSION 3.0.0)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/CMake")
include(GNUInstallDirs)
include(CMakePackageConfigHelpers)
include(CTest)
include(CatchConfigOptions)
if(CATCH_DEVELOPMENT_BUILD)
include(CTest)
endif()
# This variable is used in some subdirectories, so we need it here, rather
# than later in the install block
@@ -73,15 +87,15 @@ if(CATCH_BUILD_EXTRA_TESTS)
add_subdirectory(tests/ExtraTests)
endif()
if(CATCH_BUILD_FUZZERS)
add_subdirectory(fuzzing)
endif()
if (CATCH_DEVELOPMENT_BUILD)
add_warnings_to_targets("${CATCH_WARNING_TARGETS}")
endif()
#option(CATCH_USE_VALGRIND "Perform SelfTests with Valgrind" OFF)
#option(CATCH_ENABLE_WERROR "Enable all warnings as errors" ON)
#
#set_property(GLOBAL PROPERTY USE_FOLDERS ON)
#
#
@@ -99,27 +113,11 @@ if (NOT_SUBPROJECT)
${CATCH_CMAKE_CONFIG_DESTINATION}
)
## TODO: Catch2 main target?
## Install some cpp file as well?
# By default, FooConfigVersion is tied to architecture that it was
# generated on. Because Catch2 is header-only, it is arch-independent
# and thus Catch2ConfigVersion should not be tied to the architecture
# it was generated on.
#
# CMake does not provide a direct customization point for this in
# `write_basic_package_version_file`, but it can be accomplished
# indirectly by temporarily redefining `CMAKE_SIZEOF_VOID_P` to an
# empty string. Note that just undefining the variable could be
# insufficient in cases where the variable was already in CMake cache
set(CATCH2_CMAKE_SIZEOF_VOID_P ${CMAKE_SIZEOF_VOID_P})
set(CMAKE_SIZEOF_VOID_P "")
write_basic_package_version_file(
"${CMAKE_CURRENT_BINARY_DIR}/Catch2ConfigVersion.cmake"
COMPATIBILITY
SameMajorVersion
)
set(CMAKE_SIZEOF_VOID_P ${CATCH2_CMAKE_SIZEOF_VOID_P})
install(
FILES

25
CMakePresets.json Normal file
View File

@@ -0,0 +1,25 @@
{
"version": 3,
"configurePresets": [
{
"name": "basic-tests",
"displayName": "Basic development build",
"description": "Enables development build with basic tests that are cheap to build and run",
"cacheVariables": {
"CATCH_DEVELOPMENT_BUILD": "ON"
}
},
{
"name": "all-tests",
"inherits": "basic-tests",
"displayName": "Full development build",
"description": "Enables development build with examples and ALL tests",
"cacheVariables": {
"CATCH_BUILD_EXAMPLES": "ON",
"CATCH_BUILD_EXTRA_TESTS": "ON",
"CATCH_BUILD_SURROGATES": "ON",
"CATCH_ENABLE_CONFIGURE_TESTS": "ON"
}
}
]
}

2484
Doxyfile Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,24 +1,41 @@
<a id="top"></a>
![catch logo](data/artwork/catch2-logo-small.png)
![Catch2 logo](data/artwork/catch2-logo-small.png)
[![Github Releases](https://img.shields.io/github/release/catchorg/catch2.svg)](https://github.com/catchorg/catch2/releases)
[![Build Status](https://travis-ci.org/catchorg/Catch2.svg?branch=master)](https://travis-ci.org/catchorg/Catch2)
[![Build status](https://ci.appveyor.com/api/projects/status/github/catchorg/Catch2?svg=true)](https://ci.appveyor.com/project/catchorg/catch2)
[![codecov](https://codecov.io/gh/catchorg/Catch2/branch/master/graph/badge.svg)](https://codecov.io/gh/catchorg/Catch2)
<!-- We can eventually bring this back, but the upload script will have to be more complex -->
<!-- [![Try online](https://img.shields.io/badge/try-online-blue.svg)](https://wandbox.org/permlink/LzYWgcPrcy9yQmed) -->
[![Linux build status](https://github.com/catchorg/Catch2/actions/workflows/linux-simple-builds.yml/badge.svg)](https://github.com/catchorg/Catch2/actions/workflows/linux-simple-builds.yml)
[![Linux build status](https://github.com/catchorg/Catch2/actions/workflows/linux-other-builds.yml/badge.svg)](https://github.com/catchorg/Catch2/actions/workflows/linux-other-builds.yml)
[![MacOS build status](https://github.com/catchorg/Catch2/actions/workflows/mac-builds.yml/badge.svg)](https://github.com/catchorg/Catch2/actions/workflows/mac-builds.yml)
[![Build Status](https://ci.appveyor.com/api/projects/status/github/catchorg/Catch2?svg=true&branch=devel)](https://ci.appveyor.com/project/catchorg/catch2)
[![Code Coverage](https://codecov.io/gh/catchorg/Catch2/branch/devel/graph/badge.svg)](https://codecov.io/gh/catchorg/Catch2)
[![Try online](https://img.shields.io/badge/try-online-blue.svg)](https://godbolt.org/z/9x9qoM)
[![Join the chat in Discord: https://discord.gg/4CWS9zD](https://img.shields.io/badge/Discord-Chat!-brightgreen.svg)](https://discord.gg/4CWS9zD)
## Catch2 is released!
## What's the Catch2?
If you've been using an earlier version of Catch, please see the
Breaking Changes section of [the release notes](https://github.com/catchorg/Catch2/releases/tag/v2.0.1)
before moving to Catch2. You might also like to read [this blog post](https://levelofindirection.com/blog/catch2-released.html) for more details.
Catch2 is mainly a unit testing framework for C++, but it also
provides basic micro-benchmarking features, and simple BDD macros.
## What's the Catch?
Catch2's main advantage is that using it is both simple and natural.
Tests autoregister themselves and do not have to be named with valid
identifiers, assertions look like normal C++ code, and sections provide
a nice way to share set-up and tear-down code in tests.
## Catch2 v3 is being developed!
You are on the `devel` branch, where the next major version, v3, of
Catch2 is being developed. As it is a significant rework, you will
find that parts of this documentation are likely still stuck on v2.
For stable (and documentation-matching) version of Catch2, [go to the
`v2.x` branch](https://github.com/catchorg/Catch2/tree/v2.x).
For migrating from the v2 releases to v3, you should look at [our
documentation](docs/migrate-v2-to-v3.md#top). It provides a simple
guidelines on getting started, and collects most common migration
problems.
Catch2 is a multi-paradigm test framework for C++.
## How to use it
This documentation comprises these three parts:
@@ -27,7 +44,9 @@ This documentation comprises these three parts:
* [Tutorial](docs/tutorial.md#top) - getting started
* [Reference section](docs/Readme.md#top) - all the details
## More
* Issues and bugs can be raised on the [Issue tracker on GitHub](https://github.com/catchorg/Catch2/issues)
* For discussion or questions please use [the dedicated Google Groups forum](https://groups.google.com/forum/?fromgroups#!forum/catch-forum) or our [Discord](https://discord.gg/4CWS9zD)
* See [who else is using Catch2](docs/opensource-users.md#top)
* For discussion or questions please use [our Discord](https://discord.gg/4CWS9zD)
* See who else is using Catch2 in [Open Source Software](docs/opensource-users.md#top)
or [commercially](docs/commercial-users.md#top).

19
SECURITY.md Normal file
View File

@@ -0,0 +1,19 @@
# Security Policy
## Supported Versions
* Versions 1.x (branch Catch1.x) are no longer supported.
* Versions 2.x (branch v2.x) are currently supported.
* `devel` branch serves for stable-ish development and is supported,
but branches `devel-*` are considered short lived and are not supported separately.
## Reporting a Vulnerability
Due to its nature as a _unit_ test framework, Catch2 shouldn't interact
with untrusted inputs and there shouldn't be many security vulnerabilities
in it.
However, if you find one you send email to martin <dot> horenovsky <at>
gmail <dot> com. If you want to encrypt the email, my pgp key is
`E29C 46F3 B8A7 5028 6079 3B7D ECC9 C20E 314B 2360`.

14
WORKSPACE Normal file
View File

@@ -0,0 +1,14 @@
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "bazel_skylib",
strip_prefix = "bazel-skylib-2a87d4a62af886fb320883aba102255aba87275e",
urls = [
"https://github.com/bazelbuild/bazel-skylib/archive/2a87d4a62af886fb320883aba102255aba87275e.tar.gz",
],
sha256 = "d847b08d6702d2779e9eb399b54ff8920fa7521dc45e3e53572d1d8907767de7",
)
load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
bazel_skylib_workspace()

View File

@@ -18,11 +18,16 @@ init:
install:
- ps: if (($env:CONFIGURATION) -eq "Debug" -And ($env:coverage) -eq "1" ) { pip --disable-pip-version-check install codecov }
# This removes our changes to PATH. Keep this step last!
- ps: if (($env:CONFIGURATION) -eq "Debug" -And ($env:coverage) -eq "1" ) { .\tools\misc\installOpenCppCoverage.ps1 }
before_build:
# We need to modify PATH again, because it was reset since the "init" step
- set PATH=C:\Python35;C:\Python35\Scripts;%PATH%
- set CXXFLAGS=%additional_flags%
# If we are building examples/extra-tests, we need to regenerate the amalgamated files
- cmd: if "%examples%"=="1" ( python .\tools\scripts\generateAmalgamatedFiles.py )
# Indirection because appveyor doesn't handle multiline batch scripts properly
# https://stackoverflow.com/questions/37627248/how-to-split-a-command-over-multiple-lines-in-appveyor-yml/37647169#37647169
# https://help.appveyor.com/discussions/questions/3888-multi-line-cmd-or-powershell-warning-ignore
@@ -46,8 +51,10 @@ test_script:
# build explicitly.
environment:
matrix:
- FLAVOR: VS 2019 x64 Debug
- FLAVOR: VS 2019 x64 Debug Surrogates Configure Tests
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
surrogates: 1
configure_tests: 1
platform: x64
configuration: Debug
@@ -102,15 +109,11 @@ environment:
platform: Win32
configuration: Debug
# This build causes error after 30 minutes, because the VS2017 image
# tries to launch just-in-time debugger on call to std::terminate, which
# is an expected part of the tests in this build.
# We can reenable it, if we handle disabling that test in CMake.
# - FLAVOR: VS 2017 Win32 Debug Examples
# APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
# examples: 1
# platform: Win32
# configuration: Debug
- FLAVOR: VS 2017 Win32 Debug Examples
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
examples: 1
platform: Win32
configuration: Debug
- FLAVOR: VS 2017 Win32 Debug WMain
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017

View File

@@ -15,7 +15,7 @@ coverage:
codecov:
branch: master
branch: devel
max_report_age: off
comment:

View File

@@ -1,6 +1,5 @@
#!/usr/bin/env python
from conans import ConanFile, CMake
from conans import ConanFile, CMake, tools
class CatchConan(ConanFile):
name = "catch2"
@@ -15,18 +14,25 @@ class CatchConan(ConanFile):
settings = "os", "compiler", "build_type", "arch"
options = {"with_main": [True, False]}
default_options = {"with_main": True}
generators = "cmake"
def _configure_cmake(self):
cmake = CMake(self)
cmake.definitions["BUILD_TESTING"] = "OFF"
cmake.definitions["CATCH_INSTALL_DOCS"] = "OFF"
cmake.definitions["CATCH_INSTALL_HELPERS"] = "ON"
cmake.definitions["CATCH_INSTALL_EXTRAS"] = "ON"
cmake.configure(build_folder="build")
return cmake
def build(self):
# We need this workaround until the toolchains feature
# to inject stuff like MD/MT
line_to_replace = 'list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/CMake")'
tools.replace_in_file("CMakeLists.txt", line_to_replace,
'''{}
include("{}/conanbuildinfo.cmake")
conan_basic_setup()'''.format(line_to_replace, self.install_folder.replace("\\", "/")))
cmake = self._configure_cmake()
cmake.build()
@@ -35,11 +41,19 @@ class CatchConan(ConanFile):
cmake = self._configure_cmake()
cmake.install()
def package_id(self):
del self.info.options.with_main
def package_info(self):
self.cpp_info.libs = [
'Catch2Main', 'Catch2'] if self.options.with_main else ['Catch2']
lib_suffix = "d" if self.settings.build_type == "Debug" else ""
self.cpp_info.names["cmake_find_package"] = "Catch2"
self.cpp_info.names["cmake_find_package_multi"] = "Catch2"
# Catch2
self.cpp_info.components["catch2base"].names["cmake_find_package"] = "Catch2"
self.cpp_info.components["catch2base"].names["cmake_find_package_multi"] = "Catch2"
self.cpp_info.components["catch2base"].names["pkg_config"] = "Catch2"
self.cpp_info.components["catch2base"].libs = ["Catch2" + lib_suffix]
# Catch2WithMain
self.cpp_info.components["catch2main"].names["cmake_find_package"] = "Catch2WithMain"
self.cpp_info.components["catch2main"].names["cmake_find_package_multi"] = "Catch2WithMain"
self.cpp_info.components["catch2main"].names["pkg_config"] = "Catch2WithMain"
self.cpp_info.components["catch2main"].libs = ["Catch2Main" + lib_suffix]
self.cpp_info.components["catch2main"].requires = ["catch2base"]

View File

@@ -25,17 +25,16 @@ Running:
* [Command line](command-line.md#top)
Odds and ends:
* [Frequently Asked Questions (FAQ)](faq.md#top)
* [CMake integration](cmake-integration.md#top)
* [CI and other miscellaneous pieces](ci-and-misc.md#top)
FAQ:
* [Why are my tests slow to compile?](slow-compiles.md#top)
* [Known limitations](limitations.md#top)
Other:
* [Why Catch?](why-catch.md#top)
* [Open Source Projects using Catch](opensource-users.md#top)
* [Commercial Projects using Catch](commercial-users.md#top)
* [Why Catch2?](why-catch.md#top)
* [Migrating from v2 to v3](migrate-v2-to-v3.md#top)
* [Open Source Projects using Catch2](opensource-users.md#top)
* [Commercial Projects using Catch2](commercial-users.md#top)
* [Contributing](contributing.md#top)
* [Release Notes](release-notes.md#top)
* [Deprecations and incoming changes](deprecations.md#top)

View File

@@ -1,7 +1,7 @@
<a id="top"></a>
# Authoring benchmarks
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
Writing benchmarks is not easy. Catch simplifies certain aspects but you'll
always need to take care about various aspects. Understanding a few things about
@@ -11,7 +11,8 @@ First off, let's go over some terminology that will be used throughout this
guide.
- *User code*: user code is the code that the user provides to be measured.
- *Run*: one run is one execution of the user code.
- *Run*: one run is one execution of the user code. Sometimes also referred
to as an _iteration_.
- *Sample*: one sample is one data point obtained by measuring the time it takes
to perform a certain number of runs. One sample can consist of more than one
run if the clock available does not have enough resolution to accurately

View File

@@ -95,7 +95,7 @@ can use `pkg-config` to get its include path: `pkg-config --cflags catch2`.
### gdb and lldb scripts
Catch2's `contrib` folder also contains two simple debugger scripts,
Catch2's `extras` folder also contains two simple debugger scripts,
`gdbinit` for `gdb` and `lldbinit` for `lldb`. If loaded into their
respective debugger, these will tell it to step over Catch2's internals
when stepping through code.

View File

@@ -2,7 +2,7 @@
# CMake integration
**Contents**<br>
[CMake target](#cmake-target)<br>
[CMake targets](#cmake-targets)<br>
[Automatic test registration](#automatic-test-registration)<br>
[CMake project options](#cmake-project-options)<br>
[Installing Catch2 from git repository](#installing-catch2-from-git-repository)<br>
@@ -15,35 +15,59 @@ integration points for our users.
2) Catch2's repository contains CMake scripts for automatic registration
of `TEST_CASE`s in CTest
## CMake target
## CMake targets
Catch2's CMake build exports an interface target `Catch2::Catch2`. Linking
against it will add the proper include path and all necessary capabilities
to the resulting binary.
Catch2's CMake build exports two targets, `Catch2::Catch2`, and
`Catch2::Catch2WithMain`. If you do not need custom `main` function,
you should be using the latter (and only the latter). Linking against
it will add the proper include paths and link your target together with
2 static libraries that implement Catch2 and its main respectively.
If you need custom `main`, you should link only against `Catch2::Catch2`.
This means that if Catch2 has been installed on the system, it should be
enough to do:
This means that if Catch2 has been installed on the system, it should
be enough to do
```cmake
find_package(Catch2 REQUIRED)
target_link_libraries(tests Catch2::Catch2)
find_package(Catch2 3 REQUIRED)
# These tests can use the Catch2-provided main
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)
# These tests need their own main
add_executable(custom-main-tests test.cpp test-main.cpp)
target_link_libraries(custom-main-tests PRIVATE Catch2::Catch2)
```
These targets are also provided when Catch2 is used as a subdirectory.
Assuming Catch2 has been cloned to `lib/Catch2`, you only need to replace
the `find_package` call with `add_subdirectory(lib/Catch2)` and the snippet
above still works.
This target is also provided when Catch2 is used as a subdirectory.
Assuming that Catch2 has been cloned to `lib/Catch2`:
Another possibility is to use [FetchContent](https://cmake.org/cmake/help/latest/module/FetchContent.html):
```cmake
add_subdirectory(lib/Catch2)
target_link_libraries(tests Catch2::Catch2)
Include(FetchContent)
FetchContent_Declare(
Catch2
GIT_REPOSITORY https://github.com/catchorg/Catch2.git
GIT_TAG v3.0.0-preview3
)
FetchContent_MakeAvailable(Catch2)
add_executable(tests test.cpp)
target_link_libraries(tests PRIVATE Catch2::Catch2WithMain)
```
## Automatic test registration
Catch2's repository also contains two CMake scripts that help users
with automatically registering their `TEST_CASE`s with CTest. They
can be found in the `contrib` folder, and are
can be found in the `extras` folder, and are
1) `Catch.cmake` (and its dependency `CatchAddTests.cmake`)
2) `ParseAndAddCatchTests.cmake`
2) `ParseAndAddCatchTests.cmake` (deprecated)
If Catch2 has been installed in system, both of these can be used after
doing `find_package(Catch2 REQUIRED)`. Otherwise you need to add them
@@ -64,13 +88,25 @@ project(baz LANGUAGES CXX VERSION 0.0.1)
find_package(Catch2 REQUIRED)
add_executable(foo test.cpp)
target_link_libraries(foo Catch2::Catch2)
target_link_libraries(foo PRIVATE Catch2::Catch2)
include(CTest)
include(Catch)
catch_discover_tests(foo)
```
When using `FetchContent`, `include(Catch)` will fail unless
`CMAKE_MODULE_PATH` is explicitly updated to include the extras
directory.
```cmake
# ... FetchContent ...
#
list(APPEND CMAKE_MODULE_PATH ${catch2_SOURCE_DIR}/extras)
include(CTest)
include(Catch)
catch_discover_tests()
```
#### Customization
`catch_discover_tests` can be given several extra argumets:
@@ -83,6 +119,10 @@ catch_discover_tests(target
[TEST_SUFFIX suffix]
[PROPERTIES name1 value1...]
[TEST_LIST var]
[REPORTER reporter]
[OUTPUT_DIR dir]
[OUTPUT_PREFIX prefix]
[OUTPUT_SUFFIX suffix]
)
```
@@ -129,13 +169,46 @@ default `<target>_TESTS`. This can be useful when the same test
executable is being used in multiple calls to `catch_discover_tests()`.
Note that this variable is only available in CTest.
* `REPORTER reporter`
Use the specified reporter when running the test case. The reporter will
be passed to the test runner as `--reporter reporter`.
* `OUTPUT_DIR dir`
If specified, the parameter is passed along as
`--out dir/<test_name>` to test executable. The actual file name is the
same as the test name. This should be used instead of
`EXTRA_ARGS --out foo` to avoid race conditions writing the result output
when using parallel test execution.
* `OUTPUT_PREFIX prefix`
May be used in conjunction with `OUTPUT_DIR`.
If specified, `prefix` is added to each output file name, like so
`--out dir/prefix<test_name>`.
* `OUTPUT_SUFFIX suffix`
May be used in conjunction with `OUTPUT_DIR`.
If specified, `suffix` is added to each output file name, like so
`--out dir/<test_name>suffix`. This can be used to add a file extension to
the output file name e.g. ".xml".
### `ParseAndAddCatchTests.cmake`
⚠ This script is [deprecated](https://github.com/catchorg/Catch2/pull/2120)
in Catch2 2.13.4 and superseded by the above approach using `catch_discover_tests`.
See [#2092](https://github.com/catchorg/Catch2/issues/2092) for details.
`ParseAndAddCatchTests` works by parsing all implementation files
associated with the provided target, and registering them via CTest's
`add_test`. This approach has some limitations, such as the fact that
commented-out tests will be registered anyway.
commented-out tests will be registered anyway. More serious, only a
subset of the assertion macros currently available in Catch can be
detected by this script and tests with any macros that cannot be
parsed are *silently ignored*.
#### Usage
@@ -147,7 +220,7 @@ project(baz LANGUAGES CXX VERSION 0.0.1)
find_package(Catch2 REQUIRED)
add_executable(foo test.cpp)
target_link_libraries(foo Catch2::Catch2)
target_link_libraries(foo PRIVATE Catch2::Catch2)
include(CTest)
include(ParseAndAddCatchTests)
@@ -186,7 +259,22 @@ ParseAndAddCatchTests(bar)
## CMake project options
Catch2's CMake project also provides some options for other projects
that consume it. These are
that consume it. These are:
* `BUILD_TESTING` -- When `ON` and the project is not used as a subproject,
Catch2's test binary will be built. Defaults to `ON`.
* `CATCH_INSTALL_DOCS` -- When `ON`, Catch2's documentation will be
included in the installation. Defaults to `ON`.
* `CATCH_INSTALL_EXTRAS` -- When `ON`, Catch2's extras folder (the CMake
scripts mentioned above, debugger helpers) will be included in the
installation. Defaults to `ON`.
* `CATCH_DEVELOPMENT_BUILD` -- When `ON`, configures the build for development
of Catch2. This means enabling test projects, warnings and so on.
Defaults to `OFF`.
Enabling `CATCH_DEVELOPMENT_BUILD` also enables further configuration
customization options:
* `CATCH_BUILD_TESTING` -- When `ON`, Catch2's SelfTest project will be
built. Defaults to `ON`. Note that Catch2 also obeys `BUILD_TESTING` CMake
@@ -194,12 +282,15 @@ variable, so _both_ of them need to be `ON` for the SelfTest to be built,
and either of them can be set to `OFF` to disable building SelfTest.
* `CATCH_BUILD_EXAMPLES` -- When `ON`, Catch2's usage examples will be
built. Defaults to `OFF`.
* `CATCH_INSTALL_DOCS` -- When `ON`, Catch2's documentation will be
included in the installation. Defaults to `ON`.
* `CATCH_INSTALL_HELPERS` -- When `ON`, Catch2's contrib folder will be
included in the installation. Defaults to `ON`.
* `BUILD_TESTING` -- When `ON` and the project is not used as a subproject,
Catch2's test binary will be built. Defaults to `ON`.
* `CATCH_BUILD_EXTRA_TESTS` -- When `ON`, Catch2's extra tests will be
built. Defaults to `OFF`.
* `CATCH_BUILD_FUZZERS` -- When `ON`, Catch2 fuzzing entry points will
be built. Defaults to `OFF`.
* `CATCH_ENABLE_WERROR` -- When `ON`, adds `-Werror` or equivalent flag
to the compilation. Defaults to `ON`.
* `CATCH_BUILD_SURROGATES` -- When `ON`, each header in Catch2 will be
compiled separately to ensure that they are self-sufficient.
Defaults to `OFF`.
## Installing Catch2 from git repository

View File

@@ -15,7 +15,6 @@
[Warnings](#warnings)<br>
[Reporting timings](#reporting-timings)<br>
[Load test names to run from a file](#load-test-names-to-run-from-a-file)<br>
[Just test names](#just-test-names)<br>
[Specify the order test cases are run](#specify-the-order-test-cases-are-run)<br>
[Specify a seed for the Random Number Generator](#specify-a-seed-for-the-random-number-generator)<br>
[Identify framework and version according to the libIdentify standard](#identify-framework-and-version-according-to-the-libidentify-standard)<br>
@@ -29,14 +28,15 @@
[Specify the section to run](#specify-the-section-to-run)<br>
[Filenames as tags](#filenames-as-tags)<br>
[Override output colouring](#override-output-colouring)<br>
[Test Sharding](#test-sharding)<br>
[Allow running the binary without tests](#allow-running-the-binary-without-tests)<br>
[Output verbosity](#output-verbosity)<br>
Catch works quite nicely without any command line options at all - but for those times when you want greater control the following options are available.
Click one of the following links to take you straight to that option - or scroll on to browse the available options.
<a href="#specifying-which-tests-to-run"> ` <test-spec> ...`</a><br />
<a href="#usage"> ` -h, -?, --help`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -l, --list-tests`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -t, --list-tags`</a><br />
<a href="#showing-results-for-successful-tests"> ` -s, --success`</a><br />
<a href="#breaking-into-the-debugger"> ` -b, --break`</a><br />
<a href="#eliding-assertions-expected-to-throw"> ` -e, --nothrow`</a><br />
@@ -55,7 +55,8 @@ Click one of the following links to take you straight to that option - or scroll
</br>
<a href="#list-test-names-only"> ` --list-test-names-only`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-tests`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-tags`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-reporters`</a><br />
<a href="#order"> ` --order`</a><br />
<a href="#rng-seed"> ` --rng-seed`</a><br />
@@ -66,7 +67,11 @@ Click one of the following links to take you straight to that option - or scroll
<a href="#benchmark-confidence-interval"> ` --benchmark-confidence-interval`</a><br />
<a href="#benchmark-no-analysis"> ` --benchmark-no-analysis`</a><br />
<a href="#benchmark-warmup-time"> ` --benchmark-warmup-time`</a><br />
<a href="#use-colour"> ` --use-colour`</a><br />
<a href="#colour-mode"> ` --colour-mode`</a><br />
<a href="#test-sharding"> ` --shard-count`</a><br />
<a href="#test-sharding"> ` --shard-index`</a><br />
<a href=#no-tests-override> ` --allow-running-no-tests`</a><br />
<a href=#output-verbosity> ` --verbosity`</a><br />
</br>
@@ -93,7 +98,8 @@ Inclusions and exclusions are evaluated in left-to-right order.
Test case examples:
<pre>thisTestOnly Matches the test case called, 'thisTestOnly'
```
thisTestOnly Matches the test case called, 'thisTestOnly'
"this test only" Matches the test case called, 'this test only'
these* Matches all cases starting with 'these'
exclude:notThis Matches all tests except, 'notThis'
@@ -101,8 +107,9 @@ exclude:notThis Matches all tests except, 'notThis'
~*private* Matches all tests except those that contain 'private'
a* ~ab* abc Matches all tests that start with 'a', except those that
start with 'ab', except 'abc', which is included
~[tag1] Matches all tests except those tagged with '[tag1]'
-# [#somefile] Matches all tests from the file 'somefile.cpp'
</pre>
```
Names within square brackets are interpreted as tags.
A series of tags form an AND expression whereas a comma-separated sequence forms an OR expression. e.g.:
@@ -116,18 +123,48 @@ Test names containing special characters, such as `,` or `[` can specify them on
<a id="choosing-a-reporter-to-use"></a>
## Choosing a reporter to use
<pre>-r, --reporter &lt;reporter></pre>
<pre>-r, --reporter &lt;reporter[::key=value]*&gt;</pre>
A reporter is an object that formats and structures the output of running tests, and potentially summarises the results. By default a console reporter is used that writes, IDE friendly, textual output. Catch comes bundled with some alternative reporters, but more can be added in client code.<br />
The bundled reporters are:
Reporters are how the output from Catch2 (results of assertions, tests,
benchmarks and so on) is formatted and written out. The default reporter
is called the "Console" reporter and is intended to provide relatively
verbose and human-friendly output.
<pre>-r console
-r compact
-r xml
-r junit
</pre>
Reporters are also individually configurable. To pass configuration options
to the reporter, you append `::key=value` to the reporter specification
as many times as you want, e.g. `--reporter xml::out=someFile.xml`.
The keys must either be prefixed by "X", in which case they are not parsed
by Catch2 and are only passed down to the reporter, or one of options
hardcoded into Catch2. Currently there are only 2,
["out"](#sending-output-to-a-file), and ["colour-mode"](#colour-mode).
_Note that the reporter might still check the X-prefixed options for
validity, and throw an error if they are wrong._
> Support for passing arguments to reporters through the `-r`, `--reporter` flag was introduced in Catch2 X.Y.Z
There are multiple built-in reporters, you can see what they do by using the
[`--list-reporter`](command-line.md#listing-available-tests-tags-or-reporters)
flag. If you need a reporter providing custom format outside of the already
provided ones, look at the ["write your own reporter" part of the reporter
documentation](reporters.md#writing-your-own-reporter).
This option may be passed multiple times to use multiple (different)
reporters at the same time. See the [reporter documentation](reporters.md#multiple-reporters)
for details on what the resulting behaviour is. Also note that at most one
reporter can be provided without the output-file part of reporter spec.
This reporter will use the "default" output destination, based on
the [`-o`, `--out`](#sending-output-to-a-file) option.
> Support for using multiple different reporters at the same time was [introduced](https://github.com/catchorg/Catch2/pull/2183) in Catch2 X.Y.Z
_Note: There is currently no way to escape `::` in the reporter spec,
and thus the reporter names, or configuration keys and values, cannot
contain `::`. As `::` in paths is relatively obscure (unlike ':'), we do
not consider this an issue._
The JUnit reporter is an xml format that follows the structure of the JUnit XML Report ANT task, as consumed by a number of third-party tools, including Continuous Integration servers such as Hudson. If not otherwise needed, the standard XML reporter is preferred as this is a streaming reporter, whereas the Junit reporter needs to hold all its results until the end so it can write the overall results into attributes of the root node.
<a id="breaking-into-the-debugger"></a>
## Breaking into the debugger
@@ -157,24 +194,48 @@ Sometimes this results in a flood of failure messages and you'd rather just see
<a id="listing-available-tests-tags-or-reporters"></a>
## Listing available tests, tags or reporters
<pre>-l, --list-tests
-t, --list-tags
```
--list-tests
--list-tags
--list-reporters
</pre>
```
```-l``` or ```--list-tests``` will list all registered tests, along with any tags.
If one or more test-specs have been supplied too then only the matching tests will be listed.
> The `--list*` options became customizable through reporters in Catch2 X.Y.Z
```-t``` or ```--list-tags``` lists all available tags, along with the number of test cases they match. Again, supplying test specs limits the tags that match.
`--list-tests` lists all registered tests matching specified test spec.
Usually this listing also includes tags, and potentially also other
information, like source location, based on verbosity and reporter's design.
`--list-tags` lists all tags from registered tests matching specified test
spec. Usually this also includes number of tests cases they match and
similar information.
`--list-reporters` lists all available reporters and their descriptions.
```--list-reporters``` lists the available reporters.
<a id="sending-output-to-a-file"></a>
## Sending output to a file
<pre>-o, --out &lt;filename>
<pre>-o, --out &lt;filename&gt;
</pre>
Use this option to send all output to a file. By default output is sent to stdout (note that uses of stdout and stderr *from within test cases* are redirected and included in the report - so even stderr will effectively end up on stdout).
Use this option to send all output to a file, instead of stdout. You can
use `-` as the filename to explicitly send the output to stdout (this is
useful e.g. when using multiple reporters).
> Support for `-` as the filename was introduced in Catch2 X.Y.Z
Filenames starting with "%" (percent symbol) are reserved by Catch2 for
meta purposes, e.g. using `%debug` as the filename opens stream that
writes to platform specific debugging/logging mechanism.
Catch2 currently recognizes 3 meta streams:
* `%debug` - writes to platform specific debugging/logging output
* `%stdout` - writes to stdout
* `%stderr` - writes to stderr
> Support for `%stdout` and `%stderr` was introduced in Catch2 X.Y.Z
<a id="naming-a-test-run"></a>
## Naming a test run
@@ -205,16 +266,24 @@ This option transforms tabs and newline characters into ```\t``` and ```\n``` re
## Warnings
<pre>-w, --warn &lt;warning name></pre>
Enables reporting of suspicious test states. There are currently two
available warnings
You can think of Catch2's warnings as the equivalent of `-Werror` (`/WX`)
flag for C++ compilers. It turns some suspicious occurences, like a section
without assertions, into errors. Because these might be intended, warnings
are not enabled by default, but user can opt in.
You can enable multiple warnings at the same time.
There are currently two warnings implemented:
```
NoAssertions // Fail test case / leaf section if no assertions
// (e.g. `REQUIRE`) is encountered.
NoTests // Return non-zero exit code when no test cases were run
// Also calls reporter's noMatchingTestCases method
NoAssertions // Fail test case / leaf section if no assertions
// (e.g. `REQUIRE`) is encountered.
UnmatchedTestSpec // Fail test run if any of the CLI test specs did
// not match any tests.
```
> `UnmatchedTestSpec` was introduced in Catch2 X.Y.Z.
<a id="reporting-timings"></a>
## Reporting timings
@@ -222,19 +291,27 @@ available warnings
When set to ```yes``` Catch will report the duration of each test case, in milliseconds. Note that it does this regardless of whether a test case passes or fails. Note, also, the certain reporters (e.g. Junit) always report test case durations regardless of this option being set or not.
<pre>-D, --min-duration &lt;value></pre>
> `--min-duration` was [introduced](https://github.com/catchorg/Catch2/pull/1910) in Catch2 2.13.0
When set, Catch will report the duration of each test case that took more
than &lt;value> seconds, in milliseconds. This option is overriden by both
`-d yes` and `-d no`, so that either all durations are reported, or none
are.
<a id="input-file"></a>
## Load test names to run from a file
<pre>-f, --input-file &lt;filename></pre>
Provide the name of a file that contains a list of test case names - one per line. Blank lines are skipped and anything after the comment character, ```#```, is ignored.
Provide the name of a file that contains a list of test case names,
one per line. Blank lines are skipped.
A useful way to generate an initial instance of this file is to use the <a href="#list-test-names-only">list-test-names-only</a> option. This can then be manually curated to specify a specific subset of tests - or in a specific order.
<a id="list-test-names-only"></a>
## Just test names
<pre>--list-test-names-only</pre>
This option lists all available tests in a non-indented form, one on each line. This makes it ideal for saving to a file and feeding back into the <a href="#input-file">```-f``` or ```--input-file```</a> option.
A useful way to generate an initial instance of this file is to combine
the [`--list-tests`](#listing-available-tests-tags-or-reporters) flag with
the [`--verbosity quiet`](#output-verbosity) option. You can also
use test specs to filter this list down to what you want first.
<a id="order"></a>
@@ -265,13 +342,20 @@ is that as long as the random seed is fixed, running only some tests
<a id="rng-seed"></a>
## Specify a seed for the Random Number Generator
<pre>--rng-seed &lt;'time'|number&gt;</pre>
<pre>--rng-seed &lt;'time'|'random-device'|number&gt;</pre>
Sets a seed for the random number generator using ```std::srand()```.
If a number is provided this is used directly as the seed so the random pattern is repeatable.
Alternatively if the keyword ```time``` is provided then the result of calling ```std::time(0)``` is used and so the pattern becomes unpredictable. In some cases, you might need to pass the keyword ```time``` in double quotes instead of single quotes.
Sets the seed for random number generators used by Catch2. These are used
e.g. to shuffle tests when user asks for tests to be in random order.
Using `time` as the argument asks Catch2 generate the seed through call
to `std::time(nullptr)`. This provides very weak randomness and multiple
runs of the binary can generate the same seed if they are started close
to each other.
Using `random-device` asks for `std::random_device` to be used instead.
If your implementation provides working `std::random_device`, it should
be preferred to using `time`. Catch2 uses `std::random_device` by default.
In either case the actual value for the seed is printed as part of Catch's output so if an issue is discovered that is sensitive to test ordering the ordering can be reproduced - even if it was originally seeded from ```std::time(0)```.
<a id="libidentify"></a>
## Identify framework and version according to the libIdentify standard
@@ -290,7 +374,7 @@ either before running any tests, after running all tests - or both, depending on
## Specify the number of benchmark samples to collect
<pre>--benchmark-samples &lt;# of samples&gt;</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
When running benchmarks a number of "samples" is collected. This is the base data for later statistical analysis.
Per sample a clock resolution dependent number of iterations of the user code is run, which is independent of the number of samples. Defaults to 100.
@@ -299,7 +383,7 @@ Per sample a clock resolution dependent number of iterations of the user code is
## Specify the number of resamples for bootstrapping
<pre>--benchmark-resamples &lt;# of resamples&gt;</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
After the measurements are performed, statistical [bootstrapping] is performed
on the samples. The number of resamples for that bootstrapping is configurable
@@ -314,7 +398,7 @@ defaults to 95%).
## Specify the confidence-interval for bootstrapping
<pre>--benchmark-confidence-interval &lt;confidence-interval&gt;</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
The confidence-interval is used for statistical bootstrapping on the samples to
calculate the upper and lower bounds of mean and standard deviation.
@@ -324,7 +408,7 @@ Must be between 0 and 1 and defaults to 0.95.
## Disable statistical analysis of collected benchmark samples
<pre>--benchmark-no-analysis</pre>
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
When this flag is specified no bootstrapping or any other statistical analysis is performed.
Instead the user code is only measured and the plain mean from the samples is reported.
@@ -333,7 +417,7 @@ Instead the user code is only measured and the plain mean from the samples is re
## Specify the amount of time in milliseconds spent on warming up each test
<pre>--benchmark-warmup-time</pre>
> [Introduced](https://github.com/catchorg/Catch2/pull/1844) in Catch 2.11.2.
> [Introduced](https://github.com/catchorg/Catch2/pull/1844) in Catch2 2.11.2.
Configure the amount of time spent warming up each test.
@@ -382,7 +466,7 @@ There are some limitations of this feature to be aware of:
- Code outside of sections being skipped will still be executed - e.g. any set-up code in the TEST_CASE before the
start of the first section.</br>
- At time of writing, wildcards are not supported in section names.
- If you specify a section without narrowing to a test case first then all test cases will be executed
- If you specify a section without narrowing to a test case first then all test cases will be executed
(but only matching sections within them).
@@ -390,21 +474,72 @@ start of the first section.</br>
## Filenames as tags
<pre>-#, --filenames-as-tags</pre>
When this option is used then every test is given an additional tag which is formed of the unqualified
When this option is used then every test is given an additional tag which is formed of the unqualified
filename it is found in, with any extension stripped, prefixed with the `#` character.
So, for example, tests within the file `~\Dev\MyProject\Ferrets.cpp` would be tagged `[#Ferrets]`.
<a id="use-colour"></a>
<a id="colour-mode"></a>
## Override output colouring
<pre>--use-colour &lt;yes|no|auto&gt;</pre>
<pre>--colour-mode &lt;ansi|win32|none|default&gt;</pre>
Catch colours output for terminals, but omits colouring when it detects that
output is being sent to a pipe. This is done to avoid interfering with automated
processing of output.
> The `--colour-mode` option replaced the old `--colour` option in Catch2 X.Y.Z
Catch2 support two different ways of colouring terminal output, and by
default it attempts to make a good guess on which implementation to use
(and whether to even use it, e.g. Catch2 tries to avoid writing colour
codes when writing the results into a file).
`--colour-mode` allows the user to explicitly select what happens.
* `--colour-mode ansi` tells Catch2 to always use ANSI colour codes, even
when writing to a file
* `--colour-mode win32` tells Catch2 to use colour implementation based
on Win32 terminal API
* `--colour-mode none` tells Catch2 to disable colours completely
* `--colour-mode default` lets Catch2 decide
`--colour-mode default` is the default setting.
<a id="test-sharding"></a>
## Test Sharding
<pre>--shard-count <#number of shards>, --shard-index <#shard index to run></pre>
> [Introduced](https://github.com/catchorg/Catch2/pull/2257) in Catch2 X.Y.Z.
When `--shard-count <#number of shards>` is used, the tests to execute will be split evenly in to the given number of sets,
identified by indicies starting at 0. The tests in the set given by `--shard-index <#shard index to run>` will be executed.
The default shard count is `1`, and the default index to run is `0`. It is an error to specify a shard index greater than
the number of shards.
This is useful when you want to split test execution across multiple processes, as is done with [Bazel test sharding](https://docs.bazel.build/versions/main/test-encyclopedia.html#test-sharding).
<a id="no-tests-override"></a>
## Allow running the binary without tests
<pre>--allow-running-no-tests</pre>
> Introduced in Catch2 X.Y.Z.
By default, Catch2 test binaries return non-0 exit code if no tests were
run, e.g. if the binary was compiled with no tests, or the provided test
spec matched no tests. This flag overrides that, so a test run with no
tests still returns 0.
## Output verbosity
```
-v, --verbosity <quiet|normal|high>
```
Changing verbosity might change how much details Catch2's reporters output.
However, you should consider changing the verbosity level as a _suggestion_.
Not all reporters support all verbosity levels, e.g. because the reporter's
format cannot meaningfully change. In that case, the verbosity level is
ignored.
Verbosity defaults to _normal_.
`--use-colour yes` forces coloured output, `--use-colour no` disables coloured
output. The default behaviour is `--use-colour auto`.
---

View File

@@ -1,22 +1,22 @@
<a id="top"></a>
# Commercial users of Catch
# Commercial users of Catch2
As well as [Open Source](opensource-users.md#top) users Catch is widely used within proprietary code bases too.
Many organisations like to keep this information internal, and that's fine,
but if you're more open it would be great if we could list the names of as
many organisations as possible that use Catch somewhere in their codebase.
Enterprise environments often tend to be far more conservative in their tool adoption -
and being aware that other companies are using Catch can ease the path in.
Catch2 is also widely used in proprietary code bases. This page contains
some of them that are willing to share this information.
So if you are aware of Catch usage in your organisation, and are fairly confident there is no issue with sharing this
fact then please let us know - either directly, via a PR or
[issue](https://github.com/philsquared/Catch/issues), or on the [forums](https://groups.google.com/forum/?fromgroups#!forum/catch-forum).
If you want to add your organisation, please check that there is no issue
with you sharing this fact.
- Bloomberg
- [Bloomlife](https://bloomlife.com)
- NASA
- [Inscopix Inc.](https://www.inscopix.com/)
- Locksley.CZ
- [Makimo](https://makimo.pl/)
- NASA
- [UX3D](https://ux3d.io)
- [King](https://king.com)
---
[Home](Readme.md#top)

View File

@@ -8,17 +8,19 @@
[stdout](#stdout)<br>
[Fallback stringifier](#fallback-stringifier)<br>
[Default reporter](#default-reporter)<br>
[Bazel support](#bazel-support)<br>
[C++11 toggles](#c11-toggles)<br>
[C++17 toggles](#c17-toggles)<br>
[Other toggles](#other-toggles)<br>
[Windows header clutter](#windows-header-clutter)<br>
[Enabling stringification](#enabling-stringification)<br>
[Disabling exceptions](#disabling-exceptions)<br>
[Overriding Catch's debug break (`-b`)](#overriding-catchs-debug-break--b)<br>
Catch is designed to "just work" as much as possible. For most people the only configuration needed is telling Catch which source file should host all the implementation code (```CATCH_CONFIG_MAIN```).
Nonetheless there are still some occasions where finer control is needed. For these occasions Catch exposes a set of macros for configuring how it is built.
Catch2 is designed to "just work" as much as possible, and most of the
configuration options below are changed automatically during compilation,
according to the detected environment. However, this detection can also
be overriden by users, using macros documented below, and/or CMake options
with the same name.
## Prefixing Catch macros
@@ -30,19 +32,18 @@ To keep test code clean and uncluttered Catch uses short macro names (e.g. ```TE
## Terminal colour
CATCH_CONFIG_COLOUR_NONE // completely disables all text colouring
CATCH_CONFIG_COLOUR_WINDOWS // forces the Win32 console API to be used
CATCH_CONFIG_COLOUR_ANSI // forces ANSI colour codes to be used
CATCH_CONFIG_COLOUR_WIN32 // Force enables compiling colouring impl based on Win32 console API
CATCH_CONFIG_NO_COLOUR_WIN32 // Force disables ...
Yes, I am English, so I will continue to spell "colour" with a 'u'.
Yes, Catch2 uses the british spelling of colour.
When sending output to the terminal, if it detects that it can, Catch will use colourised text. On Windows the Win32 API, ```SetConsoleTextAttribute```, is used. On POSIX systems ANSI colour escape codes are inserted into the stream.
Catch2 attempts to autodetect whether the Win32 console colouring API,
`SetConsoleTextAttribute`, is available, and if it is available it compiles
in a console colouring implementation that uses it.
For finer control you can define one of the above identifiers (these are mutually exclusive - but that is not checked so may behave unexpectedly if you mix them):
This option can be used to override Catch2's autodetection and force the
compilation either ON or OFF.
Note that when ANSI colour codes are used "unistd.h" must be includable - along with a definition of ```isatty()```
Typically you should place the ```#define``` before #including "catch.hpp" in your main source file - but if you prefer you can define it for your whole project by whatever your IDE or build system provides for you to do so.
## Console width
@@ -56,7 +57,7 @@ By default a console width of 80 is assumed but this can be controlled by defini
CATCH_CONFIG_NOSTDOUT
To support platforms that do not provide `std::cout`, `std::cerr` and
`std::clog`, Catch does not usem the directly, but rather calls
`std::clog`, Catch does not use them directly, but rather calls
`Catch::cout`, `Catch::cerr` and `Catch::clog`. You can replace their
implementation by defining `CATCH_CONFIG_NOSTDOUT` and implementing
them yourself, their signatures are:
@@ -96,6 +97,12 @@ This means that defining `CATCH_CONFIG_DEFAULT_REPORTER` to `"console"`
is equivalent with the out-of-the-box experience.
## Bazel support
When `CATCH_CONFIG_BAZEL_SUPPORT` is defined, Catch2 will register a `JUnit`
reporter writing to a path pointed by `XML_OUTPUT_FILE` provided by Bazel.
> `CATCH_CONFIG_BAZEL_SUPPORT` was [introduced](https://github.com/catchorg/Catch2/pull/2399) in Catch2 X.Y.Z.
## C++11 toggles
CATCH_CONFIG_CPP11_TO_STRING // Use `std::to_string`
@@ -117,7 +124,7 @@ Catch's selection, by defining either `CATCH_CONFIG_CPP11_TO_STRING` or
CATCH_CONFIG_CPP17_OPTIONAL // Override std::optional support detection (checked by CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER)
CATCH_CONFIG_CPP17_BYTE // Override std::byte support detection (Catch provides a StringMaker specialization by default)
> `CATCH_CONFIG_CPP17_STRING_VIEW` was [introduced](https://github.com/catchorg/Catch2/issues/1376) in Catch 2.4.1.
> `CATCH_CONFIG_CPP17_STRING_VIEW` was [introduced](https://github.com/catchorg/Catch2/issues/1376) in Catch2 2.4.1.
Catch contains basic compiler/standard detection and attempts to use
some C++17 features whenever appropriate. This automatic detection
@@ -141,7 +148,7 @@ by using `_NO_` in the macro, e.g. `CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS`.
CATCH_CONFIG_ANDROID_LOGWRITE // Use android's logging system for debug output
CATCH_CONFIG_GLOBAL_NEXTAFTER // Use nextafter{,f,l} instead of std::nextafter
> [`CATCH_CONFIG_ANDROID_LOGWRITE`](https://github.com/catchorg/Catch2/issues/1743) and [`CATCH_CONFIG_GLOBAL_NEXTAFTER`](https://github.com/catchorg/Catch2/pull/1739) were introduced in Catch 2.10.0
> [`CATCH_CONFIG_ANDROID_LOGWRITE`](https://github.com/catchorg/Catch2/issues/1743) and [`CATCH_CONFIG_GLOBAL_NEXTAFTER`](https://github.com/catchorg/Catch2/pull/1739) were introduced in Catch2 2.10.0
Currently Catch enables `CATCH_CONFIG_WINDOWS_SEH` only when compiled with MSVC, because some versions of MinGW do not have the necessary Win32 API support.
@@ -179,13 +186,6 @@ This feature is considered experimental and might change at any point.
_Inspired by Doctest's `DOCTEST_CONFIG_DISABLE`_
## Windows header clutter
On Windows Catch includes `windows.h`. To minimize global namespace clutter in the implementation file, it defines `NOMINMAX` and `WIN32_LEAN_AND_MEAN` before including it. You can control this behaviour via two macros:
CATCH_CONFIG_NO_NOMINMAX // Stops Catch from using NOMINMAX macro
CATCH_CONFIG_NO_WIN32_LEAN_AND_MEAN // Stops Catch from using WIN32_LEAN_AND_MEAN macro
## Enabling stringification
@@ -197,13 +197,13 @@ By default, Catch does not stringify some types from the standard library. This
CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER // Provide StringMaker specialization for std::optional (on C++17)
CATCH_CONFIG_ENABLE_ALL_STRINGMAKERS // Defines all of the above
> `CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1380) in Catch 2.4.1.
> `CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1380) in Catch2 2.4.1.
> `CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1510) in Catch 2.6.0.
> `CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1510) in Catch2 2.6.0.
## Disabling exceptions
> Introduced in Catch 2.4.0.
> Introduced in Catch2 2.4.0.
By default, Catch2 uses exceptions to signal errors and to abort tests
when an assertion from the `REQUIRE` family of assertions fails. We also
@@ -239,7 +239,7 @@ namespace Catch {
## Overriding Catch's debug break (`-b`)
> [Introduced](https://github.com/catchorg/Catch2/pull/1846) in Catch 2.11.2.
> [Introduced](https://github.com/catchorg/Catch2/pull/1846) in Catch2 2.11.2.
You can override Catch2's break-into-debugger code by defining the
`CATCH_BREAK_INTO_DEBUGGER()` macro. This can be used if e.g. Catch2 does

View File

@@ -1,117 +1,137 @@
<a id="top"></a>
# Contributing to Catch
# Contributing to Catch2
**Contents**<br>
[Branches](#branches)<br>
[Directory structure](#directory-structure)<br>
[Using Git(Hub)](#using-github)<br>
[Testing your changes](#testing-your-changes)<br>
[Documenting your code](#documenting-your-code)<br>
[Code constructs to watch out for](#code-constructs-to-watch-out-for)<br>
[Writing documentation](#writing-documentation)<br>
[Writing code](#writing-code)<br>
[CoC](#coc)<br>
So you want to contribute something to Catch? That's great! Whether it's a bug fix, a new feature, support for
additional compilers - or just a fix to the documentation - all contributions are very welcome and very much appreciated.
Of course so are bug reports and other comments and questions.
So you want to contribute something to Catch2? That's great! Whether it's
a bug fix, a new feature, support for additional compilers - or just
a fix to the documentation - all contributions are very welcome and very
much appreciated. Of course so are bug reports, other comments, and
questions, but generally it is a better idea to ask questions in our
[Discord](https://discord.gg/4CWS9zD), than in the issue tracker.
If you are contributing to the code base there are a few simple guidelines to keep in mind. This also includes notes to
help you find your way around. As this is liable to drift out of date please raise an issue or, better still, a pull
request for this file, if you notice that.
## Branches
This page covers some guidelines and helpful tips for contributing
to the codebase itself.
Ongoing development is currently on _master_. At some point an integration branch will be set-up and PRs should target
that - but for now it's all against master. You may see feature branches come and go from time to time, too.
## Using Git(Hub)
## Directory structure
Ongoing development happens in the `devel` branch for Catch2 v3, and in
`v2.x` for maintenance updates to the v2 versions.
_Users_ of Catch primarily use the single header version. _Maintainers_ should work with the full source (which is still,
primarily, in headers). This can be found in the `include` folder. There are a set of test files, currently under
`projects/SelfTest`. The test app can be built via CMake from the `CMakeLists.txt` file in the root, or you can generate
project files for Visual Studio, XCode, and others (instructions in the `projects` folder). If you have access to CLion,
it can work with the CMake file directly.
Commits should be small and atomic. A commit is atomic when, after it is
applied, the codebase, tests and all, still works as expected. Small
commits are also preferred, as they make later operations with git history,
whether it is bisecting, reverting, or something else, easier.
As well as the runtime test files you'll also see a `SurrogateCpps` directory under `projects/SelfTest`.
This contains a set of .cpp files that each `#include` a single header.
While these files are not essential to compilation they help to keep the implementation headers self-contained.
At time of writing this set is not complete but has reasonable coverage.
If you add additional headers please try to remember to add a surrogate cpp for it.
_When submitting a pull request please do not include changes to the
amalgamated distribution files. This means do not include them in your
git commits!_
The other directories are `tools/scripts` which contains a set of python scripts to help in testing Catch as well as
generating the single include, and `docs`, which contains the documentation as a set of markdown files.
When addressing review comments in a MR, please do not rebase/squash the
commits immediately. Doing so makes it harder to review the new changes,
slowing down the process of merging a MR. Instead, when addressing review
comments, you should append new commits to the branch and only squash
them into other commits when the MR is ready to be merged. We recommend
creating new commits with `git commit --fixup` (or `--squash`) and then
later squashing them with `git rebase --autosquash` to make things easier.
__When submitting a pull request please do not include changes to the single include, or to the version number file
as these are managed by the scripts!__
## Testing your changes
Obviously all changes to Catch's code should be tested. If you added new
functionality, you should add tests covering and showcasing it. Even if you have
only made changes to Catch internals (i.e. you implemented some performance
improvements), you should still test your changes.
_Note: Running Catch2's tests requires Python3_
This means 2 things
* Compiling Catch's SelfTest project:
Catch2 has multiple layers of tests that are then run as part of our CI.
The most obvious one are the unit tests compiled into the `SelfTest`
binary. These are then used in "Approval tests", which run (almost) all
tests from `SelfTest` through a specific reporter and then compare the
generated output with a known good output ("Baseline"). By default, new
tests should be placed here.
However, not all tests can be written as plain unit tests. For example,
checking that Catch2 orders tests randomly when asked to, and that this
random ordering is subset-invariant, is better done as an integration
test using an external check script. Catch2 integration tests are written
using CTest, either as a direct command invocation + pass/fail regex,
or by delegating the check to a Python script.
Catch2 is slowly gaining more and more types of tests, currently Catch2
project also has buildable examples, "ExtraTests", and CMake config tests.
Examples present a small and self-contained snippets of code that
use Catch2's facilities for specific purpose. Currently they are assumed
passing if they compile.
ExtraTests then are expensive tests, that we do not want to run all the
time. This can be either because they take a long time to run, or because
they take a long time to compile, e.g. because they test compile time
configuration and require separate compilation.
Finally, CMake config tests test that you set Catch2's compile-time
configuration options through CMake, using CMake options of the same name.
None of these tests are enabled by default. To enable them, add
`-DCATCH_BUILD_EXAMPLES=ON`, `-DCATCH_BUILD_EXTRA_TESTS=ON`, and
`-DCATCH_ENABLE_CONFIGURE_TESTS=ON` when configuration the CMake build.
Bringing this all together, the steps below should configure, build,
and run all tests in the `Debug` compilation.
<!-- snippet: catch2-build-and-test -->
<a id='snippet-catch2-build-and-test'></a>
```sh
# 1. Regenerate the amalgamated distribution
./tools/scripts/generateAmalgamatedFiles.py
# 2. Configure the full test build
cmake -Bdebug-build -H. -DCMAKE_BUILD_TYPE=Debug -DCATCH_DEVELOPMENT_BUILD=ON -DCATCH_BUILD_EXAMPLES=ON -DCATCH_BUILD_EXTRA_TESTS=ON
# 3. Run the actual build
cmake --build debug-build
# 4. Run the tests using CTest
cd debug-build
ctest -j 4 --output-on-failure -C Debug
```
$ cd Catch2
$ cmake -Bdebug-build -H. -DCMAKE_BUILD_TYPE=Debug
$ cmake --build debug-build
<sup><a href='/tools/scripts/buildAndTest.sh#L6-L19' title='File snippet `catch2-build-and-test` was extracted from'>snippet source</a> | <a href='#snippet-catch2-build-and-test' title='Navigate to start of snippet `catch2-build-and-test`'>anchor</a></sup>
<!-- endSnippet -->
For convenience, the above commands are in the script `tools/scripts/buildAndTest.sh`, and can be run like this:
```bash
cd Catch2
./tools/scripts/buildAndTest.sh
```
because code that does not compile is evidently incorrect. Obviously,
you are not expected to have access to all the compilers and platforms
supported by Catch2, but you should at least smoke test your changes
on your platform. Our CI pipeline will check your PR against most of
the supported platforms, but it takes an hour to finish -- compiling
locally takes just a few minutes.
A Windows version of the script is available at `tools\scripts\buildAndTest.cmd`.
If you added new tests, you will likely see `ApprovalTests` failure.
After you check that the output difference is expected, you should
run `tools/scripts/approve.py` to confirm them, and include these changes
in your commit.
* Running the tests via CTest:
```
$ cd debug-build
$ ctest -j 2 --output-on-failure
```
__Note:__ When running your tests with multi-configuration generators like
Visual Studio, you might get errors "Test not available without configuration."
You then have to pick one configuration (e.g. ` -C Debug`) in the `ctest` call.
If you added new tests, approval tests are very likely to fail. If they
do not, it means that your changes weren't run as part of them. This
_might_ be intentional, but usually is not.
The approval tests compare current output of the SelfTest binary in various
configurations against known good outputs. The reason it fails is,
_usually_, that you've added new tests but have not yet approved the changes
they introduce. This is done with the `scripts/approve.py` script, but
before you do so, you need to check that the introduced changes are indeed
intentional.
## Documenting your code
## Writing documentation
If you have added new feature to Catch2, it needs documentation, so that
other people can use it as well. This section collects some technical
information that you will need for updating Catch2's documentation, and
possibly some generic advise as well.
### Technicalities
First, the technicalities:
* We introduced version tags to the documentation, which show users in
which version a specific feature was introduced. This means that newly
written documentation should be tagged with a placeholder, that will
be replaced with the actual version upon release. There are 2 styles
of placeholders used through the documentation, you should pick one that
fits your text better (if in doubt, take a look at the existing version
tags for other features).
* `> [Introduced](link-to-issue-or-PR) in Catch X.Y.Z` - this
placeholder is usually used after a section heading
* `> X (Y and Z) was [introduced](link-to-issue-or-PR) in Catch X.Y.Z`
- this placeholder is used when you need to tag a subpart of something,
e.g. list
* Crosslinks to different pages should target the `top` anchor, like this
`[link to contributing](contributing.md#top)`.
* If you have introduced a new document, there is a simple template you
should use. It provides you with the top anchor mentioned above, and also
with a backlink to the top of the documentation:
should use. It provides you with the top anchor mentioned to link to
(more below), and also with a backlink to the top of the documentation:
```markdown
<a id="top"></a>
# Cool feature
@@ -123,6 +143,23 @@ Text that explains how to use the cool feature.
[Home](Readme.md#top)
```
* Crosslinks to different pages should target the `top` anchor, like this
`[link to contributing](contributing.md#top)`.
* We introduced version tags to the documentation, which show users in
which version a specific feature was introduced. This means that newly
written documentation should be tagged with a placeholder, that will
be replaced with the actual version upon release. There are 2 styles
of placeholders used through the documentation, you should pick one that
fits your text better (if in doubt, take a look at the existing version
tags for other features).
* `> [Introduced](link-to-issue-or-PR) in Catch2 X.Y.Z` - this
placeholder is usually used after a section heading
* `> X (Y and Z) was [introduced](link-to-issue-or-PR) in Catch2 X.Y.Z`
- this placeholder is used when you need to tag a subpart of something,
e.g. a list
* For pages with more than 4 subheadings, we provide a table of contents
(ToC) at the top of the page. Because GitHub markdown does not support
automatic generation of ToC, it has to be handled semi-manually. Thus,
@@ -130,24 +167,75 @@ if you've added a new subheading to some page, you should add it to the
ToC. This can be done either manually, or by running the
`updateDocumentToC.py` script in the `scripts/` folder.
### Contents
Now, for the generic tips:
* Usage examples are good
* Don't be afraid to introduce new pages
* Try to be reasonably consistent with the surrounding documentation
Now, for some content tips:
* Usage examples are good. However, having large code snippets inline
can make the documentation less readable, and so the inline snippets
should be kept reasonably short. To provide more complex compilable
examples, consider adding new .cpp file to `examples/`.
* Don't be afraid to introduce new pages. The current documentation
tends towards long pages, but a lot of that is caused by legacy, and
we know that some of the pages are overly big and unfocused.
* When adding information to an existing page, please try to keep your
formatting, style and changes consistent with the rest of the page.
* Any documentation has multiple different audiences, that desire
different information from the text. The 3 basic user-types to try and
cover are:
* A beginner to Catch2, who requires closer guidance for the usage of Catch2.
* Advanced user of Catch2, who want to customize their usage.
* Experts, looking for full reference of Catch2's capabilities.
## Writing code
If want to contribute code, this section contains some simple rules
and tips on things like code formatting, code constructions to avoid,
and so on.
### C++ standard version
Catch2 currently targets C++14 as the minimum supported C++ version.
Features from higher language versions should be used only sparingly,
when the benefits from using them outweight the maintenance overhead.
Example of good use of polyfilling features is our use of `conjunction`,
where if available we use `std::conjunction` and otherwise provide our
own implementation. The reason it is good is that the surface area for
maintenance is quite small, and `std::conjunction` can directly use
compiler built-ins, thus providing significant compilation benefits.
Example of bad use of polyfilling features would be to keep around two
sets of metaprogramming in the stringification implementation, once
using C++14 compliant TMP and once using C++17's `if constexpr`. While
the C++17 would provide significant compilation speedups, the maintenance
cost would be too high.
## Code constructs to watch out for
### Formatting
To make code formatting simpler for the contributors, Catch2 provides
its own config for `clang-format`. However, because it is currently
impossible to replicate existing Catch2's formatting in clang-format,
using it to reformat a whole file would cause massive diffs. To keep
the size of your diffs reasonable, you should only use clang-format
on the newly changed code.
### Code constructs to watch out for
This section is a (sadly incomplete) listing of various constructs that
are problematic and are not always caught by our CI infrastructure.
### Naked exceptions and exceptions-related function
#### Naked exceptions and exceptions-related function
If you are throwing an exception, it should be done via `CATCH_ERROR`
or `CATCH_RUNTIME_ERROR` in `catch_enforce.h`. These macros will handle
or `CATCH_RUNTIME_ERROR` in `internal/catch_enforce.hpp`. These macros will handle
the differences between compilation with or without exceptions for you.
However, some platforms (IAR) also have problems with exceptions-related
functions, such as `std::current_exceptions`. We do not have IAR in our
@@ -155,7 +243,20 @@ CI, but luckily there should not be too many reasons to use these.
However, if you do, they should be kept behind a
`CATCH_CONFIG_DISABLE_EXCEPTIONS` macro.
### Unqualified usage of functions from C's stdlib
#### Avoid `std::move` and `std::forward`
`std::move` and `std::forward` provide nice semantic name for a specific
`static_cast`. However, being function templates they have surprisingly
high cost during compilation, and can also have a negative performance
impact for low-optimization builds.
You should be using `CATCH_MOVE` and `CATCH_FORWARD` macros from
`internal/catch_move_and_forward.hpp` instead. They expand into the proper
`static_cast`, and avoid the overhead of `std::move` and `std::forward`.
#### Unqualified usage of functions from C's stdlib
If you are using a function from C's stdlib, please include the header
as `<cfoo>` and call the function qualified. The common knowledge that
@@ -163,7 +264,49 @@ there is no difference is wrong, QNX and VxWorks won't compile if you
include the header as `<cfoo>` and call the function unqualified.
----
#### User-Defined Literals (UDL) for Catch2' types
Due to messy standardese and ... not great ... implementation of
`-Wreserved-identifier` in Clang, avoid declaring UDLs as
```cpp
Approx operator "" _a(long double);
```
and instead declare them as
```cpp
Approx operator ""_a(long double);
```
Notice that the second version does not have a space between the `""` and
the literal suffix.
### New source file template
If you are adding new source file, there is a template you should use.
Specifically, every source file should start with the licence header:
```cpp
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
```
The include guards for header files should follow the pattern `{FILENAME}_INCLUDED`.
This means that for file `catch_matchers_foo`, the include guard should
be `CATCH_MATCHERS_FOO_INCLUDED`, for `catch_generators_bar`, the include
guard should be `CATCH_GENERATORS_BAR_INCLUDED`, and so on.
## CoC
This project has a [CoC](../CODE_OF_CONDUCT.md). Please adhere to it
while contributing to Catch2.
-----------
_This documentation will always be in-progress as new information comes
up, but we are trying to keep it as up to date as possible._

View File

@@ -9,17 +9,16 @@ either of these is a breaking change, and thus will not happen until
at least the next major release.
### `ParseAndAddCatchTests.cmake`
The CMake/CTest integration using `ParseAndAddCatchTests.cmake` is deprecated,
as it can be replaced by `Catch.cmake` that provides the function
`catch_discover_tests` to get tests directly from a CMake target via the
command line interface instead of parsing C++ code with regular expressions.
## Planned changes
### `CHECKED_IF` and `CHECKED_ELSE`
To make the `CHECKED_IF` and `CHECKED_ELSE` macros more useful, they will
be marked as "OK to fail" (`Catch::ResultDisposition::SuppressFail` flag
will be added), which means that their failure will not fail the test,
making the `else` actually useful.
### Console Colour API
The API for Catch2's console colour will be changed to take an extra

View File

@@ -1,74 +1,43 @@
<a id="top"></a>
# Event Listeners
A `Listener` is a class you can register with Catch that will then be passed events,
such as a test case starting or ending, as they happen during a test run.
`Listeners` are actually types of `Reporters`, with a few small differences:
1. Once registered in code they are automatically used - you don't need to specify them on the command line
2. They are called in addition to (just before) any reporters, and you can register multiple listeners.
3. They derive from `Catch::TestEventListenerBase`, which has default stubs for all the events,
so you are not forced to implement events you're not interested in.
4. You register a listener with `CATCH_REGISTER_LISTENER`
An event listener is a bit like a reporter, in that it responds to various
reporter events in Catch2, but it is not expected to write any output.
Instead, an event listener performs actions within the test process, such
as performing global initialization (e.g. of a C library), or cleaning out
in-memory logs if they are not needed (the test case passed).
Unlike reporters, each registered event listener is always active. Event
listeners are always notified before reporter(s).
## Implementing a Listener
Simply derive a class from `Catch::TestEventListenerBase` and implement the methods you are interested in, either in
the main source file (i.e. the one that defines `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`), or in a
file that defines `CATCH_CONFIG_EXTERNAL_INTERFACES`.
To write your own event listener, you should derive from `Catch::TestEventListenerBase`,
as it provides empty stubs for all reporter events, allowing you to
only override events you care for. Afterwards you have to register it
with Catch2 using `CATCH_REGISTER_LISTENER` macro, so that Catch2 knows
about it and instantiates it before running tests.
Then register it using `CATCH_REGISTER_LISTENER`.
Example event listener:
```cpp
#include <catch2/reporters/catch_reporter_event_listener.hpp>
#include <catch2/reporters/catch_reporter_registrars.hpp>
For example ([complete source code](../examples/210-Evt-EventListeners.cpp)):
class testRunListener : public Catch::EventListenerBase {
public:
using Catch::EventListenerBase::EventListenerBase;
```c++
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
struct MyListener : Catch::TestEventListenerBase {
using TestEventListenerBase::TestEventListenerBase; // inherit constructor
void testCaseStarting( Catch::TestCaseInfo const& testInfo ) override {
// Perform some setup before a test case is run
}
void testCaseEnded( Catch::TestCaseStats const& testCaseStats ) override {
// Tear-down after a test case is run
void testRunStarting(Catch::TestRunInfo const&) override {
lib_foo_init();
}
};
CATCH_REGISTER_LISTENER( MyListener )
CATCH_REGISTER_LISTENER(testRunListener)
```
_Note that you should not use any assertion macros within a Listener!_
## Events that can be hooked
[You can find the list of events that the listeners can react to on its
own page](reporter-events.md#top).
The following are the methods that can be overridden in the Listener:
```c++
// The whole test run, starting and ending
virtual void testRunStarting( TestRunInfo const& testRunInfo );
virtual void testRunEnded( TestRunStats const& testRunStats );
// Test cases starting and ending
virtual void testCaseStarting( TestCaseInfo const& testInfo );
virtual void testCaseEnded( TestCaseStats const& testCaseStats );
// Sections starting and ending
virtual void sectionStarting( SectionInfo const& sectionInfo );
virtual void sectionEnded( SectionStats const& sectionStats );
// Assertions before/ after
virtual void assertionStarting( AssertionInfo const& assertionInfo );
virtual bool assertionEnded( AssertionStats const& assertionStats );
// A test is being skipped (because it is "hidden")
virtual void skipTest( TestCaseInfo const& testInfo );
```
More information about the events (e.g. name of the test case) is contained in the structs passed as arguments -
just look in the source code to see what fields are available.
---

27
docs/faq.md Normal file
View File

@@ -0,0 +1,27 @@
<a id="top"></a>
# Frequently Asked Questions (FAQ)
## How do I run global setup/teardown only if tests will be run?
Write a custom [event listener](event-listeners.md#top) and place the
global setup/teardown code into the `testRun*` events.
## How do I clean up global state between running different tests?
Write a custom [event listener](event-listeners.md#top) and place the
cleanup code into either `testCase*` or `testCasePartial*` events,
depending on how often the cleanup needs to happen.
## Why cannot I derive from the built-in reporters?
They are not made to be overriden, in that we do not attempt to maintain
a consistent internal state if a member function is overriden, and by
forbidding users from using them as a base class, we can refactor them
as needed later.
---
[Home](Readme.md#top)

View File

@@ -1,7 +1,7 @@
<a id="top"></a>
# Data Generators
> Introduced in Catch 2.6.0.
> Introduced in Catch2 2.6.0.
Data generators (also known as _data driven/parametrized test cases_)
let you reuse the same set of assertions across different input values.
@@ -12,23 +12,88 @@ are run once per each value in a generator.
This is best explained with an example:
```cpp
TEST_CASE("Generators") {
auto i = GENERATE(1, 2, 3);
SECTION("one") {
auto j = GENERATE( -3, -2, -1 );
REQUIRE(j < i);
}
auto i = GENERATE(1, 3, 5);
REQUIRE(is_odd(i));
}
```
The assertion in this test case will be run 9 times, because there
are 3 possible values for `i` (1, 2, and 3) and there are 3 possible
values for `j` (-3, -2, and -1).
The "Generators" `TEST_CASE` will be entered 3 times, and the value of
`i` will be 1, 3, and 5 in turn. `GENERATE`s can also be used multiple
times at the same scope, in which case the result will be a cartesian
product of all elements in the generators. This means that in the snippet
below, the test case will be run 6 (2\*3) times.
```cpp
TEST_CASE("Generators") {
auto i = GENERATE(1, 2);
auto j = GENERATE(3, 4, 5);
}
```
There are 2 parts to generators in Catch2, the `GENERATE` macro together
with the already provided generators, and the `IGenerator<T>` interface
that allows users to implement their own generators.
## Combining `GENERATE` and `SECTION`.
`GENERATE` can be seen as an implicit `SECTION`, that goes from the place
`GENERATE` is used, to the end of the scope. This can be used for various
effects. The simplest usage is shown below, where the `SECTION` "one"
runs 4 (2\*2) times, and `SECTION` "two" is run 6 times (2\*3).
```cpp
TEST_CASE("Generators") {
auto i = GENERATE(1, 2);
SECTION("one") {
auto j = GENERATE(-3, -2);
REQUIRE(j < i);
}
SECTION("two") {
auto k = GENERATE(4, 5, 6);
REQUIRE(i != k);
}
}
```
The specific order of the `SECTION`s will be "one", "one", "two", "two",
"two", "one"...
The fact that `GENERATE` introduces a virtual `SECTION` can also be used
to make a generator replay only some `SECTION`s, without having to
explicitly add a `SECTION`. As an example, the code below reports 3
assertions, because the "first" section is run once, but the "second"
section is run twice.
```cpp
TEST_CASE("GENERATE between SECTIONs") {
SECTION("first") { REQUIRE(true); }
auto _ = GENERATE(1, 2);
SECTION("second") { REQUIRE(true); }
}
```
This can lead to surprisingly complex test flows. As an example, the test
below will report 14 assertions:
```cpp
TEST_CASE("Complex mix of sections and generates") {
auto i = GENERATE(1, 2);
SECTION("A") {
SUCCEED("A");
}
auto j = GENERATE(3, 4);
SECTION("B") {
SUCCEED("B");
}
auto k = GENERATE(5, 6);
SUCCEED();
}
```
> The ability to place `GENERATE` between two `SECTION`s was [introduced](https://github.com/catchorg/Catch2/issues/1938) in Catch2 2.13.0.
## Provided generators
Catch2's provided generator functionality consists of three parts,
@@ -49,12 +114,12 @@ a test case,
* 4 specific purpose generators
* `RandomIntegerGenerator<Integral>` -- generates random Integrals from range
* `RandomFloatGenerator<Float>` -- generates random Floats from range
* `RangeGenerator<T>` -- generates all values inside an arithmetic range
* `RangeGenerator<T>(first, last)` -- generates all values inside a `[first, last)` arithmetic range
* `IteratorGenerator<T>` -- copies and returns values from an iterator range
> `ChunkGenerator<T>`, `RandomIntegerGenerator<Integral>`, `RandomFloatGenerator<Float>` and `RangeGenerator<T>` were introduced in Catch 2.7.0.
> `ChunkGenerator<T>`, `RandomIntegerGenerator<Integral>`, `RandomFloatGenerator<Float>` and `RangeGenerator<T>` were introduced in Catch2 2.7.0.
> `IteratorGenerator<T>` was introduced in Catch 2.10.0.
> `IteratorGenerator<T>` was introduced in Catch2 2.10.0.
The generators also have associated helper functions that infer their
type, making their usage much nicer. These are
@@ -74,11 +139,11 @@ type, making their usage much nicer. These are
* `from_range(InputIterator from, InputIterator to)` for `IteratorGenerator<T>`
* `from_range(Container const&)` for `IteratorGenerator<T>`
> `chunk()`, `random()` and both `range()` functions were introduced in Catch 2.7.0.
> `chunk()`, `random()` and both `range()` functions were introduced in Catch2 2.7.0.
> `from_range` has been introduced in Catch 2.10.0
> `from_range` has been introduced in Catch2 2.10.0
> `range()` for floating point numbers has been introduced in Catch 2.11.0
> `range()` for floating point numbers has been introduced in Catch2 2.11.0
And can be used as shown in the example below to create a generator
that returns 100 odd random number:
@@ -111,7 +176,7 @@ scope and thus capturing references is dangerous. If you need to use
variables inside the generator expression, make sure you thought through
the lifetime implications and use `GENERATE_COPY` or `GENERATE_REF`.**
> `GENERATE_COPY` and `GENERATE_REF` were introduced in Catch 2.7.1.
> `GENERATE_COPY` and `GENERATE_REF` were introduced in Catch2 2.7.1.
You can also override the inferred type by using `as<type>` as the first
argument to the macro. This can be useful when dealing with string literals,

View File

@@ -94,39 +94,57 @@ to remove this limitation in the future.
### Process isolation in a test
Catch does not support running tests in isolated (forked) processes. While this might in the future, the fact that Windows does not support forking and only allows full-on process creation and the desire to keep code as similar as possible across platforms, mean that this is likely to take significant development time, that is not currently available.
### Running multiple tests in parallel
Catch's test execution is strictly serial. If you find yourself with a test suite that takes too long to run and you want to make it parallel, there are 2 feasible solutions
* You can split your tests into multiple binaries and then run these binaries in parallel.
* You can have Catch list contained test cases and then run the same test binary multiple times in parallel, passing each instance list of test cases it should run.
Both of these solutions have their problems, but should let you wring parallelism out of your test suite.
### Running multiple tests in parallel
Catch2 keeps test execution in one process strictly serial, and there
are no plans to change this. If you find yourself with a test suite
that takes too long to run and yo uwant to make it parallel, you have
to run multiple processes side by side.
There are 2 basic ways to do that,
* you can split your tests into multiple binaries, and run those binaries
in parallel
* you can run the same test binary multiple times, but run a different
subset of the tests in each process
There are multiple ways to achieve the latter, the easiest way is to use
[test sharding](command-line.md#test-sharding).
## 3rd party bugs
This section outlines known bugs in 3rd party components (this means compilers, standard libraries, standard runtimes).
### Visual Studio 2017 -- raw string literal in assert fails to compile
There is a known bug in Visual Studio 2017 (VC 15), that causes compilation error when preprocessor attempts to stringize a raw string literal (`#` preprocessor is applied to it). This snippet is sufficient to trigger the compilation error:
There is a known bug in Visual Studio 2017 (VC 15), that causes compilation
error when preprocessor attempts to stringize a raw string literal
(`#` preprocessor directive is applied to it). This snippet is sufficient
to trigger the compilation error:
```cpp
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
#include <catch2/catch_test_macros.hpp>
TEST_CASE("test") {
CHECK(std::string(R"("\)") == "\"\\");
}
```
Catch provides a workaround, it is possible to disable stringification of original expressions by defining `CATCH_CONFIG_DISABLE_STRINGIFICATION`:
Catch2 provides a workaround, by letting the user disable stringification
of the original expression by defining `CATCH_CONFIG_DISABLE_STRINGIFICATION`,
like so:
```cpp
#define CATCH_CONFIG_FAST_COMPILE
#define CATCH_CONFIG_DISABLE_STRINGIFICATION
#include "catch.hpp"
#include <catch2/catch_test_macros.hpp>
TEST_CASE("test") {
CHECK(std::string(R"("\)") == "\"\\");
}
```
_Do note that this changes the output somewhat_
_Do note that this changes the output:_
```
catchwork\test1.cpp(6):
PASSED:
@@ -135,26 +153,11 @@ with expansion:
""\" == ""\"
```
### Visual Studio 2015 -- Alignment compilation error (C2718)
VS 2015 has a known bug, where `declval<T>` can cause compilation error
if `T` has alignment requirements that it cannot meet.
A workaround is to explicitly specialize `Catch::is_range` for given
type (this avoids code path that uses `declval<T>` in a SFINAE context).
### Visual Studio 2015 -- Wrong line number reported in debug mode
VS 2015 has a known bug where `__LINE__` macro can be improperly expanded under certain circumstances, while compiling multi-file project in Debug mode.
A workaround is to compile the binary in Release mode.
### Clang/G++ -- skipping leaf sections after an exception
Some versions of `libc++` and `libstdc++` (or their runtimes) have a bug with `std::uncaught_exception()` getting stuck returning `true` after rethrow, even if there are no active exceptions. One such case is this snippet, which skipped the sections "a" and "b", when compiled against `libcxxrt` from master
```cpp
#define CATCH_CONFIG_MAIN
#include <catch.hpp>
#include <catch2/catch_test_macros.hpp>
TEST_CASE("a") {
CHECK_THROWS(throw 3);
@@ -170,11 +173,6 @@ TEST_CASE("b") {
If you are seeing a problem like this, i.e. a weird test paths that trigger only under Clang with `libc++`, or only under very specific version of `libstdc++`, it is very likely you are seeing this. The only known workaround is to use a fixed version of your standard library.
### Clang/G++ -- `Matches` string matcher always returns false
This is a bug in `libstdc++-4.8`, where all matching methods from `<regex>` return false. Since `Matches` uses `<regex>` internally, if the underlying implementation does not work, it doesn't work either.
Workaround: Use newer version of `libstdc++`.
### libstdc++, `_GLIBCXX_DEBUG` macro and random ordering of tests

View File

@@ -3,7 +3,6 @@
## Already available
- Catch main: [Catch-provided main](../examples/000-CatchMain.cpp)
- Test Case: [Single-file](../examples/010-TestCase.cpp)
- Test Case: [Multiple-file 1](../examples/020-TestCase-1.cpp), [2](../examples/020-TestCase-2.cpp)
- Assertion: [REQUIRE, CHECK](../examples/030-Asn-Require-Check.cpp)
@@ -14,6 +13,7 @@
- Configuration: [Provide your own output streams](../examples/231-Cfg-OutputStreams.cpp)
- Generators: [Create your own generator](../examples/300-Gen-OwnGenerator.cpp)
- Generators: [Use map to convert types in GENERATE expression](../examples/301-Gen-MapTypeConversion.cpp)
- Generators: [Run test with a table of input values](../examples/302-Gen-Table.cpp)
- Generators: [Use variables in generator expressions](../examples/310-Gen-VariablesInGenerators.cpp)
- Generators: [Use custom variable capture in generator expressions](../examples/311-Gen-CustomCapture.cpp)

View File

@@ -30,7 +30,7 @@ When the last `CHECK` fails in the "Bar" test case, then only one message will b
## Logging without local scope
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch 2.7.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch2 2.7.0.
`UNSCOPED_INFO` is similar to `INFO` with two key differences:
@@ -106,7 +106,7 @@ This semicolon will be removed with next major version. It is highly advised to
**UNSCOPED_INFO(** _message expression_ **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch 2.7.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch2 2.7.0.
Similar to `INFO`, but messages are not limited to their own scope: They are removed from the buffer after each assertion, section or test case, whichever comes first.

View File

@@ -35,10 +35,10 @@ operators, that is `&&`, `||`, and `!`, like so:
```cpp
using Catch::Matchers::EndsWith;
using Catch::Matchers::Contains;
using Catch::Matchers::ContainsSubstring;
REQUIRE_THAT( getSomeString(),
EndsWith("as a service") && Contains("web scale"));
EndsWith("as a service") && ContainsSubstring("web scale"));
```
The example above asserts that the string returned from `getSomeString`
@@ -58,13 +58,13 @@ to a use-after-free (UAF):
```cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_string.h>
#include <catch2/matchers/catch_matchers_string.hpp>
TEST_CASE("Bugs, bugs, bugs", "[Bug]"){
std::string str = "Bugs as a service";
auto match_expression = Catch::Matchers::EndsWith( "as a service" ) ||
(Catch::Matchers::StartsWith( "Big data" ) && !Catch::Matchers::Contains( "web scale" ) );
(Catch::Matchers::StartsWith( "Big data" ) && !Catch::Matchers::ContainsSubstring( "web scale" ) );
REQUIRE_THAT(str, match_expression);
}
```
@@ -88,7 +88,7 @@ Out of the box, Catch2 provides the following matchers:
Catch2 provides 5 different matchers that work with `std::string`,
* `StartsWith(std::string str, CaseSensitive)`,
* `EndsWith(std::string str, CaseSensitive)`,
* `Contains(std::string str, CaseSensitive)`,
* `ContainsSubstring(std::string str, CaseSensitive)`,
* `Equals(std::string str, CaseSensitive)`, and
* `Matches(std::string str, CaseSensitive)`.
@@ -99,9 +99,9 @@ somewhere inside it.
The `Equals` matcher matches a string if (and only if) the argument
string is equal to `str`.
Finally, the `Matches` matcher performs an ECMASCript regex match using
Finally, the `Matches` matcher performs an ECMAScript regex match using
`str` against the argument string. It is important to know that
the match is performed agains the string as a whole, meaning that
the match is performed against the string as a whole, meaning that
the regex `"abc"` will not match input string `"abcd"`. To match
`"abcd"`, you need to use e.g. `"abc.*"` as your regex.
@@ -125,7 +125,7 @@ These are
* `Equals` which checks whether the result is exactly equal (order matters) to a specific vector
* `UnorderedEquals` which checks whether the result is equal to a specific vector under a permutation
* `Approx` which checks whether the result is "approx-equal" (order matters, but comparison is done via `Approx`) to a specific vector
> Approx matcher was [introduced](https://github.com/catchorg/Catch2/issues/1499) in Catch 2.7.2.
> Approx matcher was [introduced](https://github.com/catchorg/Catch2/issues/1499) in Catch2 2.7.2.
An example usage:
```cpp
@@ -145,10 +145,10 @@ Catch2 provides 3 matchers that target floating point numbers. These
are:
* `WithinAbs(double target, double margin)`,
* `WithinUlps(FloatingPoint target, uint64_t maxUlpDiff)`, and
* `WithinULP(FloatingPoint target, uint64_t maxUlpDiff)`, and
* `WithinRel(FloatingPoint target, FloatingPoint eps)`.
> `WithinRel` matcher was introduced in Catch 2.10.0
> `WithinRel` matcher was introduced in Catch2 2.10.0
`WithinAbs` creates a matcher that accepts floating point numbers whose
@@ -161,6 +161,11 @@ away from the `target` value. The short version of what this means
is that there is no more than `maxUlpDiff - 1` representeable floating
point numbers between the argument for matching and the `target` value.
**Important**: The WithinULP matcher requires the platform to use the
[IEEE-754](https://en.wikipedia.org/wiki/IEEE_754) representation for
floating point numbers.
`WithinRel` creates a matcher that accepts floating point numbers that
are _approximately equal_ with the `target` with tolerance of `eps.`
Specifically, it matches if
@@ -243,7 +248,7 @@ Note that `DerivedException` in the example above has to derive from
### Generic range Matchers
> Generic range matchers were introduced in Catch X.Y.Z
> Generic range matchers were introduced in Catch2 X.Y.Z
Catch2 also provides some matchers that use the new style matchers
definitions to handle generic range-like types. These are:
@@ -294,7 +299,7 @@ within certain range. We will call it `IsBetweenMatcher<T>`:
```c++
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers.h>
#include <catch2/matchers/catch_matchers.hpp>
// ...
@@ -345,7 +350,7 @@ style matchers arbitrarily.
## Writing custom matchers (new style)
> New style matchers were introduced in Catch X.Y.Z
> New style matchers were introduced in Catch2 X.Y.Z
To create a new-style matcher, you have to create your own type that
derives from `Catch::Matchers::MatcherGenericBase`. Your type has to

84
docs/migrate-v2-to-v3.md Normal file
View File

@@ -0,0 +1,84 @@
<a id="top"></a>
# Migrating from v2 to v3
v3 is the next major version of Catch2 and brings three significant changes:
* Catch2 is now split into multiple headers
* Catch2 is now compiled as a static library
* C++14 is the minimum required C++ version
There are many reasons why we decided to go from the old single-header
distribution model to a more standard library distribution model. The
big one is compile-time performance, but moving over to a split header
distribution model also improves the future maintainability and
extendability of the codebase. For example v3 adds a new kind of matchers
without impacting the compilation times of users that do not use matchers
in their tests. The new model is also more friendly towards package
managers, such as vcpkg and Conan.
The result of this move is a significant improvement in compilation
times, e.g. the inclusion overhead of Catch2 in the common case has been
reduced by roughly 80%. The improved ease of maintenance also led to
various runtime performance improvements and the introduction of new features.
For details, look at [the release notes of 3.0.1](release-notes.md#301).
_Note that we still provide one header + one TU distribution but do
not consider it the primarily supported option. You should also expect
that the compilation times will be worse if you use this option._
## How to migrate projects from v2 to v3
To migrate to v3, there are two basic approaches to do so.
1. Use `catch_amalgamated.hpp` and `catch_amalgamated.cpp`.
2. Build Catch2 as a proper (static) library, and move to piecewise headers
Doing 1 means downloading the [amalgamated header](/extras/catch_amalgamated.hpp)
and the [amalgamated sources](/extras/catch_amalgamated.cpp) from `extras`,
dropping them into your test project, and rewriting your includes from
`<catch2/catch.hpp>` to `"catch_amalgamated.hpp"` (or something similar,
based on how you set up your paths).
The disadvantage of using this approach are increased compilation times,
at least compared to the second approach, but it does let you avoid
dealing with consuming libraries in your build system of choice.
However, we recommend doing 2, and taking extra time to migrate to v3
properly. This lets you reap the benefits of significantly improved
compilation times in the v3 version. The basic steps to do so are:
1. Change your CMakeLists.txt to link against `Catch2WithMain` target if
you use Catch2's default main. (If you do not, keep linking against
the `Catch2` target.)
2. Delete TU with `CATCH_CONFIG_RUNNER` or `CATCH_CONFIG_MAIN` defined,
as it is no longer needed.
3. Change `#include <catch2/catch.hpp>` to `#include <catch2/catch_all.hpp>`
4. Check that everything compiles. You might have to modify namespaces,
or perform some other changes (see the
[Things that can break during porting](#things-that-can-break-during-porting)
section for the most common things).
5. Start migrating your test TUs from including `<catch2/catch_all.hpp>`
to piecemeal includes. You will likely want to start by including
`<catch2/catch_test_macros.hpp>`, and then go from there. (see
[other notes](#other-notes) for further ideas)
## Other notes
* The main test include is now `<catch2/catch_test_macros.hpp>`
* Big "subparts" like Matchers, or Generators, have their own folder, and
also their own "big header", so if you just want to include all matchers,
you can include `<catch2/matchers/catch_matchers_all.hpp>`,
or `<catch2/generators/catch_generators_all.hpp>`
## Things that can break during porting
* The namespace on Matchers were cleaned up, they are no longer first declared
deep within an internal namespace and then brought up. All Matchers now live
in `Catch::Matchers`.
* The reporter interfaces changed in a breaking manner. If you wrote custom
reporter or listener, you might need to modify them a bit.
---
[Home](Readme.md#top)

View File

@@ -1,22 +1,28 @@
<a id="top"></a>
# Open Source projects using Catch
# Open Source projects using Catch2
Catch is great for open source. With its [liberal license](../LICENSE.txt) and single-header, dependency-free, distribution
it's easy to just drop the header into your project and start writing tests - what's not to like?
Catch2 is great for open source. It is licensed under the [Boost Software
License (BSL)](../LICENSE.txt), has no further dependencies and supports
two file distribution.
As a result Catch is now being used in many Open Source projects, including some quite well known ones.
This page is an attempt to track those projects. Obviously it can never be complete.
This effort largely relies on the maintainers of the projects themselves updating this page and submitting a PR
(or, if you prefer contact one of the maintainers of Catch directly, use the
[forums](https://groups.google.com/forum/?fromgroups#!forum/catch-forum)), or raise an [issue](https://github.com/philsquared/Catch/issues) to let us know).
Of course users of those projects might want to update this page too. That's fine - as long you're confident the project maintainers won't mind.
If you're an Open Source project maintainer and see your project listed here but would rather it wasn't -
just let us know via any of the previously mentioned means - although I'm sure there won't be many who feel that way.
As a result, Catch2 is used for testing in many different Open Source
projects. This page lists at least some of them, even though it will
obviously never be complete (and does not have the ambition to be
complete). Note that the list below is intended to be in alphabetical
order, to avoid implications of relative importance of the projects.
_Please only add projects here if you are their maintainer, or have the
maintainer's explicit consent._
Listing a project here does not imply endorsement and the plan is to keep these ordered alphabetically to avoid an implication of relative importance.
## Libraries & Frameworks
### [accessorpp](https://github.com/wqking/accessorpp)
C++ library for implementing property and data binding.
### [alpaka](https://github.com/alpaka-group/alpaka)
A header-only C++14 abstraction library for accelerator development.
### [ApprovalTests.cpp](https://github.com/approvals/ApprovalTests.cpp)
C++11 implementation of Approval Tests, for quick, convenient testing of legacy code.
@@ -47,15 +53,24 @@ Header-only C++11 library to encode/decode base64, base64url, base32, base32hex
### [DtCraft](https://github.com/twhuang-uiuc/DtCraft)
A High-performance Cluster Computing Engine.
### [eventpp](https://github.com/wqking/eventpp)
C++ event library for callbacks, event dispatcher, and event queue. With eventpp you can easily implement signal and slot mechanism, publisher and subscriber pattern, or observer pattern.
### [forest](https://github.com/xorz57/forest)
Template Library of Tree Data Structures.
### [Fuxedo](https://github.com/fuxedo/fuxedo)
Open source Oracle Tuxedo-like XATMI middleware for C and C++.
### [HIP CPU Runtime](https://github.com/ROCm-Developer-Tools/HIP-CPU)
A header-only library that allows CPUs to execute unmodified HIP code. It is generic and does not assume a particular CPU vendor or architecture.
### [Inja](https://github.com/pantor/inja)
A header-only template engine for modern C++.
### [LLAMA](https://github.com/alpaka-group/llama)
A C++17 template header-only library for the abstraction of memory access patterns.
### [libcluon](https://github.com/chrberger/libcluon)
A single-header-only library written in C++14 to glue distributed software components (UDP, TCP, shared memory) supporting natively Protobuf, LCM/ZCM, MsgPack, and JSON for dynamic message transformations in-between.
@@ -68,6 +83,9 @@ A small C++ library wrapper for the native C ODBC API.
### [Nonius](https://github.com/libnonius/nonius)
A header-only framework for benchmarking small snippets of C++ code.
### [OpenALpp](https://github.com/Laguna1989/OpenALpp)
A modern OOP C++14 audio library built on OpenAL for Windows, Linux and web (emscripten).
### [polymorphic_value](https://github.com/jbcoe/polymorphic_value)
A polymorphic value-type for C++.
@@ -94,9 +112,15 @@ A thread-safe header-only mocking framework for C++14.
## Applications & Tools
### [App Mesh](https://github.com/laoshanxi/app-mesh)
A high available cloud native micro-service application management platform implemented by modern C++.
### [ArangoDB](https://github.com/arangodb/arangodb)
ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values.
### [d-SEAMS](https://github.com/d-SEAMS/seams-core)
Open source molecular dynamics simulation structure analysis suite of tools in modern C++.
### [Giada - Your Hardcore Loop Machine](https://github.com/monocasual/giada)
Minimal, open-source and cross-platform audio tool for live music production.

View File

@@ -15,6 +15,8 @@ stringification machinery to the _expr_ and records the result. As with
evaluates to `true`. `CHECKED_ELSE( expr )` work similarly, but the block
is entered only if the _expr_ evaluated to `false`.
> `CHECKED_X` macros were changed to not count as failure in Catch2 X.Y.Z.
Example:
```cpp
int a = ...;
@@ -57,9 +59,9 @@ TEST_CASE( "SUCCEED showcase" ) {
}
```
* `STATIC_REQUIRE`
* `STATIC_REQUIRE` and `STATIC_CHECK`
> [Introduced](https://github.com/catchorg/Catch2/issues/1362) in Catch 2.4.2.
> `STATIC_REQUIRE` was [introduced](https://github.com/catchorg/Catch2/issues/1362) in Catch2 2.4.2.
`STATIC_REQUIRE( expr )` is a macro that can be used the same way as a
`static_assert`, but also registers the success with Catch2, so it is
@@ -75,6 +77,20 @@ TEST_CASE("STATIC_REQUIRE showcase", "[traits]") {
}
```
> `STATIC_CHECK` was [introduced](https://github.com/catchorg/Catch2/pull/2318) in Catch2 X.Y.Z.
`STATIC_CHECK( expr )` is equivalent to `STATIC_REQUIRE( expr )`, with the
difference that when `CATCH_CONFIG_RUNTIME_STATIC_REQUIRE` is defined, it
becomes equivalent to `CHECK` instead of `REQUIRE`.
Example:
```cpp
TEST_CASE("STATIC_CHECK showcase", "[traits]") {
STATIC_CHECK( std::is_void<void>::value );
STATIC_CHECK_FALSE( std::is_void<int>::value );
}
```
## Test case related macros
* `METHOD_AS_TEST_CASE`
@@ -117,24 +133,9 @@ is initiated. This means that it either needs to be done in a global
constructor, or before Catch2's session is created in user's own main._
* `ANON_TEST_CASE`
`ANON_TEST_CASE` is a `TEST_CASE` replacement that will autogenerate
unique name. The advantage of this is that you do not have to think
of a name for the test case,`the disadvantage is that the name doesn't
necessarily remain stable across different links, and thus it might be
hard to run directly.
Example:
```cpp
ANON_TEST_CASE() {
SUCCEED("Hello from anonymous test case");
}
```
* `DYNAMIC_SECTION`
> Introduced in Catch 2.3.0.
> Introduced in Catch2 2.3.0.
`DYNAMIC_SECTION` is a `SECTION` where the user can use `operator<<` to
create the final name for that section. This can be useful with e.g.

View File

@@ -2,63 +2,69 @@
# Supplying main() yourself
**Contents**<br>
[Let Catch take full control of args and config](#let-catch-take-full-control-of-args-and-config)<br>
[Amending the config](#amending-the-config)<br>
[Let Catch2 take full control of args and config](#let-catch2-take-full-control-of-args-and-config)<br>
[Amending the Catch2 config](#amending-the-catch2-config)<br>
[Adding your own command line options](#adding-your-own-command-line-options)<br>
[Version detection](#version-detection)<br>
The easiest way to use Catch is to let it supply ```main()``` for you and handle configuring itself from the command line.
The easiest way to use Catch2 is to use its own `main` function, and let
it handle the command line arguments. This is done by linking against
Catch2Main library, e.g. through the [CMake target](cmake-integration.md#cmake-targets),
or pkg-config files.
This is achieved by writing ```#define CATCH_CONFIG_MAIN``` before the ```#include "catch.hpp"``` in *exactly one* source file.
If you want to provide your own `main`, then you should link against
the static library (target) only, without the main part. You will then
have to write your own `main` and call into Catch2 test runner manually.
Sometimes, though, you need to write your own version of main(). You can do this by writing ```#define CATCH_CONFIG_RUNNER``` instead. Now you are free to write ```main()``` as normal and call into Catch yourself manually.
Below are some basic recipes on what you can do supplying your own main.
You now have a lot of flexibility - but here are three recipes to get your started:
## Let Catch take full control of args and config
## Let Catch2 take full control of args and config
If you just need to have code that executes before and/ or after Catch this is the simplest option.
This is useful if you just need to have code that executes before/after
Catch2 runs tests.
```c++
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
```cpp
#include <catch2/catch_session.hpp>
int main( int argc, char* argv[] ) {
// global setup...
// your setup ...
int result = Catch::Session().run( argc, argv );
// global clean-up...
// your clean-up...
return result;
}
```
## Amending the config
_Note that if you only want to run some set up before tests are run, it
might be simpler to use [event listeners](event-listeners.md#top) instead._
If you still want Catch to process the command line, but you want to programmatically tweak the config, you can do so in one of two ways:
## Amending the Catch2 config
If you want Catch2 to process command line arguments, but also want to
programmatically change the resulting configuration of Catch2 run,
you can do it in two ways:
```c++
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
int main( int argc, char* argv[] )
{
int main( int argc, char* argv[] ) {
Catch::Session session; // There must be exactly one instance
// writing to session.configData() here sets defaults
// this is the preferred way to set them
int returnCode = session.applyCommandLine( argc, argv );
if( returnCode != 0 ) // Indicates a command line error
return returnCode;
// writing to session.configData() or session.Config() here
// writing to session.configData() or session.Config() here
// overrides command line args
// only do this if you know you need to
int numFailed = session.run();
// numFailed is clamped to 255 as some unices only use the lower 8 bits.
// This clamping has already been applied, so just return it here
// You can also do any post run clean-up here
@@ -66,38 +72,32 @@ int main( int argc, char* argv[] )
}
```
Take a look at the definitions of Config and ConfigData to see what you can do with them.
If you want full control of the configuration, don't call `applyCommandLine`.
To take full control of the config simply omit the call to ```applyCommandLine()```.
## Adding your own command line options
Catch embeds a powerful command line parser called [Clara](https://github.com/philsquared/Clara).
As of Catch2 (and Clara 1.0) Clara allows you to write _composable_ option and argument parsers,
so extending Catch's own command line options is now easy.
You can add new command line options to Catch2, by composing the premade
CLI parser (called Clara), and add your own options.
```c++
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
int main( int argc, char* argv[] )
{
```cpp
int main( int argc, char* argv[] ) {
Catch::Session session; // There must be exactly one instance
int height = 0; // Some user variable you want to be able to set
// Build a new parser on top of Catch's
using namespace Catch::clara;
auto cli
= session.cli() // Get Catch's composite command line parser
// Build a new parser on top of Catch2's
using namespace Catch::Clara;
auto cli
= session.cli() // Get Catch2's command line parser
| Opt( height, "height" ) // bind variable to a new option, with a hint string
["-g"]["--height"] // the option names it will respond to
("how high?"); // description string for the help output
// Now pass the new composite back to Catch so it uses that
session.cli( cli );
// Let Catch (using Clara) parse the command line
// Now pass the new composite back to Catch2 so it uses that
session.cli( cli );
// Let Catch2 (using Clara) parse the command line
int returnCode = session.applyCommandLine( argc, argv );
if( returnCode != 0 ) // Indicates a command line error
return returnCode;
@@ -110,12 +110,13 @@ int main( int argc, char* argv[] )
}
```
See the [Clara documentation](https://github.com/philsquared/Clara/blob/master/README.md) for more details.
See the [Clara documentation](https://github.com/catchorg/Clara/blob/master/README.md)
for more details on how to use the Clara parser.
## Version detection
Catch provides a triplet of macros providing the header's version,
Catch2 provides a triplet of macros providing the header's version,
* `CATCH_VERSION_MAJOR`
* `CATCH_VERSION_MINOR`

View File

@@ -2,7 +2,18 @@
# Release notes
**Contents**<br>
[3.0.1](#301)<br>
[3.0.1 (in progress)](#301-in-progress)<br>
[2.13.7](#2137)<br>
[2.13.6](#2136)<br>
[2.13.5](#2135)<br>
[2.13.4](#2134)<br>
[2.13.3](#2133)<br>
[2.13.2](#2132)<br>
[2.13.1](#2131)<br>
[2.13.0](#2130)<br>
[2.12.4](#2124)<br>
[2.12.3](#2123)<br>
[2.12.2](#2122)<br>
[2.12.1](#2121)<br>
[2.12.0](#2120)<br>
[2.11.3](#2113)<br>
@@ -44,19 +55,9 @@
This also means that to get all of Catch2's functionality in a test file,
you have to include multiple headers.**
For quick'n'dirty migration, you can replace the old `#include <catch2/catch.hpp>`
with `#include <catch2/catch_all.hpp>`. This is a (one of) convenience
header(s) that brings in _all_ of headers in Catch2. By doing this,
you should be able to migrate instantly, but at the cost of (significantly)
increased compilation times. You should prefer piecemeal including
headers that are actually required by your test code.
The basic set of functionality (`TEST_CASE`, `SECTION`, `REQUIRE`) is in
`catch2/catch_test_macros.hpp`. Matchers are in `matchers` subfolder,
generators in `generators` subfolder, and so on.
Note that documentation has not yet been updated to account for the
new design.
You probably want to look into the [migration docs](migrate-v2-to-v3.md#top),
which were written to help people coming from v2.x.x versions to the
v3 releases.
### FAQ
@@ -64,14 +65,15 @@ new design.
* Why is Catch2 moving to separate headers?
* The short answer is future extensibility and scalability. The long answer is complex and can be found on my blog, but at the most basic level, it is that providing single-header distribution is at odds with providing variety of useful features. When Catch2 was distributed in a single header, adding a new Matcher would cause overhead for everyone, but was useful only to a subset of users. This meant that the barrier to entry for new Matchers/Generators/etc is high in single header model, but much smaller in the new model.
* Will Catch2 again distribute single-header version in the future?
* No. But I intend to provide sqlite-style distribution option, with 1 header and 1 "unity" .cpp file. Do note that the header will have similar problem to the `catch_all.hpp` header.
* No. But we do provide sqlite-style amalgamated distribution option. This means that you can download just 1 .cpp file and 1 header and place them next to your own sources. However, doing this has downsides similar to using the `catch_all.hpp` header.
* Why the big breaking change caused by replacing `catch.hpp` with `catch_all.hpp`?
* The convenience header `catch_all.hpp` exists for two reasons. One of them is to provide a way for quick migration from Catch2, the second one is to provide a simple way to test things with Catch2. Using it for migration has one drawback in that it is **big**. This means that including it _will_ cause significant compile time drag, and so using it to migrate should be a concious decision by the user, not something they can just stumble into unknowingly.
* The convenience header `catch_all.hpp` exists for two reasons. One of them is to provide a way for quick migration from Catch2, the second one is to provide a simple way to test things with Catch2. Using it for migration has one drawback in that it is **big**. This means that including it _will_ cause significant compile time drag, and so using it to migrate should be a conscious decision by the user, not something they can just stumble into unknowingly.
### (Potentially) Breaking changes
* **Catch2 now uses statically compiled library as its distribution model**
* **Including `catch.hpp` no longer works**
* **Catch2 now uses C++14 as the minimum support language version**
* `ANON_TEST_CASE` has been removed, use `TEST_CASE` with no arguments instead (#1220)
* `--list*` commands no longer have non-zero return code (#1410)
* `--list-test-names-only` has been removed (#1190)
@@ -110,7 +112,24 @@ new design.
* You should instead include the appropriate headers as needed.
* `CATCH_CONFIG_IMPL` has been removed.
* The implementation is now compiled into a static library.
* Event Listener interface has changed
* `TestEventListenerBase` was renamed to `EventListenerBase`
* `EventListenerBase` now directly derives from `IStreamingReporter`, instead of deriving from `StreamingReporterBase`
* `GENERATE` decays its arguments (#2012, #2040)
* This means that `str` in `auto str = GENERATE("aa", "bb", "cc");` is inferred to `char const*` rather than `const char[2]`.
* `--list-*` flags write their output to file specified by the `-o` flag
* Many changes to reporter interfaces
* With the exception of the XmlReporter, the outputs of first party reporters should remain the same
* New pair of events were added
* One obsolete event was removed
* The base class has been renamed
* The built-in reporter class hierarchy has been redone
* Catch2 generates a random seed if one hasn't been specified by the user
* The short flag for `--list-tests`, `-l`, has been removed.
* This is not a commonly used flag and does not need to use up valuable single-letter space.
* The short flag for `--list-tags`, `-t`, has been removed.
* This is not a commonly used flag and does not need to use up valuable single-letter space.
* The `--colour` option has been replaced with `--colour-mode` option
### Improvements
@@ -118,28 +137,243 @@ new design.
* This includes having templated `match` member function
* See the [rewritten Matchers documentation](matchers.md#top) for details
* Catch2 currently provides _some_ generic matchers, but there should be more before final release of v3
* So far, `IsEmpty`, `SizeIs`, and `Contains` are provided.
* At least `ElementsAre` and `UnorderedElementsAre` are planned.
* `IsEmpty`, `SizeIs` which check that the range has specific properties
* `Contains`, which checks whether a range contains a specific element
* `AllMatch`, `AnyMatch`, `NoneMatch` range matchers, which apply matchers over a range of elements
* Significant compilation time improvements
* including `catch_test_macros.hpp` is 80% cheaper than including `catch.hpp`
* Some runtime performance optimizations
* In all tested cases the v3 branch was faster, so the table below shows the speedup of v3 to v2 at the same task
<a id="v3-runtime-optimization-table"></a>
| task | debug build | release build |
|:------------------------------------------- | ------------:| -------------:|
| Run 1M `REQUIRE(true)` | 1.10 ± 0.01 | 1.02 ± 0.06 |
| Run 100 tests, 3^3 sections, 1 REQUIRE each | 1.27 ± 0.01 | 1.04 ± 0.01 |
| Run 3k tests, no names, no tags | 1.29 ± 0.01 | 1.05 ± 0.01 |
| Run 3k tests, names, tags | 1.49 ± 0.01 | 1.22 ± 0.01 |
| Run 1 out of 3k tests no names, no tags | 1.68 ± 0.02 | 1.19 ± 0.22 |
| Run 1 out of 3k tests, names, tags | 1.79 ± 0.02 | 2.06 ± 0.23 |
* POSIX platforms use `gmtime_r`, rather than `gmtime` when constructing a date string (#2008, #2165)
* `--list-*` flags write their output to file specified by the `-o` flag (#2061, #2163)
* `Approx::operator()` is now properly `const`
* Catch2's internal helper variables no longer use reserved identifiers (#578)
* `--rng-seed` now accepts string `"random-device"` to generate random seed using `std::random_device`
* Catch2 now supports test sharding (#2257)
* You can ask for the tests to be split into N groups and only run one of them.
* This greatly simplifies parallelization of tests in a binary through external runner.
* The embedded CLI parser now supports repeatedly callable lambdas
* A lambda-based option parser can opt into being repeatedly specifiable.
* Added `STATIC_CHECK` macro, similar to `STATIC_REQUIRE` (#2318)
* When deferred tu runtime, it behaves like `CHECK`, and not like `REQUIRE`.
* You can have multiple tests with the same name, as long as other parts of the test identity differ (#1915, #1999, #2175)
* Test identity includes test's name, test's tags and and test's class name if applicable.
* Added new warning, `UnmatchedTestSpec`, to error on test specs with no matching tests
* The `-w`, `--warn` warning flags can now be provided multiple times to enable multiple warnings
* The case-insensitive handling of tags is now more reliable and takes up less memory
* Test case and assertion counting can no longer reasonably overflow on 32 bit systems
* The count is now kept in `uint64_t` on all platforms, instead of using `size_t` type.
* The `-o`, `--out` output destination specifiers recognize `-` as stdout
* You have to provide it as `--out=-` to avoid CLI error about missing option
* The new reporter specification also recognizes `-` as stdout
* Multiple reporters can now run at the same time and write to different files (#1712, #2183)
* To support this, the `-r`, `--reporter` flag now also accepts optional output destination
* For full overview of the semantics of using multiple reporters, look into the reporter documentation
* To enable the new syntax, reporter names can no longer contain `::`.
* Console colour support has been rewritten and significantly improved
* The colour implementation based on ANSI colour codes is always available
* Colour implementations respect their associated stream
* previously e.g. Win32 impl would change console colour even if Catch2 was writing to a file
* The colour API is resilient against changing evaluation order of expressions
* The associated CLI flag and compile-time configuration options have changed
* For details see the docs for command-line and compile-time Catch2 configuration
* Added a support for Bazel integration with `XML_OUTPUT_FILE` env var (#2399)
* This has to be enabled during compilation.
### Fixes
* The `INFO` macro no longer contains superfluous semicolon (#1456)
* The `--list*` family of command line flags now return 0 on success (#1410, #1146)
* Various ways of failing a benchmark are now counted and reporter properly
* The ULP matcher now handles comparing numbers with different signs properly (#2152)
* Universal ADL-found operators should no longer break decomposition (#2121)
* Reporter selection is properly case-insensitive
* Previously it forced lower cased name, which would fail for reporters with upper case characters in name
* The cumulative reporter base stores benchmark results alongside assertion results
* Catch2's SE handling should no longer interferes with ASan on Windows (#2334)
* Fixed Windows console colour handling for tests that redirect stdout (#2345)
### Other changes
* `CATCH_CONFIG_DISABLE_MATCHERS` no longer exists.
* If you do not want to use Matchers in a TU, do not include their header.
* `CATCH_CONFIG_ENABLE_CHRONO_STRINGMAKER` no longer exists.
* `StringMaker` specializations for <chrono> are always provided
* `StringMaker` specializations for `<chrono>` are always provided
* Catch2's CMake now provides 2 targets, `Catch2` and `Catch2WithMain`.
* `Catch2` is the statically compiled implementation by itself
* `Catch2WithMain` also links in the default main
* Catch2's pkg-config integration also provides 2 packages
* `catch2` is the statically compiled implementation by itself
* `catch2-with-main` also links in the default main
* Passing invalid test specifications passed to Catch2 are now reported before tests are run, and are a hard error.
* Running 0 tests (e.g. due to empty binary, or test spec not matching anything) returns non-0 exit code
* Flag `--allow-running-no-tests` overrides this behaviour.
* `NoTests` warning has been removed because it is fully subsumed by this change.
* Catch2's compile-time configuration options (`CATCH_CONFIG_FOO`) can be set through CMake options of the same name
* They use the same semantics as C++ defines, including the `CATCH_CONFIG_NO_FOO` overrides,
* `-DCATCH_CONFIG_DEFAULT_REPORTER=compact` changes default reporter to "compact"
* `-DCATCH_CONFIG_NO_ANDROID_LOGWRITE=ON` forces android logwrite to off
* `-DCATCH_CONFIG_ANDROID_LOGWRITE=OFF` does nothing (the define will not exist)
## 2.13.7
### Fixes
* Added missing `<iterator>` include in benchmarking. (#2231)
* Fixed noexcept build with benchmarking enabled (#2235)
* Fixed build for compilers with C++17 support but without C++17 library support (#2195)
* JUnit only uses 3 decimal places when reporting durations (#2221)
* `!mayfail` tagged tests are now marked as `skipped` in JUnit reporter output (#2116)
## 2.13.6
### Fixes
* Disabling all signal handlers no longer breaks compilation (#2212, #2213)
### Miscellaneous
* `catch_discover_tests` should handle escaped semicolon (`;`) better (#2214, #2215)
## 2.13.5
### Improvements
* Detection of MAC and IPHONE platforms has been improved (#2140, #2157)
* Added workaround for bug in XLC 16.1.0.1 (#2155)
* Add detection for LCC when it is masquerading as GCC (#2199)
* Modified posix signal handling so it supports newer libcs (#2178)
* `MINSIGSTKSZ` was no longer usable in constexpr context.
### Fixes
* Fixed compilation of benchmarking when `min` and `max` macros are defined (#2159)
* Including `windows.h` without `NOMINMAX` remains a really bad idea, don't do it
### Miscellaneous
* The check whether Catch2 is being built as a subproject is now more reliable (#2202, #2204)
* The problem was that if the variable name used internally was defined the project including Catch2 as subproject, it would not be properly overwritten for Catch2's CMake.
## 2.13.4
### Improvements
* Improved the hashing algorithm used for shuffling test cases (#2070)
* `TEST_CASE`s that differ only in the last character should be properly shuffled
* Note that this means that v2.13.4 gives you a different order of test cases than 2.13.3, even given the same seed.
### Miscellaneous
* Deprecated `ParseAndAddCatchTests` CMake integration (#2092)
* It is impossible to implement it properly for all the different test case variants Catch2 provides, and there are better options provided.
* Use `catch_discover_tests` instead, which uses runtime information about available tests.
* Fixed bug in `catch_discover_tests` that would cause it to fail when used in specific project structures (#2119)
* Added Bazel build file
* Added an experimental static library target to CMake
## 2.13.3
### Fixes
* Fixed possible infinite loop when combining generators with section filter (`-c` option) (#2025)
### Miscellaneous
* Fixed `ParseAndAddCatchTests` not finding `TEST_CASE`s without tags (#2055, #2056)
* `ParseAndAddCatchTests` supports `CMP0110` policy for changing behaviour of `add_test` (#2057)
* This was the shortlived change in CMake 3.18.0 that temporarily broke `ParseAndAddCatchTests`
## 2.13.2
### Improvements
* Implemented workaround for AppleClang shadowing bug (#2030)
* Implemented workaround for NVCC ICE (#2005, #2027)
### Fixes
* Fixed detection of `std::uncaught_exceptions` support under non-msvc platforms (#2021)
* Fixed the experimental stdout/stderr capture under Windows (#2013)
### Miscellaneous
* `catch_discover_tests` has been improved significantly (#2023, #2039)
* You can now specify which reporter should be used
* You can now modify where the output will be written
* `WORKING_DIRECTORY` setting is respected
* `ParseAndAddCatchTests` now supports `TEMPLATE_TEST_CASE` macros (#2031)
* Various documentation fixes and improvements (#2022, #2028, #2034)
## 2.13.1
### Improvements
* `ParseAndAddCatchTests` handles CMake v3.18.0 correctly (#1984)
* Improved autodetection of `std::byte` (#1992)
* Simplified implementation of templated test cases (#2007)
* This should have a tiny positive effect on its compilation throughput
### Fixes
* Automatic stringification of ranges handles sentinel ranges properly (#2004)
## 2.13.0
### Improvements
* `GENERATE` can now follow a `SECTION` at the same level of nesting (#1938)
* The `SECTION`(s) before the `GENERATE` will not be run multiple times, the following ones will.
* Added `-D`/`--min-duration` command line flag (#1910)
* If a test takes longer to finish than the provided value, its name and duration will be printed.
* This flag is overriden by setting `-d`/`--duration`.
### Fixes
* `TAPReporter` no longer skips successful assertions (#1983)
## 2.12.4
### Improvements
* Added support for MacOS on ARM (#1971)
## 2.12.3
### Fixes
* `GENERATE` nested in a for loop no longer creates multiple generators (#1913)
* Fixed copy paste error breaking `TEMPLATE_TEST_CASE_SIG` for 6 or more arguments (#1954)
* Fixed potential UB when handling non-ASCII characters in CLI args (#1943)
### Improvements
* There can be multiple calls to `GENERATE` on a single line
* Improved `fno-except` support for platforms that do not provide shims for exception-related std functions (#1950)
* E.g. the Green Hills C++ compiler
* XmlReporter now also reports test-case-level statistics (#1958)
* This is done via a new element, `OverallResultsCases`
### Miscellaneous
* Added `.clang-format` file to the repo (#1182, #1920)
* Rewrote contributing docs
* They should explain the different levels of testing and so on much better
## 2.12.2
### Fixes
* Fixed compilation failure if `is_range` ADL found deleted function (#1929)
* Fixed potential UB in `CAPTURE` if the expression contained non-ASCII characters (#1925)
### Improvements
* `std::result_of` is not used if `std::invoke_result` is available (#1934)
* JUnit reporter writes out `status` attribute for tests (#1899)
* Suppresed clang-tidy's `hicpp-vararg` warning (#1921)
* Catch2 was already suppressing the `cppcoreguidelines-pro-type-vararg` alias of the warning
## 2.12.1
@@ -686,7 +920,7 @@ than `single_include/catch.hpp`.**
* CLR objects (`T^`) can now be stringified (#1216)
* This affects code compiled as C++/CLI
* Added `PredicateMatcher`, a matcher that takes an arbitrary predicate function (#1236)
* See [documentation for details](https://github.com/catchorg/Catch2/blob/master/docs/matchers.md)
* See [documentation for details](https://github.com/catchorg/Catch2/blob/devel/docs/matchers.md)
### Others
* Modified CMake-installed pkg-config to allow `#include <catch.hpp>`(#1239)
@@ -714,7 +948,7 @@ than `single_include/catch.hpp`.**
* Added an option to warn (+ exit with error) when no tests were ran (#1158)
* Use as `-w NoTests`
* Added provisional support for Emscripten (#1114)
* [Added a way to override the fallback stringifier](https://github.com/catchorg/Catch2/blob/master/docs/configuration.md#fallback-stringifier) (#1024)
* [Added a way to override the fallback stringifier](https://github.com/catchorg/Catch2/blob/devel/docs/configuration.md#fallback-stringifier) (#1024)
* This allows project's own stringification machinery to be easily reused for Catch
* `Catch::Session::run()` now accepts `char const * const *`, allowing it to accept array of string literals (#1031, #1178)
* The embedded version of Clara was bumped to v1.1.3

View File

@@ -40,14 +40,10 @@ After version number is incremented, single-include header is regenerated and re
After pushing changes to GitHub, GitHub release *needs* to be created.
Tag version and release title should be same as the new version,
description should contain the release notes for the current release.
Single header version of `catch.hpp` *needs* to be attached as a binary,
as that is where the official download link links to. Preferably
it should use linux line endings. All non-bundled reporters (Automake, TAP,
TeamCity, SonarQube) should also be attached as binaries, as they might be
dependent on a specific version of the single-include header.
We also attach the two amalgamated files as "binaries".
Since 2.5.0, the release tag and the "binaries" (headers) should be PGP
signed.
Since 2.5.0, the release tag and the "binaries" (amalgamated files) should
be PGP signed.
#### Signing a tag
@@ -57,16 +53,14 @@ is the version being released, e.g. `git tag -s v2.6.0`.
Use the version name as the short message and the release notes as
the body (long) message.
#### Signing the headers
#### Signing the amalgamated files
This will create ASCII-armored signatures for the headers that are
uploaded to the GitHub release:
This will create ASCII-armored signatures for the two amalgamated files
that are uploaded to the GitHub release:
```
$ gpg2 --armor --output catch.hpp.asc --detach-sig catch.hpp
$ gpg2 --armor --output catch_reporter_automake.hpp.asc --detach-sig catch_reporter_automake.hpp
$ gpg2 --armor --output catch_reporter_teamcity.hpp.asc --detach-sig catch_reporter_teamcity.hpp
$ gpg2 --armor --output catch_reporter_tap.hpp.asc --detach-sig catch_reporter_tap.hpp
gpg --armor --output extras/catch_amalgamated.hpp.asc --detach-sig extras/catch_amalgamated.hpp
gpg --armor --output extras/catch_amalgamated.cpp.asc --detach-sig extras/catch_amalgamated.cpp
```
_GPG does not support signing multiple files in single invocation._

175
docs/reporter-events.md Normal file
View File

@@ -0,0 +1,175 @@
<a id="top"></a>
# Reporter events
**Contents**<br>
[Test running events](#test-running-events)<br>
[Benchmarking events](#benchmarking-events)<br>
[Listings events](#listings-events)<br>
[Miscellaneous events](#miscellaneous-events)<br>
Reporter events are one of the customization points for user code. They
are used by [reporters](reporters.md#top) to customize Catch2's output,
and by [event listeners](event-listeners.md#top) to perform in-process
actions under some conditions.
There are currently 21 reporter events in Catch2, split between 4 distinct
event groups:
* test running events (10 events)
* benchmarking (4 events)
* listings (3 events)
* miscellaneous (4 events)
## Test running events
Test running events are always paired so that for each `fooStarting` event,
there is a `fooEnded` event. This means that the 10 test running events
consist of 5 pairs of events:
* `testRunStarting` and `testRunEnded`,
* `testCaseStarting` and `testCaseEnded`,
* `testCasePartialStarting` and `testCasePartialEnded`,
* `sectionStarting` and `sectionEnded`,
* `assertionStarting` and `assertionEnded`
### `testRun` events
```cpp
void testRunStarting( TestRunInfo const& testRunInfo );
void testRunEnded( TestRunStats const& testRunStats );
```
The `testRun` events bookend the entire test run. `testRunStarting` is
emitted before the first test case is executed, and `testRunEnded` is
emitted after all the test cases have been executed.
### `testCase` events
```cpp
void testCaseStarting( TestCaseInfo const& testInfo );
void testCaseEnded( TestCaseStats const& testCaseStats );
```
The `testCase` events bookend one _full_ run of a specific test case.
Individual runs through a test case, e.g. due to `SECTION`s or `GENERATE`s,
are handled by a different event.
### `testCasePartial` events
> Introduced in Catch2 X.Y.Z
```cpp
void testCasePartialStarting( TestCaseInfo const& testInfo, uint64_t partNumber );
void testCasePartialEnded(TestCaseStats const& testCaseStats, uint64_t partNumber );
```
`testCasePartial` events bookend one _partial_ run of a specific test case.
This means that for any given test case, these events can be emitted
multiple times, e.g. due to multiple leaf sections.
In regards to nesting with `testCase` events, `testCasePartialStarting`
will never be emitted before the corresponding `testCaseStarting`, and
`testCasePartialEnded` will always be emitted before the corresponding
`testCaseEnded`.
### `section` events
```cpp
void sectionStarting( SectionInfo const& sectionInfo );
void sectionEnded( SectionStats const& sectionStats );
```
`section` events are emitted only for active `SECTION`s, that is, sections
that are entered. Sections that are skipped in this test case run-through
do not cause events to be emitted.
_Note that test cases always contain one implicit section. The event for
this section is emitted after the corresponding `testCasePartialStarting`
event._
### `assertion` events
```cpp
void assertionStarting( AssertionInfo const& assertionInfo );
void assertionEnded( AssertionStats const& assertionStats );
```
`assertionStarting` is called after the expression is captured, but before
the assertion expression is evaluated. This might seem like a minor
distinction, but what it means is that if you have assertion like
`REQUIRE( a + b == c + d )`, then what happens is that `a + b` and `c + d`
are evaluated before `assertionStarting` is emitted, while the `==` is
evaluated after the event.
## Benchmarking events
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
```cpp
void benchmarkPreparing( StringRef name ) override;
void benchmarkStarting( BenchmarkInfo const& benchmarkInfo ) override;
void benchmarkEnded( BenchmarkStats<> const& benchmarkStats ) override;
void benchmarkFailed( StringRef error ) override;
```
Due to the benchmark lifecycle being bit more complicated, the benchmarking
events have their own category, even though they could be seen as parallel
to the `assertion*` events. You should expect running a benchmark to
generate at least 2 of the events above.
To understand the explanation below, you should read the [benchmarking
documentation](benchmarks.md#top) first.
* `benchmarkPreparing` event is sent after the environmental probe
finishes, but before the user code is first estimated.
* `benchmarkStarting` event is sent after the user code is estimated,
but has not been benchmarked yet.
* `benchmarkEnded` event is sent after the user code has been benchmarked,
and contains the benchmarking results.
* `benchmarkFailed` event is sent if either the estimation or the
benchmarking itself fails.
## Listings events
> Introduced in Catch2 X.Y.Z.
Listings events are events that correspond to the test binary being
invoked with `--list-foo` flag.
There are currently 3 listing events, one for reporters, one for tests,
and one for tags. Note that they are not exclusive to each other.
```cpp
void listReporters( std::vector<ReporterDescription> const& descriptions );
void listTests( std::vector<TestCaseHandle> const& tests );
void listTags( std::vector<TagInfo> const& tagInfos );
```
## Miscellaneous events
```cpp
void reportInvalidTestSpec( StringRef unmatchedSpec );
void fatalErrorEncountered( StringRef error );
void noMatchingTestCases( StringRef unmatchedSpec );
```
These are one-off events that do not neatly fit into other categories.
`reportInvalidTestSpec` is sent for each [test specification command line
argument](command-line.md#specifying-which-tests-to-run) that wasn't
parsed into a valid spec.
`fatalErrorEncountered` is sent when Catch2's POSIX signal handling
or Windows SE handler is called into with a fatal signal/exception.
`noMatchingTestCases` is sent for each user provided test specification
that did not match any registered tests.
---
[Home](Readme.md#top)

View File

@@ -1,46 +1,212 @@
<a id="top"></a>
# Reporters
Catch has a modular reporting system and comes bundled with a handful of useful reporters built in.
You can also write your own reporters.
Reporters are a customization point for most of Catch2's output, e.g.
formatting and writing out [assertions (whether passing or failing),
sections, test cases, benchmarks, and so on](reporter-events.md#top).
Catch2 comes with a bunch of reporters by default (currently 8), and
you can also write your own reporter. Because multiple reporters can
be active at the same time, your own reporters do not even have to handle
all reporter event, just the ones you are interested in, e.g. benchmarks.
## Using different reporters
The reporter to use can easily be controlled from the command line.
To specify a reporter use [`-r` or `--reporter`](command-line.md#choosing-a-reporter-to-use), followed by the name of the reporter, e.g.:
You can see which reporters are available by running the test binary
with `--list-reporters`. You can then pick one of them with the [`-r`,
`--reporter` option](command-line.md#choosing-a-reporter-to-use), followed
by the name of the desired reporter, like so:
```
-r xml
--reporter xml
```
If you don't specify a reporter then the console reporter is used by default.
There are four reporters built in to the single include:
You can also select multiple reporters to be used at the same time.
In that case you should read the [section on using multiple
reporters](#multiple-reporters) to avoid any surprises from doing so.
* `console` writes as lines of text, formatted to a typical terminal width, with colours if a capable terminal is detected.
* `compact` similar to `console` but optimised for minimal output - each entry on one line
* `junit` writes xml that corresponds to Ant's [junitreport](http://help.catchsoftware.com/display/ET/JUnit+Format) target. Useful for build systems that understand Junit.
Because of the way the junit format is structured the run must complete before anything is written.
* `xml` writes an xml format tailored to Catch. Unlike `junit` this is a streaming format so results are delivered progressively.
There are a few additional reporters, for specific build systems, in the Catch repository (in `include\reporters`) which you can `#include` in your project if you would like to make use of them.
Do this in one source file - the same one you have `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`.
<a id="multiple-reporters"></a>
## Using multiple reporters
* `teamcity` writes the native, streaming, format that [TeamCity](https://www.jetbrains.com/teamcity/) understands.
Use this when building as part of a TeamCity build to see results as they happen ([code example](../examples/207-Rpt-TeamCityReporter.cpp)).
* `tap` writes in the TAP ([Test Anything Protocol](https://en.wikipedia.org/wiki/Test_Anything_Protocol)) format.
* `automake` writes in a format that correspond to [automake .trs](https://www.gnu.org/software/automake/manual/html_node/Log-files-generation-and-test-results-recording.html) files
* `sonarqube` writes the [SonarQube Generic Test Data](https://docs.sonarqube.org/latest/analysis/generic-test/) XML format.
> Support for having multiple parallel reporters was [introduced](https://github.com/catchorg/Catch2/pull/2183) in Catch2 X.Y.Z
Catch2 supports using multiple reporters at the same time while having
them write into different destinations. The two main uses of this are
* having both human-friendly and machine-parseable (e.g. in JUnit format)
output from one run of binary
* having "partial" reporters that are highly specialized, e.g. having one
reporter that writes out benchmark results as markdown tables and does
nothing else, while also having standard testing output separately
Specifying multiple reporter looks like this:
```
--reporter JUnit::out=result-junit.xml --reporter console::out=-::colour-mode=ansi
```
This tells Catch2 to use two reporters, `JUnit` reporter that writes
its machine-readable XML output to file `result-junit.xml`, and the
`console` reporter that writes its user-friendly output to stdout and
uses ANSI colour codes for colouring the output.
Using multiple reporters (or one reporter and one-or-more [event
listeners](event-listener.md#top)) can have surprisingly complex semantics
when using customization points provided to reporters by Catch2, namely
capturing stdout/stderr from test cases.
As long as at least one reporter (or listener) asks Catch2 to capture
stdout/stderr, captured stdout and stderr will be available to all
reporters and listeners.
Because this might be surprising to the users, if at least one active
_reporter_ is non-capturing, then Catch2 tries to roughly emulate
non-capturing behaviour by printing out the captured stdout/stderr
just before `testCasePartialEnded` event is sent out to the active
reporters and listeners. This means that stdout/stderr is no longer
printed out from tests as it is being written, but instead it is written
out in batch after each runthrough of a test case is finished.
You see what reporters are available from the command line by running with `--list-reporters`.
By default all these reports are written to stdout, but can be redirected to a file with [`-o` or `--out`](command-line.md#sending-output-to-a-file)
## Writing your own reporter
You can write your own custom reporter and register it with Catch.
At time of writing the interface is subject to some changes so is not, yet, documented here.
If you are determined you shouldn't have too much trouble working it out from the existing implementations -
but do keep in mind upcoming changes (these will be minor, simplifying, changes such as not needing to forward calls to the base class).
You can also write your own custom reporter and tell Catch2 to use it.
When writing your reporter, you have two options:
* Derive from `Catch::ReporterBase`. When doing this, you will have
to provide handling for all [reporter events](reporter-events.md#top).
* Derive from one of the provided [utility reporter bases in
Catch2](#utility-reporter-bases).
Generally we recommend doing the latter, as it is less work.
Apart from overriding handling of the individual reporter events, reporters
have access to some extra customization points, described below.
### Utility reporter bases
Catch2 currently provides two utility reporter bases:
* `Catch::StreamingReporterBase`
* `Catch::CumulativeReporterBase`
`StreamingReporterBase` is useful for reporters that can format and write
out the events as they come in. It provides (usually empty) implementation
for all reporter events, and if you let it handle the relevant events,
it also handles storing information about active test run and test case.
`CumulativeReporterBase` is a base for reporters that need to see the whole
test run, before they can start writing the output, such as the JUnit
and SonarQube reporters. This post-facto approach requires the assertions
to be stringified when it is finished, so that the assertion can be written
out later. Because the stringification can be expensive, and not all
cumulative reporters need the assertions, this base provides customization
point to change whether the assertions are saved or not, separate for
passing and failing assertions.
_Generally we recommend that if you override a member function from either
of the bases, you call into the base's implementation first. This is not
necessarily in all cases, but it is safer and easier._
Writing your own reporter then looks like this:
```cpp
#include <catch2/reporters/catch_reporter_streaming_base.hpp>
#include <catch2/catch_test_case_info.hpp>
#include <catch2/reporters/catch_reporter_registrars.hpp>
#include <iostream>
class PartialReporter : public Catch::StreamingReporterBase {
public:
using StreamingReporterBase::StreamingReporterBase;
static std::string getDescription() {
return "Reporter for testing TestCasePartialStarting/Ended events";
}
void testCasePartialStarting(Catch::TestCaseInfo const& testInfo,
uint64_t partNumber) override {
std::cout << "TestCaseStartingPartial: " << testInfo.name << '#' << partNumber << '\n';
}
void testCasePartialEnded(Catch::TestCaseStats const& testCaseStats,
uint64_t partNumber) override {
std::cout << "TestCasePartialEnded: " << testCaseStats.testInfo->name << '#' << partNumber << '\n';
}
};
CATCH_REGISTER_REPORTER("partial", PartialReporter)
```
This create a simple reporter that responds to `testCasePartial*` events,
and calls itself "partial" reporter, so it can be invoked with
`--reporter partial` command line flag.
### `ReporterPreferences`
Each reporter instance contains instance of `ReporterPreferences`, a type
that holds flags for the behaviour of Catch2 when this reporter run.
Currently there are two customization options:
* `shouldRedirectStdOut` - whether the reporter wants to handle
writes to stdout/stderr from user code, or not. This is useful for
reporters that output machine-parseable output, e.g. the JUnit
reporter, or the XML reporter.
* `shouldReportAllAssertions` - whether the reporter wants to handle
`assertionEnded` events for passing assertions as well as failing
assertions. Usually reporters do not report successful assertions
and don't need them for their output, but sometimes the desired output
format includes passing assertions even without the `-s` flag.
### Per-reporter configuration
> Per-reporter configuration was introduced in Catch2 X.Y.Z
Catch2 supports some configuration to happen per reporter. The configuration
options fall into one of two categories:
* Catch2-recognized options
* Reporter-specific options
The former is a small set of universal options that Catch2 handles for
the reporters, e.g. output file or console colour mode. The latter are
options that the reporters have to handle themselves, but the keys and
values can be arbitrary strings, as long as they don't contain `::`. This
allows writing reporters that can be significantly customized at runtime.
Reporter-specific options always have to be prefixed with "X" (large
letter X).
### Other expected functionality of a reporter
When writing a custom reporter, there are few more things that you should
keep in mind. These are not important for correctness, but they are
important for the reporter to work _nicely_.
* Catch2 provides a simple verbosity option for users. There are three
verbosity levels, "quiet", "normal", and "high", and if it makes sense
for reporter's output format, it should respond to these by changing
what, and how much, it writes out.
* Catch2 operates with an rng-seed. Knowing what seed a test run had
is important if you want to replicate it, so your reporter should
report the rng-seed, if at all possible given the target output format.
* Catch2 also operates with test filters, or test specs. If a filter
is present, you should also report the filter, if at all possible given
the target output format.
---

View File

@@ -1,72 +0,0 @@
<a id="top"></a>
# Why do my tests take so long to compile?
**Contents**<br>
[Short answer](#short-answer)<br>
[Long answer](#long-answer)<br>
[Practical example](#practical-example)<br>
[Other possible solutions](#other-possible-solutions)<br>
Several people have reported that test code written with Catch takes much longer to compile than they would expect. Why is that?
Catch is implemented entirely in headers. There is a little overhead due to this - but not as much as you might think - and you can minimise it simply by organising your test code as follows:
## Short answer
Exactly one source file must ```#define``` either ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER``` before ```#include```-ing Catch. In this file *do not write any test cases*! In most cases that means this file will just contain two lines (the ```#define``` and the ```#include```).
## Long answer
Usually C++ code is split between a header file, containing declarations and prototypes, and an implementation file (.cpp) containing the definition, or implementation, code. Each implementation file, along with all the headers that it includes (and which those headers include, etc), is expanded into a single entity called a translation unit - which is then passed to the compiler and compiled down to an object file.
But functions and methods can also be written inline in header files. The downside to this is that these definitions will then be compiled in *every* translation unit that includes the header.
Because Catch is implemented *entirely* in headers you might think that the whole of Catch must be compiled into every translation unit that uses it! Actually it's not quite as bad as that. Catch mitigates this situation by effectively maintaining the traditional separation between the implementation code and declarations. Internally the implementation code is protected by ```#ifdef```s and is conditionally compiled into only one translation unit. This translation unit is that one that ```#define```s ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER```. Let's call this the main source file.
As a result the main source file *does* compile the whole of Catch every time! So it makes sense to dedicate this file to *only* ```#define```-ing the identifier and ```#include```-ing Catch (and implementing the runner code, if you're doing that). Keep all your test cases in other files. This way you won't pay the recompilation cost for the whole of Catch.
## Practical example
Assume you have the `Factorial` function from the [tutorial](tutorial.md#top) in `factorial.cpp` (with forward declaration in `factorial.h`) and want to test it and keep the compile times down when adding new tests. Then you should have 2 files, `tests-main.cpp` and `tests-factorial.cpp`:
```cpp
// tests-main.cpp
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
```
```cpp
// tests-factorial.cpp
#include "catch.hpp"
#include "factorial.h"
TEST_CASE( "Factorials are computed", "[factorial]" ) {
REQUIRE( Factorial(1) == 1 );
REQUIRE( Factorial(2) == 2 );
REQUIRE( Factorial(3) == 6 );
REQUIRE( Factorial(10) == 3628800 );
}
```
After compiling `tests-main.cpp` once, it is enough to link it with separately compiled `tests-factorial.cpp`. This means that adding more tests to `tests-factorial.cpp`, will not result in recompiling Catch's main and the resulting compilation times will decrease substantially.
```
$ g++ tests-main.cpp -c
$ g++ factorial.cpp -c
$ g++ tests-main.o factorial.o tests-factorial.cpp -o tests && ./tests -r compact
Passed 1 test case with 4 assertions.
```
Now, the next time we change the file `tests-factorial.cpp` (say we add `REQUIRE( Factorial(0) == 1)`), it is enough to recompile the tests instead of recompiling main as well:
```
$ g++ tests-main.o factorial.o tests-factorial.cpp -o tests && ./tests -r compact
tests-factorial.cpp:11: failed: Factorial(0) == 1 for: 0 == 1
Failed 1 test case, failed 1 assertion.
```
## Other possible solutions
You can also opt to sacrifice some features in order to speed-up Catch's compilation times. For details see the [documentation on Catch's compile-time configuration](configuration.md#other-toggles).
---
[Home](Readme.md#top)

View File

@@ -44,7 +44,15 @@ The tag expression, ```"[widget]"``` selects A, B & D. ```"[gadget]"``` selects
For more detail on command line selection see [the command line docs](command-line.md#specifying-which-tests-to-run)
Tag names are not case sensitive and can contain any ASCII characters. This means that tags `[tag with spaces]` and `[I said "good day"]` are both allowed tags and can be filtered on. Escapes are not supported however and `[\]]` is not a valid tag.
Tag names are not case sensitive and can contain any ASCII characters.
This means that tags `[tag with spaces]` and `[I said "good day"]`
are both allowed tags and can be filtered on. However, escapes are not
supported however and `[\]]` is not a valid tag.
The same tag can be specified multiple times for a single test case,
but only one of the instances of identical tags will be kept. Which one
is kept is functionally random.
### Special Tags
@@ -90,15 +98,65 @@ This macro maps onto ```TEST_CASE``` and works in the same way, except that the
* **WHEN(** _something_ **)**
* **THEN(** _something_ **)**
These macros map onto ```SECTION```s except that the section names are the _something_s prefixed by "given: ", "when: " or "then: " respectively.
These macros map onto ```SECTION```s except that the section names are the _something_ texts prefixed by
"given: ", "when: " or "then: " respectively. These macros also map onto the AAA or A<sup>3</sup> test pattern
(standing either for [Assemble-Activate-Assert](http://wiki.c2.com/?AssembleActivateAssert) or
[Arrange-Act-Assert](http://wiki.c2.com/?ArrangeActAssert)), and in this context, the macros provide both code
documentation and reporting of these parts of a test case without the need for extra comments or code to do so.
Semantically, a `GIVEN` clause may have multiple _independent_ `WHEN` clauses within it. This allows a test
to have, e.g., one set of "given" objects and multiple subtests using those objects in various ways in each
of the `WHEN` clauses without repeating the initialisation from the `GIVEN` clause. When there are _dependent_
clauses -- such as a second `WHEN` clause that should only happen _after_ the previous `WHEN` clause has been
executed and validated -- there are additional macros starting with `AND_`:
* **AND_GIVEN(** _something_ **)**
* **AND_WHEN(** _something_ **)**
* **AND_THEN(** _something_ **)**
Similar to ```GIVEN```, ```WHEN``` and ```THEN``` except that the prefixes start with "and ". These are used to chain ```GIVEN```s, ```WHEN```s and ```THEN```s together.
These are used to chain ```GIVEN```s, ```WHEN```s and ```THEN```s together. The `AND_*` clause is placed
_inside_ the clause on which it depends. There can be multiple _independent_ clauses that are all _dependent_
on a single outer clause.
```cpp
SCENARIO( "vector can be sized and resized" ) {
GIVEN( "An empty vector" ) {
auto v = std::vector<std::string>{};
> `AND_GIVEN` was [introduced](https://github.com/catchorg/Catch2/issues/1360) in Catch 2.4.0.
// Validate assumption of the GIVEN clause
THEN( "The size and capacity start at 0" ) {
REQUIRE( v.size() == 0 );
REQUIRE( v.capacity() == 0 );
}
// Validate one use case for the GIVEN object
WHEN( "push_back() is called" ) {
v.push_back("hullo");
THEN( "The size changes" ) {
REQUIRE( v.size() == 1 );
REQUIRE( v.capacity() >= 1 );
}
}
}
}
```
This code will result in two runs through the scenario:
```
Scenario : vector can be sized and resized
Given : An empty vector
Then : The size and capacity start at 0
Scenario : vector can be sized and resized
Given : An empty vector
When : push_back() is called
Then : The size changes
```
See also [runnable example on godbolt](https://godbolt.org/z/e5vPPM),
with a more complicated (and failing) example.
> `AND_GIVEN` was [introduced](https://github.com/catchorg/Catch2/issues/1360) in Catch2 2.4.0.
When any of these macros are used the console reporter recognises them and formats the test case header such that the Givens, Whens and Thens are aligned to aid readability.
@@ -112,7 +170,7 @@ by types, in the form of `TEMPLATE_TEST_CASE`,
* **TEMPLATE_TEST_CASE(** _test name_ , _tags_, _type1_, _type2_, ..., _typen_ **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1437) in Catch 2.5.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1437) in Catch2 2.5.0.
_test name_ and _tag_ are exactly the same as they are in `TEST_CASE`,
with the difference that the tag string must be provided (however, it
@@ -164,7 +222,7 @@ TEMPLATE_TEST_CASE( "vectors can be sized and resized", "[vector][template]", in
* **TEMPLATE_PRODUCT_TEST_CASE(** _test name_ , _tags_, (_template-type1_, _template-type2_, ..., _template-typen_), (_template-arg1_, _template-arg2_, ..., _template-argm_) **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1468) in Catch 2.6.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1468) in Catch2 2.6.0.
_template-type1_ through _template-typen_ is list of template template
types which should be combined with each of _template-arg1_ through
@@ -203,13 +261,9 @@ TEMPLATE_PRODUCT_TEST_CASE("Product with differing arities", "[template][product
}
```
_While there is an upper limit on the number of types you can specify
in single `TEMPLATE_TEST_CASE` or `TEMPLATE_PRODUCT_TEST_CASE`, the limit
is very high and should not be encountered in practice._
* **TEMPLATE_LIST_TEST_CASE(** _test name_, _tags_, _type list_ **)**
> [Introduced](https://github.com/catchorg/Catch2/issues/1627) in Catch 2.9.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1627) in Catch2 2.9.0.
_type list_ is a generic list of types on which test case should be instantiated.
List can be `std::tuple`, `boost::mpl::list`, `boost::mp11::mp_list` or anything with
@@ -229,7 +283,7 @@ TEMPLATE_LIST_TEST_CASE("Template test case with test types specified inside std
## Signature based parametrised test cases
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch 2.8.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0.
In addition to [type parametrised test cases](#type-parametrised-test-cases) Catch2 also supports
signature base parametrised test cases, in form of `TEMPLATE_TEST_CASE_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_SIG`.
@@ -251,7 +305,7 @@ Currently Catch2 support up to 11 template parameters in signature
* **TEMPLATE_TEST_CASE_SIG(** _test name_ , _tags_, _signature_, _type1_, _type2_, ..., _typen_ **)**
Inside `TEMPLATE_TEST_CASE_SIG` test case you can use the names of template parameters as defined in _signature_.
Inside `TEMPLATE_TEST_CASE_SIG` test case you can use the names of template parameters as defined in _signature_.
```cpp
TEMPLATE_TEST_CASE_SIG("TemplateTestSig: arrays can be created from NTTP arguments", "[vector][template][nttp]",

View File

@@ -88,7 +88,7 @@ the limit is very high and should not be encountered in practice._
## Signature-based parametrised test fixtures
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch 2.8.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0.
Catch2 also provides `TEMPLATE_TEST_CASE_METHOD_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG` to support
fixtures using non-type template parameters. These test cases work similar to `TEMPLATE_TEST_CASE_METHOD` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD`,

View File

@@ -64,14 +64,14 @@ namespace Catch {
By default all exceptions deriving from `std::exception` will be translated to strings by calling the `what()` method. For exception types that do not derive from `std::exception` - or if `what()` does not return a suitable string - use `CATCH_TRANSLATE_EXCEPTION`. This defines a function that takes your exception type, by reference, and returns a string. It can appear anywhere in the code - it doesn't have to be in the same translation unit. For example:
```cpp
CATCH_TRANSLATE_EXCEPTION( MyType& ex ) {
CATCH_TRANSLATE_EXCEPTION( MyType const& ex ) {
return ex.message();
}
```
## Enums
> Introduced in Catch 2.8.0.
> Introduced in Catch2 2.8.0.
Enums that already have a `<<` overload for `std::ostream` will convert to strings as expected.
If you only need to convert enums to strings for test reporting purposes you can provide a `StringMaker` specialisations as any other type.
@@ -110,7 +110,7 @@ TEST_CASE() {
## Floating point precision
> [Introduced](https://github.com/catchorg/Catch2/issues/1614) in Catch 2.8.0.
> [Introduced](https://github.com/catchorg/Catch2/issues/1614) in Catch2 2.8.0.
Catch provides a built-in `StringMaker` specialization for both `float`
and `double`. By default, it uses what we think is a reasonable precision,

View File

@@ -3,32 +3,20 @@
**Contents**<br>
[Getting Catch2](#getting-catch2)<br>
[Where to put it?](#where-to-put-it)<br>
[Writing tests](#writing-tests)<br>
[Test cases and sections](#test-cases-and-sections)<br>
[BDD-Style](#bdd-style)<br>
[Scaling up](#scaling-up)<br>
[Type parametrised test cases](#type-parametrised-test-cases)<br>
[BDD style testing](#bdd-style-testing)<br>
[Data and Type driven tests](#data-and-type-driven-tests)<br>
[Next steps](#next-steps)<br>
## Getting Catch2
The simplest way to get Catch2 is to download the latest [single header version](https://raw.githubusercontent.com/catchorg/Catch2/master/single_include/catch2/catch.hpp). The single header is generated by merging a set of individual headers but it is still just normal source code in a header file.
Ideally you should be using Catch2 through its [CMake integration](cmake-integration.md#top).
Catch2 also provides pkg-config files and single TU distribution, but this
documentation will assume you are using CMake. If you are using single-TU
distribution instead, remember to replace the included header with `catch_amalgamated.hpp`.
Alternative ways of getting Catch2 include using your system package
manager, or installing it using [its CMake package](cmake-integration.md#installing-catch2-from-git-repository).
The full source for Catch2, including test projects, documentation, and other things, is hosted on GitHub. [http://catch-lib.net](http://catch-lib.net) will redirect you there.
## Where to put it?
Catch2 is header only. All you need to do is drop the file somewhere reachable from your project - either in some central location you can set your header search path to find, or directly into your project tree itself! This is a particularly good option for other Open-Source projects that want to use Catch for their test suite. See [this blog entry for more on that](https://levelofindirection.com/blog/unit-testing-in-cpp-and-objective-c-just-got-ridiculously-easier-still.html).
The rest of this tutorial will assume that the Catch2 single-include header (or the include folder) is available unqualified - but you may need to prefix it with a folder name if necessary.
_If you have installed Catch2 from system package manager, or CMake
package, you need to include the header as `#include <catch2/catch.hpp>`_
## Writing tests
@@ -40,11 +28,8 @@ unsigned int Factorial( unsigned int number ) {
}
```
To keep things simple we'll put everything in a single file (<a href="#scaling-up">see later for more on how to structure your test files</a>).
```c++
#define CATCH_CONFIG_MAIN // This tells Catch to provide a main() - only do this in one cpp file
#include "catch.hpp"
#include <catch2/catch_test_macros.hpp>
unsigned int Factorial( unsigned int number ) {
return number <= 1 ? number : Factorial(number-1)*number;
@@ -60,13 +45,10 @@ TEST_CASE( "Factorials are computed", "[factorial]" ) {
This will compile to a complete executable which responds to [command line arguments](command-line.md#top). If you just run it with no arguments it will execute all test cases (in this case there is just one), report any failures, report a summary of how many tests passed and failed and return the number of failed tests (useful for if you just want a yes/ no answer to: "did it work").
If you run this as written it will pass. Everything is good. Right?
Well, there is still a bug here. In fact the first version of this tutorial I posted here genuinely had the bug in! So it's not completely contrived (thanks to Daryle Walker (```@CTMacUser```) for pointing this out).
What is the bug? Well what is the factorial of zero?
[The factorial of zero is one](http://mathforum.org/library/drmath/view/57128.html) - which is just one of those things you have to know (and remember!).
Let's add that to the test case:
Anyway, as the tests above as written will pass, but there is a bug.
The problem is that `Factorial(0)` should return 1 (due to [its
definition](https://en.wikipedia.org/wiki/Factorial#Factorial_of_zero)).
Let's add that as an assertion to the test case:
```c++
TEST_CASE( "Factorials are computed", "[factorial]" ) {
@@ -78,7 +60,8 @@ TEST_CASE( "Factorials are computed", "[factorial]" ) {
}
```
Now we get a failure - something like:
After another compile & run cycle, we will see a test failure. The output
will look something like:
```
Example.cpp:9: FAILED:
@@ -87,37 +70,51 @@ with expansion:
0 == 1
```
Note that we get the actual return value of Factorial(0) printed for us (0) - even though we used a natural expression with the == operator. That lets us immediately see what the problem is.
Let's change the factorial function to:
Note that the output contains both the original expression,
`REQUIRE( Factorial(0) == 1 )` and the actual value returned by the call
to the `Factorial` function: `0`.
We can fix this bug by slightly modifying the `Factorial` function to:
```c++
unsigned int Factorial( unsigned int number ) {
return number > 1 ? Factorial(number-1)*number : 1;
}
```
Now all the tests pass.
Of course there are still more issues to deal with. For example we'll hit problems when the return value starts to exceed the range of an unsigned int. With factorials that can happen quite quickly. You might want to add tests for such cases and decide how to handle them. We'll stop short of doing that here.
### What did we do here?
Although this was a simple test it's been enough to demonstrate a few things about how Catch is used. Let's take a moment to consider those before we move on.
Although this was a simple test it's been enough to demonstrate a few
things about how Catch2 is used. Let's take a moment to consider those
before we move on.
* We introduce test cases with the `TEST_CASE` macro. This macro takes
one or two string arguments - a free form test name and, optionally,
one or more tags (for more see [Test cases and Sections](#test-cases-and-sections)).
* The test automatically self-registers with the test runner, and user
does not have do anything more to ensure that it is picked up by the test
framework. _Note that you can run specific test, or set of tests,
through the [command line](command-line#top)._
* The individual test assertions are written using the `REQUIRE` macro.
It accepts a boolean expression, and uses expression templates to
internally decompose it, so that it can be individually stringified
on test failure.
On the last point, note that there are more testing macros available,
because not all useful checks can be expressed as a simple boolean
expression. As an example, checking that an expression throws an exception
is done with the `REQUIRE_THROWS` macro. More on that later.
1. All we did was ```#define``` one identifier and ```#include``` one header and we got everything - even an implementation of ```main()``` that will [respond to command line arguments](command-line.md#top). You can only use that ```#define``` in one implementation file, for (hopefully) obvious reasons. Once you have more than one file with unit tests in you'll just ```#include "catch.hpp"``` and go. Usually it's a good idea to have a dedicated implementation file that just has ```#define CATCH_CONFIG_MAIN``` and ```#include "catch.hpp"```. You can also provide your own implementation of main and drive Catch yourself (see [Supplying-your-own-main()](own-main.md#top)).
2. We introduce test cases with the ```TEST_CASE``` macro. This macro takes one or two arguments - a free form test name and, optionally, one or more tags (for more see <a href="#test-cases-and-sections">Test cases and Sections</a>). The test name must be unique. You can run sets of tests by specifying a wildcarded test name or a tag expression. See the [command line docs](command-line.md#top) for more information on running tests.
3. The name and tags arguments are just strings. We haven't had to declare a function or method - or explicitly register the test case anywhere. Behind the scenes a function with a generated name is defined for you, and automatically registered using static registry classes. By abstracting the function name away we can name our tests without the constraints of identifier names.
4. We write our individual test assertions using the ```REQUIRE``` macro. Rather than a separate macro for each type of condition we express the condition naturally using C/C++ syntax. Behind the scenes a simple set of expression templates captures the left-hand-side and right-hand-side of the expression so we can display the values in our test report. As we'll see later there _are_ other assertion macros - but because of this technique the number of them is drastically reduced.
<a id="test-cases-and-sections"></a>
## Test cases and sections
Most test frameworks have a class-based fixture mechanism. That is, test cases map to methods on a class and common setup and teardown can be performed in ```setup()``` and ```teardown()``` methods (or constructor/ destructor in languages, like C++, that support deterministic destruction).
Like most test frameworks, Catch2 supports a class-based fixture mechanism,
where individual tests are methods on class and setup/teardown can be
done in constructor/destructor of the type.
While Catch fully supports this way of working there are a few problems with the approach. In particular the way your code must be split up, and the blunt granularity of it, may cause problems. You can only have one setup/ teardown pair across a set of methods, but sometimes you want slightly different setup in each method, or you may even want several levels of setup (a concept which we will clarify later on in this tutorial). It was <a href="http://jamesnewkirk.typepad.com/posts/2007/09/why-you-should-.html">problems like these</a> that led James Newkirk, who led the team that built NUnit, to start again from scratch and <a href="http://jamesnewkirk.typepad.com/posts/2007/09/announcing-xuni.html">build xUnit</a>).
Catch takes a different approach (to both NUnit and xUnit) that is a more natural fit for C++ and the C family of languages. This is best explained through an example ([code](../examples/100-Fix-Section.cpp)):
However, their use in Catch2 is rare, because idiomatic Catch2 tests
instead use _sections_ to share setup and teardown code between test code.
This is best explained through an example ([code](../examples/100-Fix-Section.cpp)):
```c++
TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
@@ -154,125 +151,75 @@ TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
}
```
For each ```SECTION``` the ```TEST_CASE``` is executed from the start - so as we enter each section we know that size is 5 and capacity is at least 5. We enforced those requirements with the ```REQUIRE```s at the top level so we can be confident in them.
This works because the ```SECTION``` macro contains an if statement that calls back into Catch to see if the section should be executed. One leaf section is executed on each run through a ```TEST_CASE```. The other sections are skipped. Next time through the next section is executed, and so on until no new sections are encountered.
For each `SECTION` the `TEST_CASE` is executed from the start. This means
that each section is entered with a freshly constructed vector `v`, that
we know has size 5 and capacity at least 5, because the two assertions
are also checked before the section is entered. Each run through a test
case will execute one, and only one, leaf section.
So far so good - this is already an improvement on the setup/teardown approach because now we see our setup code inline and use the stack.
Section can also be nested, in which case the parent section can be
entered multiple times, once for each leaf section. Nested sections are
most useful when you have multiple tests that share part of the set up.
To continue on the vector example above, you could add a check that
`std::vector::reserve` does not remove unused excess capacity, like this:
The power of sections really shows, however, when we need to execute a sequence of checked operations. Continuing the vector example, we might want to verify that attempting to reserve a capacity smaller than the current capacity of the vector changes nothing. We can do that, naturally, like so:
```c++
```cpp
SECTION( "reserving bigger changes capacity but not size" ) {
v.reserve( 10 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );
SECTION( "reserving smaller again does not change capacity" ) {
SECTION( "reserving down unused capacity does not change capacity" ) {
v.reserve( 7 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );
}
}
```
Sections can be nested to an arbitrary depth (limited only by your stack size). Each leaf section (i.e. a section that contains no nested sections) will be executed exactly once, on a separate path of execution from any other leaf section (so no leaf section can interfere with another). A failure in a parent section will prevent nested sections from running - but then that's the idea.
Another way to look at sections is that they are a way to define a tree
of paths through the test. Each section represents a node, and the final
tree is walked in depth-first manner, with each path only visiting only
one leaf node.
## BDD-Style
If you name your test cases and sections appropriately you can achieve a BDD-style specification structure. This became such a useful way of working that first class support has been added to Catch. Scenarios can be specified using ```SCENARIO```, ```GIVEN```, ```WHEN``` and ```THEN``` macros, which map on to ```TEST_CASE```s and ```SECTION```s, respectively. For more details see [Test cases and sections](test-cases-and-sections.md#top).
The vector example can be adjusted to use these macros like so ([example code](../examples/120-Bdd-ScenarioGivenWhenThen.cpp)):
```c++
SCENARIO( "vectors can be sized and resized", "[vector]" ) {
GIVEN( "A vector with some items" ) {
std::vector<int> v( 5 );
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 5 );
WHEN( "the size is increased" ) {
v.resize( 10 );
THEN( "the size and capacity change" ) {
REQUIRE( v.size() == 10 );
REQUIRE( v.capacity() >= 10 );
}
}
WHEN( "the size is reduced" ) {
v.resize( 0 );
THEN( "the size changes but not capacity" ) {
REQUIRE( v.size() == 0 );
REQUIRE( v.capacity() >= 5 );
}
}
WHEN( "more capacity is reserved" ) {
v.reserve( 10 );
THEN( "the capacity changes but not the size" ) {
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 10 );
}
}
WHEN( "less capacity is reserved" ) {
v.reserve( 0 );
THEN( "neither size nor capacity are changed" ) {
REQUIRE( v.size() == 5 );
REQUIRE( v.capacity() >= 5 );
}
}
}
}
```
Conveniently, these tests will be reported as follows when run:
```
Scenario: vectors can be sized and resized
Given: A vector with some items
When: more capacity is reserved
Then: the capacity changes but not the size
```
<a id="scaling-up"></a>
## Scaling up
To keep the tutorial simple we put all our code in a single file. This is fine to get started - and makes jumping into Catch even quicker and easier. As you write more real-world tests, though, this is not really the best approach.
The requirement is that the following block of code ([or equivalent](own-main.md#top)):
```c++
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
```
appears in _exactly one_ source file. Use as many additional cpp files (or whatever you call your implementation files) as you need for your tests, partitioned however makes most sense for your way of working. Each additional file need only ```#include "catch.hpp"``` - do not repeat the ```#define```!
In fact it is usually a good idea to put the block with the ```#define``` [in its own source file](slow-compiles.md#top) (code example [main](../examples/020-TestCase-1.cpp), [tests](../examples/020-TestCase-2.cpp)).
Do not write your tests in header files!
There is no practical limit on nesting sections, as long as your compiler
can handle them, but keep in mind that overly nested sections can become
unreadable. From experience, having section nest more than 3 levels is
usually very hard to follow and not worth the removed duplication.
## Type parametrised test cases
## BDD style testing
Test cases in Catch2 can be also parametrised by type, via the
`TEMPLATE_TEST_CASE` and `TEMPLATE_PRODUCT_TEST_CASE` macros,
which behave in the same way the `TEST_CASE` macro, but are run for
every type or type combination.
Catch2 also provides some basic support for BDD-style testing. There are
macro aliases for `TEST_CASE` and `SECTIONS` that you can use so that
the resulting tests read as BDD spec. `SCENARIO` acts as a `TEST_CASE`
with "Scenario: " name prefix. Then there are `GIVEN`, `WHEN`, `THEN`
(and their variants with `AND_` prefix), which act as a `SECTION`,
similarly prefixed with the macro name.
For more details, see our documentation on [test cases and
sections](test-cases-and-sections.md#type-parametrised-test-cases).
For more details on the macros look at the [test cases and
sections](test-cases-and-sections.md#top) part of the reference docs,
or at the [vector example done with BDD macros](../examples/120-Bdd-ScenarioGivenWhenThen.cpp).
## Data and Type driven tests
Test cases in Catch2 can also be driven by types, input data, or both
at the same time.
For more details look into the Catch2 reference, either at the
[type parametrized test cases](test-cases-and-sections.md#type-parametrised-test-cases),
or [data generators](generators.md#top).
## Next steps
This has been a brief introduction to get you up and running with Catch, and to point out some of the key differences between Catch and other frameworks you may already be familiar with. This will get you going quite far already and you are now in a position to dive in and write some tests.
This page is a brief introduction to get you up and running with Catch2,
and to show the basic features of Catch2. The features mentioned here
can get you quite far, but there are many more. However, you can read
about these as you go, in the ever-growing [reference section](Readme.md#top)
of the documentation.
Of course there is more to learn - most of which you should be able to page-fault in as you go. Please see the ever-growing [Reference section](Readme.md#top) for what's available.
---

View File

@@ -6,40 +6,53 @@ including (but not limited to),
[Google Test](http://code.google.com/p/googletest/),
[Boost.Test](http://www.boost.org/doc/libs/1_49_0/libs/test/doc/html/index.html),
[CppUnit](http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=Main_Page),
[Cute](http://www.cute-test.com),
[Cute](http://www.cute-test.com), and
[many, many more](http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C.2B.2B).
So what does Catch bring to the party that differentiates it from these? Apart from a Catchy name, of course.
So what does Catch2 bring to the party that differentiates it from these? Apart from the catchy name, of course.
## Key Features
* Quick and Really easy to get started. Just download catch.hpp, `#include` it and you're away.
* No external dependencies. As long as you can compile C++11 and have a C++ standard library available.
* Quick and easy to get started. Just download two files, add them into your project and you're away.
* No external dependencies. As long as you can compile C++14 and have the C++ standard library available.
* Write test cases as, self-registering, functions (or methods, if you prefer).
* Divide test cases into sections, each of which is run in isolation (eliminates the need for fixtures).
* Use BDD-style Given-When-Then sections as well as traditional unit test cases.
* Only one core assertion macro for comparisons. Standard C/C++ operators are used for the comparison - yet the full expression is decomposed and lhs and rhs values are logged.
* Tests are named using free-form strings - no more couching names in legal identifiers.
## Other core features
* Tests can be tagged for easily running ad-hoc groups of tests.
* Failures can (optionally) break into the debugger on Windows and Mac.
* Failures can (optionally) break into the debugger on common platforms.
* Output is through modular reporter objects. Basic textual and XML reporters are included. Custom reporters can easily be added.
* JUnit xml output is supported for integration with third-party tools, such as CI servers.
* A default main() function is provided, but you can supply your own for complete control (e.g. integration into your own test runner GUI).
* A command line parser is provided and can still be used if you choose to provided your own main() function.
* Catch can test itself.
* Alternative assertion macro(s) report failures but don't abort the test case
* Floating point tolerance comparisons are built in using an expressive Approx() syntax.
* Good set of facilities for floating point comparisons (`Catch::Approx` and full set of matchers)
* Internal and friendly macros are isolated so name clashes can be managed
* Matchers
* Data generators (data driven test support)
* Hamcrest-style Matchers for testing complex properties
* Microbenchmarking support
## Who else is using Catch?
See the list of [open source projects using Catch](opensource-users.md#top).
## Who else is using Catch2?
See the [tutorial](tutorial.md#top) to get more of a taste of using Catch in practice
A whole lot of people. According to the 2021 Jetbrains C++ ecosystem survey,
about 11% of C++ programmers use Catch2 for unit testing, making it the
second most popular unit testing framework.
You can also take a look at the (incomplete) list of [open source projects](opensource-users.md#top)
or the (very incomplete) list of [commercial users of Catch2](commercial-users.md#top)
for some idea on who else also uses Catch2.
---
See the [tutorial](tutorial.md#top) to get more of a taste of using
Catch2 in practice.
---

View File

@@ -1,13 +0,0 @@
// 000-CatchMain.cpp
// It is generally recommended to have a single file provide the main
// of a testing binary, and other test files to link against it.
// Let Catch provide main():
#include <catch2/internal/catch_default_main.hpp>
// That's it
// Compile implementation of Catch for use with files that do contain tests:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -c 000-CatchMain.cpp
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% -c 000-CatchMain.cpp

View File

@@ -8,7 +8,7 @@
// - REQUIRE_FALSE() stops at first failure.
// - CHECK_FALSE() continues after failure.
// main() provided in 000-CatchMain.cpp
// main() provided by linkage to Catch2WithMain
#include <catch2/catch_test_macros.hpp>
@@ -53,8 +53,8 @@ TEST_CASE( "Assert that something is false (continue after failure)", "[check-fa
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 030-Asn-Require-Check 030-Asn-Require-Check.cpp 000-CatchMain.o && 030-Asn-Require-Check --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 030-Asn-Require-Check.cpp 000-CatchMain.obj && 030-Asn-Require-Check --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 030-Asn-Require-Check 030-Asn-Require-Check.cpp && 030-Asn-Require-Check --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 030-Asn-Require-Check.cpp && 030-Asn-Require-Check --success
// Expected compact output (all assertions):
//

View File

@@ -4,7 +4,7 @@
// - Sections (this file)
// - Traditional class-based fixtures
// main() provided in 000-CatchMain.cpp
// main() provided by linkage to Catch2WithMain
#include <catch2/catch_test_macros.hpp>
#include <vector>
@@ -45,8 +45,8 @@ TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 100-Fix-Section 100-Fix-Section.cpp 000-CatchMain.o && 100-Fix-Section --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 100-Fix-Section.cpp 000-CatchMain.obj && 100-Fix-Section --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 100-Fix-Section 100-Fix-Section.cpp && 100-Fix-Section --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 100-Fix-Section.cpp && 100-Fix-Section --success
// Expected compact output (all assertions):
//

View File

@@ -4,7 +4,7 @@
// - Sections
// - Traditional class-based fixtures (this file)
// main() provided in 000-CatchMain.cpp
// main() provided by linkage to Catch2WithMain
#include <catch2/catch_test_macros.hpp>
@@ -52,8 +52,8 @@ TEST_CASE_METHOD( UniqueTestsFixture, "Create Employee/Normal", "[create]" ) {
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 110-Fix-ClassFixture 110-Fix-ClassFixture.cpp 000-CatchMain.o && 110-Fix-ClassFixture --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 110-Fix-ClassFixture.cpp 000-CatchMain.obj && 110-Fix-ClassFixture --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 110-Fix-ClassFixture 110-Fix-ClassFixture.cpp && 110-Fix-ClassFixture --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 110-Fix-ClassFixture.cpp && 110-Fix-ClassFixture --success
// Expected compact output (all assertions):
//

View File

@@ -1,6 +1,6 @@
// 120-Bdd-ScenarioGivenWhenThen.cpp
// main() provided in 000-CatchMain.cpp
// main() provided by linkage with Catch2WithMain
#include <catch2/catch_test_macros.hpp>
@@ -48,8 +48,8 @@ SCENARIO( "vectors can be sized and resized", "[vector]" ) {
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 120-Bdd-ScenarioGivenWhenThen 120-Bdd-ScenarioGivenWhenThen.cpp 000-CatchMain.o && 120-Bdd-ScenarioGivenWhenThen --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 120-Bdd-ScenarioGivenWhenThen.cpp 000-CatchMain.obj && 120-Bdd-ScenarioGivenWhenThen --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 120-Bdd-ScenarioGivenWhenThen 120-Bdd-ScenarioGivenWhenThen.cpp && 120-Bdd-ScenarioGivenWhenThen --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 120-Bdd-ScenarioGivenWhenThen.cpp && 120-Bdd-ScenarioGivenWhenThen --success
// Expected compact output (all assertions):
//

View File

@@ -5,14 +5,10 @@
// 2. My listener and registration
// 3. Test cases
// main() provided in 000-CatchMain.cpp
// Let Catch provide the required interfaces:
#define CATCH_CONFIG_EXTERNAL_INTERFACES
#include <catch2/catch_test_macros.hpp>
#include <catch2/reporters/catch_reporter_bases.hpp>
#include <catch2/catch_reporter_registrars.hpp>
#include <catch2/reporters/catch_reporter_event_listener.hpp>
#include <catch2/reporters/catch_reporter_registrars.hpp>
#include <catch2/catch_test_case_info.hpp>
#include <iostream>
// -----------------------------------------------------------------------
@@ -26,7 +22,7 @@ std::string ws(int const level) {
}
std::ostream& operator<<(std::ostream& out, Catch::Tag t) {
return out << "original: " << t.original << "lower cased: " << t.lowerCased;
return out << "original: " << t.original;
}
template< typename T >
@@ -307,9 +303,9 @@ char const * dashed_line =
"--------------------------------------------------------------------------";
struct MyListener : Catch::TestEventListenerBase {
struct MyListener : Catch::EventListenerBase {
using TestEventListenerBase::TestEventListenerBase; // inherit constructor
using EventListenerBase::EventListenerBase; // inherit constructor
// Get rid of Wweak-tables
~MyListener();
@@ -370,10 +366,9 @@ struct MyListener : Catch::TestEventListenerBase {
print( std::cout, 1, "- assertionInfo", assertionInfo );
}
bool assertionEnded( Catch::AssertionStats const& assertionStats ) override {
void assertionEnded( Catch::AssertionStats const& assertionStats ) override {
std::cout << "\nEvent: assertionEnded:\n";
print( std::cout, 1, "- assertionStats", assertionStats );
return true;
}
};
@@ -425,8 +420,8 @@ TEST_CASE_METHOD( Fixture, "3: Testcase with class-based fixture", "[tag-C][tag-
}
// Compile & run:
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 210-Evt-EventListeners 210-Evt-EventListeners.cpp 000-CatchMain.o && 210-Evt-EventListeners --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 210-Evt-EventListeners.cpp 000-CatchMain.obj && 210-Evt-EventListeners --success
// - g++ -std=c++11 -Wall -I$(CATCH_SINGLE_INCLUDE) -o 210-Evt-EventListeners 210-Evt-EventListeners.cpp && 210-Evt-EventListeners --success
// - cl -EHsc -I%CATCH_SINGLE_INCLUDE% 210-Evt-EventListeners.cpp && 210-Evt-EventListeners --success
// Expected compact output (all assertions):
//

View File

@@ -43,7 +43,9 @@ int const& RandomIntGenerator::get() const {
// is a value-wrapper around std::unique_ptr<IGenerator<int>>.
Catch::Generators::GeneratorWrapper<int> random(int low, int high) {
return Catch::Generators::GeneratorWrapper<int>(
std::make_unique<RandomIntGenerator>(low, high)
new RandomIntGenerator(low, high)
// Another possibility:
// Catch::Detail::make_unique<RandomIntGenerator>(low, high)
);
}
@@ -58,7 +60,7 @@ TEST_CASE("Generating random ints", "[example][generator]") {
REQUIRE(i <= 100);
}
SECTION("Creating the random generator directly") {
auto i = GENERATE(take(100, GeneratorWrapper<int>(std::unique_ptr<IGenerator<int>>(new RandomIntGenerator(-100, 100)))));
auto i = GENERATE(take(100, GeneratorWrapper<int>(Catch::Detail::make_unique<RandomIntGenerator>(-100, 100))));
REQUIRE(i >= -100);
REQUIRE(i <= 100);
}

View File

@@ -1,6 +1,9 @@
// 301-Gen-MapTypeConversion.cpp
// Shows how to use map to modify generator's return type.
// Specifically we wrap a std::string returning generator with a generator
// that converts the strings using stoi, so the returned type is actually
// an int.
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators_adapters.hpp>
@@ -40,7 +43,7 @@ std::string const& LineGenerator::get() const {
// is a value-wrapper around std::unique_ptr<IGenerator<std::string>>.
Catch::Generators::GeneratorWrapper<std::string> lines(std::string /* ignored for example */) {
return Catch::Generators::GeneratorWrapper<std::string>(
std::make_unique<LineGenerator>()
new LineGenerator()
);
}

View File

@@ -0,0 +1,55 @@
// 302-Gen-Table.cpp
// Shows how to use table to run a test many times with different inputs. Lifted from examples on
// issue #850.
#include <catch2/catch_test_macros.hpp>
#include <catch2/generators/catch_generators.hpp>
#include <string>
struct TestSubject {
// this is the method we are going to test. It returns the length of the
// input string.
size_t GetLength( const std::string& input ) const { return input.size(); }
};
TEST_CASE("Table allows pre-computed test inputs and outputs", "[example][generator]") {
using std::make_tuple;
// do setup here as normal
TestSubject subj;
SECTION("This section is run for each row in the table") {
std::string test_input;
size_t expected_output;
std::tie( test_input, expected_output ) =
GENERATE( table<std::string, size_t>(
{ /* In this case one of the parameters to our test case is the
* expected output, but this is not required. There could be
* multiple expected values in the table, which can have any
* (fixed) number of columns.
*/
make_tuple( "one", 3 ),
make_tuple( "two", 3 ),
make_tuple( "three", 5 ),
make_tuple( "four", 4 ) } ) );
// run the test
auto result = subj.GetLength(test_input);
// capture the input data to go with the outputs.
CAPTURE(test_input);
// check it matches the pre-calculated data
REQUIRE(result == expected_output);
} // end section
}
/* Possible simplifications where less legacy toolchain support is needed:
*
* - With libstdc++6 or newer, the make_tuple() calls can be ommitted
* (technically C++17 but does not require -std in GCC/Clang). See
* https://stackoverflow.com/questions/12436586/tuple-vector-and-initializer-list
*
* - In C++17 mode std::tie() and the preceding variable delcarations can be
* replaced by structured bindings: auto [test_input, expected] = GENERATE(
* table<std::string, size_t>({ ...
*/
// Compiling and running this file will result in 4 successful assertions

View File

@@ -29,7 +29,7 @@ TEST_CASE("Generate random doubles across different ranges",
// This will take r1 by reference and r2 by value.
// Note that there are no advantages for doing so in this example,
// it is done only for expository purposes.
auto number = Catch::Generators::generate( CATCH_INTERNAL_LINEINFO,
auto number = Catch::Generators::generate( "custom capture generator", CATCH_INTERNAL_LINEINFO,
[&r1, r2]{
using namespace Catch::Generators;
return makeGenerators(take(50, random(std::get<0>(r1), std::get<1>(r2))));

View File

@@ -32,6 +32,7 @@ set( SOURCES_IDIOMATIC_EXAMPLES
210-Evt-EventListeners.cpp
300-Gen-OwnGenerator.cpp
301-Gen-MapTypeConversion.cpp
302-Gen-Table.cpp
310-Gen-VariablesInGenerators.cpp
311-Gen-CustomCapture.cpp
)

View File

@@ -33,6 +33,10 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
[TEST_SUFFIX suffix]
[PROPERTIES name1 value1...]
[TEST_LIST var]
[REPORTER reporter]
[OUTPUT_DIR dir]
[OUTPUT_PREFIX prefix}
[OUTPUT_SUFFIX suffix]
)
``catch_discover_tests`` sets up a post-build command on the test executable
@@ -90,6 +94,28 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
executable is being used in multiple calls to ``catch_discover_tests()``.
Note that this variable is only available in CTest.
``REPORTER reporter``
Use the specified reporter when running the test case. The reporter will
be passed to the Catch executable as ``--reporter reporter``.
``OUTPUT_DIR dir``
If specified, the parameter is passed along as
``--out dir/<test_name>`` to Catch executable. The actual file name is the
same as the test name. This should be used instead of
``EXTRA_ARGS --out foo`` to avoid race conditions writing the result output
when using parallel test execution.
``OUTPUT_PREFIX prefix``
May be used in conjunction with ``OUTPUT_DIR``.
If specified, ``prefix`` is added to each output file name, like so
``--out dir/prefix<test_name>``.
``OUTPUT_SUFFIX suffix``
May be used in conjunction with ``OUTPUT_DIR``.
If specified, ``suffix`` is added to each output file name, like so
``--out dir/<test_name>suffix``. This can be used to add a file extension to
the output e.g. ".xml".
#]=======================================================================]
#------------------------------------------------------------------------------
@@ -97,7 +123,7 @@ function(catch_discover_tests TARGET)
cmake_parse_arguments(
""
""
"TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST"
"TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;REPORTER;OUTPUT_DIR;OUTPUT_PREFIX;OUTPUT_SUFFIX"
"TEST_SPEC;EXTRA_ARGS;PROPERTIES"
${ARGN}
)
@@ -110,7 +136,7 @@ function(catch_discover_tests TARGET)
endif()
## Generate a unique name based on the extra arguments
string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS}")
string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS} ${_REPORTER} ${_OUTPUT_DIR} ${_OUTPUT_PREFIX} ${_OUTPUT_SUFFIX}")
string(SUBSTRING ${args_hash} 0 7 args_hash)
# Define rule to generate test list for aforementioned test executable
@@ -134,6 +160,10 @@ function(catch_discover_tests TARGET)
-D "TEST_PREFIX=${_TEST_PREFIX}"
-D "TEST_SUFFIX=${_TEST_SUFFIX}"
-D "TEST_LIST=${_TEST_LIST}"
-D "TEST_REPORTER=${_REPORTER}"
-D "TEST_OUTPUT_DIR=${_OUTPUT_DIR}"
-D "TEST_OUTPUT_PREFIX=${_OUTPUT_PREFIX}"
-D "TEST_OUTPUT_SUFFIX=${_OUTPUT_SUFFIX}"
-D "CTEST_FILE=${ctest_tests_file}"
-P "${_CATCH_DISCOVER_TESTS_SCRIPT}"
VERBATIM
@@ -172,4 +202,5 @@ endfunction()
set(_CATCH_DISCOVER_TESTS_SCRIPT
${CMAKE_CURRENT_LIST_DIR}/CatchAddTests.cmake
CACHE INTERNAL "Catch2 full path to CatchAddTests.cmake helper file"
)

View File

@@ -6,13 +6,20 @@ set(suffix "${TEST_SUFFIX}")
set(spec ${TEST_SPEC})
set(extra_args ${TEST_EXTRA_ARGS})
set(properties ${TEST_PROPERTIES})
set(reporter ${TEST_REPORTER})
set(output_dir ${TEST_OUTPUT_DIR})
set(output_prefix ${TEST_OUTPUT_PREFIX})
set(output_suffix ${TEST_OUTPUT_SUFFIX})
set(script)
set(suite)
set(tests)
function(add_command NAME)
set(_args "")
foreach(_arg ${ARGN})
# use ARGV* instead of ARGN, because ARGN splits arrays into multiple arguments
math(EXPR _last_arg ${ARGC}-1)
foreach(_n RANGE 1 ${_last_arg})
set(_arg "${ARGV${_n}}")
if(_arg MATCHES "[^-./:a-zA-Z0-9_]")
set(_args "${_args} [==[${_arg}]==]") # form a bracket_argument
else()
@@ -29,16 +36,12 @@ if(NOT EXISTS "${TEST_EXECUTABLE}")
)
endif()
execute_process(
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-test-names-only
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-tests --verbosity quiet
OUTPUT_VARIABLE output
RESULT_VARIABLE result
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
)
# Catch --list-test-names-only reports the number of tests, so 0 is... surprising
if(${result} EQUAL 0)
message(WARNING
"Test executable '${TEST_EXECUTABLE}' contains no tests!\n"
)
elseif(${result} LESS 0)
if(NOT ${result} EQUAL 0)
message(FATAL_ERROR
"Error running test executable '${TEST_EXECUTABLE}':\n"
" Result: ${result}\n"
@@ -48,6 +51,40 @@ endif()
string(REPLACE "\n" ";" output "${output}")
# Run test executable to get list of available reporters
execute_process(
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-reporters
OUTPUT_VARIABLE reporters_output
RESULT_VARIABLE reporters_result
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
)
if(NOT ${reporters_result} EQUAL 0)
message(FATAL_ERROR
"Error running test executable '${TEST_EXECUTABLE}':\n"
" Result: ${reporters_result}\n"
" Output: ${reporters_output}\n"
)
endif()
string(FIND "${reporters_output}" "${reporter}" reporter_is_valid)
if(reporter AND ${reporter_is_valid} EQUAL -1)
message(FATAL_ERROR
"\"${reporter}\" is not a valid reporter!\n"
)
endif()
# Prepare reporter
if(reporter)
set(reporter_arg "--reporter ${reporter}")
endif()
# Prepare output dir
if(output_dir AND NOT IS_ABSOLUTE ${output_dir})
set(output_dir "${TEST_WORKING_DIR}/${output_dir}")
if(NOT EXISTS ${output_dir})
file(MAKE_DIRECTORY ${output_dir})
endif()
endif()
# Parse output
foreach(line ${output})
set(test ${line})
@@ -56,6 +93,12 @@ foreach(line ${output})
foreach(char , [ ])
string(REPLACE ${char} "\\${char}" test_name ${test_name})
endforeach(char)
# ...add output dir
if(output_dir)
string(REGEX REPLACE "[^A-Za-z0-9_]" "_" test_name_clean ${test_name})
set(output_dir_arg "--out ${output_dir}/${output_prefix}${test_name_clean}${output_suffix}")
endif()
# ...and add to script
add_command(add_test
"${prefix}${test}${suffix}"
@@ -63,6 +106,8 @@ foreach(line ${output})
"${TEST_EXECUTABLE}"
"${test_name}"
${extra_args}
"${reporter_arg}"
"${output_dir_arg}"
)
add_command(set_tests_properties
"${prefix}${test}${suffix}"

View File

@@ -1,9 +1,11 @@
#==================================================================================================#
# supported macros #
# - TEST_CASE, #
# - TEMPLATE_TEST_CASE #
# - SCENARIO, #
# - TEST_CASE_METHOD, #
# - CATCH_TEST_CASE, #
# - CATCH_TEMPLATE_TEST_CASE #
# - CATCH_SCENARIO, #
# - CATCH_TEST_CASE_METHOD. #
# #
@@ -106,7 +108,8 @@ function(ParseAndAddCatchTests_ParseFile SourceFile TestTarget)
ParseAndAddCatchTests_RemoveComments(Contents)
# Find definition of test names
string(REGEX MATCHALL "[ \t]*(CATCH_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)[ \t]*\\([^\)]+\\)+[ \t\n]*{+[ \t]*(//[^\n]*[Tt][Ii][Mm][Ee][Oo][Uu][Tt][ \t]*[0-9]+)*" Tests "${Contents}")
# https://regex101.com/r/JygOND/1
string(REGEX MATCHALL "[ \t]*(CATCH_)?(TEMPLATE_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)[ \t]*\\([ \t\n]*\"[^\"]*\"[ \t\n]*(,[ \t\n]*\"[^\"]*\")?(,[ \t\n]*[^\,\)]*)*\\)[ \t\n]*\{+[ \t]*(//[^\n]*[Tt][Ii][Mm][Ee][Oo][Uu][Tt][ \t]*[0-9]+)*" Tests "${Contents}")
if(PARSE_CATCH_TESTS_ADD_TO_CONFIGURE_DEPENDS AND Tests)
ParseAndAddCatchTests_PrintDebugMessage("Adding ${SourceFile} to CMAKE_CONFIGURE_DEPENDS property")
@@ -117,13 +120,21 @@ function(ParseAndAddCatchTests_ParseFile SourceFile TestTarget)
)
endif()
# check CMP0110 policy for new add_test() behavior
if(POLICY CMP0110)
cmake_policy(GET CMP0110 _cmp0110_value) # new add_test() behavior
else()
# just to be thorough explicitly set the variable
set(_cmp0110_value)
endif()
foreach(TestName ${Tests})
# Strip newlines
string(REGEX REPLACE "\\\\\n|\n" "" TestName "${TestName}")
# Get test type and fixture if applicable
string(REGEX MATCH "(CATCH_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)[ \t]*\\([^,^\"]*" TestTypeAndFixture "${TestName}")
string(REGEX MATCH "(CATCH_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)" TestType "${TestTypeAndFixture}")
string(REGEX MATCH "(CATCH_)?(TEMPLATE_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)[ \t]*\\([^,^\"]*" TestTypeAndFixture "${TestName}")
string(REGEX MATCH "(CATCH_)?(TEMPLATE_)?(TEST_CASE_METHOD|SCENARIO|TEST_CASE)" TestType "${TestTypeAndFixture}")
string(REGEX REPLACE "${TestType}\\([ \t]*" "" TestFixture "${TestTypeAndFixture}")
# Get string parts of test definition
@@ -189,24 +200,39 @@ function(ParseAndAddCatchTests_ParseFile SourceFile TestTarget)
# Escape commas in the test spec
string(REPLACE "," "\\," Name ${Name})
# Work around CMake 3.18.0 change in `add_test()`, before the escaped quotes were necessary,
# only with CMake 3.18.0 the escaped double quotes confuse the call. This change is reverted in 3.18.1
# And properly introduced in 3.19 with the CMP0110 policy
if(_cmp0110_value STREQUAL "NEW" OR ${CMAKE_VERSION} VERSION_EQUAL "3.18")
ParseAndAddCatchTests_PrintDebugMessage("CMP0110 set to NEW, no need for add_test(\"\") workaround")
else()
ParseAndAddCatchTests_PrintDebugMessage("CMP0110 set to OLD adding \"\" for add_test() workaround")
set(CTestName "\"${CTestName}\"")
endif()
# Handle template test cases
if("${TestTypeAndFixture}" MATCHES ".*TEMPLATE_.*")
set(Name "${Name} - *")
endif()
# Add the test and set its properties
add_test(NAME "\"${CTestName}\"" COMMAND ${OptionalCatchTestLauncher} $<TARGET_FILE:${TestTarget}> ${Name} ${AdditionalCatchParameters})
add_test(NAME "${CTestName}" COMMAND ${OptionalCatchTestLauncher} $<TARGET_FILE:${TestTarget}> ${Name} ${AdditionalCatchParameters})
# Old CMake versions do not document VERSION_GREATER_EQUAL, so we use VERSION_GREATER with 3.8 instead
if(PARSE_CATCH_TESTS_NO_HIDDEN_TESTS AND ${HiddenTagFound} AND ${CMAKE_VERSION} VERSION_GREATER "3.8")
ParseAndAddCatchTests_PrintDebugMessage("Setting DISABLED test property")
set_tests_properties("\"${CTestName}\"" PROPERTIES DISABLED ON)
set_tests_properties("${CTestName}" PROPERTIES DISABLED ON)
else()
set_tests_properties("\"${CTestName}\"" PROPERTIES FAIL_REGULAR_EXPRESSION "No tests ran"
set_tests_properties("${CTestName}" PROPERTIES FAIL_REGULAR_EXPRESSION "No tests ran"
LABELS "${Labels}")
endif()
set_property(
TARGET ${TestTarget}
APPEND
PROPERTY ParseAndAddCatchTests_TESTS "\"${CTestName}\"")
PROPERTY ParseAndAddCatchTests_TESTS "${CTestName}")
set_property(
SOURCE ${SourceFile}
APPEND
PROPERTY ParseAndAddCatchTests_TESTS "\"${CTestName}\"")
PROPERTY ParseAndAddCatchTests_TESTS "${CTestName}")
endif()
@@ -215,6 +241,7 @@ endfunction()
# entry point
function(ParseAndAddCatchTests TestTarget)
message(DEPRECATION "ParseAndAddCatchTest: function deprecated because of possibility of missed test cases. Consider using 'catch_discover_tests' from 'Catch.cmake'")
ParseAndAddCatchTests_PrintDebugMessage("Started parsing ${TestTarget}")
get_target_property(SourceFiles ${TestTarget} SOURCES)
ParseAndAddCatchTests_PrintDebugMessage("Found the following sources: ${SourceFiles}")

10260
extras/catch_amalgamated.cpp Normal file

File diff suppressed because it is too large Load Diff

11988
extras/catch_amalgamated.hpp Normal file

File diff suppressed because it is too large Load Diff

20
fuzzing/CMakeLists.txt Normal file
View File

@@ -0,0 +1,20 @@
# License: Boost 1.0
# By Paul Dreik 2020
# add a library that brings in the main() function from libfuzzer
# and has all the dependencies, so the individual fuzzers can be
# added one line each.
add_library(fuzzhelper NullOStream.h NullOStream.cpp)
target_link_libraries(fuzzhelper PUBLIC Catch2::Catch2)
# use C++17 so we can get string_view
target_compile_features(fuzzhelper PUBLIC cxx_std_17)
# This should be possible to set from the outside to be oss-fuzz compatible,
# fix later. For now, target libFuzzer only.
target_link_options(fuzzhelper PUBLIC "-fsanitize=fuzzer")
foreach(fuzzer TestSpecParser XmlWriter textflow)
add_executable(fuzz_${fuzzer} fuzz_${fuzzer}.cpp)
target_link_libraries(fuzz_${fuzzer} PRIVATE fuzzhelper)
endforeach()

10
fuzzing/NullOStream.cpp Normal file
View File

@@ -0,0 +1,10 @@
#include "NullOStream.h"
void NullOStream::avoidOutOfLineVirtualCompilerWarning()
{
}
int NullStreambuf::overflow(int c){
setp(dummyBuffer, dummyBuffer + sizeof(dummyBuffer));
return (c == traits_type::eof()) ? '\0' : c;
}

20
fuzzing/NullOStream.h Normal file
View File

@@ -0,0 +1,20 @@
#pragma once
#include <ostream>
#include <streambuf>
// from https://stackoverflow.com/a/8244052
class NullStreambuf : public std::streambuf {
char dummyBuffer[64];
protected:
virtual int overflow(int c) override final;
};
class NullOStream final : private NullStreambuf, public std::ostream {
public:
NullOStream() : std::ostream(this) {}
NullStreambuf *rdbuf() { return this; }
virtual void avoidOutOfLineVirtualCompilerWarning();
};

33
fuzzing/build_fuzzers.sh Executable file
View File

@@ -0,0 +1,33 @@
#!/bin/sh
#
# Builds the fuzzers
#
# By Paul Dreik 20200923
set -exu
CATCHROOT=$(readlink -f $(dirname $0)/..)
BUILDDIR=$CATCHROOT/build-fuzzers
mkdir -p $BUILDDIR
cd $BUILDDIR
if which /usr/lib/ccache/clang++ >/dev/null 2>&1 ; then
CXX=/usr/lib/ccache/clang++
else
CXX=clang++
fi
cmake $CATCHROOT \
-DCMAKE_CXX_COMPILER=$CXX \
-DCMAKE_CXX_FLAGS="-fsanitize=fuzzer-no-link,address,undefined -O3 -g" \
-DCATCH_DEVELOPMENT_BUILD=On \
-DCATCH_BUILD_EXAMPLES=Off \
-DCATCH_BUILD_EXTRA_TESTS=Off \
-DCATCH_BUILD_TESTING=Off \
-DBUILD_TESTING=Off \
-DCATCH_ENABLE_WERROR=Off \
-DCATCH_BUILD_FUZZERS=On
cmake --build . -j $(nproc)

View File

@@ -0,0 +1,16 @@
//License: Boost 1.0
//By Paul Dreik 2020
#include <catch2/internal/catch_test_spec_parser.hpp>
#include <catch2/internal/catch_tag_alias_registry.hpp>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
Catch::TagAliasRegistry tar;
Catch::TestSpecParser tsp(tar);
std::string buf(Data,Data+Size);
tsp.parse(buf);
return 0;
}

View File

@@ -0,0 +1,16 @@
//License: Boost 1.0
//By Paul Dreik 2020
#include <catch2/internal/catch_xmlwriter.hpp>
#include "NullOStream.h"
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
std::string buf(Data,Data+Size);
NullOStream nul;
Catch::XmlEncode encode(buf);
encode.encodeTo(nul);
return 0;
}

47
fuzzing/fuzz_textflow.cpp Normal file
View File

@@ -0,0 +1,47 @@
//License: Boost 1.0
//By Paul Dreik 2020
#include <catch2/internal/catch_textflow.hpp>
#include "NullOStream.h"
#include <string>
#include <string_view>
template<class Callback>
void split(const char *Data, size_t Size, Callback callback) {
using namespace std::literals;
constexpr auto sep="\n~~~\n"sv;
std::string_view remainder(Data,Size);
for (;;) {
auto pos=remainder.find(sep);
if(pos==std::string_view::npos) {
//not found. use the remainder and exit
callback(remainder);
return;
} else {
//found. invoke callback on the first part, then proceed with the rest.
callback(remainder.substr(0,pos));
remainder=remainder.substr(pos+sep.size());
}
}
}
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
Catch::TextFlow::Columns columns;
// break the input on separator
split((const char*)Data,Size,[&](std::string_view word) {
columns+=Catch::TextFlow::Column(std::string(word));
});
NullOStream nul;
nul << columns;
return 0;
}

9
mdsnippets.json Normal file
View File

@@ -0,0 +1,9 @@
{
"ReadOnly": false,
"TocLevel": 5,
"Exclude": [
"cmake-build"
],
"WriteHeader": false,
"Convention": "InPlaceOverwrite"
}

View File

@@ -1,4 +1,11 @@
include(MiscFunctions)
include(CatchMiscFunctions)
# CMake derives a Visual Studio project GUID from the file path but can be overridden via a property
# (see https://gitlab.kitware.com/cmake/cmake/-/commit/c85367f4). Using a non-constant GUID
# can cause problems if other projects/repos want to reference the vcxproj file,
# so we force a constant GUID here.
set(Catch2_GUID_CMAKE "8d538cbe-01bf-4a2e-a98a-6c368fdf13d7" CACHE INTERNAL "Project GUID")
set(Catch2WithMain_GUID_CMAKE "8bd3552a-2cfb-4a59-ab15-2031b97ada1e" CACHE INTERNAL "Project GUID")
# Please keep these ordered alphabetically
set(BENCHMARK_HEADERS
@@ -31,23 +38,33 @@ set(BENCHMARK_SOURCES
SOURCE_GROUP("benchmark" FILES ${BENCHMARK_HEADERS} ${BENCHMARK_SOURCES})
set(INTERNAL_HEADERS
"${CMAKE_BINARY_DIR}/generated-includes/catch2/catch_user_config.hpp"
"${SOURCES_DIR}/catch_user_config.hpp.in"
${SOURCES_DIR}/catch_all.hpp
${SOURCES_DIR}/matchers/catch_matchers_all.hpp
${SOURCES_DIR}/generators/catch_generators_all.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_all.hpp
${SOURCES_DIR}/matchers/internal/catch_matchers_impl.hpp
${SOURCES_DIR}/internal/catch_case_insensitive_comparisons.hpp
${SOURCES_DIR}/internal/catch_console_width.hpp
${SOURCES_DIR}/internal/catch_container_nonmembers.hpp
${SOURCES_DIR}/internal/catch_noncopyable.hpp
${SOURCES_DIR}/catch_approx.hpp
${SOURCES_DIR}/internal/catch_assertion_handler.hpp
${SOURCES_DIR}/catch_assertion_info.hpp
${SOURCES_DIR}/catch_assertion_result.hpp
${SOURCES_DIR}/internal/catch_test_macro_impl.hpp
${SOURCES_DIR}/internal/catch_test_failure_exception.hpp
${SOURCES_DIR}/internal/catch_case_sensitive.hpp
${SOURCES_DIR}/internal/catch_clara.hpp
${SOURCES_DIR}/internal/catch_clara_upstream.hpp
${SOURCES_DIR}/internal/catch_commandline.hpp
${SOURCES_DIR}/internal/catch_common.hpp
${SOURCES_DIR}/internal/catch_source_line_info.hpp
${SOURCES_DIR}/internal/catch_compiler_capabilities.hpp
${SOURCES_DIR}/catch_config.hpp
${SOURCES_DIR}/internal/catch_config_android_logwrite.hpp
${SOURCES_DIR}/internal/catch_config_counter.hpp
${SOURCES_DIR}/internal/catch_config_uncaught_exceptions.hpp
${SOURCES_DIR}/internal/catch_config_wchar.hpp
${SOURCES_DIR}/internal/catch_console_colour.hpp
${SOURCES_DIR}/internal/catch_context.hpp
${SOURCES_DIR}/internal/catch_debug_console.hpp
@@ -58,6 +75,10 @@ set(INTERNAL_HEADERS
${SOURCES_DIR}/internal/catch_errno_guard.hpp
${SOURCES_DIR}/internal/catch_exception_translator_registry.hpp
${SOURCES_DIR}/internal/catch_fatal_condition_handler.hpp
${SOURCES_DIR}/internal/catch_floating_point_helpers.hpp
${SOURCES_DIR}/internal/catch_istream.hpp
${SOURCES_DIR}/internal/catch_unique_name.hpp
${SOURCES_DIR}/internal/catch_sharding.hpp
${SOURCES_DIR}/generators/catch_generator_exception.hpp
${SOURCES_DIR}/generators/catch_generators.hpp
${SOURCES_DIR}/generators/catch_generators_adapters.hpp
@@ -70,7 +91,8 @@ set(INTERNAL_HEADERS
${SOURCES_DIR}/interfaces/catch_interfaces_generatortracker.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_registry_hub.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_runner.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter_factory.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter_registry.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_tag_alias_registry.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_testcase.hpp
${SOURCES_DIR}/internal/catch_lazy_expr.hpp
@@ -80,30 +102,37 @@ set(INTERNAL_HEADERS
${SOURCES_DIR}/matchers/catch_matchers_container_properties.hpp
${SOURCES_DIR}/matchers/catch_matchers_contains.hpp
${SOURCES_DIR}/matchers/catch_matchers_exception.hpp
${SOURCES_DIR}/matchers/catch_matchers_floating.hpp
${SOURCES_DIR}/matchers/catch_matchers_floating_point.hpp
${SOURCES_DIR}/matchers/catch_matchers_predicate.hpp
${SOURCES_DIR}/matchers/catch_matchers_quantifiers.hpp
${SOURCES_DIR}/matchers/catch_matchers_string.hpp
${SOURCES_DIR}/matchers/catch_matchers_templated.hpp
${SOURCES_DIR}/matchers/catch_matchers_vector.hpp
${SOURCES_DIR}/catch_message.hpp
${SOURCES_DIR}/internal/catch_message_info.hpp
${SOURCES_DIR}/internal/catch_meta.hpp
${SOURCES_DIR}/internal/catch_option.hpp
${SOURCES_DIR}/internal/catch_move_and_forward.hpp
${SOURCES_DIR}/internal/catch_optional.hpp
${SOURCES_DIR}/internal/catch_output_redirect.hpp
${SOURCES_DIR}/internal/catch_platform.hpp
${SOURCES_DIR}/internal/catch_polyfills.hpp
${SOURCES_DIR}/internal/catch_preprocessor.hpp
${SOURCES_DIR}/internal/catch_preprocessor_remove_parens.hpp
${SOURCES_DIR}/internal/catch_random_number_generator.hpp
${SOURCES_DIR}/catch_reporter_registrars.hpp
${SOURCES_DIR}/internal/catch_random_seed_generation.hpp
${SOURCES_DIR}/reporters/catch_reporter_registrars.hpp
${SOURCES_DIR}/internal/catch_reporter_registry.hpp
${SOURCES_DIR}/internal/catch_reporter_spec_parser.hpp
${SOURCES_DIR}/internal/catch_result_type.hpp
${SOURCES_DIR}/internal/catch_run_context.hpp
${SOURCES_DIR}/internal/catch_section.hpp
${SOURCES_DIR}/internal/catch_stdstreams.hpp
${SOURCES_DIR}/catch_section_info.hpp
${SOURCES_DIR}/catch_session.hpp
${SOURCES_DIR}/internal/catch_singletons.hpp
${SOURCES_DIR}/internal/catch_startup_exception_registry.hpp
${SOURCES_DIR}/internal/catch_stream.hpp
${SOURCES_DIR}/internal/catch_reusable_string_stream.hpp
${SOURCES_DIR}/internal/catch_stream_end_stop.hpp
${SOURCES_DIR}/internal/catch_string_manip.hpp
${SOURCES_DIR}/internal/catch_stringref.hpp
${SOURCES_DIR}/catch_tag_alias.hpp
@@ -118,26 +147,32 @@ set(INTERNAL_HEADERS
${SOURCES_DIR}/internal/catch_test_registry.hpp
${SOURCES_DIR}/catch_test_spec.hpp
${SOURCES_DIR}/internal/catch_test_spec_parser.hpp
${SOURCES_DIR}/internal/catch_text.hpp
${SOURCES_DIR}/internal/catch_textflow.hpp
${SOURCES_DIR}/catch_timer.hpp
${SOURCES_DIR}/internal/catch_to_string.hpp
${SOURCES_DIR}/catch_tostring.hpp
${SOURCES_DIR}/catch_totals.hpp
${SOURCES_DIR}/catch_translate_exception.hpp
${SOURCES_DIR}/internal/catch_uncaught_exceptions.hpp
${SOURCES_DIR}/internal/catch_unique_ptr.hpp
${SOURCES_DIR}/internal/catch_void_type.hpp
${SOURCES_DIR}/catch_version.hpp
${SOURCES_DIR}/catch_version_macros.hpp
${SOURCES_DIR}/internal/catch_wildcard_pattern.hpp
${SOURCES_DIR}/internal/catch_windows_h_proxy.hpp
${SOURCES_DIR}/internal/catch_xmlwriter.hpp
${SOURCES_DIR}/internal/catch_test_case_info_hasher.hpp
)
set(IMPL_SOURCES
${SOURCES_DIR}/catch_approx.cpp
${SOURCES_DIR}/internal/catch_assertion_handler.cpp
${SOURCES_DIR}/catch_assertion_result.cpp
${SOURCES_DIR}/matchers/internal/catch_matchers_combined_tu.cpp
${SOURCES_DIR}/internal/catch_clara.cpp
${SOURCES_DIR}/internal/catch_commandline.cpp
${SOURCES_DIR}/internal/catch_common.cpp
${SOURCES_DIR}/internal/catch_source_line_info.cpp
${SOURCES_DIR}/catch_config.cpp
${SOURCES_DIR}/internal/catch_case_insensitive_comparisons.cpp
${SOURCES_DIR}/internal/catch_console_colour.cpp
${SOURCES_DIR}/internal/catch_context.cpp
${SOURCES_DIR}/internal/catch_debug_console.cpp
@@ -146,11 +181,13 @@ set(IMPL_SOURCES
${SOURCES_DIR}/internal/catch_enum_values_registry.cpp
${SOURCES_DIR}/internal/catch_exception_translator_registry.cpp
${SOURCES_DIR}/internal/catch_fatal_condition_handler.cpp
${SOURCES_DIR}/internal/catch_floating_point_helpers.cpp
${SOURCES_DIR}/internal/catch_istream.cpp
${SOURCES_DIR}/generators/internal/catch_generators_combined_tu.cpp
${SOURCES_DIR}/interfaces/catch_interfaces_combined_tu.cpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter.cpp
${SOURCES_DIR}/internal/catch_list.cpp
${SOURCES_DIR}/matchers/catch_matchers_floating.cpp
${SOURCES_DIR}/matchers/catch_matchers_floating_point.cpp
${SOURCES_DIR}/matchers/catch_matchers_string.cpp
${SOURCES_DIR}/matchers/catch_matchers_templated.cpp
${SOURCES_DIR}/catch_message.cpp
@@ -158,13 +195,16 @@ set(IMPL_SOURCES
${SOURCES_DIR}/catch_registry_hub.cpp
${SOURCES_DIR}/internal/catch_combined_tu.cpp
${SOURCES_DIR}/internal/catch_random_number_generator.cpp
${SOURCES_DIR}/internal/catch_random_seed_generation.cpp
${SOURCES_DIR}/internal/catch_reporter_registry.cpp
${SOURCES_DIR}/internal/catch_reporter_spec_parser.cpp
${SOURCES_DIR}/internal/catch_result_type.cpp
${SOURCES_DIR}/internal/catch_run_context.cpp
${SOURCES_DIR}/internal/catch_section.cpp
${SOURCES_DIR}/internal/catch_stdstreams.cpp
${SOURCES_DIR}/catch_session.cpp
${SOURCES_DIR}/internal/catch_singletons.cpp
${SOURCES_DIR}/internal/catch_stream.cpp
${SOURCES_DIR}/internal/catch_reusable_string_stream.cpp
${SOURCES_DIR}/internal/catch_stringref.cpp
${SOURCES_DIR}/internal/catch_string_manip.cpp
${SOURCES_DIR}/internal/catch_tag_alias_registry.cpp
@@ -172,6 +212,7 @@ set(IMPL_SOURCES
${SOURCES_DIR}/internal/catch_test_case_registry_impl.cpp
${SOURCES_DIR}/internal/catch_test_case_tracker.cpp
${SOURCES_DIR}/internal/catch_test_registry.cpp
${SOURCES_DIR}/internal/catch_textflow.cpp
${SOURCES_DIR}/catch_test_spec.cpp
${SOURCES_DIR}/internal/catch_test_spec_parser.cpp
${SOURCES_DIR}/catch_timer.cpp
@@ -180,6 +221,7 @@ set(IMPL_SOURCES
${SOURCES_DIR}/catch_version.cpp
${SOURCES_DIR}/internal/catch_wildcard_pattern.cpp
${SOURCES_DIR}/internal/catch_xmlwriter.cpp
${SOURCES_DIR}/internal/catch_test_case_info_hasher.cpp
)
set(INTERNAL_FILES ${IMPL_SOURCES} ${INTERNAL_HEADERS})
@@ -187,24 +229,31 @@ set(INTERNAL_FILES ${IMPL_SOURCES} ${INTERNAL_HEADERS})
set(REPORTER_HEADERS
${SOURCES_DIR}/reporters/catch_reporters_all.hpp
${SOURCES_DIR}/reporters/catch_reporter_automake.hpp
${SOURCES_DIR}/reporters/catch_reporter_bases.hpp
${SOURCES_DIR}/reporters/catch_reporter_common_base.hpp
${SOURCES_DIR}/reporters/catch_reporter_compact.hpp
${SOURCES_DIR}/reporters/catch_reporter_console.hpp
${SOURCES_DIR}/reporters/catch_reporter_cumulative_base.hpp
${SOURCES_DIR}/reporters/catch_reporter_event_listener.hpp
${SOURCES_DIR}/reporters/catch_reporter_helpers.hpp
${SOURCES_DIR}/reporters/catch_reporter_junit.hpp
${SOURCES_DIR}/reporters/catch_reporter_listening.hpp
${SOURCES_DIR}/reporters/catch_reporter_multi.hpp
${SOURCES_DIR}/reporters/catch_reporter_sonarqube.hpp
${SOURCES_DIR}/reporters/catch_reporter_streaming_base.hpp
${SOURCES_DIR}/reporters/catch_reporter_tap.hpp
${SOURCES_DIR}/reporters/catch_reporter_teamcity.hpp
${SOURCES_DIR}/reporters/catch_reporter_xml.hpp
)
set(REPORTER_SOURCES
${SOURCES_DIR}/reporters/catch_reporter_automake.cpp
${SOURCES_DIR}/reporters/catch_reporter_bases.cpp
${SOURCES_DIR}/reporters/catch_reporter_combined_tu.cpp
${SOURCES_DIR}/reporters/catch_reporter_common_base.cpp
${SOURCES_DIR}/reporters/catch_reporter_compact.cpp
${SOURCES_DIR}/reporters/catch_reporter_console.cpp
${SOURCES_DIR}/reporters/catch_reporter_cumulative_base.cpp
${SOURCES_DIR}/reporters/catch_reporter_junit.cpp
${SOURCES_DIR}/reporters/catch_reporter_listening.cpp
${SOURCES_DIR}/reporters/catch_reporter_multi.cpp
${SOURCES_DIR}/reporters/catch_reporter_sonarqube.cpp
${SOURCES_DIR}/reporters/catch_reporter_streaming_base.cpp
${SOURCES_DIR}/reporters/catch_reporter_tap.cpp
${SOURCES_DIR}/reporters/catch_reporter_teamcity.cpp
${SOURCES_DIR}/reporters/catch_reporter_xml.cpp
@@ -219,12 +268,15 @@ add_library(Catch2 STATIC
${BENCHMARK_HEADERS}
${BENCHMARK_SOURCES}
)
add_build_reproducibility_settings(Catch2)
add_library(Catch2::Catch2 ALIAS Catch2)
if (ANDROID)
target_link_libraries(Catch2 INTERFACE log)
endif()
set_target_properties(Catch2 PROPERTIES DEBUG_POSTFIX "d")
# depend on bunch of C++11 and C++14 features to have C++14 enabled by default
target_compile_features(Catch2
PUBLIC
@@ -250,9 +302,15 @@ target_compile_features(Catch2
cxx_variadic_macros
)
configure_file(
"${SOURCES_DIR}/catch_user_config.hpp.in"
"${CMAKE_BINARY_DIR}/generated-includes/catch2/catch_user_config.hpp"
)
target_include_directories(Catch2
PUBLIC
$<BUILD_INTERFACE:${SOURCES_DIR}/..>
$<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/generated-includes>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)
@@ -260,12 +318,14 @@ target_include_directories(Catch2
add_library(Catch2WithMain STATIC
${SOURCES_DIR}/internal/catch_main.cpp
)
add_build_reproducibility_settings(Catch2WithMain)
add_library(Catch2::Catch2WithMain ALIAS Catch2WithMain)
target_link_libraries(Catch2WithMain PUBLIC Catch2)
set_target_properties(Catch2WithMain
PROPERTIES
OUTPUT_NAME "Catch2Main"
)
set_target_properties(Catch2WithMain PROPERTIES DEBUG_POSTFIX "d")
if (NOT_SUBPROJECT)
# create and install an export set for catch target as Catch2::Catch
@@ -289,7 +349,15 @@ if (NOT_SUBPROJECT)
${CATCH_CMAKE_CONFIG_DESTINATION}
)
# Install the headers
install(DIRECTORY ${SOURCES_DIR} DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} FILES_MATCHING PATTERN "*.h*")
install(
DIRECTORY
"${SOURCES_DIR}"
"${CMAKE_BINARY_DIR}/generated-includes/catch2" # Also install the generated header
DESTINATION
"${CMAKE_INSTALL_INCLUDEDIR}"
FILES_MATCHING
PATTERN "*.hpp"
)
endif()
# Some tests require a full recompilation of Catch2 lib with different
@@ -308,6 +376,7 @@ if (CATCH_BUILD_EXAMPLES OR CATCH_BUILD_EXTRA_TESTS)
target_include_directories(Catch2_buildall_interface
INTERFACE
$<BUILD_INTERFACE:${SOURCES_DIR}/..>
$<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/generated-includes>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)
target_compile_features(Catch2_buildall_interface

View File

@@ -1,19 +1,21 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Benchmark
#ifndef TWOBLUECUBES_CATCH_BENCHMARK_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_BENCHMARK_HPP_INCLUDED
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#include <catch2/catch_config.hpp>
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_BENCHMARK_HPP_INCLUDED
#define CATCH_BENCHMARK_HPP_INCLUDED
#include <catch2/interfaces/catch_interfaces_config.hpp>
#include <catch2/internal/catch_compiler_capabilities.hpp>
#include <catch2/internal/catch_context.hpp>
#include <catch2/interfaces/catch_interfaces_reporter.hpp>
#include <catch2/internal/catch_unique_name.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <catch2/benchmark/catch_chronometer.hpp>
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_environment.hpp>
@@ -33,12 +35,12 @@
namespace Catch {
namespace Benchmark {
struct Benchmark {
Benchmark(std::string &&name)
: name(std::move(name)) {}
Benchmark(std::string&& benchmarkName)
: name(CATCH_MOVE(benchmarkName)) {}
template <class FUN>
Benchmark(std::string &&name, FUN &&func)
: fun(std::move(func)), name(std::move(name)) {}
Benchmark(std::string&& benchmarkName , FUN &&func)
: fun(CATCH_MOVE(func)), name(CATCH_MOVE(benchmarkName)) {}
template <typename Clock>
ExecutionPlan<FloatDuration<Clock>> prepare(const IConfig &cfg, Environment<FloatDuration<Clock>> env) const {
@@ -51,7 +53,7 @@ namespace Catch {
template <typename Clock = default_clock>
void run() {
IConfigPtr cfg = getCurrentContext().getConfig();
auto const* cfg = getCurrentContext().getConfig();
auto env = Detail::measure_environment<Clock>();
@@ -80,16 +82,18 @@ namespace Catch {
auto analysis = Detail::analyse(*cfg, env, samples.begin(), samples.end());
BenchmarkStats<FloatDuration<Clock>> stats{ info, analysis.samples, analysis.mean, analysis.standard_deviation, analysis.outliers, analysis.outlier_variance };
getResultCapture().benchmarkEnded(stats);
} CATCH_CATCH_ANON (TestFailureException) {
getResultCapture().benchmarkFailed("Benchmark failed due to failed assertion"_sr);
} CATCH_CATCH_ALL{
if (translateActiveException() != Detail::benchmarkErrorMsg) // benchmark errors have been reported, otherwise rethrow.
std::rethrow_exception(std::current_exception());
getResultCapture().benchmarkFailed(translateActiveException());
// We let the exception go further up so that the
// test case is marked as failed.
std::rethrow_exception(std::current_exception());
}
}
// sets lambda to be used in fun *and* executes benchmark!
template <typename Fun,
typename std::enable_if<!Detail::is_related<Fun, Benchmark>::value, int>::type = 0>
template <typename Fun, std::enable_if_t<!Detail::is_related<Fun, Benchmark>::value, int> = 0>
Benchmark & operator=(Fun func) {
fun = Detail::BenchmarkFunction(func);
run();
@@ -121,17 +125,17 @@ namespace Catch {
#if defined(CATCH_CONFIG_PREFIX_ALL)
#define CATCH_BENCHMARK(...) \
INTERNAL_CATCH_BENCHMARK(INTERNAL_CATCH_UNIQUE_NAME(____C_A_T_C_H____B_E_N_C_H____), INTERNAL_CATCH_GET_1_ARG(__VA_ARGS__,,), INTERNAL_CATCH_GET_2_ARG(__VA_ARGS__,,))
INTERNAL_CATCH_BENCHMARK(INTERNAL_CATCH_UNIQUE_NAME(CATCH2_INTERNAL_BENCHMARK_), INTERNAL_CATCH_GET_1_ARG(__VA_ARGS__,,), INTERNAL_CATCH_GET_2_ARG(__VA_ARGS__,,))
#define CATCH_BENCHMARK_ADVANCED(name) \
INTERNAL_CATCH_BENCHMARK_ADVANCED(INTERNAL_CATCH_UNIQUE_NAME(____C_A_T_C_H____B_E_N_C_H____), name)
INTERNAL_CATCH_BENCHMARK_ADVANCED(INTERNAL_CATCH_UNIQUE_NAME(CATCH2_INTERNAL_BENCHMARK_), name)
#else
#define BENCHMARK(...) \
INTERNAL_CATCH_BENCHMARK(INTERNAL_CATCH_UNIQUE_NAME(____C_A_T_C_H____B_E_N_C_H____), INTERNAL_CATCH_GET_1_ARG(__VA_ARGS__,,), INTERNAL_CATCH_GET_2_ARG(__VA_ARGS__,,))
INTERNAL_CATCH_BENCHMARK(INTERNAL_CATCH_UNIQUE_NAME(CATCH2_INTERNAL_BENCHMARK_), INTERNAL_CATCH_GET_1_ARG(__VA_ARGS__,,), INTERNAL_CATCH_GET_2_ARG(__VA_ARGS__,,))
#define BENCHMARK_ADVANCED(name) \
INTERNAL_CATCH_BENCHMARK_ADVANCED(INTERNAL_CATCH_UNIQUE_NAME(____C_A_T_C_H____B_E_N_C_H____), name)
INTERNAL_CATCH_BENCHMARK_ADVANCED(INTERNAL_CATCH_UNIQUE_NAME(CATCH2_INTERNAL_BENCHMARK_), name)
#endif
#endif // TWOBLUECUBES_CATCH_BENCHMARK_HPP_INCLUDED
#endif // CATCH_BENCHMARK_HPP_INCLUDED

View File

@@ -1,3 +1,10 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
/** \file
* This is a convenience header for Catch2's benchmarking. It includes
* **all** of Catch2 headers related to benchmarking.

View File

@@ -1,20 +1,20 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// User-facing chronometer
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_CHRONOMETER_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_CHRONOMETER_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_CHRONOMETER_HPP_INCLUDED
#define CATCH_CHRONOMETER_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_optimizer.hpp>
#include <catch2/benchmark/detail/catch_complete_invoke.hpp>
#include <catch2/internal/catch_meta.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
namespace Catch {
namespace Benchmark {
@@ -43,13 +43,13 @@ namespace Catch {
struct Chronometer {
public:
template <typename Fun>
void measure(Fun&& fun) { measure(std::forward<Fun>(fun), is_callable<Fun(int)>()); }
void measure(Fun&& fun) { measure(CATCH_FORWARD(fun), is_callable<Fun(int)>()); }
int runs() const { return k; }
int runs() const { return repeats; }
Chronometer(Detail::ChronometerConcept& meter, int k)
Chronometer(Detail::ChronometerConcept& meter, int repeats_)
: impl(&meter)
, k(k) {}
, repeats(repeats_) {}
private:
template <typename Fun>
@@ -61,15 +61,15 @@ namespace Catch {
void measure(Fun&& fun, std::true_type) {
Detail::optimizer_barrier();
impl->start();
for (int i = 0; i < k; ++i) invoke_deoptimized(fun, i);
for (int i = 0; i < repeats; ++i) invoke_deoptimized(fun, i);
impl->finish();
Detail::optimizer_barrier();
}
Detail::ChronometerConcept* impl;
int k;
int repeats;
};
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_CHRONOMETER_HPP_INCLUDED
#endif // CATCH_CHRONOMETER_HPP_INCLUDED

View File

@@ -1,15 +1,14 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Clocks
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_CLOCK_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_CLOCK_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_CLOCK_HPP_INCLUDED
#define CATCH_CLOCK_HPP_INCLUDED
#include <chrono>
#include <ratio>
@@ -37,4 +36,4 @@ namespace Catch {
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_CLOCK_HPP_INCLUDED
#endif // CATCH_CLOCK_HPP_INCLUDED

View File

@@ -1,15 +1,16 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Constructor and destructor helpers
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_CONSTRUCTOR_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_CONSTRUCTOR_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_CONSTRUCTOR_HPP_INCLUDED
#define CATCH_CONSTRUCTOR_HPP_INCLUDED
#include <catch2/internal/catch_move_and_forward.hpp>
#include <type_traits>
@@ -19,7 +20,7 @@ namespace Catch {
template <typename T, bool Destruct>
struct ObjectStorage
{
using TStorage = typename std::aligned_storage<sizeof(T), std::alignment_of<T>::value>::type;
using TStorage = std::aligned_storage_t<sizeof(T), std::alignment_of<T>::value>;
ObjectStorage() : data() {}
@@ -30,7 +31,7 @@ namespace Catch {
ObjectStorage(ObjectStorage&& other)
{
new(&data) T(std::move(other.stored_object()));
new(&data) T(CATCH_MOVE(other.stored_object()));
}
~ObjectStorage() { destruct_on_exit<T>(); }
@@ -38,11 +39,11 @@ namespace Catch {
template <typename... Args>
void construct(Args&&... args)
{
new (&data) T(std::forward<Args>(args)...);
new (&data) T(CATCH_FORWARD(args)...);
}
template <bool AllowManualDestruction = !Destruct>
typename std::enable_if<AllowManualDestruction>::type destruct()
std::enable_if_t<AllowManualDestruction> destruct()
{
stored_object().~T();
}
@@ -50,10 +51,10 @@ namespace Catch {
private:
// If this is a constructor benchmark, destruct the underlying object
template <typename U>
void destruct_on_exit(typename std::enable_if<Destruct, U>::type* = 0) { destruct<true>(); }
void destruct_on_exit(std::enable_if_t<Destruct, U>* = 0) { destruct<true>(); }
// Otherwise, don't
template <typename U>
void destruct_on_exit(typename std::enable_if<!Destruct, U>::type* = 0) { }
void destruct_on_exit(std::enable_if_t<!Destruct, U>* = 0) { }
T& stored_object() {
return *static_cast<T*>(static_cast<void*>(&data));
@@ -76,4 +77,4 @@ namespace Catch {
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_CONSTRUCTOR_HPP_INCLUDED
#endif // CATCH_CONSTRUCTOR_HPP_INCLUDED

View File

@@ -1,15 +1,14 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Environment information
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_ENVIRONMENT_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_ENVIRONMENT_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_ENVIRONMENT_HPP_INCLUDED
#define CATCH_ENVIRONMENT_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_outlier_classification.hpp>
@@ -35,4 +34,4 @@ namespace Catch {
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_ENVIRONMENT_HPP_INCLUDED
#endif // CATCH_ENVIRONMENT_HPP_INCLUDED

View File

@@ -1,15 +1,14 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Statistics estimates
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_ESTIMATE_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_ESTIMATE_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_ESTIMATE_HPP_INCLUDED
#define CATCH_ESTIMATE_HPP_INCLUDED
namespace Catch {
namespace Benchmark {
@@ -28,4 +27,4 @@ namespace Catch {
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_ESTIMATE_HPP_INCLUDED
#endif // CATCH_ESTIMATE_HPP_INCLUDED

View File

@@ -1,17 +1,16 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Execution plan
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_EXECUTION_PLAN_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_EXECUTION_PLAN_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#include <catch2/catch_config.hpp>
#ifndef CATCH_EXECUTION_PLAN_HPP_INCLUDED
#define CATCH_EXECUTION_PLAN_HPP_INCLUDED
#include <catch2/interfaces/catch_interfaces_config.hpp>
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_environment.hpp>
#include <catch2/benchmark/detail/catch_benchmark_function.hpp>
@@ -19,6 +18,7 @@
#include <catch2/benchmark/detail/catch_run_for_at_least.hpp>
#include <algorithm>
#include <iterator>
namespace Catch {
namespace Benchmark {
@@ -55,4 +55,4 @@ namespace Catch {
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_EXECUTION_PLAN_HPP_INCLUDED
#endif // CATCH_EXECUTION_PLAN_HPP_INCLUDED

View File

@@ -1,22 +1,22 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Hinting the optimizer
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_OPTIMIZER_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_OPTIMIZER_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_OPTIMIZER_HPP_INCLUDED
#define CATCH_OPTIMIZER_HPP_INCLUDED
#if defined(_MSC_VER)
# include <atomic> // atomic_thread_fence
#endif
#include <catch2/internal/catch_move_and_forward.hpp>
#include <type_traits>
#include <utility>
namespace Catch {
namespace Benchmark {
@@ -57,15 +57,15 @@ namespace Catch {
}
template <typename Fn, typename... Args>
inline auto invoke_deoptimized(Fn&& fn, Args&&... args) -> typename std::enable_if<!std::is_same<void, decltype(fn(args...))>::value>::type {
deoptimize_value(std::forward<Fn>(fn) (std::forward<Args...>(args...)));
inline auto invoke_deoptimized(Fn&& fn, Args&&... args) -> std::enable_if_t<!std::is_same<void, decltype(fn(args...))>::value> {
deoptimize_value(CATCH_FORWARD(fn) (CATCH_FORWARD(args)...));
}
template <typename Fn, typename... Args>
inline auto invoke_deoptimized(Fn&& fn, Args&&... args) -> typename std::enable_if<std::is_same<void, decltype(fn(args...))>::value>::type {
std::forward<Fn>(fn) (std::forward<Args...>(args...));
inline auto invoke_deoptimized(Fn&& fn, Args&&... args) -> std::enable_if_t<std::is_same<void, decltype(fn(args...))>::value> {
CATCH_FORWARD(fn) (CATCH_FORWARD(args)...);
}
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_OPTIMIZER_HPP_INCLUDED
#endif // CATCH_OPTIMIZER_HPP_INCLUDED

View File

@@ -1,14 +1,14 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Outlier information
#ifndef TWOBLUECUBES_CATCH_OUTLIERS_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_OUTLIERS_HPP_INCLUDED
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_OUTLIER_CLASSIFICATION_HPP_INCLUDED
#define CATCH_OUTLIER_CLASSIFICATION_HPP_INCLUDED
namespace Catch {
namespace Benchmark {
@@ -26,4 +26,4 @@ namespace Catch {
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_OUTLIERS_HPP_INCLUDED
#endif // CATCH_OUTLIERS_CLASSIFICATION_HPP_INCLUDED

View File

@@ -1,23 +1,22 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Benchmark results
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_BENCHMARK_RESULTS_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_BENCHMARK_RESULTS_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_SAMPLE_ANALYSIS_HPP_INCLUDED
#define CATCH_SAMPLE_ANALYSIS_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_estimate.hpp>
#include <catch2/benchmark/catch_outlier_classification.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <algorithm>
#include <vector>
#include <string>
#include <iterator>
namespace Catch {
@@ -36,7 +35,7 @@ namespace Catch {
samples2.reserve(samples.size());
std::transform(samples.begin(), samples.end(), std::back_inserter(samples2), [](Duration d) { return Duration2(d); });
return {
std::move(samples2),
CATCH_MOVE(samples2),
mean,
standard_deviation,
outliers,
@@ -47,4 +46,4 @@ namespace Catch {
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_BENCHMARK_RESULTS_HPP_INCLUDED
#endif // CATCH_SAMPLE_ANALYSIS_HPP_INCLUDED

View File

@@ -1,19 +1,21 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Run and analyse one benchmark
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_DETAIL_ANALYSE_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_DETAIL_ANALYSE_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_ANALYSE_HPP_INCLUDED
#define CATCH_ANALYSE_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_environment.hpp>
#include <catch2/benchmark/catch_sample_analysis.hpp>
#include <catch2/benchmark/detail/catch_stats.hpp>
#include <catch2/interfaces/catch_interfaces_config.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <algorithm>
#include <iterator>
@@ -26,7 +28,7 @@ namespace Catch {
SampleAnalysis<Duration> analyse(const IConfig &cfg, Environment<Duration>, Iterator first, Iterator last) {
if (!cfg.benchmarkNoAnalysis()) {
std::vector<double> samples;
samples.reserve(last - first);
samples.reserve(static_cast<size_t>(last - first));
std::transform(first, last, std::back_inserter(samples), [](Duration d) { return d.count(); });
auto analysis = Catch::Benchmark::Detail::analyse_samples(cfg.benchmarkConfidenceInterval(), cfg.benchmarkResamples(), samples.begin(), samples.end());
@@ -44,15 +46,15 @@ namespace Catch {
samples2.reserve(samples.size());
std::transform(samples.begin(), samples.end(), std::back_inserter(samples2), [](double d) { return Duration(d); });
return {
std::move(samples2),
CATCH_MOVE(samples2),
wrap_estimate(analysis.mean),
wrap_estimate(analysis.standard_deviation),
outliers,
analysis.outlier_variance,
};
} else {
std::vector<Duration> samples;
samples.reserve(last - first);
std::vector<Duration> samples;
samples.reserve(static_cast<size_t>(last - first));
Duration mean = Duration(0);
int i = 0;
@@ -63,7 +65,7 @@ namespace Catch {
mean /= i;
return {
std::move(samples),
CATCH_MOVE(samples),
Estimate<Duration>{mean, mean, mean, 0.0},
Estimate<Duration>{Duration(0), Duration(0), Duration(0), 0.0},
OutlierClassification{},
@@ -75,4 +77,4 @@ namespace Catch {
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_DETAIL_ANALYSE_HPP_INCLUDED
#endif // CATCH_ANALYSE_HPP_INCLUDED

View File

@@ -1,33 +1,29 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Dumb std::function implementation for consistent call overhead
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_DETAIL_BENCHMARK_FUNCTION_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_DETAIL_BENCHMARK_FUNCTION_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_BENCHMARK_FUNCTION_HPP_INCLUDED
#define CATCH_BENCHMARK_FUNCTION_HPP_INCLUDED
#include <catch2/benchmark/catch_chronometer.hpp>
#include <catch2/benchmark/detail/catch_complete_invoke.hpp>
#include <catch2/internal/catch_meta.hpp>
#include <catch2/internal/catch_unique_ptr.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <cassert>
#include <type_traits>
#include <utility>
#include <memory>
namespace Catch {
namespace Benchmark {
namespace Detail {
template <typename T>
using Decay = typename std::decay<T>::type;
template <typename T, typename U>
struct is_related
: std::is_same<Decay<T>, Decay<U>> {};
: std::is_same<std::decay_t<T>, std::decay_t<U>> {};
/// We need to reinvent std::function because every piece of code that might add overhead
/// in a measurement context needs to have consistent performance characteristics so that we
@@ -40,7 +36,7 @@ namespace Catch {
private:
struct callable {
virtual void call(Chronometer meter) const = 0;
virtual callable* clone() const = 0;
virtual Catch::Detail::unique_ptr<callable> clone() const = 0;
virtual ~callable(); // = default;
callable() = default;
@@ -49,10 +45,12 @@ namespace Catch {
};
template <typename Fun>
struct model : public callable {
model(Fun&& fun) : fun(std::move(fun)) {}
model(Fun const& fun) : fun(fun) {}
model(Fun&& fun_) : fun(CATCH_MOVE(fun_)) {}
model(Fun const& fun_) : fun(fun_) {}
model<Fun>* clone() const override { return new model<Fun>(*this); }
Catch::Detail::unique_ptr<callable> clone() const override {
return Catch::Detail::make_unique<model<Fun>>( *this );
}
void call(Chronometer meter) const override {
call(meter, is_callable<Fun(Chronometer)>());
@@ -77,33 +75,34 @@ namespace Catch {
: f(new model<do_nothing>{ {} }) {}
template <typename Fun,
typename std::enable_if<!is_related<Fun, BenchmarkFunction>::value, int>::type = 0>
std::enable_if_t<!is_related<Fun, BenchmarkFunction>::value, int> = 0>
BenchmarkFunction(Fun&& fun)
: f(new model<typename std::decay<Fun>::type>(std::forward<Fun>(fun))) {}
: f(new model<std::decay_t<Fun>>(CATCH_FORWARD(fun))) {}
BenchmarkFunction(BenchmarkFunction&& that)
: f(std::move(that.f)) {}
BenchmarkFunction( BenchmarkFunction&& that ) noexcept:
f( CATCH_MOVE( that.f ) ) {}
BenchmarkFunction(BenchmarkFunction const& that)
: f(that.f->clone()) {}
BenchmarkFunction& operator=(BenchmarkFunction&& that) {
f = std::move(that.f);
BenchmarkFunction&
operator=( BenchmarkFunction&& that ) noexcept {
f = CATCH_MOVE( that.f );
return *this;
}
BenchmarkFunction& operator=(BenchmarkFunction const& that) {
f.reset(that.f->clone());
f = that.f->clone();
return *this;
}
void operator()(Chronometer meter) const { f->call(meter); }
private:
std::unique_ptr<callable> f;
Catch::Detail::unique_ptr<callable> f;
};
} // namespace Detail
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_DETAIL_BENCHMARK_FUNCTION_HPP_INCLUDED
#endif // CATCH_BENCHMARK_FUNCTION_HPP_INCLUDED

View File

@@ -1,22 +1,22 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Invoke with a special case for void
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_DETAIL_COMPLETE_INVOKE_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_DETAIL_COMPLETE_INVOKE_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#include <catch2/internal/catch_enforce.hpp>
#ifndef CATCH_COMPLETE_INVOKE_HPP_INCLUDED
#define CATCH_COMPLETE_INVOKE_HPP_INCLUDED
#include <catch2/internal/catch_test_failure_exception.hpp>
#include <catch2/internal/catch_meta.hpp>
#include <catch2/interfaces/catch_interfaces_capture.hpp>
#include <catch2/interfaces/catch_interfaces_registry_hub.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <type_traits>
#include <utility>
namespace Catch {
namespace Benchmark {
@@ -33,39 +33,31 @@ namespace Catch {
struct CompleteInvoker {
template <typename Fun, typename... Args>
static Result invoke(Fun&& fun, Args&&... args) {
return std::forward<Fun>(fun)(std::forward<Args>(args)...);
return CATCH_FORWARD(fun)(CATCH_FORWARD(args)...);
}
};
template <>
struct CompleteInvoker<void> {
template <typename Fun, typename... Args>
static CompleteType_t<void> invoke(Fun&& fun, Args&&... args) {
std::forward<Fun>(fun)(std::forward<Args>(args)...);
CATCH_FORWARD(fun)(CATCH_FORWARD(args)...);
return {};
}
};
template <typename Sig>
using ResultOf_t = typename std::result_of<Sig>::type;
// invoke and not return void :(
template <typename Fun, typename... Args>
CompleteType_t<ResultOf_t<Fun(Args...)>> complete_invoke(Fun&& fun, Args&&... args) {
return CompleteInvoker<ResultOf_t<Fun(Args...)>>::invoke(std::forward<Fun>(fun), std::forward<Args>(args)...);
CompleteType_t<FunctionReturnType<Fun, Args...>> complete_invoke(Fun&& fun, Args&&... args) {
return CompleteInvoker<FunctionReturnType<Fun, Args...>>::invoke(CATCH_FORWARD(fun), CATCH_FORWARD(args)...);
}
extern const std::string benchmarkErrorMsg;
} // namespace Detail
template <typename Fun>
Detail::CompleteType_t<Detail::ResultOf_t<Fun()>> user_code(Fun&& fun) {
CATCH_TRY{
return Detail::complete_invoke(std::forward<Fun>(fun));
} CATCH_CATCH_ALL{
getResultCapture().benchmarkFailed(translateActiveException());
CATCH_RUNTIME_ERROR(Detail::benchmarkErrorMsg);
}
Detail::CompleteType_t<FunctionReturnType<Fun>> user_code(Fun&& fun) {
return Detail::complete_invoke(CATCH_FORWARD(fun));
}
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_DETAIL_COMPLETE_INVOKE_HPP_INCLUDED
#endif // CATCH_COMPLETE_INVOKE_HPP_INCLUDED

View File

@@ -1,15 +1,14 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Environment measurement
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_DETAIL_ESTIMATE_CLOCK_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_DETAIL_ESTIMATE_CLOCK_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_ESTIMATE_CLOCK_HPP_INCLUDED
#define CATCH_ESTIMATE_CLOCK_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_environment.hpp>
@@ -17,10 +16,10 @@
#include <catch2/benchmark/detail/catch_measure.hpp>
#include <catch2/benchmark/detail/catch_run_for_at_least.hpp>
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/internal/catch_unique_ptr.hpp>
#include <algorithm>
#include <iterator>
#include <tuple>
#include <vector>
#include <cmath>
@@ -30,11 +29,11 @@ namespace Catch {
template <typename Clock>
std::vector<double> resolution(int k) {
std::vector<TimePoint<Clock>> times;
times.reserve(k + 1);
times.reserve(static_cast<size_t>(k + 1));
std::generate_n(std::back_inserter(times), k + 1, now<Clock>{});
std::vector<double> deltas;
deltas.reserve(k);
deltas.reserve(static_cast<size_t>(k));
std::transform(std::next(times.begin()), times.end(), times.begin(),
std::back_inserter(deltas),
[](TimePoint<Clock> a, TimePoint<Clock> b) { return static_cast<double>((a - b).count()); });
@@ -68,7 +67,9 @@ namespace Catch {
}
template <typename Clock>
EnvironmentEstimate<FloatDuration<Clock>> estimate_clock_cost(FloatDuration<Clock> resolution) {
auto time_limit = std::min(resolution * clock_cost_estimation_tick_limit, FloatDuration<Clock>(clock_cost_estimation_time_limit));
auto time_limit = (std::min)(
resolution * clock_cost_estimation_tick_limit,
FloatDuration<Clock>(clock_cost_estimation_time_limit));
auto time_clock = [](int k) {
return Detail::measure<Clock>([k] {
for (int i = 0; i < k; ++i) {
@@ -82,7 +83,7 @@ namespace Catch {
auto&& r = run_for_at_least<Clock>(std::chrono::duration_cast<ClockDuration<Clock>>(clock_cost_estimation_time), iters, time_clock);
std::vector<double> times;
int nsamples = static_cast<int>(std::ceil(time_limit / r.elapsed));
times.reserve(nsamples);
times.reserve(static_cast<size_t>(nsamples));
std::generate_n(std::back_inserter(times), nsamples, [time_clock, &r] {
return static_cast<double>((time_clock(r.iterations) / r.iterations).count());
});
@@ -94,7 +95,14 @@ namespace Catch {
template <typename Clock>
Environment<FloatDuration<Clock>> measure_environment() {
static Environment<FloatDuration<Clock>>* env = nullptr;
#if defined(__clang__)
# pragma clang diagnostic push
# pragma clang diagnostic ignored "-Wexit-time-destructors"
#endif
static Catch::Detail::unique_ptr<Environment<FloatDuration<Clock>>> env;
#if defined(__clang__)
# pragma clang diagnostic pop
#endif
if (env) {
return *env;
}
@@ -103,11 +111,11 @@ namespace Catch {
auto resolution = Detail::estimate_clock_resolution<Clock>(iters);
auto cost = Detail::estimate_clock_cost<Clock>(resolution.mean);
env = new Environment<FloatDuration<Clock>>{ resolution, cost };
env = Catch::Detail::make_unique<Environment<FloatDuration<Clock>>>( Environment<FloatDuration<Clock>>{resolution, cost} );
return *env;
}
} // namespace Detail
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_DETAIL_ESTIMATE_CLOCK_HPP_INCLUDED
#endif // CATCH_ESTIMATE_CLOCK_HPP_INCLUDED

View File

@@ -1,35 +1,33 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Measure
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_DETAIL_MEASURE_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_DETAIL_MEASURE_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_MEASURE_HPP_INCLUDED
#define CATCH_MEASURE_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/detail/catch_complete_invoke.hpp>
#include <catch2/benchmark/detail/catch_timing.hpp>
#include <utility>
#include <catch2/internal/catch_move_and_forward.hpp>
namespace Catch {
namespace Benchmark {
namespace Detail {
template <typename Clock, typename Fun, typename... Args>
TimingOf<Clock, Fun(Args...)> measure(Fun&& fun, Args&&... args) {
TimingOf<Clock, Fun, Args...> measure(Fun&& fun, Args&&... args) {
auto start = Clock::now();
auto&& r = Detail::complete_invoke(fun, std::forward<Args>(args)...);
auto&& r = Detail::complete_invoke(fun, CATCH_FORWARD(args)...);
auto end = Clock::now();
auto delta = end - start;
return { delta, std::forward<decltype(r)>(r), 1 };
return { delta, CATCH_FORWARD(r), 1 };
}
} // namespace Detail
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_DETAIL_MEASURE_HPP_INCLUDED
#endif // CATCH_MEASURE_HPP_INCLUDED

View File

@@ -1,18 +1,17 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// repeat algorithm
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_DETAIL_REPEAT_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_DETAIL_REPEAT_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_REPEAT_HPP_INCLUDED
#define CATCH_REPEAT_HPP_INCLUDED
#include <type_traits>
#include <utility>
#include <catch2/internal/catch_move_and_forward.hpp>
namespace Catch {
namespace Benchmark {
@@ -27,11 +26,11 @@ namespace Catch {
Fun fun;
};
template <typename Fun>
repeater<typename std::decay<Fun>::type> repeat(Fun&& fun) {
return { std::forward<Fun>(fun) };
repeater<std::decay_t<Fun>> repeat(Fun&& fun) {
return { CATCH_FORWARD(fun) };
}
} // namespace Detail
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_DETAIL_REPEAT_HPP_INCLUDED
#endif // CATCH_REPEAT_HPP_INCLUDED

View File

@@ -1,15 +1,14 @@
/*
* Created by Joachim on 16/04/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Run a function for a minimum amount of time
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
#ifndef TWOBLUECUBES_CATCH_RUN_FOR_AT_LEAST_HPP_INCLUDED
#define TWOBLUECUBES_CATCH_RUN_FOR_AT_LEAST_HPP_INCLUDED
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#ifndef CATCH_RUN_FOR_AT_LEAST_HPP_INCLUDED
#define CATCH_RUN_FOR_AT_LEAST_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_chronometer.hpp>
@@ -17,40 +16,43 @@
#include <catch2/benchmark/detail/catch_complete_invoke.hpp>
#include <catch2/benchmark/detail/catch_timing.hpp>
#include <catch2/internal/catch_meta.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <utility>
#include <type_traits>
namespace Catch {
namespace Benchmark {
namespace Detail {
template <typename Clock, typename Fun>
TimingOf<Clock, Fun(int)> measure_one(Fun&& fun, int iters, std::false_type) {
TimingOf<Clock, Fun, int> measure_one(Fun&& fun, int iters, std::false_type) {
return Detail::measure<Clock>(fun, iters);
}
template <typename Clock, typename Fun>
TimingOf<Clock, Fun(Chronometer)> measure_one(Fun&& fun, int iters, std::true_type) {
TimingOf<Clock, Fun, Chronometer> measure_one(Fun&& fun, int iters, std::true_type) {
Detail::ChronometerModel<Clock> meter;
auto&& result = Detail::complete_invoke(fun, Chronometer(meter, iters));
return { meter.elapsed(), std::move(result), iters };
return { meter.elapsed(), CATCH_MOVE(result), iters };
}
template <typename Clock, typename Fun>
using run_for_at_least_argument_t = typename std::conditional<is_callable<Fun(Chronometer)>::value, Chronometer, int>::type;
using run_for_at_least_argument_t = std::conditional_t<is_callable<Fun(Chronometer)>::value, Chronometer, int>;
[[noreturn]]
void throw_optimized_away_error();
template <typename Clock, typename Fun>
TimingOf<Clock, Fun(run_for_at_least_argument_t<Clock, Fun>)> run_for_at_least(ClockDuration<Clock> how_long, int seed, Fun&& fun) {
auto iters = seed;
TimingOf<Clock, Fun, run_for_at_least_argument_t<Clock, Fun>>
run_for_at_least(ClockDuration<Clock> how_long,
const int initial_iterations,
Fun&& fun) {
auto iters = initial_iterations;
while (iters < (1 << 30)) {
auto&& Timing = measure_one<Clock>(fun, iters, is_callable<Fun(Chronometer)>());
if (Timing.elapsed >= how_long) {
return { Timing.elapsed, std::move(Timing.result), iters };
return { Timing.elapsed, CATCH_MOVE(Timing.result), iters };
}
iters *= 2;
}
@@ -60,4 +62,4 @@ namespace Catch {
} // namespace Benchmark
} // namespace Catch
#endif // TWOBLUECUBES_CATCH_RUN_FOR_AT_LEAST_HPP_INCLUDED
#endif // CATCH_RUN_FOR_AT_LEAST_HPP_INCLUDED

View File

@@ -1,18 +1,19 @@
/*
* Created by Martin on 15/06/2019.
* Adapted from donated nonius code.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
// Statistical analysis tools
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#include <catch2/benchmark/detail/catch_stats.hpp>
#include <catch2/internal/catch_compiler_capabilities.hpp>
#include <cassert>
#include <cstddef>
#include <iterator>
#include <random>
@@ -21,6 +22,27 @@
#endif
namespace {
using Catch::Benchmark::Detail::sample;
template <typename URng, typename Estimator>
sample resample(URng& rng, unsigned int resamples, std::vector<double>::iterator first, std::vector<double>::iterator last, Estimator& estimator) {
auto n = static_cast<size_t>(last - first);
std::uniform_int_distribution<decltype(n)> dist(0, n - 1);
sample out;
out.reserve(resamples);
std::generate_n(std::back_inserter(out), resamples, [n, first, &estimator, &dist, &rng] {
std::vector<double> resampled;
resampled.reserve(n);
std::generate_n(std::back_inserter(resampled), n, [first, &dist, &rng] { return first[static_cast<std::ptrdiff_t>(dist(rng))]; });
return estimator(resampled.begin(), resampled.end());
});
std::sort(out.begin(), out.end());
return out;
}
double erf_inv(double x) {
// Code accompanying the article "Approximating the erfinv function" in GPU Computing Gems, Volume 2
double w, p;
@@ -98,11 +120,15 @@ namespace {
double standard_deviation(std::vector<double>::iterator first, std::vector<double>::iterator last) {
auto m = Catch::Benchmark::Detail::mean(first, last);
double variance = std::accumulate(first, last, 0., [m](double a, double b) {
double diff = b - m;
return a + diff * diff;
}) / (last - first);
return std::sqrt(variance);
double variance = std::accumulate( first,
last,
0.,
[m]( double a, double b ) {
double diff = b - m;
return a + diff * diff;
} ) /
( last - first );
return std::sqrt( variance );
}
}
@@ -150,7 +176,7 @@ namespace Catch {
double sb = stddev.point;
double mn = mean.point / n;
double mg_min = mn / 2.;
double sg = std::min(mg_min / 4., sb / std::sqrt(n));
double sg = (std::min)(mg_min / 4., sb / std::sqrt(n));
double sg2 = sg * sg;
double sb2 = sb * sb;
@@ -161,7 +187,7 @@ namespace Catch {
double k0 = -n * nd;
double k1 = sb2 - n * sg2 + nd;
double det = k1 * k1 - 4 * sg2 * k0;
return (int)(-2. * k0 / (k1 + std::sqrt(det)));
return static_cast<int>(-2. * k0 / (k1 + std::sqrt(det)));
};
auto var_out = [n, sb2, sg2](double c) {
@@ -169,11 +195,11 @@ namespace Catch {
return (nc / n) * (sb2 - nc * sg2);
};
return std::min(var_out(1), var_out(std::min(c_max(0.), c_max(mg_min)))) / sb2;
return (std::min)(var_out(1), var_out((std::min)(c_max(0.), c_max(mg_min)))) / sb2;
}
bootstrap_analysis analyse_samples(double confidence_level, int n_resamples, std::vector<double>::iterator first, std::vector<double>::iterator last) {
bootstrap_analysis analyse_samples(double confidence_level, unsigned int n_resamples, std::vector<double>::iterator first, std::vector<double>::iterator last) {
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS
static std::random_device entropy;

Some files were not shown because too many files have changed in this diff Show More