Compare commits

...

140 Commits

Author SHA1 Message Date
Chris Thrasher
a5245ac76b Remove redundant destructors
Classes will automatically inherit the virtual-ness of their base
class destructors. If the base class already has a virtual
destructor and the derived class needs default destructor semantics
then the derived class can omit defining the destructor in favor of
the compiler automatically defining it.

This has an additional benefit of reenabling move semantics. The
presence of a user-specified destructor automatically disables move
operations.
2023-12-30 18:56:54 -07:00
Martin Hořeňovský
b7b71ffd3a Merge pull request #2786 from catchorg/redundant_link_libraries
Fix clang warning about redundant link libraries
2023-12-29 21:31:03 +01:00
Chris Thrasher
048d7f7796 Fix clang warning about redundant link libraries 2023-12-28 16:54:30 -06:00
Vertexwahn
a9a94bec13 Bazel support: Bump bazel-skylib version to 1.5.0 2023-12-28 23:44:26 +01:00
影月 零
c8262e1f40 Handle linking to liblog when [cross] building for Android using meson. (#2784)
* Alter meson for Android cross builds.

* Handle liblog linking on Termux as well as the Android NDK.
2023-12-28 23:40:42 +01:00
Martin Hořeňovský
08bdd43fcd Merge pull request #2785 from jpalus/optional-reproducible-build
Make compiler flags for reproducible builds optional
2023-12-28 21:20:41 +01:00
Jan Palus
1512dac7e4 Make compiler flags for reproducible builds optional
no change in default behavior: -ffile-prefix-map is still added if
supported by compiler but add option for disabling it as it breaks
generic tools for extracting debuginfo/debugsource from build artifacts
like: https://sourceware.org/git/?p=debugedit.git;a=blob;f=scripts/find-debuginfo.in
2023-12-28 00:41:54 +01:00
Martin Hořeňovský
b52d97855d Move-enable various parts of TextFlow
This removes about 200 pointless copies from printing the help
message (the original motivation for the change), and also nicely
improves performance of the various reporters that depend on
TextFlow.
2023-12-27 21:43:49 +01:00
Martin Hořeňovský
eaafd07674 Avoid copying TokenStreams while parsing
The code is now even worse mess than before, due to the ad-hoc
implementation of Result-ish type based on virtual functions in
Clara, but it has dropped the allocations for empty binary down
to 151.
2023-12-27 21:14:12 +01:00
Martin Hořeňovský
5d637d4c6b Args and TokenStream use StringRefs instead of std::strings
Because TokenStream is copied around a lot, moving it to use
`StringRef` removes _a lot_ of allocations per `Opt` in the parser.
Args are not copied around much, but changing them as well makes it
obvious that they do not participate in the ownership.

The changes add up to removing ~180 allocations for "empty"
invocation of the test binary.
(`./tests/ExtraTests/NoTests --allow-running-no-tests -o /dev/null`
is down to 317 allocs)
2023-12-27 16:24:33 +01:00
Martin Hořeňovský
cd3c7ebe87 Use StringRef for Opt's optNames
Removes another ~70 allocations.
2023-12-26 16:03:22 +01:00
Martin Hořeňovský
5d5f42f99b Enable moving Opts into Parser
This also required keeping the reference type in the fluent interface
for Opts.

Removes another ~40 allocations.
2023-12-26 15:41:07 +01:00
Martin Hořeňovský
c57e349d1d Avoid copying Clara::Parser for every inserted Opt
This prevents the full construction from being O(N^2) in number
of `Opt`s, and also reduces the number of allocations for running
no tests significantly:

`tests/SelfTest`: 7705 -> 6095
`tests/ExtraTests/NoTests` 2215 -> 605
2023-12-26 15:11:48 +01:00
Martin Hořeňovský
822c44a203 Cleanup help string construction in Clara
* Clara::Opt::getHelpColumns returns single item

  It could never return multiple items, but for some reason it
  was wrapping that single item in a vector.

* Use ReusableStringStream in Clara
* Reserve HelpColumns ahead of time
* Use StringRef for descriptions in HelpColumn type

The combination of these changes ends up removing about 7% (~200)
of allocations when Catch2 has to prepare output for `-h`.
2023-12-26 14:57:29 +01:00
Martin Hořeňovský
2295d2c8cc Store Opt/Arg hint and description as StringRef
There is no good reason for these to be std::strings, as these
are just (optional) constants for nice user output. This ends up
reducing the allocations significantly.

When measuring allocations when running no tests, the changes are
`tests/SelfTest` 9213 -> 7705
`tests/ExtraTests/NoTests` 3723 -> 2215
2023-12-26 14:40:45 +01:00
Martin Hořeňovský
2b69a3e216 Remove superfluous catch_session.hpp include in catch_sharding.hpp 2023-12-26 14:02:19 +01:00
Martin Hořeňovský
c809cb4d1c Update Doxyfile for newer doxygen 2023-12-23 11:27:46 +01:00
Martin Hořeňovský
9aadc3a53d Cleanup includes in sources
Mainly just removes some unused includes, but sometimes the include
is replaced by a smaller header instead.
2023-12-23 11:23:32 +01:00
Martin Hořeňovský
64ade68ca2 Remove superfluous duration_casts from benchmarking 2023-12-21 18:40:00 +01:00
Martin Hořeňovský
680064d391 Don't store right end of the interval in uniform_integer_distribution 2023-12-21 18:39:18 +01:00
Martin Hořeňovský
3acb8b30f1 More detailed examples for lifetimes in combined matcher exprs
Example taken from #2777
2023-12-13 17:24:23 +01:00
Martin Hořeňovský
3f23192e55 Fix typo in release notes 2023-12-13 17:24:18 +01:00
Igor Machado Coelho
d40a3289e5 Support MODULE.bazel (#2781) 2023-12-13 17:22:56 +01:00
Martin Hořeňovský
53d0d913a4 v3.5.0 2023-12-11 00:55:40 +01:00
Martin Hořeňovský
1648c30ec3 Look just for 'Catch2 X.Y.Z' in doc placeholder update 2023-12-11 00:52:22 +01:00
Simhon Chourasia
d4e9fb8aa5 Highlight that SECTIONs rerun the entire test case from beginning (#2749) 2023-12-10 22:35:54 +01:00
Martin Hořeňovský
b606bc2802 Remove obsolete section in limitations.md
We no longer use `std::shuffle` to implement random test order, so
a debug mode bug in old versions of libstdc++ cannot apply to us.
2023-12-10 22:09:48 +01:00
Blake-Madden
4ab0af8baf Fix minor typos in documentation (#2769)
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-12-10 22:01:13 +01:00
Martin Hořeňovský
b7d70ddcd6 Ensure we always read 32 bit seed from std::random_device 2023-12-10 21:37:12 +01:00
Martin Hořeňovský
a6f22c5169 Remove static instance of std::random_device in Benchmark::analyse_samples 2023-12-10 21:22:43 +01:00
Martin Hořeňovský
1887d42e3d Use our PCG32 RNG instead of mt19937 in Benchmark::analyse_samples 2023-12-10 21:18:23 +01:00
Martin Hořeňovský
1774dbfd53 Make it clearer that the JSON reporter is WIP 2023-12-10 21:04:56 +01:00
Martin Hořeňovský
cb07ff9a7e Fix uniform_floating_point_distribution for unit ranges 2023-12-10 19:53:46 +01:00
Martin Hořeňovský
ae4fe16b81 Make the user-facing random Generators reproducible
Thanks to the new distributions, this is almost trivial change.
2023-12-10 19:53:44 +01:00
Martin Hořeňovský
28c66fdc5a Make uniform_floating_point_distribution reproducible
By moving to use our `uniform_integer_distribution`, which is
reproducible across different platforms, instead of the stdlib
one which is not, we can provide reproducible results for `float`s
and `double`s. Still no reproducibility for `long double`s, because
those are too different across different platforms.
2023-12-10 19:53:41 +01:00
Martin Hořeňovský
ed9d672b5c Add uniform_integer_distribution 2023-12-10 19:53:39 +01:00
Martin Hořeňovský
04a829b0e1 Add helpers for implementing uniform integer distribution
* Utility for extended mult n x n bits -> 2n bits
* Utility to adapt output from URBG to target (unsigned) integral
  type
* Utility to reorder signed values into unsigned type while keeping
  the order.
2023-12-10 19:53:36 +01:00
Martin Hořeňovský
ab1b079e4d Add uniform_floating_point_distribution 2023-12-10 18:44:12 +01:00
Martin Hořeňovský
d139b4ff7c Add implementation of helpers for uniform float distribution
Specifically we add
 * `gamma(a, b)`, which returns the magnitude of largest 1-ULP
   step in range [a, b].
 * `count_equidistant_float(a, b, distance)`, which returns the
   number of equi-distant floats in range [a, b].
2023-12-10 18:44:10 +01:00
Martin Hořeňovský
bfd9f0f5a6 Move nextafter polyfill to polyfills.hpp 2023-12-10 18:44:06 +01:00
Martin Hořeňovský
9a1e73568c Add test showing literals and complex generators in one GENERATE 2023-12-10 18:32:45 +01:00
Ikko Eltociear Ashimine
21d2da23bc Fix typo in tostring.md
specialiation -> specialization
2023-12-09 21:00:24 +01:00
Martin Hořeňovský
d1d7414eb9 Always run apt-get update before apt-get install
The base images for GitHub Actions are updated weekly, but
sometimes that is not enough to be able to install the packages
we require. The recommended fix for this is to always run
`apt-get update` before `apt-get install`.
2023-12-09 12:05:45 +01:00
Martin Hořeňovský
dacbf4fd6c Drop VS 2017 support
We do not support specific compilers, but rather compilers with
reasonable quality of their C++14 support. While developing the
new random generators, I ran into issues with VS2017 where it
rejects perfectly valid C++14 code, **and** the error does not
point me in the right direction to try and work around the issue.

It is time for VS2017 to go.
2023-12-08 23:54:00 +01:00
Pablo Duboue
0520ff4436 [DOC] Replaced broken link (fixes #2770)
The original link is no longer available but a fork is still on GitHub.
2023-12-02 22:05:48 +01:00
SupSuper
4a7be16c8c Fix compilation on Xbox platforms
Xbox does not support getenv
2023-12-02 22:03:48 +01:00
Martin Hořeňovský
32d9ae24bc JSONWriter deals in StringRefs instead of std::strings
Together with liberal use of `_sr` UDL to compile-time convert
string literals into StringRefs, this will reduce the number of
allocation and remove most of the strcpy calls inherent in
converting string lits into `std::string`s.
2023-11-17 09:54:03 +01:00
Sergei Iskakov
de7ba4e889 fn need to be in parenthesis
Otherwise intel c++ 19.1 cause an error
"expression preceding parentheses of apparent call must have (pointer-to-) function type CATCH2"
2023-11-16 12:33:35 +01:00
Martin Hořeňovský
733b901dd2 Fix special character escaping in JsonWriter 2023-11-14 23:35:22 +01:00
Beartama
7bf136b501 Add JSON reporter (#2706)
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-11-14 22:52:15 +01:00
Gerald Senarclens de Grancy
2c68a0d05f lifted suggested version
Did this, because with 3.0.1, the files
- catch_xmlwriter.cpp
- catch_string_manip.hpp
- catch_test_case_info.hpp

were missing

This led to some obvious and some obscure compile errors when compiling
with g++ 13.2.0 (std c++17 or c++20)
One of these errors being "Elaborated-type-specifier for a scoped enum
must not use the ‘class’ keyword"

Since this is a rather bad experience and debugging it is
time-consuming, I suggest to simply boost the recommended version in the
snippet which is likely copied by new Catch2 users
2023-11-14 17:10:31 +01:00
Krzysiek Karbowiak
01cac90c62 Bump up actions/checkout version to v4 (#2760) 2023-11-09 10:31:49 +01:00
Krzysiek Karbowiak
b735dfce2d Increase build parallelism on macOS (#2759)
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-11-07 22:17:47 +01:00
Martin Hořeňovský
caffe79a31 Fix missing include in catch_message.hpp
Because the issue comes from the expansions of `UNSCOPED_INFO`,
surrogate TUs could not catch this bug, and in common usage, the
include transitively comes from `catch_test_macros.hpp`.

Fixes #2758
2023-11-04 00:33:18 +01:00
Chris Thrasher
a8cf3e6710 Mark CATCH_CONFIG_ options as advanced
These options are rather low-level and don't need to be seen in the
CMake cache unless you opt into seeing all other advanced options.

This removes a lot of cache entries from the screen when using a GUI
or TUI to view the cache thus making it easier for users to focus on
the cache variables they're more likely to change on a frequent
basis.
2023-10-31 17:27:46 -06:00
Martin Hořeňovský
79d39a1954 Fix tests for C++23's multi-arg index operator
Closes #2744
2023-10-28 21:49:58 +02:00
Martin Hořeňovský
6ebc013b8c Fix UDL definitions for C++23
Technically, the declaration should not have a space between
the quotes and the underscore, because `_foo` is a reserved
identifier, but `""_foo` is not. In general it works, but newer
Clang versions warn about this, because WG21 wants to deprecate
and later remove this form completely.
2023-10-28 21:35:03 +02:00
Alex Merry
966d361551 Improve formatting of test specification docs
The existing formatting created one-element lists separated by paragraphs, when it would make more sense to have the paragraphs that are providing more information about one of those list entries be part of the list entry itself.

I think this makes the documentation easier to read in both markdown and html form, and should also improve the structure for assistive technologies.
2023-10-24 16:01:56 +02:00
Per Lundberg
766541d12d why-catch.md: Add JetBrains survey link 2023-09-27 20:14:35 +02:00
Holger Kaelberer
7b793314e5 Catch.cmake: Support CMake multi-config with PRE_TEST discovery mode (#2739)
Closes #2746 

---------

Co-authored-by: Holger Kaelberer <Holger.Kaelberer@bmw.de>
2023-09-25 19:40:35 +02:00
Christian Tacke
0fb817e41f fix some bugprone-macro-parentheses warnings
When using the public headers of catch2 in another project
that uses clang-tidy with some checks enabled, then some
warnings in catch2's headers are also reported.

This fixes a bunch of bugprone-macro-parentheses warnings.

See: https://clang.llvm.org/extra/clang-tidy/checks/bugprone/macro-parentheses.html
2023-09-25 10:56:45 +02:00
Martin Hořeňovský
f161110be4 Merge pull request #2747 from xandox/devel
correct argument references in CatchAddTests.cmake
2023-09-25 10:52:10 +02:00
Ilya Arzhannikov
db495acdbb correct argument references in CatchAddTests.cmake 2023-09-20 15:47:11 +02:00
Martin Hořeňovský
9c541ca72e Add test for multiple streaming parts in UNSCOPED_INFO 2023-09-17 10:44:31 +02:00
Martin Hořeňovský
92672591c1 Make jackknife TU-local to stats.cpp 2023-09-16 21:29:47 +02:00
Martin Hořeňovský
56fcd584c1 Make directCompare TU-local to stats.cpp 2023-09-16 21:18:44 +02:00
Jonathan Allen Grant
aafe09bc1c Update meson.build to fix #2722 (#2742) 2023-09-13 10:06:14 +02:00
Martin Hořeňovský
47a2c96938 Reduce the number of templates in Benchmarking
The basic idea was to reduce the number of things dependent on the `Clock`
type. To that end, I replaced `Duration<Clock>` with `IDuration` typedef
for `std::nanoseconds`, and `FloatDuration<Clock>` with `FDuration`
typedef for `Duration<double, std::nano>`. We can generally assume that
any clock's duration can be expressed in nanoseconds, as long as we insert
`duration_cast`s into the right places.

Note that we cannot remove all dependence on `Clock` as a template
arguments, because functions that actually measure the elapsed time have
to use the Clock.

We also changed some template function arguments to pass plain function
pointers, so that the actual implementation can be placed into a cpp file.
2023-09-08 10:04:31 +02:00
Martin Hořeňovský
fb96279aed Remove superfluous stdlib includes from catch_benchmark.hpp 2023-09-08 10:04:24 +02:00
Martin Hořeňovský
e14a08d734 Remove unused typedef from Benchmark::Environment 2023-09-08 10:04:22 +02:00
Martin Hořeňovský
9bba07cb87 Replace vector iterator args in benchmarks with ptr args 2023-09-08 10:04:19 +02:00
Martin Hořeňovský
b4ffba5087 Update sample output in docs/benchmarks.md 2023-09-08 10:04:17 +02:00
Martin Jeřábek
3a5cde55b7 implement stringify for std::nullopt_t 2023-08-30 16:21:02 +02:00
Martin Hořeňovský
2a19ae16b8 Rewrite commandline test spec docs
Closes #2738
2023-08-30 16:18:34 +02:00
Martin Hořeňovský
f24d39e42b Support C arrays and ADL ranges in from_range generator
Closes #2737
2023-08-29 15:38:13 +02:00
Martin Hořeňovský
85eb4652b4 Add nice license headers to files in examples/ and fuzzing/
Related to #2730
2023-08-24 16:34:31 +02:00
Ryan Pavlik
5bba3e4038 Edited amalgamated file generator, to block REUSE from getting confused
It struggled with the copyright lines included as string literals.
2023-08-19 14:31:27 +02:00
Martin Hořeňovský
e09de7222c Small cleanup in XML reporter 2023-08-14 12:41:52 +02:00
Martin Hořeňovský
a64ff326bf Change 'estimated' to 'est run time' in console reporter output 2023-08-14 10:22:59 +02:00
Martin Hořeňovský
ad56463477 Flush stream after benchmarkStarting in ConsoleReporter
This means that the user will see the estimation of full benchmark
running time when it is available, unlike now when it often only
ends up flushed after the benchmark is fully finished.

This means that the user will almost immediately see the start
of table like this

```
benchmark name                       samples       iterations    estimated
                                     mean          low mean      high mean
                                     std dev       low std dev   high std dev
-------------------------------------------------------------------------------
Fill vector generated                          100            54     3.0834 ms
```

This presents significant improvement in user experience especially
for long running benchmarks.
2023-08-13 23:09:02 +02:00
Martin Hořeňovský
9538d16005 Mention missing catch_user_config.hpp in FAQ 2023-08-11 15:36:23 +02:00
Martin Hořeňovský
a94bee771e Add missing line for v3.4.0 to ToC in release-notes.md 2023-08-11 15:36:03 +02:00
Martin Hořeňovský
d7304f0c41 Constify section hints in static-analysis mode
This prevents a `misc-const-correctness` in clang-tidy
2023-08-10 21:02:18 +02:00
rosstang
cd60a0301c Assert Info reset need to also reset result disposition to normal to handle uncaught exception correctly (#2723)
* AssertionEnd does not reset the assertion info yet. That is done after populateReaction. And reset assertion info would also reset the result disposition to normal, so that any uncaught exception would be reported as failure

* Approving test output changes due to added unit tests

* Unit tests to throw std::runtime_error instead of std::exception

* Add a unit test to test incomplete assertion handler

---------

Co-authored-by: Ross <ross.tang@gfo-x.com>
2023-08-07 22:07:31 +02:00
Martin Hořeňovský
b593be2116 Always default empty destructors 2023-08-05 18:21:38 +02:00
Vitalii Trubchaninov
ed4acded38 Don't define tryTranslators function if exception are disabled
"-Wunused-function -Wall" produces if this function is defined when exceptions are disabled
2023-08-04 15:28:55 +02:00
Riom
4acc51828f Introduce CATCH_CONFIG_PREFIX_MESSAGES to only prefix a few logging related macros. (#2544)
* Add missing include for VxWorks build.

std::min is defined in algorithm provides std::min. It appears to be transitively included for most platforms. For VxWorks however this explicit include is required.

* Add option CATCH_CONFIG_PREFIX_MESSAGES to selectively prefix message macros only.

In contrast to CATCH_CONFIG_PREFIX_ALL, this will only prefix the following macros:
I.e. INFO, UNSCOPED_INFO, WARN and CATCH_CAPTURE

This is mainly useful for codebases that use INFO or WARN for their own logging macros.
2023-07-19 17:04:43 +02:00
Martin Hořeňovský
6e79e682b7 v3.4.0 2023-07-13 13:37:30 +02:00
Martin Hořeňovský
683c85772f Clean up explanation in tests 2023-07-12 11:49:43 +02:00
Martin Hořeňovský
1b049bdba4 2 more TEST_CASEs to DiscoverTests/register-tests.cpp 2023-07-04 00:06:24 +02:00
Martin Hořeňovský
e4b16053a6 Escape Catch2 test names in catch_discover_tests tests 2023-06-15 14:19:39 +02:00
Robin Christ
42ee66b5e6 Fix handling of semicolon and backslash characters in CMake test discovery (#2676)
This PR fixes the handling of semicolon and backslash characters in test names in the CMake test discovery

Closes #2674
2023-06-14 23:40:10 +02:00
Martin Hořeňovský
a0c6a28460 Fix possible FP in catch_discover_tests tests 2023-06-14 23:31:41 +02:00
Martin Hořeňovský
c8363143e7 Add test scaffolding for catch_discover_tests 2023-06-14 21:14:33 +02:00
Martin Hořeňovský
7a52dfa77b Fix typo in cross-docs links 2023-06-11 19:37:15 +02:00
Vertexwahn
9131736630 Bazel support: Update skylib 2023-06-08 13:30:50 +02:00
Martin Hořeňovský
0631b607ee Test & document SKIP in generator constructor
Closes #1593
2023-05-31 15:12:43 +02:00
Martin Hořeňovský
dff7513b28 Static analysis cleanup in tests 2023-05-29 21:45:30 +02:00
Martin Hořeňovský
bf5aa7b383 Experimental static analysis support in TEST_CASE and SECTION
Closes #2681
2023-05-29 21:45:28 +02:00
Martin Hořeňovský
dba9197ec7 Add new config option: STATIC_ANALYSIS_SUPPORT 2023-05-29 00:55:20 +02:00
Martin Hořeňovský
f60c15364b Add macro for suppressing Wshadow 2023-05-28 21:07:31 +02:00
Martin Hořeňovský
b3cf1bfb5d Avoid unused variable warning in GeneratorsImpl tests 2023-05-28 21:05:01 +02:00
Martin Hořeňovský
73b93ce6bc Include catch_user_config.hpp in all catch_config_* files 2023-05-28 21:04:16 +02:00
Martin Hořeňovský
8008625d7e Merge pull request #2693 from Ali-Amir/u/ali/optional-meson-unit-tests
Add option to disable building unit tests in Meson build file.
2023-05-27 12:00:52 +02:00
Ali-Amir Aldan
ce7b153021 Add option to disable building unit tests in Meson build file. 2023-05-26 10:05:06 -07:00
Cristian Morales Vega
535205e2ac Suppress -Wunused-result warning in gcc
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66425.
2023-05-23 23:31:55 +02:00
Martin Hořeňovský
689fdcd7dc Fix some tests never being run 2023-05-20 22:03:48 +02:00
Martin Hořeňovský
a153fce724 Improve error messages for TEST_CASE tag parsing errors
Also removes a duplicated test case checking for empty tag error.

Related to #2650
2023-05-20 21:13:48 +02:00
Martin Hořeňovský
06c0e1cfab Merge pull request #2689 from ThePhD/fix/includes/header-exception
🛠💚 Add <exception> header where strictly necessary
2023-05-16 18:41:19 +02:00
ThePhD
05d7eb5a00 🛠 Add <exception> header where strictly necessary 2023-05-16 12:18:57 -04:00
Valeri
f53bb3ae7b meson: require version >=0.54.1
See discussion in https://github.com/mesonbuild/wrapdb/pull/1016.
2023-05-15 17:41:56 +02:00
Martin Hořeňovský
ce8a7b3390 Merge pull request #2687 from ChrisThrasher/sfml
Add SFML to the list of open source users
2023-05-15 13:44:59 +02:00
Chris Thrasher
6dce539fad Add SFML to the list of open source users 2023-05-13 14:34:34 -06:00
Yaroslav
5a40b2275c Update CatchConfigOptions.cmake
Fix CMake warning

CMake Warning (dev) at catch2-src/CMake/CatchConfigOptions.cmake:71 (set):
  uninitialized variable 'BUILD_SHARED_LIBS'
2023-05-12 13:52:45 +02:00
Martin Hořeňovský
598895d048 Fix Wredundant-decls
Closes #2682
2023-05-12 09:51:13 +02:00
Martin Hořeňovský
0dc82e08df Move CATCH_INTERNAL_STRINGIFY macro into its own header 2023-05-07 20:58:54 +02:00
Martin Hořeňovský
8ca504cbc9 Move AssertionResult when passing it inside RunContext 2023-05-06 23:58:48 +02:00
Martin Hořeňovský
c57b5cdf43 Move-enable Catch::optional
This avoids copies in couple places through Catch2, e.g. reporter
spec handling, and moving around `AssertionResult` in `RunContext`.
2023-05-06 15:22:02 +02:00
Martin Hořeňovský
d84777c9cb Fix assertionStarting events being sent after the expr is evaluated
Closes #2678
2023-05-06 11:48:41 +02:00
Martin Hořeňovský
51fdbedd13 Internal linkage for outlier_variance 2023-05-01 13:21:47 +02:00
Martin Hořeňovský
10f0a58643 Some template instantiation reductions 2023-05-01 13:21:45 +02:00
Martin Hořeňovský
fe64c28925 Reduce compilation costs of benchmarks
We replaced some simple std::algorithm usage by loops, and reduced
header inclusion.
2023-05-01 13:21:41 +02:00
Martin Hořeňovský
7d07efc92b Clean up iterator usage in benchmarks
Specifically we turned `mean`, `classify_outliers`, `jackknife`,
into concrete functions that take only `const_iterator` from vecs,
instead of generic iterators over anything.

I also changed `resample` to take `const_iterator` instead of
plain `iterator`, and similar for `standard_deviation`, and
`analyse_samples`.
2023-05-01 13:21:36 +02:00
Martin Hořeňovský
f3c678c0ab Constexprify constants in estimate_clock.hpp 2023-05-01 13:21:33 +02:00
Vertexwahn
46539b6d9b Fix spelling 2023-04-29 12:55:51 +02:00
Holger Kaelberer
10596b2278 Fix unreachable-code-return warnings 2023-04-20 14:53:03 +02:00
Holger Kaelberer
897fe2a01b cmake: Improve unreachable-code warnings
Enable CI to report -Wunreachable-code-aggressive warnings in clang
builds which covers all,  -Wunreachable-code, -code-break,
-code-return.
2023-04-20 14:53:03 +02:00
Holger Kaelberer
aad926baf8 Catch.cmake: Add new DISCOVERY_MODE option to catch_discover_tests
Introducing a new DISCOVERY_MODE mode option, which provides greater
control over when catch_discover_tests perforsm test discovery.

It has two supported modes:
* POST_BUILD: The default behavior, which adds a POST_BUILD command
  to perform test discovery after the test has been built as was
  always done so far.

* PRE_TEST: New mode, which delays test discovery until test execution.
  The generated include file generates the appropriate CTest files at
  runtime and regenerates the CTest files when the executable is
  updated.
  This mode can be used in build-environments that don't allow for
  executing the linked binaries at build-time (like in a
  cross-compilation environment).

DISCOVERY_MODE can be controlled in two ways:
1. Setting the DISCOVERY_MODE when calling catch_discover_tests.

2. Setting the global CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE prior
   to calling gtest_discover_tests.

Closes #2493
2023-04-19 23:51:42 +02:00
Holger Kaelberer
4e8399d835 CatchAddTests.cmake: Refactor into callable method
Move test discovery logic into new catch_discover_tests_impl method
and make CatchAddTests aware of whether it is being launched in
CMake's script mode.

When launched in script mode, catch_discover_tests_impl is called
passing arguments obtained from the definitions passed into the call
to cmake. This preserves the existing behavior assumed by Catch.cmake.

Looking ahead, it also allows CatchAddTests to be included in
generated files and call catch_discover_tests_impl to perform test
discovery at test runtime with the new PRE_TEST discovery mode
introduced later.
2023-04-19 23:51:42 +02:00
Martin Hořeňovský
9a2a4eadc0 Bump xml-format-version in XML reporter 2023-04-10 21:59:50 +02:00
Martin Hořeňovský
fb806da76f Add lineinfo to XML reporter output for INFO/WARN
Closes #1251
2023-04-10 16:32:07 +02:00
Arne Mertz
50bf00e266 Fix reporter detection in catch_discover_tests
Closes #2668
Instead of grepping for the reporter, use it in the process and check for the error code
2023-04-05 10:12:50 +02:00
Martin Hořeňovský
9f08097f55 Cleanup internal includes by splitting out some event structs
* Split out BenchmarkInfo and BenchmarkStats to their own header
* Outline BenchmarkStats<> declaration to separate header
* Split out TestRunInfo into its own header

These changes let us remove the large `interfaces_reporter.hpp`
include from `benchmark.hpp`, and replace it with
`interfaces_capture.hpp` in `run_context.hpp`.

I also cleaned out `interfaces_repoter.hpp` from reporter headers
that depend on `reporter_common_base.hpp`. This will not change
anything in the actual inclusion set, but makes it logically
more consistent.
2023-03-31 19:31:51 +02:00
Martin Hořeňovský
1f881ab464 Split ITestInvoker into its own header 2023-03-23 16:50:11 +01:00
Martin Hořeňovský
c487b27d9d Reduce misc includes all around 2023-03-23 16:50:07 +01:00
Martin Hořeňovský
3230760db2 Cleanup in translating exceptions to messages 2023-03-23 12:26:44 +01:00
Martin Hořeňovský
b3ebce715e Cleanup benchmarking includes 2023-03-23 11:59:40 +01:00
Martin Hořeňovský
d0f70fdfd6 Unify IReporterRegistry and ReporterRegistry
To keep the compilation firewall effect, the implementations
are hidden behind a PIMPL. In this case it is probably not
worth it, but we can inline it later if needed.
2023-03-22 23:58:44 +01:00
Martin Hořeňovský
4f4ad8ada9 Sprinkle some constexpr around 2023-03-22 19:12:32 +01:00
Martin Hořeňovský
5b665be643 Cut out catch_interfaces_capture.hpp include from the main include 2023-03-22 19:12:03 +01:00
239 changed files with 12382 additions and 4026 deletions

View File

@@ -8,3 +8,4 @@ build:vs2022 --cxxopt=/std:c++17
build:windows --config=vs2022
build:linux --config=gcc11
build:macos --cxxopt=-std=c++2b

View File

@@ -11,7 +11,7 @@ jobs:
compilation_mode: [fastbuild, dbg, opt]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Mount bazel cache
uses: actions/cache@v3

View File

@@ -18,10 +18,12 @@ jobs:
other_pkgs: clang-11
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Prepare environment
run: sudo apt-get install -y meson ninja-build ${{matrix.other_pkgs}}
run: |
sudo apt-get update
sudo apt-get install -y meson ninja-build ${{matrix.other_pkgs}}
- name: Configure build
env:

View File

@@ -29,13 +29,13 @@ jobs:
build_type: Debug
std: 14
other_pkgs: g++-7
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON -DCATCH_ENABLE_CMAKE_HELPER_TESTS=ON
- cxx: g++-7
build_description: Extras + Examples
build_type: Release
std: 14
other_pkgs: g++-7
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON -DCATCH_ENABLE_CMAKE_HELPER_TESTS=ON
# Extras and examples with Clang-10
- cxx: clang++-10
@@ -43,13 +43,13 @@ jobs:
build_type: Debug
std: 17
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON -DCATCH_ENABLE_CMAKE_HELPER_TESTS=ON
- cxx: clang++-10
build_description: Extras + Examples
build_type: Release
std: 17
other_pkgs: clang-10
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON
cmake_configurations: -DCATCH_BUILD_EXTRA_TESTS=ON -DCATCH_BUILD_EXAMPLES=ON -DCATCH_ENABLE_CMAKE_HELPER_TESTS=ON
# Configure tests with Clang-10
- cxx: clang++-10
@@ -70,10 +70,12 @@ jobs:
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Prepare environment
run: sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
run: |
sudo apt-get update
sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
- name: Configure build
working-directory: ${{runner.workspace}}

View File

@@ -83,7 +83,7 @@ jobs:
other_pkgs: g++-10
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Add repositories for older GCC
run: |
@@ -92,7 +92,9 @@ jobs:
if: ${{ matrix.cxx == 'g++-5' || matrix.cxx == 'g++-6' }}
- name: Prepare environment
run: sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
run: |
sudo apt-get update
sudo apt-get install -y ninja-build ${{matrix.other_pkgs}}
- name: Configure build
working-directory: ${{runner.workspace}}

View File

@@ -22,7 +22,7 @@ jobs:
extra_tests: ON
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Configure build
working-directory: ${{runner.workspace}}
@@ -42,11 +42,10 @@ jobs:
- name: Build tests + lib
working-directory: ${{runner.workspace}}/build
run: make -j 2
run: make -j `sysctl -n hw.ncpu`
- name: Run tests
env:
CTEST_OUTPUT_ON_FAILURE: 1
working-directory: ${{runner.workspace}}/build
# Hardcode 2 cores we know are there
run: ctest -C ${{matrix.build_type}} -j 2
run: ctest -C ${{matrix.build_type}} -j `sysctl -n hw.ncpu`

View File

@@ -9,7 +9,7 @@ jobs:
steps:
- name: Checkout source code
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Setup Dependencies
uses: actions/setup-python@v2

View File

@@ -13,7 +13,7 @@ jobs:
build_type: [Debug, Release]
std: [14, 17]
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
- name: Configure build
working-directory: ${{runner.workspace}}

View File

@@ -43,12 +43,15 @@ expand_template(
"#cmakedefine CATCH_CONFIG_NO_GLOBAL_NEXTAFTER": "",
"#cmakedefine CATCH_CONFIG_NO_POSIX_SIGNALS": "",
"#cmakedefine CATCH_CONFIG_NO_USE_ASYNC": "",
"#cmakedefine CATCH_CONFIG_NO_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT": "",
"#cmakedefine CATCH_CONFIG_NO_WCHAR": "",
"#cmakedefine CATCH_CONFIG_NO_WINDOWS_SEH": "",
"#cmakedefine CATCH_CONFIG_NOSTDOUT": "",
"#cmakedefine CATCH_CONFIG_POSIX_SIGNALS": "",
"#cmakedefine CATCH_CONFIG_PREFIX_ALL": "",
"#cmakedefine CATCH_CONFIG_PREFIX_MESSAGES": "",
"#cmakedefine CATCH_CONFIG_SHARED_LIBRARY": "",
"#cmakedefine CATCH_CONFIG_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT": "",
"#cmakedefine CATCH_CONFIG_USE_ASYNC": "",
"#cmakedefine CATCH_CONFIG_WCHAR": "",
"#cmakedefine CATCH_CONFIG_WINDOWS_CRTDBG": "",

View File

@@ -18,10 +18,12 @@
macro(AddOverridableConfigOption OptionBaseName)
option(CATCH_CONFIG_${OptionBaseName} "Read docs/configuration.md for details" OFF)
option(CATCH_CONFIG_NO_${OptionBaseName} "Read docs/configuration.md for details" OFF)
mark_as_advanced(CATCH_CONFIG_${OptionBaseName} CATCH_CONFIG_NO_${OptionBaseName})
endmacro()
macro(AddConfigOption OptionBaseName)
option(CATCH_CONFIG_${OptionBaseName} "Read docs/configuration.md for details" OFF)
mark_as_advanced(CATCH_CONFIG_${OptionBaseName})
endmacro()
set(_OverridableOptions
@@ -41,6 +43,7 @@ set(_OverridableOptions
"WCHAR"
"WINDOWS_SEH"
"GETENV"
"EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT"
)
foreach(OptionName ${_OverridableOptions})
@@ -61,6 +64,7 @@ set(_OtherConfigOptions
"FAST_COMPILE"
"NOSTDOUT"
"PREFIX_ALL"
"PREFIX_MESSAGES"
"WINDOWS_CRTDBG"
)
@@ -68,11 +72,17 @@ set(_OtherConfigOptions
foreach(OptionName ${_OtherConfigOptions})
AddConfigOption(${OptionName})
endforeach()
set(CATCH_CONFIG_SHARED_LIBRARY ${BUILD_SHARED_LIBS})
if(DEFINED BUILD_SHARED_LIBS)
set(CATCH_CONFIG_SHARED_LIBRARY ${BUILD_SHARED_LIBS})
else()
set(CATCH_CONFIG_SHARED_LIBRARY "")
endif()
set(CATCH_CONFIG_DEFAULT_REPORTER "console" CACHE STRING "Read docs/configuration.md for details. The name of the reporter should be without quotes.")
set(CATCH_CONFIG_CONSOLE_WIDTH "80" CACHE STRING "Read docs/configuration.md for details. Must form a valid integer literal.")
mark_as_advanced(CATCH_CONFIG_SHARED_LIBRARY CATCH_CONFIG_DEFAULT_REPORTER CATCH_CONFIG_CONSOLE_WIDTH)
# There is no good way to both turn this into a CMake cache variable,
# and keep reasonable default semantics inside the project. Thus we do
# not define it and users have to provide it as an outside variable.

View File

@@ -46,7 +46,6 @@ function(add_warnings_to_targets targets)
set(CHECKED_WARNING_FLAGS
"-Wabsolute-value"
"-Wall"
"-Wc++20-compat"
"-Wcall-to-pure-virtual-from-ctor-dtor"
"-Wcast-align"
"-Wcatch-value"
@@ -74,6 +73,7 @@ function(add_warnings_to_targets targets)
"-Woverloaded-virtual"
"-Wparentheses"
"-Wpedantic"
"-Wredundant-decls"
"-Wreorder"
"-Wreturn-std-move"
"-Wshadow"
@@ -83,7 +83,7 @@ function(add_warnings_to_targets targets)
"-Wundef"
"-Wuninitialized"
"-Wunneeded-internal-declaration"
"-Wunreachable-code"
"-Wunreachable-code-aggressive"
"-Wunused"
"-Wunused-function"
"-Wunused-parameter"

View File

@@ -11,6 +11,7 @@ endif()
option(CATCH_INSTALL_DOCS "Install documentation alongside library" ON)
option(CATCH_INSTALL_EXTRAS "Install extras (CMake scripts, debugger helpers) alongside library" ON)
option(CATCH_DEVELOPMENT_BUILD "Build tests, enable warnings, enable Werror, etc" OFF)
option(CATCH_ENABLE_REPRODUCIBLE_BUILD "Add compiler flags for improving build reproducibility" ON)
include(CMakeDependentOption)
cmake_dependent_option(CATCH_BUILD_TESTING "Build the SelfTest project" ON "CATCH_DEVELOPMENT_BUILD" OFF)
@@ -21,6 +22,7 @@ cmake_dependent_option(CATCH_ENABLE_COVERAGE "Generate coverage for codecov.io"
cmake_dependent_option(CATCH_ENABLE_WERROR "Enables Werror during build" ON "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_BUILD_SURROGATES "Enable generating and building surrogate TUs for the main headers" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_CONFIGURE_TESTS "Enable CMake configuration tests. WARNING: VERY EXPENSIVE" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
cmake_dependent_option(CATCH_ENABLE_CMAKE_HELPER_TESTS "Enable CMake helper tests. WARNING: VERY EXPENSIVE" OFF "CATCH_DEVELOPMENT_BUILD" OFF)
# Catch2's build breaks if done in-tree. You probably should not build
@@ -31,7 +33,7 @@ if (CMAKE_BINARY_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
endif()
project(Catch2
VERSION 3.3.2 # CML version placeholder, don't delete
VERSION 3.5.0 # CML version placeholder, don't delete
LANGUAGES CXX
# HOMEPAGE_URL is not supported until CMake version 3.12, which
# we do not target yet.

View File

@@ -18,7 +18,8 @@
"CATCH_BUILD_EXAMPLES": "ON",
"CATCH_BUILD_EXTRA_TESTS": "ON",
"CATCH_BUILD_SURROGATES": "ON",
"CATCH_ENABLE_CONFIGURE_TESTS": "ON"
"CATCH_ENABLE_CONFIGURE_TESTS": "ON",
"CATCH_ENABLE_CMAKE_HELPER_TESTS": "ON"
}
}
]

306
Doxyfile
View File

@@ -1,4 +1,4 @@
# Doxyfile 1.8.16
# Doxyfile 1.9.1
# This file describes the settings to be used by the documentation system
# doxygen (www.doxygen.org) for a project.
@@ -32,7 +32,7 @@ DOXYFILE_ENCODING = UTF-8
# title of most generated pages and in a few other places.
# The default value is: My Project.
PROJECT_NAME = "Catch2"
PROJECT_NAME = Catch2
# The PROJECT_NUMBER tag can be used to enter a project or revision number. This
# could be handy for archiving the generated documentation or if some version
@@ -51,6 +51,7 @@ PROJECT_BRIEF = "Popular C++ unit testing framework"
# pixels and the maximum width should not exceed 200 pixels. Doxygen will copy
# the logo to the output directory.
PROJECT_LOGO =
# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) path
# into which the generated documentation will be written. If a relative path is
@@ -216,6 +217,14 @@ QT_AUTOBRIEF = YES
MULTILINE_CPP_IS_BRIEF = NO
# By default Python docstrings are displayed as preformatted text and doxygen's
# special commands cannot be used. By setting PYTHON_DOCSTRING to NO the
# doxygen's special commands can be used and the contents of the docstring
# documentation blocks is shown as doxygen documentation.
# The default value is: YES.
PYTHON_DOCSTRING = YES
# If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the
# documentation from any documented member that it re-implements.
# The default value is: YES.
@@ -251,13 +260,7 @@ TAB_SIZE = 4
# a double escape (\\{ and \\})
ALIASES = "complexity=@par Complexity:" \
"noexcept=**Noexcept**"
# This tag can be used to specify a number of word-keyword mappings (TCL only).
# A mapping has the form "name=value". For example adding "class=itcl::class"
# will allow you to use the command class in the itcl::class meaning.
TCL_SUBST =
noexcept=**Noexcept**
# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources
# only. Doxygen will then generate output that is more tailored for C. For
@@ -299,19 +302,22 @@ OPTIMIZE_OUTPUT_SLICE = NO
# parses. With this tag you can assign which parser to use for a given
# extension. Doxygen has a built-in mapping, but you can override or extend it
# using this tag. The format is ext=language, where ext is a file extension, and
# language is one of the parsers supported by doxygen: IDL, Java, Javascript,
# Csharp (C#), C, C++, D, PHP, md (Markdown), Objective-C, Python, Slice,
# language is one of the parsers supported by doxygen: IDL, Java, JavaScript,
# Csharp (C#), C, C++, D, PHP, md (Markdown), Objective-C, Python, Slice, VHDL,
# Fortran (fixed format Fortran: FortranFixed, free formatted Fortran:
# FortranFree, unknown formatted Fortran: Fortran. In the later case the parser
# tries to guess whether the code is fixed or free formatted code, this is the
# default for Fortran type files), VHDL, tcl. For instance to make doxygen treat
# .inc files as Fortran files (default is PHP), and .f files as C (default is
# Fortran), use: inc=Fortran f=C.
# default for Fortran type files). For instance to make doxygen treat .inc files
# as Fortran files (default is PHP), and .f files as C (default is Fortran),
# use: inc=Fortran f=C.
#
# Note: For files without extension you can use no_extension as a placeholder.
#
# Note that for custom extensions you also need to set FILE_PATTERNS otherwise
# the files are not read by doxygen.
# the files are not read by doxygen. When specifying no_extension you should add
# * to the FILE_PATTERNS.
#
# Note see also the list of default file extension mappings.
EXTENSION_MAPPING =
@@ -445,6 +451,19 @@ TYPEDEF_HIDES_STRUCT = NO
LOOKUP_CACHE_SIZE = 0
# The NUM_PROC_THREADS specifies the number threads doxygen is allowed to use
# during processing. When set to 0 doxygen will based this on the number of
# cores available in the system. You can set it explicitly to a value larger
# than 0 to get more control over the balance between CPU load and processing
# speed. At this moment only the input processing can be done using multiple
# threads. Since this is still an experimental feature the default is set to 1,
# which efficively disables parallel processing. Please report any issues you
# encounter. Generating dot graphs in parallel is controlled by the
# DOT_NUM_THREADS setting.
# Minimum value: 0, maximum value: 32, default value: 1.
NUM_PROC_THREADS = 1
#---------------------------------------------------------------------------
# Build related configuration options
#---------------------------------------------------------------------------
@@ -508,6 +527,13 @@ EXTRACT_LOCAL_METHODS = NO
EXTRACT_ANON_NSPACES = NO
# If this flag is set to YES, the name of an unnamed parameter in a declaration
# will be determined by the corresponding definition. By default unnamed
# parameters remain unnamed in the output.
# The default value is: YES.
RESOLVE_UNNAMED_PARAMS = YES
# If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all
# undocumented members inside documented classes or files. If set to NO these
# members will be included in the various overviews, but no documentation
@@ -525,8 +551,8 @@ HIDE_UNDOC_MEMBERS = NO
HIDE_UNDOC_CLASSES = NO
# If the HIDE_FRIEND_COMPOUNDS tag is set to YES, doxygen will hide all friend
# (class|struct|union) declarations. If set to NO, these declarations will be
# included in the documentation.
# declarations. If set to NO, these declarations will be included in the
# documentation.
# The default value is: NO.
HIDE_FRIEND_COMPOUNDS = NO
@@ -545,11 +571,18 @@ HIDE_IN_BODY_DOCS = NO
INTERNAL_DOCS = NO
# If the CASE_SENSE_NAMES tag is set to NO then doxygen will only generate file
# names in lower-case letters. If set to YES, upper-case letters are also
# allowed. This is useful if you have classes or files whose names only differ
# in case and if your file system supports case sensitive file names. Windows
# (including Cygwin) ands Mac users are advised to set this option to NO.
# With the correct setting of option CASE_SENSE_NAMES doxygen will better be
# able to match the capabilities of the underlying filesystem. In case the
# filesystem is case sensitive (i.e. it supports files in the same directory
# whose names only differ in casing), the option must be set to YES to properly
# deal with such files in case they appear in the input. For filesystems that
# are not case sensitive the option should be be set to NO to properly deal with
# output files written for symbols that only differ in casing, such as for two
# classes, one named CLASS and the other named Class, and to also support
# references to files without having to specify the exact matching casing. On
# Windows (including Cygwin) and MacOS, users should typically set this option
# to NO, whereas on Linux or other Unix flavors it should typically be set to
# YES.
# The default value is: system dependent.
CASE_SENSE_NAMES = NO
@@ -788,7 +821,10 @@ WARN_IF_DOC_ERROR = YES
WARN_NO_PARAMDOC = YES
# If the WARN_AS_ERROR tag is set to YES then doxygen will immediately stop when
# a warning is encountered.
# a warning is encountered. If the WARN_AS_ERROR tag is set to FAIL_ON_WARNINGS
# then doxygen will continue running as if WARN_AS_ERROR tag is set to NO, but
# at the end of the doxygen process doxygen will return with a non-zero status.
# Possible values are: NO, YES and FAIL_ON_WARNINGS.
# The default value is: NO.
WARN_AS_ERROR = NO
@@ -819,13 +855,13 @@ WARN_LOGFILE = doxygen.errors
# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING
# Note: If this tag is empty the current directory is searched.
INPUT = "src/catch2"
INPUT = src/catch2
# This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
# libiconv (or the iconv built into libc) for the transcoding. See the libiconv
# documentation (see: https://www.gnu.org/software/libiconv/) for the list of
# possible encodings.
# documentation (see:
# https://www.gnu.org/software/libiconv/) for the list of possible encodings.
# The default value is: UTF-8.
INPUT_ENCODING = UTF-8
@@ -838,13 +874,61 @@ INPUT_ENCODING = UTF-8
# need to set EXTENSION_MAPPING for the extension otherwise the files are not
# read by doxygen.
#
# Note the list of default checked file patterns might differ from the list of
# default file extension mappings.
#
# If left blank the following patterns are tested:*.c, *.cc, *.cxx, *.cpp,
# *.c++, *.java, *.ii, *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h,
# *.hh, *.hxx, *.hpp, *.h++, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, *.inc,
# *.m, *.markdown, *.md, *.mm, *.dox, *.py, *.pyw, *.f90, *.f95, *.f03, *.f08,
# *.f, *.for, *.tcl, *.vhd, *.vhdl, *.ucf, *.qsf and *.ice.
# *.m, *.markdown, *.md, *.mm, *.dox (to be provided as doxygen C comment),
# *.py, *.pyw, *.f90, *.f95, *.f03, *.f08, *.f18, *.f, *.for, *.vhd, *.vhdl,
# *.ucf, *.qsf and *.ice.
# FILE_PATTERNS =
FILE_PATTERNS = *.c \
*.cc \
*.cxx \
*.cpp \
*.c++ \
*.java \
*.ii \
*.ixx \
*.ipp \
*.i++ \
*.inl \
*.idl \
*.ddl \
*.odl \
*.h \
*.hh \
*.hxx \
*.hpp \
*.h++ \
*.cs \
*.d \
*.php \
*.php4 \
*.php5 \
*.phtml \
*.inc \
*.m \
*.markdown \
*.md \
*.mm \
*.dox \
*.py \
*.pyw \
*.f90 \
*.f95 \
*.f03 \
*.f08 \
*.f18 \
*.f \
*.for \
*.vhd \
*.vhdl \
*.ucf \
*.qsf \
*.ice
# The RECURSIVE tag can be used to specify whether or not subdirectories should
# be searched for input files as well.
@@ -968,6 +1052,7 @@ FILTER_SOURCE_PATTERNS =
# (index.html). This can be useful if you have a project on for instance GitHub
# and want to reuse the introduction page also for the doxygen output.
USE_MDFILE_AS_MAINPAGE =
#---------------------------------------------------------------------------
# Configuration options related to source browsing
@@ -1055,6 +1140,44 @@ USE_HTAGS = NO
VERBATIM_HEADERS = YES
# If the CLANG_ASSISTED_PARSING tag is set to YES then doxygen will use the
# clang parser (see:
# http://clang.llvm.org/) for more accurate parsing at the cost of reduced
# performance. This can be particularly helpful with template rich C++ code for
# which doxygen's built-in parser lacks the necessary type information.
# Note: The availability of this option depends on whether or not doxygen was
# generated with the -Duse_libclang=ON option for CMake.
# The default value is: NO.
CLANG_ASSISTED_PARSING = NO
# If clang assisted parsing is enabled and the CLANG_ADD_INC_PATHS tag is set to
# YES then doxygen will add the directory of each input to the include path.
# The default value is: YES.
CLANG_ADD_INC_PATHS = YES
# If clang assisted parsing is enabled you can provide the compiler with command
# line options that you would normally use when invoking the compiler. Note that
# the include paths will already be set by doxygen for the files and directories
# specified with INPUT and INCLUDE_PATH.
# This tag requires that the tag CLANG_ASSISTED_PARSING is set to YES.
CLANG_OPTIONS =
# If clang assisted parsing is enabled you can provide the clang parser with the
# path to the directory containing a file called compile_commands.json. This
# file is the compilation database (see:
# http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html) containing the
# options used when the source files were built. This is equivalent to
# specifying the -p option to a clang tool, such as clang-check. These options
# will then be passed to the parser. Any options specified with CLANG_OPTIONS
# will be added as well.
# Note: The availability of this option depends on whether or not doxygen was
# generated with the -Duse_libclang=ON option for CMake.
CLANG_DATABASE_PATH =
#---------------------------------------------------------------------------
# Configuration options related to the alphabetical class index
#---------------------------------------------------------------------------
@@ -1066,13 +1189,6 @@ VERBATIM_HEADERS = YES
ALPHABETICAL_INDEX = YES
# The COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns in
# which the alphabetical index list will be split.
# Minimum value: 1, maximum value: 20, default value: 5.
# This tag requires that the tag ALPHABETICAL_INDEX is set to YES.
COLS_IN_ALPHA_INDEX = 5
# In case all classes in a project start with a common prefix, all classes will
# be put under the same header in the alphabetical index. The IGNORE_PREFIX tag
# can be used to specify a prefix (or a list of prefixes) that should be ignored
@@ -1211,9 +1327,9 @@ HTML_TIMESTAMP = NO
# If the HTML_DYNAMIC_MENUS tag is set to YES then the generated HTML
# documentation will contain a main index with vertical navigation menus that
# are dynamically created via Javascript. If disabled, the navigation index will
# are dynamically created via JavaScript. If disabled, the navigation index will
# consists of multiple levels of tabs that are statically embedded in every HTML
# page. Disable this option to support browsers that do not have Javascript,
# page. Disable this option to support browsers that do not have JavaScript,
# like the Qt help browser.
# The default value is: YES.
# This tag requires that the tag GENERATE_HTML is set to YES.
@@ -1243,10 +1359,11 @@ HTML_INDEX_NUM_ENTRIES = 100
# If the GENERATE_DOCSET tag is set to YES, additional index files will be
# generated that can be used as input for Apple's Xcode 3 integrated development
# environment (see: https://developer.apple.com/xcode/), introduced with OSX
# 10.5 (Leopard). To create a documentation set, doxygen will generate a
# Makefile in the HTML output directory. Running make will produce the docset in
# that directory and running make install will install the docset in
# environment (see:
# https://developer.apple.com/xcode/), introduced with OSX 10.5 (Leopard). To
# create a documentation set, doxygen will generate a Makefile in the HTML
# output directory. Running make will produce the docset in that directory and
# running make install will install the docset in
# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find it at
# startup. See https://developer.apple.com/library/archive/featuredarticles/Doxy
# genXcode/_index.html for more information.
@@ -1288,8 +1405,8 @@ DOCSET_PUBLISHER_NAME = Publisher
# If the GENERATE_HTMLHELP tag is set to YES then doxygen generates three
# additional HTML index files: index.hhp, index.hhc, and index.hhk. The
# index.hhp is a project file that can be read by Microsoft's HTML Help Workshop
# (see: https://www.microsoft.com/en-us/download/details.aspx?id=21138) on
# Windows.
# (see:
# https://www.microsoft.com/en-us/download/details.aspx?id=21138) on Windows.
#
# The HTML Help Workshop contains a compiler that can convert all HTML output
# generated by doxygen into a single compiled HTML file (.chm). Compiled HTML
@@ -1364,7 +1481,8 @@ QCH_FILE =
# The QHP_NAMESPACE tag specifies the namespace to use when generating Qt Help
# Project output. For more information please see Qt Help Project / Namespace
# (see: https://doc.qt.io/archives/qt-4.8/qthelpproject.html#namespace).
# (see:
# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#namespace).
# The default value is: org.doxygen.Project.
# This tag requires that the tag GENERATE_QHP is set to YES.
@@ -1372,8 +1490,8 @@ QHP_NAMESPACE = org.doxygen.Project
# The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating Qt
# Help Project output. For more information please see Qt Help Project / Virtual
# Folders (see: https://doc.qt.io/archives/qt-4.8/qthelpproject.html#virtual-
# folders).
# Folders (see:
# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#virtual-folders).
# The default value is: doc.
# This tag requires that the tag GENERATE_QHP is set to YES.
@@ -1381,16 +1499,16 @@ QHP_VIRTUAL_FOLDER = doc
# If the QHP_CUST_FILTER_NAME tag is set, it specifies the name of a custom
# filter to add. For more information please see Qt Help Project / Custom
# Filters (see: https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-
# filters).
# Filters (see:
# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-filters).
# This tag requires that the tag GENERATE_QHP is set to YES.
QHP_CUST_FILTER_NAME =
# The QHP_CUST_FILTER_ATTRS tag specifies the list of the attributes of the
# custom filter to add. For more information please see Qt Help Project / Custom
# Filters (see: https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-
# filters).
# Filters (see:
# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-filters).
# This tag requires that the tag GENERATE_QHP is set to YES.
QHP_CUST_FILTER_ATTRS =
@@ -1402,9 +1520,9 @@ QHP_CUST_FILTER_ATTRS =
QHP_SECT_FILTER_ATTRS =
# The QHG_LOCATION tag can be used to specify the location of Qt's
# qhelpgenerator. If non-empty doxygen will try to run qhelpgenerator on the
# generated .qhp file.
# The QHG_LOCATION tag can be used to specify the location (absolute path
# including file name) of Qt's qhelpgenerator. If non-empty doxygen will try to
# run qhelpgenerator on the generated .qhp file.
# This tag requires that the tag GENERATE_QHP is set to YES.
QHG_LOCATION =
@@ -1481,6 +1599,17 @@ TREEVIEW_WIDTH = 250
EXT_LINKS_IN_WINDOW = NO
# If the HTML_FORMULA_FORMAT option is set to svg, doxygen will use the pdf2svg
# tool (see https://github.com/dawbarton/pdf2svg) or inkscape (see
# https://inkscape.org) to generate formulas as SVG images instead of PNGs for
# the HTML output. These images will generally look nicer at scaled resolutions.
# Possible values are: png (the default) and svg (looks nicer but requires the
# pdf2svg or inkscape tool).
# The default value is: png.
# This tag requires that the tag GENERATE_HTML is set to YES.
HTML_FORMULA_FORMAT = png
# Use this tag to change the font size of LaTeX formulas included as images in
# the HTML documentation. When you change the font size after a successful
# doxygen run you need to manually remove any form_*.png images from the HTML
@@ -1501,8 +1630,14 @@ FORMULA_FONTSIZE = 10
FORMULA_TRANSPARENT = YES
# The FORMULA_MACROFILE can contain LaTeX \newcommand and \renewcommand commands
# to create new LaTeX commands to be used in formulas as building blocks. See
# the section "Including formulas" for details.
FORMULA_MACROFILE =
# Enable the USE_MATHJAX option to render LaTeX formulas using MathJax (see
# https://www.mathjax.org) which uses client side Javascript for the rendering
# https://www.mathjax.org) which uses client side JavaScript for the rendering
# instead of using pre-rendered bitmaps. Use this if you do not have LaTeX
# installed or if you want to formulas look prettier in the HTML output. When
# enabled you may also need to install MathJax separately and configure the path
@@ -1514,7 +1649,7 @@ USE_MATHJAX = YES
# When MathJax is enabled you can set the default output format to be used for
# the MathJax output. See the MathJax site (see:
# http://docs.mathjax.org/en/latest/output.html) for more details.
# http://docs.mathjax.org/en/v2.7-latest/output.html) for more details.
# Possible values are: HTML-CSS (which is slower, but has the best
# compatibility), NativeMML (i.e. MathML) and SVG.
# The default value is: HTML-CSS.
@@ -1530,7 +1665,7 @@ MATHJAX_FORMAT = HTML-CSS
# Content Delivery Network so you can quickly see the result without installing
# MathJax. However, it is strongly recommended to install a local copy of
# MathJax from https://www.mathjax.org before deployment.
# The default value is: https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/.
# The default value is: https://cdn.jsdelivr.net/npm/mathjax@2.
# This tag requires that the tag USE_MATHJAX is set to YES.
MATHJAX_RELPATH = http://cdn.mathjax.org/mathjax/latest
@@ -1545,7 +1680,8 @@ MATHJAX_EXTENSIONS = TeX/AMSmath \
# The MATHJAX_CODEFILE tag can be used to specify a file with javascript pieces
# of code that will be used on startup of the MathJax code. See the MathJax site
# (see: http://docs.mathjax.org/en/latest/output.html) for more details. For an
# (see:
# http://docs.mathjax.org/en/v2.7-latest/output.html) for more details. For an
# example see the documentation.
# This tag requires that the tag USE_MATHJAX is set to YES.
@@ -1573,7 +1709,7 @@ MATHJAX_CODEFILE =
SEARCHENGINE = YES
# When the SERVER_BASED_SEARCH tag is enabled the search engine will be
# implemented using a web server instead of a web client using Javascript. There
# implemented using a web server instead of a web client using JavaScript. There
# are two flavors of web server based searching depending on the EXTERNAL_SEARCH
# setting. When disabled, doxygen will generate a PHP script for searching and
# an index file used by the script. When EXTERNAL_SEARCH is enabled the indexing
@@ -1592,7 +1728,8 @@ SERVER_BASED_SEARCH = NO
#
# Doxygen ships with an example indexer (doxyindexer) and search engine
# (doxysearch.cgi) which are based on the open source search engine library
# Xapian (see: https://xapian.org/).
# Xapian (see:
# https://xapian.org/).
#
# See the section "External Indexing and Searching" for details.
# The default value is: NO.
@@ -1605,8 +1742,9 @@ EXTERNAL_SEARCH = NO
#
# Doxygen ships with an example indexer (doxyindexer) and search engine
# (doxysearch.cgi) which are based on the open source search engine library
# Xapian (see: https://xapian.org/). See the section "External Indexing and
# Searching" for details.
# Xapian (see:
# https://xapian.org/). See the section "External Indexing and Searching" for
# details.
# This tag requires that the tag SEARCHENGINE is set to YES.
SEARCHENGINE_URL =
@@ -1770,9 +1908,11 @@ LATEX_EXTRA_FILES =
PDF_HYPERLINKS = YES
# If the USE_PDFLATEX tag is set to YES, doxygen will use pdflatex to generate
# the PDF file directly from the LaTeX files. Set this option to YES, to get a
# higher quality PDF documentation.
# If the USE_PDFLATEX tag is set to YES, doxygen will use the engine as
# specified with LATEX_CMD_NAME to generate the PDF file directly from the LaTeX
# files. Set this option to YES, to get a higher quality PDF documentation.
#
# See also section LATEX_CMD_NAME for selecting the engine.
# The default value is: YES.
# This tag requires that the tag GENERATE_LATEX is set to YES.
@@ -2204,7 +2344,7 @@ HIDE_UNDOC_RELATIONS = YES
# http://www.graphviz.org/), a graph visualization toolkit from AT&T and Lucent
# Bell Labs. The other options in this section have no effect if this option is
# set to NO
# The default value is: NO.
# The default value is: YES.
HAVE_DOT = YES
@@ -2283,10 +2423,32 @@ UML_LOOK = NO
# but if the number exceeds 15, the total amount of fields shown is limited to
# 10.
# Minimum value: 0, maximum value: 100, default value: 10.
# This tag requires that the tag HAVE_DOT is set to YES.
# This tag requires that the tag UML_LOOK is set to YES.
UML_LIMIT_NUM_FIELDS = 10
# If the DOT_UML_DETAILS tag is set to NO, doxygen will show attributes and
# methods without types and arguments in the UML graphs. If the DOT_UML_DETAILS
# tag is set to YES, doxygen will add type and arguments for attributes and
# methods in the UML graphs. If the DOT_UML_DETAILS tag is set to NONE, doxygen
# will not generate fields with class member information in the UML graphs. The
# class diagrams will look similar to the default class diagrams but using UML
# notation for the relationships.
# Possible values are: NO, YES and NONE.
# The default value is: NO.
# This tag requires that the tag UML_LOOK is set to YES.
DOT_UML_DETAILS = NO
# The DOT_WRAP_THRESHOLD tag can be used to set the maximum number of characters
# to display on a single line. If the actual line length exceeds this threshold
# significantly it will wrapped across multiple lines. Some heuristics are apply
# to avoid ugly line breaks.
# Minimum value: 0, maximum value: 1000, default value: 17.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_WRAP_THRESHOLD = 17
# If the TEMPLATE_RELATIONS tag is set to YES then the inheritance and
# collaboration graphs will show the relations between templates and their
# instances.
@@ -2360,7 +2522,9 @@ DIRECTORY_GRAPH = NO
# Note: If you choose svg you need to set HTML_FILE_EXTENSION to xhtml in order
# to make the SVG files visible in IE 9+ (other browsers do not have this
# requirement).
# Possible values are: png, jpg, gif, svg, png:gd, png:gd:gd, png:cairo,
# Possible values are: png, png:cairo, png:cairo:cairo, png:cairo:gd, png:gd,
# png:gd:gd, jpg, jpg:cairo, jpg:cairo:gd, jpg:gd, jpg:gd:gd, gif, gif:cairo,
# gif:cairo:gd, gif:gd, gif:gd:gd, svg, png:gd, png:gd:gd, png:cairo,
# png:cairo:gd, png:cairo:cairo, png:cairo:gdiplus, png:gdiplus and
# png:gdiplus:gdiplus.
# The default value is: png.
@@ -2476,9 +2640,11 @@ DOT_MULTI_TARGETS = YES
GENERATE_LEGEND = YES
# If the DOT_CLEANUP tag is set to YES, doxygen will remove the intermediate dot
# If the DOT_CLEANUP tag is set to YES, doxygen will remove the intermediate
# files that are used to generate the various graphs.
#
# Note: This setting is not only used for dot files but also for msc and
# plantuml temporary files.
# The default value is: YES.
# This tag requires that the tag HAVE_DOT is set to YES.
DOT_CLEANUP = YES

3
MODULE.bazel Normal file
View File

@@ -0,0 +1,3 @@
module(name = "catch2")
bazel_dep(name = "bazel_skylib", version = "1.5.0")

View File

@@ -4,10 +4,10 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "bazel_skylib",
sha256 = "b8a1527901774180afc798aeb28c4634bdccf19c4d98e7bdd1ce79d1fe9aaad7",
sha256 = "cd55a062e763b9349921f0f5db8c3933288dc8ba4f76dd9416aac68acee3cb94",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.4.1/bazel-skylib-1.4.1.tar.gz",
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.4.1/bazel-skylib-1.4.1.tar.gz",
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.5.0/bazel-skylib-1.5.0.tar.gz",
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.5.0/bazel-skylib-1.5.0.tar.gz",
],
)

View File

@@ -70,14 +70,3 @@ environment:
additional_flags: "/permissive- /std:c++latest"
platform: x64
configuration: Debug
- FLAVOR: VS 2017 x64 Debug
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
platform: x64
configuration: Debug
- FLAVOR: VS 2017 x64 Release Coverage
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
coverage: 1
platform: x64
configuration: Debug

View File

@@ -93,7 +93,7 @@ Fibonacci
-------------------------------------------------------------------------------
C:\path\to\Catch2\Benchmark.tests.cpp(10)
...............................................................................
benchmark name samples iterations estimated
benchmark name samples iterations est run time
mean low mean high mean
std dev low std dev high std dev
-------------------------------------------------------------------------------

View File

@@ -82,7 +82,7 @@ variable set to "1".
### CodeCoverage module (GCOV, LCOV...)
If you are using GCOV tool to get testing coverage of your code, and are not sure how to integrate it with CMake and Catch, there should be an external example over at https://github.com/fkromer/catch_cmake_coverage
If you are using GCOV tool to get testing coverage of your code, and are not sure how to integrate it with CMake and Catch, there should be an external example over at https://github.com/claremacrae/catch_cmake_coverage
### pkg-config

View File

@@ -51,7 +51,7 @@ Include(FetchContent)
FetchContent_Declare(
Catch2
GIT_REPOSITORY https://github.com/catchorg/Catch2.git
GIT_TAG v3.0.1 # or a later release
GIT_TAG v3.4.0 # or a later release
)
FetchContent_MakeAvailable(Catch2)
@@ -126,6 +126,7 @@ catch_discover_tests(target
[OUTPUT_DIR dir]
[OUTPUT_PREFIX prefix]
[OUTPUT_SUFFIX suffix]
[DISCOVERY_MODE <POST_BUILD|PRE_TEST>]
)
```
@@ -198,6 +199,16 @@ If specified, `suffix` is added to each output file name, like so
`--out dir/<test_name>suffix`. This can be used to add a file extension to
the output file name e.g. ".xml".
* `DISCOVERY_MODE mode`
If specified allows control over when test discovery is performed.
For a value of `POST_BUILD` (default) test discovery is performed at build time.
For a value of `PRE_TEST` test discovery is delayed until just prior to test
execution (useful e.g. in cross-compilation environments).
``DISCOVERY_MODE`` defaults to the value of the
``CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE`` variable if it is not passed when
calling ``catch_discover_tests``. This provides a mechanism for globally
selecting a preferred test discovery behavior.
### `ParseAndAddCatchTests.cmake`

View File

@@ -85,43 +85,102 @@ Click one of the following links to take you straight to that option - or scroll
<pre>&lt;test-spec> ...</pre>
Test cases, wildcarded test cases, tags and tag expressions are all passed directly as arguments. Tags are distinguished by being enclosed in square brackets.
By providing a test spec, you filter which tests will be run. If you call
Catch2 without any test spec, then it will run all non-hidden test
cases. A test case is hidden if it has the `[!benchmark]` tag, any tag
with a dot at the start, e.g. `[.]` or `[.foo]`.
If no test specs are supplied then all test cases, except "hidden" tests, are run.
A test is hidden by giving it any tag starting with (or just) a period (```.```) - or, in the deprecated case, tagged ```[hide]``` or given name starting with `'./'`. To specify hidden tests from the command line ```[.]``` or ```[hide]``` can be used *regardless of how they were declared*.
There are three basic test specs that can then be combined into more
complex specs:
Specs must be enclosed in quotes if they contain spaces. If they do not contain spaces the quotes are optional.
* Full test name, e.g. `"Test 1"`.
Wildcards consist of the `*` character at the beginning and/or end of test case names and can substitute for any number of any characters (including none).
This allows only test cases whose name is "Test 1".
Test specs are case insensitive.
* Wildcarded test name, e.g. `"*Test"`, or `"Test*"`, or `"*Test*"`.
If a spec is prefixed with `exclude:` or the `~` character then the pattern matches an exclusion. This means that tests matching the pattern are excluded from the set - even if a prior inclusion spec included them. Subsequent inclusion specs will take precedence, however.
Inclusions and exclusions are evaluated in left-to-right order.
This allows any test case whose name ends with, starts with, or contains
in the middle the string "Test". Note that the wildcard can only be at
the start or end.
Test case examples:
* Tag name, e.g. `[some-tag]`.
This allows any test case tagged with "[some-tag]". Remember that some
tags are special, e.g. those that start with "." or with "!".
You can also combine the basic test specs to create more complex test
specs. You can:
* Concatenate specs to apply all of them, e.g. `[some-tag][other-tag]`.
This allows test cases that are tagged with **both** "[some-tag]" **and**
"[other-tag]". A test case with just "[some-tag]" will not pass the filter,
nor will test case with just "[other-tag]".
* Comma-join specs to apply any of them, e.g. `[some-tag],[other-tag]`.
This allows test cases that are tagged with **either** "[some-tag]" **or**
"[other-tag]". A test case with both will obviously also pass the filter.
Note that commas take precendence over simple concatenation. This means
that `[a][b],[c]` accepts tests that are tagged with either both "[a]" and
"[b]", or tests that are tagged with just "[c]".
* Negate the spec by prepending it with `~`, e.g. `~[some-tag]`.
This rejects any test case that is tagged with "[some-tag]". Note that
rejection takes precedence over other filters.
Note that negations always binds to the following _basic_ test spec.
This means that `~[foo][bar]` negates only the "[foo]" tag and not the
"[bar]" tag.
Note that when Catch2 is deciding whether to include a test, first it
checks whether the test matches any negative filters. If it does,
the test is rejected. After that, the behaviour depends on whether there
are positive filters as well. If there are no positive filters, all
remaining non-hidden tests are included. If there are positive filters,
only tests that match the positive filters are included.
You can also match test names with special characters by escaping them
with a backslash (`"\"`), e.g. a test named `"Do A, then B"` is matched
by "Do A\, then B" test spec. Backslash also escapes itself.
### Examples
Given these TEST_CASEs,
```
thisTestOnly Matches the test case called, 'thisTestOnly'
"this test only" Matches the test case called, 'this test only'
these* Matches all cases starting with 'these'
exclude:notThis Matches all tests except, 'notThis'
~notThis Matches all tests except, 'notThis'
~*private* Matches all tests except those that contain 'private'
a* ~ab* abc Matches all tests that start with 'a', except those that
start with 'ab', except 'abc', which is included
~[tag1] Matches all tests except those tagged with '[tag1]'
-# [#somefile] Matches all tests from the file 'somefile.cpp'
TEST_CASE("Test 1") {}
TEST_CASE("Test 2", "[.foo]") {}
TEST_CASE("Test 3", "[.bar]") {}
TEST_CASE("Test 4", "[.][foo][bar]") {}
```
Names within square brackets are interpreted as tags.
A series of tags form an AND expression whereas a comma-separated sequence forms an OR expression. e.g.:
this is the result of these filters
```
./tests # Selects only the first test, others are hidden
./tests "Test 1" # Selects only the first test, other do not match
./tests ~"Test 1" # Selects no tests. Test 1 is rejected, other tests are hidden
./tests "Test *" # Selects all tests.
./tests [bar] # Selects tests 3 and 4. Other tests are not tagged [bar]
./tests ~[foo] # Selects test 1, because it is the only non-hidden test without [foo] tag
./tests [foo][bar] # Selects test 4.
./tests [foo],[bar] # Selects tests 2, 3, 4.
./tests ~[foo][bar] # Selects test 3. 2 and 4 are rejected due to having [foo] tag
./tests ~"Test 2"[foo] # Selects test 4, because test 2 is explicitly rejected
./tests [foo][bar],"Test 1" # Selects tests 1 and 4.
./tests "Test 1*" # Selects test 1, wildcard can match zero characters
```
<pre>[one][two],[three]</pre>
This matches all tests tagged `[one]` and `[two]`, as well as all tests tagged `[three]`
_Note: Using plain asterisk on a command line can cause issues with shell
expansion. Make sure that the asterisk is passed to Catch2 and is not
interpreted by the shell._
Test names containing special characters, such as `,` or `[` can specify them on the command line using `\`.
`\` also escapes itself.
<a id="choosing-a-reporter-to-use"></a>
## Choosing a reporter to use

View File

@@ -15,6 +15,7 @@
[Enabling stringification](#enabling-stringification)<br>
[Disabling exceptions](#disabling-exceptions)<br>
[Overriding Catch's debug break (`-b`)](#overriding-catchs-debug-break--b)<br>
[Static analysis support](#static-analysis-support)<br>
Catch2 is designed to "just work" as much as possible, and most of the
configuration options below are changed automatically during compilation,
@@ -25,7 +26,8 @@ with the same name.
## Prefixing Catch macros
CATCH_CONFIG_PREFIX_ALL
CATCH_CONFIG_PREFIX_ALL // Prefix all macros with CATCH_
CATCH_CONFIG_PREFIX_MESSAGES // Prefix only INFO, UNSCOPED_INFO, WARN and CAPTURE
To keep test code clean and uncluttered Catch uses short macro names (e.g. ```TEST_CASE``` and ```REQUIRE```). Occasionally these may conflict with identifiers from platform headers or the system under test. In this case the above identifier can be defined. This will cause all the Catch user macros to be prefixed with ```CATCH_``` (e.g. ```CATCH_TEST_CASE``` and ```CATCH_REQUIRE```).
@@ -264,6 +266,31 @@ The macro will be used as is, that is, `CATCH_BREAK_INTO_DEBUGGER();`
must compile and must break into debugger.
## Static analysis support
> Introduced in Catch2 3.4.0.
Some parts of Catch2, e.g. `SECTION`s, can be hard for static analysis
tools to reason about. Catch2 can change its internals to help static
analysis tools reason about the tests.
Catch2 automatically detects some static analysis tools (initial
implementation checks for clang-tidy and Coverity), but you can override
its detection (in either direction) via
```
CATCH_CONFIG_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT // force enables static analysis help
CATCH_CONFIG_NO_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT // force disables static analysis help
```
_As the name suggests, this is currently experimental, and thus we provide
no backwards compatibility guarantees._
**DO NOT ENABLE THIS FOR BUILDS YOU INTEND TO RUN.** The changed internals
are not meant to be runnable, only "scannable".
---
[Home](Readme.md#top)

View File

@@ -10,6 +10,7 @@
[Does Catch2 support running tests in parallel?](#does-catch2-support-running-tests-in-parallel)<br>
[Can I compile Catch2 into a dynamic library?](#can-i-compile-catch2-into-a-dynamic-library)<br>
[What repeatability guarantees does Catch2 provide?](#what-repeatability-guarantees-does-catch2-provide)<br>
[My build cannot find `catch2/catch_user_config.hpp`, how can I fix it?](#my-build-cannot-find-catch2catch_user_confighpp-how-can-i-fix-it)<br>
## How do I run global setup/teardown only if tests will be run?
@@ -28,7 +29,7 @@ depending on how often the cleanup needs to happen.
## Why cannot I derive from the built-in reporters?
They are not made to be overridden, in that we do not attempt to maintain
a consistent internal state if a member function is overriden, and by
a consistent internal state if a member function is overridden, and by
forbidding users from using them as a base class, we can refactor them
as needed later.
@@ -83,12 +84,30 @@ and it is also generally repeatable across versions, but we might break
it from time to time. E.g. we broke repeatability with previous versions
in v2.13.4 so that test cases with similar names are shuffled better.
Random generators currently rely on platform's stdlib, specifically
the distributions from `<random>`. We thus provide no extra guarantee
above what your platform does. **Important: `<random>`'s distributions
Since Catch2 3.5.0 the random generators use custom distributions,
that should be repeatable across different platforms, with few caveats.
For details see the section on random generators in the [Generator
documentation](generators.md#random-number-generators-details).
Before this version, random generators relied on distributions from
platform's stdlib. We thus can provide no extra guarantee on top of the
ones given by your platform. **Important: `<random>`'s distributions
are not specified to be repeatable across different platforms.**
## My build cannot find `catch2/catch_user_config.hpp`, how can I fix it?
`catch2/catch_user_config.hpp` is a generated header that contains user
compile time configuration. It is generated by CMake/Meson/Bazel during
build. If you are not using either of these, your three options are to
1) Build Catch2 separately using build tool that will generate the header
2) Use the amalgamated files to build Catch2
3) Use CMake to configure a build. This will generate the header and you
can copy it into your own checkout of Catch2.
---
[Home](Readme.md#top)

View File

@@ -134,7 +134,7 @@ type, making their usage much nicer. These are
* `map<T>(func, GeneratorWrapper<U>&&)` for `MapGenerator<T, U, Func>` (map `U` to `T`)
* `chunk(chunk-size, GeneratorWrapper<T>&&)` for `ChunkGenerator<T>`
* `random(IntegerOrFloat a, IntegerOrFloat b)` for `RandomIntegerGenerator` or `RandomFloatGenerator`
* `range(Arithemtic start, Arithmetic end)` for `RangeGenerator<Arithmetic>` with a step size of `1`
* `range(Arithmetic start, Arithmetic end)` for `RangeGenerator<Arithmetic>` with a step size of `1`
* `range(Arithmetic start, Arithmetic end, Arithmetic step)` for `RangeGenerator<Arithmetic>` with a custom step size
* `from_range(InputIterator from, InputIterator to)` for `IteratorGenerator<T>`
* `from_range(Container const&)` for `IteratorGenerator<T>`
@@ -189,6 +189,31 @@ TEST_CASE("type conversion", "[generators]") {
}
```
### Random number generators: details
> This section applies from Catch2 3.5.0. Before that, random generators
> were a thin wrapper around distributions from `<random>`.
All of the `random(a, b)` generators in Catch2 currently generate uniformly
distributed number in closed interval \[a; b\]. This is different from
`std::uniform_real_distribution`, which should return numbers in interval
\[a; b) (but due to rounding can end up returning b anyway), but the
difference is intentional, so that `random(a, a)` makes sense. If there is
enough interest from users, we can provide API to pick any of CC, CO, OC,
or OO ranges.
Unlike `std::uniform_int_distribution`, Catch2's generators also support
various single-byte integral types, such as `char` or `bool`.
Given the same seed, the output from the integral generators is
reproducible across different platforms. For floating point generators,
we only promise reproducibility on platforms that obey the IEEE 754
standard, and where `float` is 4 bytes and `double` is 8 bytes. We provide
no guarantees for `long double`, as the internals of `long double` can
vary wildly across different platforms.
## Generator interface
You can also implement your own generators, by deriving from the
@@ -221,3 +246,21 @@ For full example of implementing your own generator, look into Catch2's
examples, specifically
[Generators: Create your own generator](../examples/300-Gen-OwnGenerator.cpp).
### Handling empty generators
The generator interface assumes that a generator always has at least one
element. This is not always true, e.g. if the generator depends on an external
datafile, the file might be missing.
There are two ways to handle this, depending on whether you want this
to be an error or not.
* If empty generator **is** an error, throw an exception in constructor.
* If empty generator **is not** an error, use the [`SKIP`](skipping-passing-failing.md#skipping-test-cases-at-runtime) in constructor.
---
[Home](Readme.md#top)

View File

@@ -173,13 +173,3 @@ TEST_CASE("b") {
If you are seeing a problem like this, i.e. weird test paths that trigger only under Clang with `libc++`, or only under very specific version of `libstdc++`, it is very likely you are seeing this. The only known workaround is to use a fixed version of your standard library.
### libstdc++, `_GLIBCXX_DEBUG` macro and random ordering of tests
Running a Catch2 binary compiled against libstdc++ with `_GLIBCXX_DEBUG`
macro defined with `--order rand` will cause a debug check to trigger and
abort the run due to self-assignment.
[This is a known bug inside libstdc++](https://stackoverflow.com/questions/22915325/avoiding-self-assignment-in-stdshuffle/23691322)
Workaround: Don't use `--order rand` when compiling against debug-enabled
libstdc++.

View File

@@ -50,25 +50,43 @@ Both of the string matchers used in the examples above live in the
`catch_matchers_string.hpp` header, so to compile the code above also
requires `#include <catch2/matchers/catch_matchers_string.hpp>`.
### Combining operators and lifetimes
**IMPORTANT**: The combining operators do not take ownership of the
matcher objects being combined. This means that if you store combined
matcher object, you have to ensure that the matchers being combined
outlive its last use. What this means is that the following code leads
to a use-after-free (UAF):
matcher objects being combined.
This means that if you store combined matcher object, you have to ensure
that the individual matchers being combined outlive the combined matcher.
Note that the negation matcher from `!` also counts as combining matcher
for this.
Explained on an example, this is fine
```cpp
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_string.hpp>
TEST_CASE("Bugs, bugs, bugs", "[Bug]"){
std::string str = "Bugs as a service";
auto match_expression = Catch::Matchers::EndsWith( "as a service" ) ||
(Catch::Matchers::StartsWith( "Big data" ) && !Catch::Matchers::ContainsSubstring( "web scale" ) );
REQUIRE_THAT(str, match_expression);
}
CHECK_THAT(value, WithinAbs(0, 2e-2) && !WithinULP(0., 1));
```
and so is this
```cpp
auto is_close_to_zero = WithinAbs(0, 2e-2);
auto is_zero = WithinULP(0., 1);
CHECK_THAT(value, is_close_to_zero && !is_zero);
```
but this is not
```cpp
auto is_close_to_zero = WithinAbs(0, 2e-2);
auto is_zero = WithinULP(0., 1);
auto is_close_to_but_not_zero = is_close_to_zero && !is_zero;
CHECK_THAT(a_value, is_close_to_but_not_zero); // UAF
```
because `!is_zero` creates a temporary instance of Negation matcher,
which the `is_close_to_but_not_zero` refers to. After the line ends,
the temporary is destroyed and the combined `is_close_to_but_not_zero`
matcher now refers to non-existent object, so using it causes use-after-free.
## Built-in matchers
@@ -286,7 +304,7 @@ comparable. (e.g. you may compare `std::vector<int>` to `std::array<char>`).
`UnorderedRangeEquals` is similar to `RangeEquals`, but the order
does not matter. For example "1, 2, 3" would match "3, 2, 1", but not
"1, 1, 2, 3" As with `RangeEquals`, `UnorderedRangeEquals` compares
the individual elements using using `operator==` by default.
the individual elements using `operator==` by default.
Both `RangeEquals` and `UnorderedRangeEquals` optionally accept a
predicate which can be used to compare the containers element-wise.

View File

@@ -95,6 +95,9 @@ A C++ client library for Consul. Consul is a distributed tool for discovering an
### [Reactive-Extensions/ RxCpp](https://github.com/Reactive-Extensions/RxCpp)
A library of algorithms for values-distributed-in-time.
### [SFML](https://github.com/SFML/SFML)
Simple and Fast Multimedia Library.
### [SOCI](https://github.com/SOCI/soci)
The C++ Database Access Library.
@@ -143,7 +146,7 @@ Newsbeuter is an open-source RSS/Atom feed reader for text terminals.
A 2D, Zombie, RPG game which is being made on our own engine.
### [raspigcd](https://github.com/pantadeusz/raspigcd)
Low level CLI app and library for execution of GCODE on Raspberry Pi without any additional microcontrolers (just RPi + Stepsticks).
Low level CLI app and library for execution of GCODE on Raspberry Pi without any additional microcontrollers (just RPi + Stepsticks).
### [SpECTRE](https://github.com/sxs-collaboration/spectre)
SpECTRE is a code for multi-scale, multi-physics problems in astrophysics and gravitational physics.

View File

@@ -2,6 +2,8 @@
# Release notes
**Contents**<br>
[3.5.0](#350)<br>
[3.4.0](#340)<br>
[3.3.2](#332)<br>
[3.3.1](#331)<br>
[3.3.0](#330)<br>
@@ -57,6 +59,69 @@
## 3.5.0
### Improvements
* Introduced `CATCH_CONFIG_PREFIX_MESSAGES` to prefix only logging macros (#2544)
* This means `INFO`, `UNSCOPED_INFO`, `WARN` and `CAPTURE`.
* Section hints in static analysis mode are now `const`
* This prevents Clang-Tidy from complaining about `misc-const-correctness`.
* `from_range` generator supports C arrays and ranges that require ADL (#2737)
* Stringification support for `std::optional` now also includes `std::nullopt` (#2740)
* The Console reporter flushes output after writing benchmark runtime estimate.
* This means that you can immediately see for how long the benchmark is expected to run.
* Added workaround to enable compilation with ICC 19.1 (#2551, #2766)
* Compiling Catch2 for XBox should work out of the box (#2772)
* Catch2 should automatically disable getenv when compiled for XBox.
* Compiling Catch2 with exceptions disabled no longer triggers `Wunused-function` (#2726)
* **`random` Generators for integral types are now reproducible across different platforms**
* Unlike `<random>`, Catch2's generators also support 1 byte integral types (`char`, `bool`, ...)
* **`random` Generators for `float` and `double` are now reproducible across different platforms**
* `long double` varies across different platforms too much to be reproducible
* This guarantee applies only to platforms with IEEE 754 floats.
### Fixes
* UDL declaration inside Catch2 are now strictly conforming to the standard
* `operator "" _a` is UB, `operator ""_a` is fine. Seriously.
* Fixed `CAPTURE` tests failing to compile in C++23 mode (#2744)
* Fixed missing include in `catch_message.hpp` (#2758)
* Fixed `CHECK_ELSE` suppressing failure from uncaught exceptions(#2723)
### Miscellaneous
* The documentation for specifying which tests to run through commandline has been completely rewritten (#2738)
* Fixed installation when building Catch2 with meson (#2722, #2742)
* Fixed `catch_discover_tests` when using custom reporter and `PRE_TEST` discovery mode (#2747)
* `catch_discover_tests` supports multi-config CMake generator in `PRE_TEST` discovery mode (#2739, #2746)
## 3.4.0
### Improvements
* `VectorEquals` supports elements that provide only `==` and not `!=` (#2648)
* Catch2 supports compiling with IAR compiler (#2651)
* Various small internal performance improvements
* Various small internal compilation time improvements
* XMLReporter now reports location info for INFO and WARN (#1251)
* This bumps up the xml format version to 3
* Documented that `SKIP` in generator constructor can be used to handle empty generator (#1593)
* Added experimental static analysis support to `TEST_CASE` and `SECTION` macros (#2681)
* The two macros are redefined in a way that helps the SA tools reason about the possible paths through a test case with sections.
* The support is controlled by the `CATCH_CONFIG_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT` option and autodetects clang-tidy and Coverity.
* `*_THROWS`, `*_THROWS_AS`, etc now suppress warning coming from `__attribute__((warn_unused_result))` on GCC (#2691)
* Unlike plain `[[nodiscard]]`, this warning is not silenced by void cast. WTF GCC?
### Fixes
* Fixed `assertionStarting` events being sent after the expr is evaluated (#2678)
* Errors in `TEST_CASE` tags are now reported nicely (#2650)
### Miscellaneous
* Bunch of improvements to `catch_discover_tests`
* Added DISCOVERY_MODE option, so the discovery can happen either post build or pre-run.
* Fixed handling of semicolons and backslashes in test names (#2674, #2676)
* meson build can disable building tests (#2693)
* meson build properly sets meson version 0.54.1 as the minimal supported version (#2688)
## 3.3.2
### Improvements
@@ -149,7 +214,7 @@
### Fixes
* Cleaned out some warnings and static analysis issues
* Suppressed `-Wcomma` warning rarely occuring in templated test cases (#2543)
* Suppressed `-Wcomma` warning rarely occurring in templated test cases (#2543)
* Constified implementation details in `INFO` (#2564)
* Made `MatcherGenericBase` copy constructor const (#2566)
* Fixed serialization of test filters so the output roundtrips
@@ -352,7 +417,7 @@ v3 releases.
* Added `STATIC_CHECK` macro, similar to `STATIC_REQUIRE` (#2318)
* When deferred tu runtime, it behaves like `CHECK`, and not like `REQUIRE`.
* You can have multiple tests with the same name, as long as other parts of the test identity differ (#1915, #1999, #2175)
* Test identity includes test's name, test's tags and and test's class name if applicable.
* Test identity includes test's name, test's tags and test's class name if applicable.
* Added new warning, `UnmatchedTestSpec`, to error on test specs with no matching tests
* The `-w`, `--warn` warning flags can now be provided multiple times to enable multiple warnings
* The case-insensitive handling of tags is now more reliable and takes up less memory
@@ -517,7 +582,7 @@ v3 releases.
* The `SECTION`(s) before the `GENERATE` will not be run multiple times, the following ones will.
* Added `-D`/`--min-duration` command line flag (#1910)
* If a test takes longer to finish than the provided value, its name and duration will be printed.
* This flag is overriden by setting `-d`/`--duration`.
* This flag is overridden by setting `-d`/`--duration`.
### Fixes
* `TAPReporter` no longer skips successful assertions (#1983)
@@ -585,7 +650,7 @@ v3 releases.
### Fixes
* Fixed computation of benchmarking column widths in ConsoleReporter (#1885, #1886)
* Suppressed clang-tidy's `cppcoreguidelines-pro-type-vararg` in assertions (#1901)
* It was a false positive trigered by the new warning support workaround
* It was a false positive triggered by the new warning support workaround
* Fixed bug in test specification parser handling of OR'd patterns using escaping (#1905)
### Miscellaneous
@@ -922,7 +987,7 @@ v3 releases.
### Contrib
* `ParseAndAddCatchTests` has learned how to use `DISABLED` CTest property (#1452)
* `ParseAndAddCatchTests` now works when there is a whitspace before the test name (#1493)
* `ParseAndAddCatchTests` now works when there is a whitespace before the test name (#1493)
### Miscellaneous

View File

@@ -96,12 +96,12 @@ void assertionStarting( AssertionInfo const& assertionInfo );
void assertionEnded( AssertionStats const& assertionStats );
```
`assertionStarting` is called after the expression is captured, but before
the assertion expression is evaluated. This might seem like a minor
distinction, but what it means is that if you have assertion like
`REQUIRE( a + b == c + d )`, then what happens is that `a + b` and `c + d`
are evaluated before `assertionStarting` is emitted, while the `==` is
evaluated after the event.
The `assertionStarting` event is emitted before the expression in the
assertion is captured or evaluated and `assertionEnded` is emitted
afterwards. This means that given assertion like `REQUIRE(a + b == c + d)`,
Catch2 first emits `assertionStarting` event, then `a + b` and `c + d`
are evaluated, then their results are captured, the comparison is evaluated,
and then `assertionEnded` event is emitted.
## Benchmarking events

View File

@@ -52,7 +52,7 @@ its machine-readable XML output to file `result-junit.xml`, and the
uses ANSI colour codes for colouring the output.
Using multiple reporters (or one reporter and one-or-more [event
listeners](event-listener.md#top)) can have surprisingly complex semantics
listeners](event-listeners.md#top)) can have surprisingly complex semantics
when using customization points provided to reporters by Catch2, namely
capturing stdout/stderr from test cases.

View File

@@ -9,7 +9,7 @@ In some situations it may not be possible to meaningfully execute a test case,
for example when the system under test is missing certain hardware capabilities.
If the required conditions can only be determined at runtime, it often
doesn't make sense to consider such a test case as either passed or failed,
because it simply can not run at all.
because it simply cannot run at all.
To properly express such scenarios, Catch2 provides a way to explicitly
_skip_ test cases, using the `SKIP` macro:
@@ -84,6 +84,12 @@ exit code, same as it does if no test cases have run. This behaviour can
be overridden using the [--allow-running-no-tests](command-line.md#no-tests-override)
flag.
### `SKIP` inside generators
You can also use the `SKIP` macro inside generator's constructor to handle
cases where the generator is empty, but you do not want to fail the test
case.
## Passing and failing test cases

View File

@@ -231,7 +231,7 @@ TEMPLATE_TEST_CASE( "vectors can be sized and resized", "[vector][template]", in
> [Introduced](https://github.com/catchorg/Catch2/issues/1468) in Catch2 2.6.0.
_template-type1_ through _template-typen_ is list of template template
_template-type1_ through _template-typen_ is list of template
types which should be combined with each of _template-arg1_ through
_template-argm_, resulting in _n * m_ test cases. Inside the test case,
the resulting type is available under the name of `TestType`.

View File

@@ -75,7 +75,7 @@ CATCH_TRANSLATE_EXCEPTION( MyType const& ex ) {
Enums that already have a `<<` overload for `std::ostream` will convert to strings as expected.
If you only need to convert enums to strings for test reporting purposes you can provide a `StringMaker` specialisations as any other type.
However, as a convenience, Catch provides the `REGISTER_ENUM` helper macro that will generate the `StringMaker` specialiation for you with minimal code.
However, as a convenience, Catch provides the `REGISTER_ENUM` helper macro that will generate the `StringMaker` specialisation for you with minimal code.
Simply provide it the (qualified) enum name, followed by all the enum values, and you're done!
E.g.

View File

@@ -119,7 +119,7 @@ This is best explained through an example ([code](../examples/100-Fix-Section.cp
```c++
TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
// This setup will be done 4 times in total, once for each section
std::vector<int> v( 5 );
REQUIRE( v.size() == 5 );
@@ -152,11 +152,12 @@ TEST_CASE( "vectors can be sized and resized", "[vector]" ) {
}
```
For each `SECTION` the `TEST_CASE` is executed from the start. This means
For each `SECTION` the `TEST_CASE` is **executed from the start**. This means
that each section is entered with a freshly constructed vector `v`, that
we know has size 5 and capacity at least 5, because the two assertions
are also checked before the section is entered. Each run through a test
case will execute one, and only one, leaf section.
are also checked before the section is entered. This behaviour may not be
ideal for tests where setup is expensive. Each run through a test case will
execute one, and only one, leaf section.
Section can also be nested, in which case the parent section can be
entered multiple times, once for each leaf section. Nested sections are

View File

@@ -30,7 +30,7 @@ So what does Catch2 bring to the party that differentiates it from these? Apart
* Output is through modular reporter objects. Basic textual and XML reporters are included. Custom reporters can easily be added.
* JUnit xml output is supported for integration with third-party tools, such as CI servers.
* A default main() function is provided, but you can supply your own for complete control (e.g. integration into your own test runner GUI).
* A command line parser is provided and can still be used if you choose to provided your own main() function.
* A command line parser is provided and can still be used if you choose to provide your own main() function.
* Alternative assertion macro(s) report failures but don't abort the test case
* Good set of facilities for floating point comparisons (`Catch::Approx` and full set of matchers)
* Internal and friendly macros are isolated so name clashes can be managed
@@ -41,8 +41,8 @@ So what does Catch2 bring to the party that differentiates it from these? Apart
## Who else is using Catch2?
A whole lot of people. According to the 2021 JetBrains C++ ecosystem survey,
about 11% of C++ programmers use Catch2 for unit testing, making it the
A whole lot of people. According to [the 2022 JetBrains C++ ecosystem survey](https://www.jetbrains.com/lp/devecosystem-2022/cpp/#Which-unit-testing-frameworks-do-you-regularly-use),
about 12% of C++ programmers use Catch2 for unit testing, making it the
second most popular unit testing framework.
You can also take a look at the (incomplete) list of [open source projects](opensource-users.md#top)

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 010-TestCase.cpp
// And write tests in the same file:
#include <catch2/catch_test_macros.hpp>

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 020-TestCase-1.cpp
#include <catch2/catch_test_macros.hpp>

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 020-TestCase-2.cpp
// main() provided by Catch in file 020-TestCase-1.cpp.

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 030-Asn-Require-Check.cpp
// Catch has two natural expression assertion macro's:

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 100-Fix-Section.cpp
// Catch has two ways to express fixtures:

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 110-Fix-ClassFixture.cpp
// Catch has two ways to express fixtures:

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 120-Bdd-ScenarioGivenWhenThen.cpp
// main() provided by linkage with Catch2WithMain

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 210-Evt-EventListeners.cpp
// Contents:

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 231-Cfg-OutputStreams.cpp
// Show how to replace the streams with a simple custom made streambuf.

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 300-Gen-OwnGenerator.cpp
// Shows how to define a custom generator.

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 301-Gen-MapTypeConversion.cpp
// Shows how to use map to modify generator's return type.

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 302-Gen-Table.cpp
// Shows how to use table to run a test many times with different inputs. Lifted from examples on
// issue #850.
@@ -44,11 +52,11 @@ TEST_CASE("Table allows pre-computed test inputs and outputs", "[example][genera
/* Possible simplifications where less legacy toolchain support is needed:
*
* - With libstdc++6 or newer, the make_tuple() calls can be ommitted
* - With libstdc++6 or newer, the make_tuple() calls can be omitted
* (technically C++17 but does not require -std in GCC/Clang). See
* https://stackoverflow.com/questions/12436586/tuple-vector-and-initializer-list
*
* - In C++17 mode std::tie() and the preceding variable delcarations can be
* - In C++17 mode std::tie() and the preceding variable declarations can be
* replaced by structured bindings: auto [test_input, expected] = GENERATE(
* table<std::string, size_t>({ ...
*/

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 310-Gen-VariablesInGenerator.cpp
// Shows how to use variables when creating generators.

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// 311-Gen-CustomCapture.cpp
// Shows how to provide custom capture list to the generator expression

View File

@@ -53,7 +53,7 @@ set(ALL_EXAMPLE_TARGETS
)
foreach( name ${ALL_EXAMPLE_TARGETS} )
target_link_libraries( ${name} Catch2 Catch2WithMain )
target_link_libraries( ${name} Catch2WithMain )
endforeach()

View File

@@ -35,8 +35,9 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
[TEST_LIST var]
[REPORTER reporter]
[OUTPUT_DIR dir]
[OUTPUT_PREFIX prefix}
[OUTPUT_PREFIX prefix]
[OUTPUT_SUFFIX suffix]
[DISCOVERY_MODE <POST_BUILD|PRE_TEST>]
)
``catch_discover_tests`` sets up a post-build command on the test executable
@@ -123,14 +124,28 @@ same as the Catch name; see also ``TEST_PREFIX`` and ``TEST_SUFFIX``.
test executable and when the tests are executed themselves. This requires
cmake/ctest >= 3.22.
`DISCOVERY_MODE mode``
Provides control over when ``catch_discover_tests`` performs test discovery.
By default, ``POST_BUILD`` sets up a post-build command to perform test discovery
at build time. In certain scenarios, like cross-compiling, this ``POST_BUILD``
behavior is not desirable. By contrast, ``PRE_TEST`` delays test discovery until
just prior to test execution. This way test discovery occurs in the target environment
where the test has a better chance at finding appropriate runtime dependencies.
``DISCOVERY_MODE`` defaults to the value of the
``CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE`` variable if it is not passed when
calling ``catch_discover_tests``. This provides a mechanism for globally selecting
a preferred test discovery behavior without having to modify each call site.
#]=======================================================================]
#------------------------------------------------------------------------------
function(catch_discover_tests TARGET)
cmake_parse_arguments(
""
""
"TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;REPORTER;OUTPUT_DIR;OUTPUT_PREFIX;OUTPUT_SUFFIX"
"TEST_PREFIX;TEST_SUFFIX;WORKING_DIRECTORY;TEST_LIST;REPORTER;OUTPUT_DIR;OUTPUT_PREFIX;OUTPUT_SUFFIX;DISCOVERY_MODE"
"TEST_SPEC;EXTRA_ARGS;PROPERTIES;DL_PATHS"
${ARGN}
)
@@ -141,57 +156,128 @@ function(catch_discover_tests TARGET)
if(NOT _TEST_LIST)
set(_TEST_LIST ${TARGET}_TESTS)
endif()
if (_DL_PATHS)
if(${CMAKE_VERSION} VERSION_LESS "3.22.0")
message(FATAL_ERROR "The DL_PATHS option requires at least cmake 3.22")
endif()
endif()
if(NOT _DISCOVERY_MODE)
if(NOT CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE)
set(CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE "POST_BUILD")
endif()
set(_DISCOVERY_MODE ${CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE})
endif()
if (NOT _DISCOVERY_MODE MATCHES "^(POST_BUILD|PRE_TEST)$")
message(FATAL_ERROR "Unknown DISCOVERY_MODE: ${_DISCOVERY_MODE}")
endif()
## Generate a unique name based on the extra arguments
string(SHA1 args_hash "${_TEST_SPEC} ${_EXTRA_ARGS} ${_REPORTER} ${_OUTPUT_DIR} ${_OUTPUT_PREFIX} ${_OUTPUT_SUFFIX}")
string(SUBSTRING ${args_hash} 0 7 args_hash)
# Define rule to generate test list for aforementioned test executable
set(ctest_include_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}_include-${args_hash}.cmake")
set(ctest_tests_file "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}_tests-${args_hash}.cmake")
set(ctest_file_base "${CMAKE_CURRENT_BINARY_DIR}/${TARGET}-${args_hash}")
set(ctest_include_file "${ctest_file_base}_include.cmake")
set(ctest_tests_file "${ctest_file_base}_tests.cmake")
get_property(crosscompiling_emulator
TARGET ${TARGET}
PROPERTY CROSSCOMPILING_EMULATOR
)
add_custom_command(
TARGET ${TARGET} POST_BUILD
BYPRODUCTS "${ctest_tests_file}"
COMMAND "${CMAKE_COMMAND}"
-D "TEST_TARGET=${TARGET}"
-D "TEST_EXECUTABLE=$<TARGET_FILE:${TARGET}>"
-D "TEST_EXECUTOR=${crosscompiling_emulator}"
-D "TEST_WORKING_DIR=${_WORKING_DIRECTORY}"
-D "TEST_SPEC=${_TEST_SPEC}"
-D "TEST_EXTRA_ARGS=${_EXTRA_ARGS}"
-D "TEST_PROPERTIES=${_PROPERTIES}"
-D "TEST_PREFIX=${_TEST_PREFIX}"
-D "TEST_SUFFIX=${_TEST_SUFFIX}"
-D "TEST_LIST=${_TEST_LIST}"
-D "TEST_REPORTER=${_REPORTER}"
-D "TEST_OUTPUT_DIR=${_OUTPUT_DIR}"
-D "TEST_OUTPUT_PREFIX=${_OUTPUT_PREFIX}"
-D "TEST_OUTPUT_SUFFIX=${_OUTPUT_SUFFIX}"
-D "TEST_DL_PATHS=${_DL_PATHS}"
-D "CTEST_FILE=${ctest_tests_file}"
-P "${_CATCH_DISCOVER_TESTS_SCRIPT}"
VERBATIM
)
file(WRITE "${ctest_include_file}"
"if(EXISTS \"${ctest_tests_file}\")\n"
" include(\"${ctest_tests_file}\")\n"
"else()\n"
" add_test(${TARGET}_NOT_BUILT-${args_hash} ${TARGET}_NOT_BUILT-${args_hash})\n"
"endif()\n"
)
if(_DISCOVERY_MODE STREQUAL "POST_BUILD")
add_custom_command(
TARGET ${TARGET} POST_BUILD
BYPRODUCTS "${ctest_tests_file}"
COMMAND "${CMAKE_COMMAND}"
-D "TEST_TARGET=${TARGET}"
-D "TEST_EXECUTABLE=$<TARGET_FILE:${TARGET}>"
-D "TEST_EXECUTOR=${crosscompiling_emulator}"
-D "TEST_WORKING_DIR=${_WORKING_DIRECTORY}"
-D "TEST_SPEC=${_TEST_SPEC}"
-D "TEST_EXTRA_ARGS=${_EXTRA_ARGS}"
-D "TEST_PROPERTIES=${_PROPERTIES}"
-D "TEST_PREFIX=${_TEST_PREFIX}"
-D "TEST_SUFFIX=${_TEST_SUFFIX}"
-D "TEST_LIST=${_TEST_LIST}"
-D "TEST_REPORTER=${_REPORTER}"
-D "TEST_OUTPUT_DIR=${_OUTPUT_DIR}"
-D "TEST_OUTPUT_PREFIX=${_OUTPUT_PREFIX}"
-D "TEST_OUTPUT_SUFFIX=${_OUTPUT_SUFFIX}"
-D "TEST_DL_PATHS=${_DL_PATHS}"
-D "CTEST_FILE=${ctest_tests_file}"
-P "${_CATCH_DISCOVER_TESTS_SCRIPT}"
VERBATIM
)
if(NOT ${CMAKE_VERSION} VERSION_LESS "3.10.0")
file(WRITE "${ctest_include_file}"
"if(EXISTS \"${ctest_tests_file}\")\n"
" include(\"${ctest_tests_file}\")\n"
"else()\n"
" add_test(${TARGET}_NOT_BUILT-${args_hash} ${TARGET}_NOT_BUILT-${args_hash})\n"
"endif()\n"
)
elseif(_DISCOVERY_MODE STREQUAL "PRE_TEST")
get_property(GENERATOR_IS_MULTI_CONFIG GLOBAL
PROPERTY GENERATOR_IS_MULTI_CONFIG
)
if(GENERATOR_IS_MULTI_CONFIG)
set(ctest_tests_file "${ctest_file_base}_tests-$<CONFIG>.cmake")
endif()
string(CONCAT ctest_include_content
"if(EXISTS \"$<TARGET_FILE:${TARGET}>\")" "\n"
" if(NOT EXISTS \"${ctest_tests_file}\" OR" "\n"
" NOT \"${ctest_tests_file}\" IS_NEWER_THAN \"$<TARGET_FILE:${TARGET}>\" OR\n"
" NOT \"${ctest_tests_file}\" IS_NEWER_THAN \"\${CMAKE_CURRENT_LIST_FILE}\")\n"
" include(\"${_CATCH_DISCOVER_TESTS_SCRIPT}\")" "\n"
" catch_discover_tests_impl(" "\n"
" TEST_EXECUTABLE" " [==[" "$<TARGET_FILE:${TARGET}>" "]==]" "\n"
" TEST_EXECUTOR" " [==[" "${crosscompiling_emulator}" "]==]" "\n"
" TEST_WORKING_DIR" " [==[" "${_WORKING_DIRECTORY}" "]==]" "\n"
" TEST_SPEC" " [==[" "${_TEST_SPEC}" "]==]" "\n"
" TEST_EXTRA_ARGS" " [==[" "${_EXTRA_ARGS}" "]==]" "\n"
" TEST_PROPERTIES" " [==[" "${_PROPERTIES}" "]==]" "\n"
" TEST_PREFIX" " [==[" "${_TEST_PREFIX}" "]==]" "\n"
" TEST_SUFFIX" " [==[" "${_TEST_SUFFIX}" "]==]" "\n"
" TEST_LIST" " [==[" "${_TEST_LIST}" "]==]" "\n"
" TEST_REPORTER" " [==[" "${_REPORTER}" "]==]" "\n"
" TEST_OUTPUT_DIR" " [==[" "${_OUTPUT_DIR}" "]==]" "\n"
" TEST_OUTPUT_PREFIX" " [==[" "${_OUTPUT_PREFIX}" "]==]" "\n"
" TEST_OUTPUT_SUFFIX" " [==[" "${_OUTPUT_SUFFIX}" "]==]" "\n"
" CTEST_FILE" " [==[" "${ctest_tests_file}" "]==]" "\n"
" TEST_DL_PATHS" " [==[" "${_DL_PATHS}" "]==]" "\n"
" CTEST_FILE" " [==[" "${CTEST_FILE}" "]==]" "\n"
" )" "\n"
" endif()" "\n"
" include(\"${ctest_tests_file}\")" "\n"
"else()" "\n"
" add_test(${TARGET}_NOT_BUILT ${TARGET}_NOT_BUILT)" "\n"
"endif()" "\n"
)
if(GENERATOR_IS_MULTI_CONFIG)
foreach(_config ${CMAKE_CONFIGURATION_TYPES})
file(GENERATE OUTPUT "${ctest_file_base}_include-${_config}.cmake" CONTENT "${ctest_include_content}" CONDITION $<CONFIG:${_config}>)
endforeach()
string(CONCAT ctest_include_multi_content
"if(NOT CTEST_CONFIGURATION_TYPE)" "\n"
" message(\"No configuration for testing specified, use '-C <cfg>'.\")" "\n"
"else()" "\n"
" include(\"${ctest_file_base}_include-\${CTEST_CONFIGURATION_TYPE}.cmake\")" "\n"
"endif()" "\n"
)
file(GENERATE OUTPUT "${ctest_include_file}" CONTENT "${ctest_include_multi_content}")
else()
file(GENERATE OUTPUT "${ctest_file_base}_include.cmake" CONTENT "${ctest_include_content}")
file(WRITE "${ctest_include_file}" "include(\"${ctest_file_base}_include.cmake\")")
endif()
endif()
if(NOT ${CMAKE_VERSION} VERSION_LESS "3.10.0")
# Add discovered tests to directory TEST_INCLUDE_FILES
set_property(DIRECTORY
APPEND PROPERTY TEST_INCLUDE_FILES "${ctest_include_file}"
@@ -204,9 +290,7 @@ function(catch_discover_tests TARGET)
PROPERTY TEST_INCLUDE_FILE "${ctest_include_file}"
)
else()
message(FATAL_ERROR
"Cannot set more than one TEST_INCLUDE_FILE"
)
message(FATAL_ERROR "Cannot set more than one TEST_INCLUDE_FILE")
endif()
endif()

View File

@@ -1,28 +1,6 @@
# Distributed under the OSI-approved BSD 3-Clause License. See accompanying
# file Copyright.txt or https://cmake.org/licensing for details.
set(prefix "${TEST_PREFIX}")
set(suffix "${TEST_SUFFIX}")
set(spec ${TEST_SPEC})
set(extra_args ${TEST_EXTRA_ARGS})
set(properties ${TEST_PROPERTIES})
set(reporter ${TEST_REPORTER})
set(output_dir ${TEST_OUTPUT_DIR})
set(output_prefix ${TEST_OUTPUT_PREFIX})
set(output_suffix ${TEST_OUTPUT_SUFFIX})
set(dl_paths ${TEST_DL_PATHS})
set(script)
set(suite)
set(tests)
if(WIN32)
set(dl_paths_variable_name PATH)
elseif(APPLE)
set(dl_paths_variable_name DYLD_LIBRARY_PATH)
else()
set(dl_paths_variable_name LD_LIBRARY_PATH)
endif()
function(add_command NAME)
set(_args "")
# use ARGV* instead of ARGN, because ARGN splits arrays into multiple arguments
@@ -38,119 +16,177 @@ function(add_command NAME)
set(script "${script}${NAME}(${_args})\n" PARENT_SCOPE)
endfunction()
# Run test executable to get list of available tests
if(NOT EXISTS "${TEST_EXECUTABLE}")
message(FATAL_ERROR
"Specified test executable '${TEST_EXECUTABLE}' does not exist"
function(catch_discover_tests_impl)
cmake_parse_arguments(
""
""
"TEST_EXECUTABLE;TEST_WORKING_DIR;TEST_DL_PATHS;TEST_OUTPUT_DIR;TEST_OUTPUT_PREFIX;TEST_OUTPUT_SUFFIX;TEST_PREFIX;TEST_REPORTER;TEST_SPEC;TEST_SUFFIX;TEST_LIST;CTEST_FILE"
"TEST_EXTRA_ARGS;TEST_PROPERTIES;TEST_EXECUTOR"
${ARGN}
)
endif()
if(dl_paths)
cmake_path(CONVERT "${dl_paths}" TO_NATIVE_PATH_LIST paths)
set(ENV{${dl_paths_variable_name}} "${paths}")
endif()
set(prefix "${_TEST_PREFIX}")
set(suffix "${_TEST_SUFFIX}")
set(spec ${_TEST_SPEC})
set(extra_args ${_TEST_EXTRA_ARGS})
set(properties ${_TEST_PROPERTIES})
set(reporter ${_TEST_REPORTER})
set(output_dir ${_TEST_OUTPUT_DIR})
set(output_prefix ${_TEST_OUTPUT_PREFIX})
set(output_suffix ${_TEST_OUTPUT_SUFFIX})
set(dl_paths ${_TEST_DL_PATHS})
set(script)
set(suite)
set(tests)
execute_process(
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-tests --verbosity quiet
OUTPUT_VARIABLE output
RESULT_VARIABLE result
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
)
if(NOT ${result} EQUAL 0)
message(FATAL_ERROR
"Error running test executable '${TEST_EXECUTABLE}':\n"
" Result: ${result}\n"
" Output: ${output}\n"
)
endif()
string(REPLACE "\n" ";" output "${output}")
# Run test executable to get list of available reporters
execute_process(
COMMAND ${TEST_EXECUTOR} "${TEST_EXECUTABLE}" ${spec} --list-reporters
OUTPUT_VARIABLE reporters_output
RESULT_VARIABLE reporters_result
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
)
if(NOT ${reporters_result} EQUAL 0)
message(FATAL_ERROR
"Error running test executable '${TEST_EXECUTABLE}':\n"
" Result: ${reporters_result}\n"
" Output: ${reporters_output}\n"
)
endif()
string(FIND "${reporters_output}" "${reporter}" reporter_is_valid)
if(reporter AND ${reporter_is_valid} EQUAL -1)
message(FATAL_ERROR
"\"${reporter}\" is not a valid reporter!\n"
)
endif()
# Prepare reporter
if(reporter)
set(reporter_arg "--reporter ${reporter}")
endif()
# Prepare output dir
if(output_dir AND NOT IS_ABSOLUTE ${output_dir})
set(output_dir "${TEST_WORKING_DIR}/${output_dir}")
if(NOT EXISTS ${output_dir})
file(MAKE_DIRECTORY ${output_dir})
if(WIN32)
set(dl_paths_variable_name PATH)
elseif(APPLE)
set(dl_paths_variable_name DYLD_LIBRARY_PATH)
else()
set(dl_paths_variable_name LD_LIBRARY_PATH)
endif()
endif()
if(dl_paths)
foreach(path ${dl_paths})
cmake_path(NATIVE_PATH path native_path)
list(APPEND environment_modifications "${dl_paths_variable_name}=path_list_prepend:${native_path}")
# Run test executable to get list of available tests
if(NOT EXISTS "${_TEST_EXECUTABLE}")
message(FATAL_ERROR
"Specified test executable '${_TEST_EXECUTABLE}' does not exist"
)
endif()
if(dl_paths)
cmake_path(CONVERT "${dl_paths}" TO_NATIVE_PATH_LIST paths)
set(ENV{${dl_paths_variable_name}} "${paths}")
endif()
execute_process(
COMMAND ${_TEST_EXECUTOR} "${_TEST_EXECUTABLE}" ${spec} --list-tests --verbosity quiet
OUTPUT_VARIABLE output
RESULT_VARIABLE result
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
)
if(NOT ${result} EQUAL 0)
message(FATAL_ERROR
"Error running test executable '${_TEST_EXECUTABLE}':\n"
" Result: ${result}\n"
" Output: ${output}\n"
)
endif()
# Make sure to escape ; (semicolons) in test names first, because
# that'd break the foreach loop for "Parse output" later and create
# wrongly splitted and thus failing test cases (false positives)
string(REPLACE ";" "\;" output "${output}")
string(REPLACE "\n" ";" output "${output}")
# Prepare reporter
if(reporter)
set(reporter_arg "--reporter ${reporter}")
# Run test executable to check whether reporter is available
# note that the use of --list-reporters is not the important part,
# we only want to check whether the execution succeeds with ${reporter_arg}
execute_process(
COMMAND ${_TEST_EXECUTOR} "${_TEST_EXECUTABLE}" ${spec} ${reporter_arg} --list-reporters
OUTPUT_VARIABLE reporter_check_output
RESULT_VARIABLE reporter_check_result
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
)
if(${reporter_check_result} EQUAL 255)
message(FATAL_ERROR
"\"${reporter}\" is not a valid reporter!\n"
)
elseif(NOT ${reporter_check_result} EQUAL 0)
message(FATAL_ERROR
"Error running test executable '${_TEST_EXECUTABLE}':\n"
" Result: ${reporter_check_result}\n"
" Output: ${reporter_check_output}\n"
)
endif()
endif()
# Prepare output dir
if(output_dir AND NOT IS_ABSOLUTE ${output_dir})
set(output_dir "${_TEST_WORKING_DIR}/${output_dir}")
if(NOT EXISTS ${output_dir})
file(MAKE_DIRECTORY ${output_dir})
endif()
endif()
if(dl_paths)
foreach(path ${dl_paths})
cmake_path(NATIVE_PATH path native_path)
list(APPEND environment_modifications "${dl_paths_variable_name}=path_list_prepend:${native_path}")
endforeach()
endif()
# Parse output
foreach(line ${output})
set(test "${line}")
# Escape characters in test case names that would be parsed by Catch2
# Note that the \ escaping must happen FIRST! Do not change the order.
set(test_name "${test}")
foreach(char \\ , [ ])
string(REPLACE ${char} "\\${char}" test_name "${test_name}")
endforeach(char)
# ...add output dir
if(output_dir)
string(REGEX REPLACE "[^A-Za-z0-9_]" "_" test_name_clean "${test_name}")
set(output_dir_arg "--out ${output_dir}/${output_prefix}${test_name_clean}${output_suffix}")
endif()
# ...and add to script
add_command(add_test
"${prefix}${test}${suffix}"
${_TEST_EXECUTOR}
"${_TEST_EXECUTABLE}"
"${test_name}"
${extra_args}
"${reporter_arg}"
"${output_dir_arg}"
)
add_command(set_tests_properties
"${prefix}${test}${suffix}"
PROPERTIES
WORKING_DIRECTORY "${_TEST_WORKING_DIR}"
${properties}
)
if(environment_modifications)
add_command(set_tests_properties
"${prefix}${test}${suffix}"
PROPERTIES
ENVIRONMENT_MODIFICATION "${environment_modifications}")
endif()
list(APPEND tests "${prefix}${test}${suffix}")
endforeach()
# Create a list of all discovered tests, which users may use to e.g. set
# properties on the tests
add_command(set ${_TEST_LIST} ${tests})
# Write CTest script
file(WRITE "${_CTEST_FILE}" "${script}")
endfunction()
if(CMAKE_SCRIPT_MODE_FILE)
catch_discover_tests_impl(
TEST_EXECUTABLE ${TEST_EXECUTABLE}
TEST_EXECUTOR ${TEST_EXECUTOR}
TEST_WORKING_DIR ${TEST_WORKING_DIR}
TEST_SPEC ${TEST_SPEC}
TEST_EXTRA_ARGS ${TEST_EXTRA_ARGS}
TEST_PROPERTIES ${TEST_PROPERTIES}
TEST_PREFIX ${TEST_PREFIX}
TEST_SUFFIX ${TEST_SUFFIX}
TEST_LIST ${TEST_LIST}
TEST_REPORTER ${TEST_REPORTER}
TEST_OUTPUT_DIR ${TEST_OUTPUT_DIR}
TEST_OUTPUT_PREFIX ${TEST_OUTPUT_PREFIX}
TEST_OUTPUT_SUFFIX ${TEST_OUTPUT_SUFFIX}
TEST_DL_PATHS ${TEST_DL_PATHS}
CTEST_FILE ${CTEST_FILE}
)
endif()
# Parse output
foreach(line ${output})
set(test ${line})
# Escape characters in test case names that would be parsed by Catch2
set(test_name ${test})
foreach(char , [ ])
string(REPLACE ${char} "\\${char}" test_name ${test_name})
endforeach(char)
# ...add output dir
if(output_dir)
string(REGEX REPLACE "[^A-Za-z0-9_]" "_" test_name_clean ${test_name})
set(output_dir_arg "--out ${output_dir}/${output_prefix}${test_name_clean}${output_suffix}")
endif()
# ...and add to script
add_command(add_test
"${prefix}${test}${suffix}"
${TEST_EXECUTOR}
"${TEST_EXECUTABLE}"
"${test_name}"
${extra_args}
"${reporter_arg}"
"${output_dir_arg}"
)
add_command(set_tests_properties
"${prefix}${test}${suffix}"
PROPERTIES
WORKING_DIRECTORY "${TEST_WORKING_DIR}"
${properties}
)
if(environment_modifications)
add_command(set_tests_properties
"${prefix}${test}${suffix}"
PROPERTIES
ENVIRONMENT_MODIFICATION "${environment_modifications}")
endif()
list(APPEND tests "${prefix}${test}${suffix}")
endforeach()
# Create a list of all discovered tests, which users may use to e.g. set
# properties on the tests
add_command(set ${TEST_LIST} ${tests})
# Write CTest script
file(WRITE "${CTEST_FILE}" "${script}")

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
#include "NullOStream.h"
void NullOStream::avoidOutOfLineVirtualCompilerWarning()

View File

@@ -1,3 +1,11 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
#pragma once
#include <ostream>

View File

@@ -1,4 +1,10 @@
//License: Boost 1.0
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
//By Paul Dreik 2020
#include <catch2/internal/catch_test_spec_parser.hpp>

View File

@@ -1,4 +1,10 @@
//License: Boost 1.0
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
//By Paul Dreik 2020
#include <catch2/internal/catch_xmlwriter.hpp>

View File

@@ -1,4 +1,10 @@
//License: Boost 1.0
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
//By Paul Dreik 2020
#include <catch2/internal/catch_textflow.hpp>

View File

@@ -8,10 +8,12 @@
project(
'catch2',
'cpp',
version: '3.3.2', # CML version placeholder, don't delete
version: '3.5.0', # CML version placeholder, don't delete
license: 'BSL-1.0',
meson_version: '>=0.50.0',
meson_version: '>=0.54.1',
)
subdir('src/catch2')
subdir('tests')
if get_option('tests')
subdir('tests')
endif

1
meson_options.txt Normal file
View File

@@ -0,0 +1 @@
option('tests', type: 'boolean', value: true, description: 'Build the unit tests')

View File

@@ -21,6 +21,8 @@ set(BENCHMARK_HEADERS
${SOURCES_DIR}/benchmark/catch_sample_analysis.hpp
${SOURCES_DIR}/benchmark/detail/catch_analyse.hpp
${SOURCES_DIR}/benchmark/detail/catch_benchmark_function.hpp
${SOURCES_DIR}/benchmark/detail/catch_benchmark_stats.hpp
${SOURCES_DIR}/benchmark/detail/catch_benchmark_stats_fwd.hpp
${SOURCES_DIR}/benchmark/detail/catch_complete_invoke.hpp
${SOURCES_DIR}/benchmark/detail/catch_estimate_clock.hpp
${SOURCES_DIR}/benchmark/detail/catch_measure.hpp
@@ -31,6 +33,7 @@ set(BENCHMARK_HEADERS
)
set(BENCHMARK_SOURCES
${SOURCES_DIR}/benchmark/catch_chronometer.cpp
${SOURCES_DIR}/benchmark/detail/catch_analyse.cpp
${SOURCES_DIR}/benchmark/detail/catch_benchmark_function.cpp
${SOURCES_DIR}/benchmark/detail/catch_run_for_at_least.cpp
${SOURCES_DIR}/benchmark/detail/catch_stats.cpp
@@ -71,6 +74,7 @@ set(IMPL_HEADERS
${SOURCES_DIR}/internal/catch_compiler_capabilities.hpp
${SOURCES_DIR}/internal/catch_config_android_logwrite.hpp
${SOURCES_DIR}/internal/catch_config_counter.hpp
${SOURCES_DIR}/internal/catch_config_static_analysis_support.hpp
${SOURCES_DIR}/internal/catch_config_uncaught_exceptions.hpp
${SOURCES_DIR}/internal/catch_config_wchar.hpp
${SOURCES_DIR}/internal/catch_console_colour.hpp
@@ -89,6 +93,7 @@ set(IMPL_HEADERS
${SOURCES_DIR}/internal/catch_getenv.hpp
${SOURCES_DIR}/internal/catch_istream.hpp
${SOURCES_DIR}/internal/catch_is_permutation.hpp
${SOURCES_DIR}/internal/catch_jsonwriter.hpp
${SOURCES_DIR}/internal/catch_lazy_expr.hpp
${SOURCES_DIR}/internal/catch_leak_detector.hpp
${SOURCES_DIR}/internal/catch_list.hpp
@@ -104,6 +109,8 @@ set(IMPL_HEADERS
${SOURCES_DIR}/internal/catch_polyfills.hpp
${SOURCES_DIR}/internal/catch_preprocessor.hpp
${SOURCES_DIR}/internal/catch_preprocessor_remove_parens.hpp
${SOURCES_DIR}/internal/catch_random_floating_point_helpers.hpp
${SOURCES_DIR}/internal/catch_random_integer_helpers.hpp
${SOURCES_DIR}/internal/catch_random_number_generator.hpp
${SOURCES_DIR}/internal/catch_random_seed_generation.hpp
${SOURCES_DIR}/internal/catch_reporter_registry.hpp
@@ -128,10 +135,13 @@ set(IMPL_HEADERS
${SOURCES_DIR}/internal/catch_test_failure_exception.hpp
${SOURCES_DIR}/internal/catch_test_macro_impl.hpp
${SOURCES_DIR}/internal/catch_test_registry.hpp
${SOURCES_DIR}/internal/catch_test_run_info.hpp
${SOURCES_DIR}/internal/catch_test_spec_parser.hpp
${SOURCES_DIR}/internal/catch_textflow.hpp
${SOURCES_DIR}/internal/catch_to_string.hpp
${SOURCES_DIR}/internal/catch_uncaught_exceptions.hpp
${SOURCES_DIR}/internal/catch_uniform_floating_point_distribution.hpp
${SOURCES_DIR}/internal/catch_uniform_integer_distribution.hpp
${SOURCES_DIR}/internal/catch_unique_name.hpp
${SOURCES_DIR}/internal/catch_unique_ptr.hpp
${SOURCES_DIR}/internal/catch_void_type.hpp
@@ -153,6 +163,7 @@ set(IMPL_SOURCES
${SOURCES_DIR}/catch_timer.cpp
${SOURCES_DIR}/catch_tostring.cpp
${SOURCES_DIR}/catch_totals.cpp
${SOURCES_DIR}/catch_translate_exception.cpp
${SOURCES_DIR}/catch_version.cpp
${SOURCES_DIR}/internal/catch_assertion_handler.cpp
${SOURCES_DIR}/internal/catch_case_insensitive_comparisons.cpp
@@ -171,6 +182,7 @@ set(IMPL_SOURCES
${SOURCES_DIR}/internal/catch_floating_point_helpers.cpp
${SOURCES_DIR}/internal/catch_getenv.cpp
${SOURCES_DIR}/internal/catch_istream.cpp
${SOURCES_DIR}/internal/catch_jsonwriter.cpp
${SOURCES_DIR}/internal/catch_lazy_expr.cpp
${SOURCES_DIR}/internal/catch_leak_detector.cpp
${SOURCES_DIR}/internal/catch_list.cpp
@@ -216,8 +228,8 @@ set(INTERFACE_HEADERS
${SOURCES_DIR}/interfaces/catch_interfaces_registry_hub.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter_factory.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter_registry.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_tag_alias_registry.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_test_invoker.hpp
${SOURCES_DIR}/interfaces/catch_interfaces_testcase.hpp
)
set(INTERFACE_SOURCES
@@ -228,7 +240,6 @@ set(INTERFACE_SOURCES
${SOURCES_DIR}/interfaces/catch_interfaces_registry_hub.cpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter.cpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter_factory.cpp
${SOURCES_DIR}/interfaces/catch_interfaces_reporter_registry.cpp
${SOURCES_DIR}/interfaces/catch_interfaces_testcase.cpp
)
set(INTERFACE_FILES ${INTERFACE_HEADERS} ${INTERFACE_SOURCES})
@@ -284,6 +295,7 @@ set(REPORTER_HEADERS
${SOURCES_DIR}/reporters/catch_reporter_cumulative_base.hpp
${SOURCES_DIR}/reporters/catch_reporter_event_listener.hpp
${SOURCES_DIR}/reporters/catch_reporter_helpers.hpp
${SOURCES_DIR}/reporters/catch_reporter_json.hpp
${SOURCES_DIR}/reporters/catch_reporter_junit.hpp
${SOURCES_DIR}/reporters/catch_reporter_multi.hpp
${SOURCES_DIR}/reporters/catch_reporter_registrars.hpp
@@ -302,6 +314,7 @@ set(REPORTER_SOURCES
${SOURCES_DIR}/reporters/catch_reporter_cumulative_base.cpp
${SOURCES_DIR}/reporters/catch_reporter_event_listener.cpp
${SOURCES_DIR}/reporters/catch_reporter_helpers.cpp
${SOURCES_DIR}/reporters/catch_reporter_json.cpp
${SOURCES_DIR}/reporters/catch_reporter_junit.cpp
${SOURCES_DIR}/reporters/catch_reporter_multi.cpp
${SOURCES_DIR}/reporters/catch_reporter_registrars.cpp
@@ -335,7 +348,9 @@ source_group("generated headers"
)
add_library(Catch2 ${ALL_FILES})
add_build_reproducibility_settings(Catch2)
if (CATCH_ENABLE_REPRODUCIBLE_BUILD)
add_build_reproducibility_settings(Catch2)
endif()
add_library(Catch2::Catch2 ALIAS Catch2)
if (ANDROID)
@@ -388,7 +403,9 @@ target_include_directories(Catch2
add_library(Catch2WithMain
${SOURCES_DIR}/internal/catch_main.cpp
)
add_build_reproducibility_settings(Catch2WithMain)
if (CATCH_ENABLE_REPRODUCIBLE_BUILD)
add_build_reproducibility_settings(Catch2WithMain)
endif()
add_library(Catch2::Catch2WithMain ALIAS Catch2WithMain)
target_link_libraries(Catch2WithMain PUBLIC Catch2)
set_target_properties(Catch2WithMain

View File

@@ -10,26 +10,28 @@
#ifndef CATCH_BENCHMARK_HPP_INCLUDED
#define CATCH_BENCHMARK_HPP_INCLUDED
#include <catch2/interfaces/catch_interfaces_config.hpp>
#include <catch2/catch_user_config.hpp>
#include <catch2/internal/catch_compiler_capabilities.hpp>
#include <catch2/internal/catch_context.hpp>
#include <catch2/interfaces/catch_interfaces_reporter.hpp>
#include <catch2/internal/catch_unique_name.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <catch2/benchmark/catch_chronometer.hpp>
#include <catch2/internal/catch_test_failure_exception.hpp>
#include <catch2/internal/catch_unique_name.hpp>
#include <catch2/interfaces/catch_interfaces_capture.hpp>
#include <catch2/interfaces/catch_interfaces_config.hpp>
#include <catch2/interfaces/catch_interfaces_registry_hub.hpp>
#include <catch2/benchmark/detail/catch_benchmark_stats.hpp>
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_environment.hpp>
#include <catch2/benchmark/catch_execution_plan.hpp>
#include <catch2/benchmark/detail/catch_estimate_clock.hpp>
#include <catch2/benchmark/detail/catch_complete_invoke.hpp>
#include <catch2/benchmark/detail/catch_analyse.hpp>
#include <catch2/benchmark/detail/catch_benchmark_function.hpp>
#include <catch2/benchmark/detail/catch_run_for_at_least.hpp>
#include <algorithm>
#include <functional>
#include <chrono>
#include <exception>
#include <string>
#include <vector>
#include <cmath>
namespace Catch {
@@ -43,16 +45,18 @@ namespace Catch {
: fun(CATCH_MOVE(func)), name(CATCH_MOVE(benchmarkName)) {}
template <typename Clock>
ExecutionPlan<FloatDuration<Clock>> prepare(const IConfig &cfg, Environment<FloatDuration<Clock>> env) const {
ExecutionPlan prepare(const IConfig &cfg, Environment env) const {
auto min_time = env.clock_resolution.mean * Detail::minimum_ticks;
auto run_time = std::max(min_time, std::chrono::duration_cast<decltype(min_time)>(cfg.benchmarkWarmupTime()));
auto&& test = Detail::run_for_at_least<Clock>(std::chrono::duration_cast<ClockDuration<Clock>>(run_time), 1, fun);
auto&& test = Detail::run_for_at_least<Clock>(std::chrono::duration_cast<IDuration>(run_time), 1, fun);
int new_iters = static_cast<int>(std::ceil(min_time * test.iterations / test.elapsed));
return { new_iters, test.elapsed / test.iterations * new_iters * cfg.benchmarkSamples(), fun, std::chrono::duration_cast<FloatDuration<Clock>>(cfg.benchmarkWarmupTime()), Detail::warmup_iterations };
return { new_iters, test.elapsed / test.iterations * new_iters * cfg.benchmarkSamples(), fun, std::chrono::duration_cast<FDuration>(cfg.benchmarkWarmupTime()), Detail::warmup_iterations };
}
template <typename Clock = default_clock>
void run() {
static_assert( Clock::is_steady,
"Benchmarking clock should be steady" );
auto const* cfg = getCurrentContext().getConfig();
auto env = Detail::measure_environment<Clock>();
@@ -79,10 +83,10 @@ namespace Catch {
return plan.template run<Clock>(*cfg, env);
});
auto analysis = Detail::analyse(*cfg, env, samples.begin(), samples.end());
BenchmarkStats<FloatDuration<Clock>> stats{ CATCH_MOVE(info), CATCH_MOVE(analysis.samples), analysis.mean, analysis.standard_deviation, analysis.outliers, analysis.outlier_variance };
auto analysis = Detail::analyse(*cfg, samples.data(), samples.data() + samples.size());
BenchmarkStats<> stats{ CATCH_MOVE(info), CATCH_MOVE(analysis.samples), analysis.mean, analysis.standard_deviation, analysis.outliers, analysis.outlier_variance };
getResultCapture().benchmarkEnded(stats);
} CATCH_CATCH_ANON (TestFailureException) {
} CATCH_CATCH_ANON (TestFailureException const&) {
getResultCapture().benchmarkFailed("Benchmark failed due to failed assertion"_sr);
} CATCH_CATCH_ALL{
getResultCapture().benchmarkFailed(translateActiveException());

View File

@@ -33,6 +33,8 @@
#include <catch2/benchmark/catch_sample_analysis.hpp>
#include <catch2/benchmark/detail/catch_analyse.hpp>
#include <catch2/benchmark/detail/catch_benchmark_function.hpp>
#include <catch2/benchmark/detail/catch_benchmark_stats.hpp>
#include <catch2/benchmark/detail/catch_benchmark_stats_fwd.hpp>
#include <catch2/benchmark/detail/catch_complete_invoke.hpp>
#include <catch2/benchmark/detail/catch_estimate_clock.hpp>
#include <catch2/benchmark/detail/catch_measure.hpp>

View File

@@ -12,7 +12,6 @@
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_optimizer.hpp>
#include <catch2/benchmark/detail/catch_complete_invoke.hpp>
#include <catch2/internal/catch_meta.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
@@ -33,7 +32,10 @@ namespace Catch {
void start() override { started = Clock::now(); }
void finish() override { finished = Clock::now(); }
ClockDuration<Clock> elapsed() const { return finished - started; }
IDuration elapsed() const {
return std::chrono::duration_cast<std::chrono::nanoseconds>(
finished - started );
}
TimePoint<Clock> started;
TimePoint<Clock> finished;

View File

@@ -11,28 +11,16 @@
#define CATCH_CLOCK_HPP_INCLUDED
#include <chrono>
#include <ratio>
namespace Catch {
namespace Benchmark {
template <typename Clock>
using ClockDuration = typename Clock::duration;
template <typename Clock>
using FloatDuration = std::chrono::duration<double, typename Clock::period>;
using IDuration = std::chrono::nanoseconds;
using FDuration = std::chrono::duration<double, std::nano>;
template <typename Clock>
using TimePoint = typename Clock::time_point;
using default_clock = std::chrono::steady_clock;
template <typename Clock>
struct now {
TimePoint<Clock> operator()() const {
return Clock::now();
}
};
using fp_seconds = std::chrono::duration<double, std::ratio<1>>;
} // namespace Benchmark
} // namespace Catch

View File

@@ -15,21 +15,13 @@
namespace Catch {
namespace Benchmark {
template <typename Duration>
struct EnvironmentEstimate {
Duration mean;
FDuration mean;
OutlierClassification outliers;
template <typename Duration2>
operator EnvironmentEstimate<Duration2>() const {
return { mean, outliers };
}
};
template <typename Clock>
struct Environment {
using clock_type = Clock;
EnvironmentEstimate<FloatDuration<Clock>> clock_resolution;
EnvironmentEstimate<FloatDuration<Clock>> clock_cost;
EnvironmentEstimate clock_resolution;
EnvironmentEstimate clock_cost;
};
} // namespace Benchmark
} // namespace Catch

View File

@@ -12,17 +12,12 @@
namespace Catch {
namespace Benchmark {
template <typename Duration>
template <typename Type>
struct Estimate {
Duration point;
Duration lower_bound;
Duration upper_bound;
Type point;
Type lower_bound;
Type upper_bound;
double confidence_interval;
template <typename Duration2>
operator Estimate<Duration2>() const {
return { point, lower_bound, upper_bound, confidence_interval };
}
};
} // namespace Benchmark
} // namespace Catch

View File

@@ -17,38 +17,38 @@
#include <catch2/benchmark/detail/catch_repeat.hpp>
#include <catch2/benchmark/detail/catch_run_for_at_least.hpp>
#include <algorithm>
#include <iterator>
#include <vector>
namespace Catch {
namespace Benchmark {
template <typename Duration>
struct ExecutionPlan {
int iterations_per_sample;
Duration estimated_duration;
FDuration estimated_duration;
Detail::BenchmarkFunction benchmark;
Duration warmup_time;
FDuration warmup_time;
int warmup_iterations;
template <typename Duration2>
operator ExecutionPlan<Duration2>() const {
return { iterations_per_sample, estimated_duration, benchmark, warmup_time, warmup_iterations };
}
template <typename Clock>
std::vector<FloatDuration<Clock>> run(const IConfig &cfg, Environment<FloatDuration<Clock>> env) const {
std::vector<FDuration> run(const IConfig &cfg, Environment env) const {
// warmup a bit
Detail::run_for_at_least<Clock>(std::chrono::duration_cast<ClockDuration<Clock>>(warmup_time), warmup_iterations, Detail::repeat(now<Clock>{}));
Detail::run_for_at_least<Clock>(
std::chrono::duration_cast<IDuration>( warmup_time ),
warmup_iterations,
Detail::repeat( []() { return Clock::now(); } )
);
std::vector<FloatDuration<Clock>> times;
times.reserve(cfg.benchmarkSamples());
std::generate_n(std::back_inserter(times), cfg.benchmarkSamples(), [this, env] {
std::vector<FDuration> times;
const auto num_samples = cfg.benchmarkSamples();
times.reserve( num_samples );
for ( size_t i = 0; i < num_samples; ++i ) {
Detail::ChronometerModel<Clock> model;
this->benchmark(Chronometer(model, iterations_per_sample));
this->benchmark( Chronometer( model, iterations_per_sample ) );
auto sample_time = model.elapsed() - env.clock_cost.mean;
if (sample_time < FloatDuration<Clock>::zero()) sample_time = FloatDuration<Clock>::zero();
return sample_time / iterations_per_sample;
});
if ( sample_time < FDuration::zero() ) {
sample_time = FDuration::zero();
}
times.push_back(sample_time / iterations_per_sample);
}
return times;
}
};

View File

@@ -70,7 +70,7 @@ namespace Catch {
template <typename Fn, typename... Args>
inline auto invoke_deoptimized(Fn&& fn, Args&&... args) -> std::enable_if_t<std::is_same<void, decltype(fn(args...))>::value> {
CATCH_FORWARD(fn) (CATCH_FORWARD(args)...);
CATCH_FORWARD((fn)) (CATCH_FORWARD(args)...);
}
} // namespace Benchmark
} // namespace Catch

View File

@@ -10,38 +10,20 @@
#ifndef CATCH_SAMPLE_ANALYSIS_HPP_INCLUDED
#define CATCH_SAMPLE_ANALYSIS_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_estimate.hpp>
#include <catch2/benchmark/catch_outlier_classification.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <catch2/benchmark/catch_clock.hpp>
#include <algorithm>
#include <vector>
#include <iterator>
namespace Catch {
namespace Benchmark {
template <typename Duration>
struct SampleAnalysis {
std::vector<Duration> samples;
Estimate<Duration> mean;
Estimate<Duration> standard_deviation;
std::vector<FDuration> samples;
Estimate<FDuration> mean;
Estimate<FDuration> standard_deviation;
OutlierClassification outliers;
double outlier_variance;
template <typename Duration2>
operator SampleAnalysis<Duration2>() const {
std::vector<Duration2> samples2;
samples2.reserve(samples.size());
std::transform(samples.begin(), samples.end(), std::back_inserter(samples2), [](Duration d) { return Duration2(d); });
return {
CATCH_MOVE(samples2),
mean,
standard_deviation,
outliers,
outlier_variance,
};
}
};
} // namespace Benchmark
} // namespace Catch

View File

@@ -0,0 +1,85 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
// Adapted from donated nonius code.
#include <catch2/benchmark/detail/catch_analyse.hpp>
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_sample_analysis.hpp>
#include <catch2/benchmark/detail/catch_stats.hpp>
#include <catch2/interfaces/catch_interfaces_config.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <vector>
namespace Catch {
namespace Benchmark {
namespace Detail {
SampleAnalysis analyse(const IConfig &cfg, FDuration* first, FDuration* last) {
if (!cfg.benchmarkNoAnalysis()) {
std::vector<double> samples;
samples.reserve(static_cast<size_t>(last - first));
for (auto current = first; current != last; ++current) {
samples.push_back( current->count() );
}
auto analysis = Catch::Benchmark::Detail::analyse_samples(
cfg.benchmarkConfidenceInterval(),
cfg.benchmarkResamples(),
samples.data(),
samples.data() + samples.size() );
auto outliers = Catch::Benchmark::Detail::classify_outliers(
samples.data(), samples.data() + samples.size() );
auto wrap_estimate = [](Estimate<double> e) {
return Estimate<FDuration> {
FDuration(e.point),
FDuration(e.lower_bound),
FDuration(e.upper_bound),
e.confidence_interval,
};
};
std::vector<FDuration> samples2;
samples2.reserve(samples.size());
for (auto s : samples) {
samples2.push_back( FDuration( s ) );
}
return {
CATCH_MOVE(samples2),
wrap_estimate(analysis.mean),
wrap_estimate(analysis.standard_deviation),
outliers,
analysis.outlier_variance,
};
} else {
std::vector<FDuration> samples;
samples.reserve(static_cast<size_t>(last - first));
FDuration mean = FDuration(0);
int i = 0;
for (auto it = first; it < last; ++it, ++i) {
samples.push_back(FDuration(*it));
mean += FDuration(*it);
}
mean /= i;
return SampleAnalysis{
CATCH_MOVE(samples),
Estimate<FDuration>{ mean, mean, mean, 0.0 },
Estimate<FDuration>{ FDuration( 0 ),
FDuration( 0 ),
FDuration( 0 ),
0.0 },
OutlierClassification{},
0.0
};
}
}
} // namespace Detail
} // namespace Benchmark
} // namespace Catch

View File

@@ -11,68 +11,15 @@
#define CATCH_ANALYSE_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/catch_environment.hpp>
#include <catch2/benchmark/catch_sample_analysis.hpp>
#include <catch2/benchmark/detail/catch_stats.hpp>
#include <catch2/interfaces/catch_interfaces_config.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <algorithm>
#include <iterator>
#include <vector>
namespace Catch {
class IConfig;
namespace Benchmark {
namespace Detail {
template <typename Duration, typename Iterator>
SampleAnalysis<Duration> analyse(const IConfig &cfg, Environment<Duration>, Iterator first, Iterator last) {
if (!cfg.benchmarkNoAnalysis()) {
std::vector<double> samples;
samples.reserve(static_cast<size_t>(last - first));
std::transform(first, last, std::back_inserter(samples), [](Duration d) { return d.count(); });
auto analysis = Catch::Benchmark::Detail::analyse_samples(cfg.benchmarkConfidenceInterval(), cfg.benchmarkResamples(), samples.begin(), samples.end());
auto outliers = Catch::Benchmark::Detail::classify_outliers(samples.begin(), samples.end());
auto wrap_estimate = [](Estimate<double> e) {
return Estimate<Duration> {
Duration(e.point),
Duration(e.lower_bound),
Duration(e.upper_bound),
e.confidence_interval,
};
};
std::vector<Duration> samples2;
samples2.reserve(samples.size());
std::transform(samples.begin(), samples.end(), std::back_inserter(samples2), [](double d) { return Duration(d); });
return {
CATCH_MOVE(samples2),
wrap_estimate(analysis.mean),
wrap_estimate(analysis.standard_deviation),
outliers,
analysis.outlier_variance,
};
} else {
std::vector<Duration> samples;
samples.reserve(static_cast<size_t>(last - first));
Duration mean = Duration(0);
int i = 0;
for (auto it = first; it < last; ++it, ++i) {
samples.push_back(Duration(*it));
mean += Duration(*it);
}
mean /= i;
return {
CATCH_MOVE(samples),
Estimate<Duration>{mean, mean, mean, 0.0},
Estimate<Duration>{Duration(0), Duration(0), Duration(0), 0.0},
OutlierClassification{},
0.0
};
}
}
SampleAnalysis analyse(const IConfig &cfg, FDuration* first, FDuration* last);
} // namespace Detail
} // namespace Benchmark
} // namespace Catch

View File

@@ -11,7 +11,6 @@
#define CATCH_BENCHMARK_FUNCTION_HPP_INCLUDED
#include <catch2/benchmark/catch_chronometer.hpp>
#include <catch2/benchmark/detail/catch_complete_invoke.hpp>
#include <catch2/internal/catch_meta.hpp>
#include <catch2/internal/catch_unique_ptr.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>

View File

@@ -0,0 +1,48 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
#ifndef CATCH_BENCHMARK_STATS_HPP_INCLUDED
#define CATCH_BENCHMARK_STATS_HPP_INCLUDED
#include <catch2/benchmark/catch_estimate.hpp>
#include <catch2/benchmark/catch_outlier_classification.hpp>
// The fwd decl & default specialization needs to be seen by VS2017 before
// BenchmarkStats itself, or VS2017 will report compilation error.
#include <catch2/benchmark/detail/catch_benchmark_stats_fwd.hpp>
#include <string>
#include <vector>
namespace Catch {
struct BenchmarkInfo {
std::string name;
double estimatedDuration;
int iterations;
unsigned int samples;
unsigned int resamples;
double clockResolution;
double clockCost;
};
// We need to keep template parameter for backwards compatibility,
// but we also do not want to use the template paraneter.
template <class Dummy>
struct BenchmarkStats {
BenchmarkInfo info;
std::vector<Benchmark::FDuration> samples;
Benchmark::Estimate<Benchmark::FDuration> mean;
Benchmark::Estimate<Benchmark::FDuration> standardDeviation;
Benchmark::OutlierClassification outliers;
double outlierVariance;
};
} // end namespace Catch
#endif // CATCH_BENCHMARK_STATS_HPP_INCLUDED

View File

@@ -0,0 +1,23 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
#ifndef CATCH_BENCHMARK_STATS_FWD_HPP_INCLUDED
#define CATCH_BENCHMARK_STATS_FWD_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
namespace Catch {
// We cannot forward declare the type with default template argument
// multiple times, so it is split out into a separate header so that
// we can prevent multiple declarations in dependees
template <typename Duration = Benchmark::FDuration>
struct BenchmarkStats;
} // end namespace Catch
#endif // CATCH_BENCHMARK_STATS_FWD_HPP_INCLUDED

View File

@@ -10,14 +10,9 @@
#ifndef CATCH_COMPLETE_INVOKE_HPP_INCLUDED
#define CATCH_COMPLETE_INVOKE_HPP_INCLUDED
#include <catch2/internal/catch_test_failure_exception.hpp>
#include <catch2/internal/catch_meta.hpp>
#include <catch2/interfaces/catch_interfaces_capture.hpp>
#include <catch2/interfaces/catch_interfaces_registry_hub.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <type_traits>
namespace Catch {
namespace Benchmark {
namespace Detail {

View File

@@ -19,7 +19,6 @@
#include <catch2/internal/catch_unique_ptr.hpp>
#include <algorithm>
#include <iterator>
#include <vector>
#include <cmath>
@@ -30,46 +29,49 @@ namespace Catch {
std::vector<double> resolution(int k) {
std::vector<TimePoint<Clock>> times;
times.reserve(static_cast<size_t>(k + 1));
std::generate_n(std::back_inserter(times), k + 1, now<Clock>{});
for ( int i = 0; i < k + 1; ++i ) {
times.push_back( Clock::now() );
}
std::vector<double> deltas;
deltas.reserve(static_cast<size_t>(k));
std::transform(std::next(times.begin()), times.end(), times.begin(),
std::back_inserter(deltas),
[](TimePoint<Clock> a, TimePoint<Clock> b) { return static_cast<double>((a - b).count()); });
for ( size_t idx = 1; idx < times.size(); ++idx ) {
deltas.push_back( static_cast<double>(
( times[idx] - times[idx - 1] ).count() ) );
}
return deltas;
}
const auto warmup_iterations = 10000;
const auto warmup_time = std::chrono::milliseconds(100);
const auto minimum_ticks = 1000;
const auto warmup_seed = 10000;
const auto clock_resolution_estimation_time = std::chrono::milliseconds(500);
const auto clock_cost_estimation_time_limit = std::chrono::seconds(1);
const auto clock_cost_estimation_tick_limit = 100000;
const auto clock_cost_estimation_time = std::chrono::milliseconds(10);
const auto clock_cost_estimation_iterations = 10000;
constexpr auto warmup_iterations = 10000;
constexpr auto warmup_time = std::chrono::milliseconds(100);
constexpr auto minimum_ticks = 1000;
constexpr auto warmup_seed = 10000;
constexpr auto clock_resolution_estimation_time = std::chrono::milliseconds(500);
constexpr auto clock_cost_estimation_time_limit = std::chrono::seconds(1);
constexpr auto clock_cost_estimation_tick_limit = 100000;
constexpr auto clock_cost_estimation_time = std::chrono::milliseconds(10);
constexpr auto clock_cost_estimation_iterations = 10000;
template <typename Clock>
int warmup() {
return run_for_at_least<Clock>(std::chrono::duration_cast<ClockDuration<Clock>>(warmup_time), warmup_seed, &resolution<Clock>)
return run_for_at_least<Clock>(warmup_time, warmup_seed, &resolution<Clock>)
.iterations;
}
template <typename Clock>
EnvironmentEstimate<FloatDuration<Clock>> estimate_clock_resolution(int iterations) {
auto r = run_for_at_least<Clock>(std::chrono::duration_cast<ClockDuration<Clock>>(clock_resolution_estimation_time), iterations, &resolution<Clock>)
EnvironmentEstimate estimate_clock_resolution(int iterations) {
auto r = run_for_at_least<Clock>(clock_resolution_estimation_time, iterations, &resolution<Clock>)
.result;
return {
FloatDuration<Clock>(mean(r.begin(), r.end())),
classify_outliers(r.begin(), r.end()),
FDuration(mean(r.data(), r.data() + r.size())),
classify_outliers(r.data(), r.data() + r.size()),
};
}
template <typename Clock>
EnvironmentEstimate<FloatDuration<Clock>> estimate_clock_cost(FloatDuration<Clock> resolution) {
EnvironmentEstimate estimate_clock_cost(FDuration resolution) {
auto time_limit = (std::min)(
resolution * clock_cost_estimation_tick_limit,
FloatDuration<Clock>(clock_cost_estimation_time_limit));
FDuration(clock_cost_estimation_time_limit));
auto time_clock = [](int k) {
return Detail::measure<Clock>([k] {
for (int i = 0; i < k; ++i) {
@@ -80,26 +82,28 @@ namespace Catch {
};
time_clock(1);
int iters = clock_cost_estimation_iterations;
auto&& r = run_for_at_least<Clock>(std::chrono::duration_cast<ClockDuration<Clock>>(clock_cost_estimation_time), iters, time_clock);
auto&& r = run_for_at_least<Clock>(clock_cost_estimation_time, iters, time_clock);
std::vector<double> times;
int nsamples = static_cast<int>(std::ceil(time_limit / r.elapsed));
times.reserve(static_cast<size_t>(nsamples));
std::generate_n(std::back_inserter(times), nsamples, [time_clock, &r] {
return static_cast<double>((time_clock(r.iterations) / r.iterations).count());
});
for ( int s = 0; s < nsamples; ++s ) {
times.push_back( static_cast<double>(
( time_clock( r.iterations ) / r.iterations )
.count() ) );
}
return {
FloatDuration<Clock>(mean(times.begin(), times.end())),
classify_outliers(times.begin(), times.end()),
FDuration(mean(times.data(), times.data() + times.size())),
classify_outliers(times.data(), times.data() + times.size()),
};
}
template <typename Clock>
Environment<FloatDuration<Clock>> measure_environment() {
Environment measure_environment() {
#if defined(__clang__)
# pragma clang diagnostic push
# pragma clang diagnostic ignored "-Wexit-time-destructors"
#endif
static Catch::Detail::unique_ptr<Environment<FloatDuration<Clock>>> env;
static Catch::Detail::unique_ptr<Environment> env;
#if defined(__clang__)
# pragma clang diagnostic pop
#endif
@@ -111,7 +115,7 @@ namespace Catch {
auto resolution = Detail::estimate_clock_resolution<Clock>(iters);
auto cost = Detail::estimate_clock_cost<Clock>(resolution.mean);
env = Catch::Detail::make_unique<Environment<FloatDuration<Clock>>>( Environment<FloatDuration<Clock>>{resolution, cost} );
env = Catch::Detail::make_unique<Environment>( Environment{resolution, cost} );
return *env;
}
} // namespace Detail

View File

@@ -10,7 +10,6 @@
#ifndef CATCH_MEASURE_HPP_INCLUDED
#define CATCH_MEASURE_HPP_INCLUDED
#include <catch2/benchmark/catch_clock.hpp>
#include <catch2/benchmark/detail/catch_complete_invoke.hpp>
#include <catch2/benchmark/detail/catch_timing.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
@@ -19,7 +18,7 @@ namespace Catch {
namespace Benchmark {
namespace Detail {
template <typename Clock, typename Fun, typename... Args>
TimingOf<Clock, Fun, Args...> measure(Fun&& fun, Args&&... args) {
TimingOf<Fun, Args...> measure(Fun&& fun, Args&&... args) {
auto start = Clock::now();
auto&& r = Detail::complete_invoke(fun, CATCH_FORWARD(args)...);
auto end = Clock::now();

View File

@@ -7,9 +7,10 @@
// SPDX-License-Identifier: BSL-1.0
#include <catch2/benchmark/detail/catch_run_for_at_least.hpp>
#include <exception>
#include <catch2/internal/catch_enforce.hpp>
#include <exception>
namespace Catch {
namespace Benchmark {
namespace Detail {

View File

@@ -24,11 +24,11 @@ namespace Catch {
namespace Benchmark {
namespace Detail {
template <typename Clock, typename Fun>
TimingOf<Clock, Fun, int> measure_one(Fun&& fun, int iters, std::false_type) {
TimingOf<Fun, int> measure_one(Fun&& fun, int iters, std::false_type) {
return Detail::measure<Clock>(fun, iters);
}
template <typename Clock, typename Fun>
TimingOf<Clock, Fun, Chronometer> measure_one(Fun&& fun, int iters, std::true_type) {
TimingOf<Fun, Chronometer> measure_one(Fun&& fun, int iters, std::true_type) {
Detail::ChronometerModel<Clock> meter;
auto&& result = Detail::complete_invoke(fun, Chronometer(meter, iters));
@@ -43,8 +43,8 @@ namespace Catch {
void throw_optimized_away_error();
template <typename Clock, typename Fun>
TimingOf<Clock, Fun, run_for_at_least_argument_t<Clock, Fun>>
run_for_at_least(ClockDuration<Clock> how_long,
TimingOf<Fun, run_for_at_least_argument_t<Clock, Fun>>
run_for_at_least(IDuration how_long,
const int initial_iterations,
Fun&& fun) {
auto iters = initial_iterations;

View File

@@ -10,10 +10,14 @@
#include <catch2/benchmark/detail/catch_stats.hpp>
#include <catch2/internal/catch_compiler_capabilities.hpp>
#include <catch2/internal/catch_floating_point_helpers.hpp>
#include <catch2/internal/catch_random_number_generator.hpp>
#include <algorithm>
#include <cassert>
#include <cmath>
#include <cstddef>
#include <iterator>
#include <numeric>
#include <random>
@@ -21,139 +25,204 @@
#include <future>
#endif
namespace {
namespace Catch {
namespace Benchmark {
namespace Detail {
namespace {
using Catch::Benchmark::Detail::sample;
template <typename URng, typename Estimator>
static sample
resample( URng& rng,
unsigned int resamples,
double const* first,
double const* last,
Estimator& estimator ) {
auto n = static_cast<size_t>( last - first );
std::uniform_int_distribution<decltype( n )> dist( 0,
n - 1 );
template <typename URng, typename Estimator>
sample resample(URng& rng, unsigned int resamples, std::vector<double>::iterator first, std::vector<double>::iterator last, Estimator& estimator) {
auto n = static_cast<size_t>(last - first);
std::uniform_int_distribution<decltype(n)> dist(0, n - 1);
sample out;
out.reserve( resamples );
// We allocate the vector outside the loop to avoid realloc
// per resample
std::vector<double> resampled;
resampled.reserve( n );
for ( size_t i = 0; i < resamples; ++i ) {
resampled.clear();
for ( size_t s = 0; s < n; ++s ) {
resampled.push_back(
first[static_cast<std::ptrdiff_t>(
dist( rng ) )] );
}
const auto estimate =
estimator( resampled.data(), resampled.data() + resampled.size() );
out.push_back( estimate );
}
std::sort( out.begin(), out.end() );
return out;
}
sample out;
out.reserve(resamples);
std::generate_n(std::back_inserter(out), resamples, [n, first, &estimator, &dist, &rng] {
std::vector<double> resampled;
resampled.reserve(n);
std::generate_n(std::back_inserter(resampled), n, [first, &dist, &rng] { return first[static_cast<std::ptrdiff_t>(dist(rng))]; });
return estimator(resampled.begin(), resampled.end());
});
std::sort(out.begin(), out.end());
return out;
}
static double outlier_variance( Estimate<double> mean,
Estimate<double> stddev,
int n ) {
double sb = stddev.point;
double mn = mean.point / n;
double mg_min = mn / 2.;
double sg = (std::min)( mg_min / 4., sb / std::sqrt( n ) );
double sg2 = sg * sg;
double sb2 = sb * sb;
auto c_max = [n, mn, sb2, sg2]( double x ) -> double {
double k = mn - x;
double d = k * k;
double nd = n * d;
double k0 = -n * nd;
double k1 = sb2 - n * sg2 + nd;
double det = k1 * k1 - 4 * sg2 * k0;
return static_cast<int>( -2. * k0 /
( k1 + std::sqrt( det ) ) );
};
auto var_out = [n, sb2, sg2]( double c ) {
double nc = n - c;
return ( nc / n ) * ( sb2 - nc * sg2 );
};
return (std::min)( var_out( 1 ),
var_out(
(std::min)( c_max( 0. ),
c_max( mg_min ) ) ) ) /
sb2;
}
static double erf_inv( double x ) {
// Code accompanying the article "Approximating the erfinv
// function" in GPU Computing Gems, Volume 2
double w, p;
w = -log( ( 1.0 - x ) * ( 1.0 + x ) );
if ( w < 6.250000 ) {
w = w - 3.125000;
p = -3.6444120640178196996e-21;
p = -1.685059138182016589e-19 + p * w;
p = 1.2858480715256400167e-18 + p * w;
p = 1.115787767802518096e-17 + p * w;
p = -1.333171662854620906e-16 + p * w;
p = 2.0972767875968561637e-17 + p * w;
p = 6.6376381343583238325e-15 + p * w;
p = -4.0545662729752068639e-14 + p * w;
p = -8.1519341976054721522e-14 + p * w;
p = 2.6335093153082322977e-12 + p * w;
p = -1.2975133253453532498e-11 + p * w;
p = -5.4154120542946279317e-11 + p * w;
p = 1.051212273321532285e-09 + p * w;
p = -4.1126339803469836976e-09 + p * w;
p = -2.9070369957882005086e-08 + p * w;
p = 4.2347877827932403518e-07 + p * w;
p = -1.3654692000834678645e-06 + p * w;
p = -1.3882523362786468719e-05 + p * w;
p = 0.0001867342080340571352 + p * w;
p = -0.00074070253416626697512 + p * w;
p = -0.0060336708714301490533 + p * w;
p = 0.24015818242558961693 + p * w;
p = 1.6536545626831027356 + p * w;
} else if ( w < 16.000000 ) {
w = sqrt( w ) - 3.250000;
p = 2.2137376921775787049e-09;
p = 9.0756561938885390979e-08 + p * w;
p = -2.7517406297064545428e-07 + p * w;
p = 1.8239629214389227755e-08 + p * w;
p = 1.5027403968909827627e-06 + p * w;
p = -4.013867526981545969e-06 + p * w;
p = 2.9234449089955446044e-06 + p * w;
p = 1.2475304481671778723e-05 + p * w;
p = -4.7318229009055733981e-05 + p * w;
p = 6.8284851459573175448e-05 + p * w;
p = 2.4031110387097893999e-05 + p * w;
p = -0.0003550375203628474796 + p * w;
p = 0.00095328937973738049703 + p * w;
p = -0.0016882755560235047313 + p * w;
p = 0.0024914420961078508066 + p * w;
p = -0.0037512085075692412107 + p * w;
p = 0.005370914553590063617 + p * w;
p = 1.0052589676941592334 + p * w;
p = 3.0838856104922207635 + p * w;
} else {
w = sqrt( w ) - 5.000000;
p = -2.7109920616438573243e-11;
p = -2.5556418169965252055e-10 + p * w;
p = 1.5076572693500548083e-09 + p * w;
p = -3.7894654401267369937e-09 + p * w;
p = 7.6157012080783393804e-09 + p * w;
p = -1.4960026627149240478e-08 + p * w;
p = 2.9147953450901080826e-08 + p * w;
p = -6.7711997758452339498e-08 + p * w;
p = 2.2900482228026654717e-07 + p * w;
p = -9.9298272942317002539e-07 + p * w;
p = 4.5260625972231537039e-06 + p * w;
p = -1.9681778105531670567e-05 + p * w;
p = 7.5995277030017761139e-05 + p * w;
p = -0.00021503011930044477347 + p * w;
p = -0.00013871931833623122026 + p * w;
p = 1.0103004648645343977 + p * w;
p = 4.8499064014085844221 + p * w;
}
return p * x;
}
static double
standard_deviation( double const* first, double const* last ) {
auto m = Catch::Benchmark::Detail::mean( first, last );
double variance =
std::accumulate( first,
last,
0.,
[m]( double a, double b ) {
double diff = b - m;
return a + diff * diff;
} ) /
( last - first );
return std::sqrt( variance );
}
static sample jackknife( double ( *estimator )( double const*,
double const* ),
double* first,
double* last ) {
const auto second = first + 1;
sample results;
results.reserve( static_cast<size_t>( last - first ) );
for ( auto it = first; it != last; ++it ) {
std::iter_swap( it, first );
results.push_back( estimator( second, last ) );
}
return results;
}
double erf_inv(double x) {
// Code accompanying the article "Approximating the erfinv function" in GPU Computing Gems, Volume 2
double w, p;
w = -log((1.0 - x) * (1.0 + x));
if (w < 6.250000) {
w = w - 3.125000;
p = -3.6444120640178196996e-21;
p = -1.685059138182016589e-19 + p * w;
p = 1.2858480715256400167e-18 + p * w;
p = 1.115787767802518096e-17 + p * w;
p = -1.333171662854620906e-16 + p * w;
p = 2.0972767875968561637e-17 + p * w;
p = 6.6376381343583238325e-15 + p * w;
p = -4.0545662729752068639e-14 + p * w;
p = -8.1519341976054721522e-14 + p * w;
p = 2.6335093153082322977e-12 + p * w;
p = -1.2975133253453532498e-11 + p * w;
p = -5.4154120542946279317e-11 + p * w;
p = 1.051212273321532285e-09 + p * w;
p = -4.1126339803469836976e-09 + p * w;
p = -2.9070369957882005086e-08 + p * w;
p = 4.2347877827932403518e-07 + p * w;
p = -1.3654692000834678645e-06 + p * w;
p = -1.3882523362786468719e-05 + p * w;
p = 0.0001867342080340571352 + p * w;
p = -0.00074070253416626697512 + p * w;
p = -0.0060336708714301490533 + p * w;
p = 0.24015818242558961693 + p * w;
p = 1.6536545626831027356 + p * w;
} else if (w < 16.000000) {
w = sqrt(w) - 3.250000;
p = 2.2137376921775787049e-09;
p = 9.0756561938885390979e-08 + p * w;
p = -2.7517406297064545428e-07 + p * w;
p = 1.8239629214389227755e-08 + p * w;
p = 1.5027403968909827627e-06 + p * w;
p = -4.013867526981545969e-06 + p * w;
p = 2.9234449089955446044e-06 + p * w;
p = 1.2475304481671778723e-05 + p * w;
p = -4.7318229009055733981e-05 + p * w;
p = 6.8284851459573175448e-05 + p * w;
p = 2.4031110387097893999e-05 + p * w;
p = -0.0003550375203628474796 + p * w;
p = 0.00095328937973738049703 + p * w;
p = -0.0016882755560235047313 + p * w;
p = 0.0024914420961078508066 + p * w;
p = -0.0037512085075692412107 + p * w;
p = 0.005370914553590063617 + p * w;
p = 1.0052589676941592334 + p * w;
p = 3.0838856104922207635 + p * w;
} else {
w = sqrt(w) - 5.000000;
p = -2.7109920616438573243e-11;
p = -2.5556418169965252055e-10 + p * w;
p = 1.5076572693500548083e-09 + p * w;
p = -3.7894654401267369937e-09 + p * w;
p = 7.6157012080783393804e-09 + p * w;
p = -1.4960026627149240478e-08 + p * w;
p = 2.9147953450901080826e-08 + p * w;
p = -6.7711997758452339498e-08 + p * w;
p = 2.2900482228026654717e-07 + p * w;
p = -9.9298272942317002539e-07 + p * w;
p = 4.5260625972231537039e-06 + p * w;
p = -1.9681778105531670567e-05 + p * w;
p = 7.5995277030017761139e-05 + p * w;
p = -0.00021503011930044477347 + p * w;
p = -0.00013871931833623122026 + p * w;
p = 1.0103004648645343977 + p * w;
p = 4.8499064014085844221 + p * w;
}
return p * x;
}
double standard_deviation(std::vector<double>::iterator first, std::vector<double>::iterator last) {
auto m = Catch::Benchmark::Detail::mean(first, last);
double variance = std::accumulate( first,
last,
0.,
[m]( double a, double b ) {
double diff = b - m;
return a + diff * diff;
} ) /
( last - first );
return std::sqrt( variance );
}
}
} // namespace
} // namespace Detail
} // namespace Benchmark
} // namespace Catch
namespace Catch {
namespace Benchmark {
namespace Detail {
#if defined( __GNUC__ ) || defined( __clang__ )
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wfloat-equal"
#endif
bool directCompare( double lhs, double rhs ) { return lhs == rhs; }
#if defined( __GNUC__ ) || defined( __clang__ )
# pragma GCC diagnostic pop
#endif
double weighted_average_quantile(int k, int q, std::vector<double>::iterator first, std::vector<double>::iterator last) {
double weighted_average_quantile( int k,
int q,
double* first,
double* last ) {
auto count = last - first;
double idx = (count - 1) * k / static_cast<double>(q);
int j = static_cast<int>(idx);
double g = idx - j;
std::nth_element(first, first + j, last);
auto xj = first[j];
if ( directCompare( g, 0 ) ) {
if ( Catch::Detail::directCompare( g, 0 ) ) {
return xj;
}
@@ -161,6 +230,48 @@ namespace Catch {
return xj + g * (xj1 - xj);
}
OutlierClassification
classify_outliers( double const* first, double const* last ) {
std::vector<double> copy( first, last );
auto q1 = weighted_average_quantile( 1, 4, copy.data(), copy.data() + copy.size() );
auto q3 = weighted_average_quantile( 3, 4, copy.data(), copy.data() + copy.size() );
auto iqr = q3 - q1;
auto los = q1 - ( iqr * 3. );
auto lom = q1 - ( iqr * 1.5 );
auto him = q3 + ( iqr * 1.5 );
auto his = q3 + ( iqr * 3. );
OutlierClassification o;
for ( ; first != last; ++first ) {
const double t = *first;
if ( t < los ) {
++o.low_severe;
} else if ( t < lom ) {
++o.low_mild;
} else if ( t > his ) {
++o.high_severe;
} else if ( t > him ) {
++o.high_mild;
}
++o.samples_seen;
}
return o;
}
double mean( double const* first, double const* last ) {
auto count = last - first;
double sum = 0.;
while (first != last) {
sum += *first;
++first;
}
return sum / static_cast<double>(count);
}
double normal_cdf( double x ) {
return std::erfc( -x / std::sqrt( 2.0 ) ) / 2.0;
}
double erfc_inv(double x) {
return erf_inv(1.0 - x);
@@ -182,50 +293,77 @@ namespace Catch {
return result;
}
Estimate<double>
bootstrap( double confidence_level,
double* first,
double* last,
sample const& resample,
double ( *estimator )( double const*, double const* ) ) {
auto n_samples = last - first;
double outlier_variance(Estimate<double> mean, Estimate<double> stddev, int n) {
double sb = stddev.point;
double mn = mean.point / n;
double mg_min = mn / 2.;
double sg = (std::min)(mg_min / 4., sb / std::sqrt(n));
double sg2 = sg * sg;
double sb2 = sb * sb;
double point = estimator( first, last );
// Degenerate case with a single sample
if ( n_samples == 1 )
return { point, point, point, confidence_level };
auto c_max = [n, mn, sb2, sg2](double x) -> double {
double k = mn - x;
double d = k * k;
double nd = n * d;
double k0 = -n * nd;
double k1 = sb2 - n * sg2 + nd;
double det = k1 * k1 - 4 * sg2 * k0;
return static_cast<int>(-2. * k0 / (k1 + std::sqrt(det)));
sample jack = jackknife( estimator, first, last );
double jack_mean =
mean( jack.data(), jack.data() + jack.size() );
double sum_squares = 0, sum_cubes = 0;
for ( double x : jack ) {
auto difference = jack_mean - x;
auto square = difference * difference;
auto cube = square * difference;
sum_squares += square;
sum_cubes += cube;
}
double accel = sum_cubes / ( 6 * std::pow( sum_squares, 1.5 ) );
long n = static_cast<long>( resample.size() );
double prob_n =
std::count_if( resample.begin(),
resample.end(),
[point]( double x ) { return x < point; } ) /
static_cast<double>( n );
// degenerate case with uniform samples
if ( Catch::Detail::directCompare( prob_n, 0. ) ) {
return { point, point, point, confidence_level };
}
double bias = normal_quantile( prob_n );
double z1 = normal_quantile( ( 1. - confidence_level ) / 2. );
auto cumn = [n]( double x ) -> long {
return std::lround( normal_cdf( x ) *
static_cast<double>( n ) );
};
auto var_out = [n, sb2, sg2](double c) {
double nc = n - c;
return (nc / n) * (sb2 - nc * sg2);
auto a = [bias, accel]( double b ) {
return bias + b / ( 1. - accel * b );
};
double b1 = bias + z1;
double b2 = bias - z1;
double a1 = a( b1 );
double a2 = a( b2 );
auto lo = static_cast<size_t>( (std::max)( cumn( a1 ), 0l ) );
auto hi =
static_cast<size_t>( (std::min)( cumn( a2 ), n - 1 ) );
return (std::min)(var_out(1), var_out((std::min)(c_max(0.), c_max(mg_min)))) / sb2;
return { point, resample[lo], resample[hi], confidence_level };
}
bootstrap_analysis analyse_samples(double confidence_level, unsigned int n_resamples, std::vector<double>::iterator first, std::vector<double>::iterator last) {
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION
CATCH_INTERNAL_SUPPRESS_GLOBALS_WARNINGS
static std::random_device entropy;
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION
auto n = static_cast<int>(last - first); // seriously, one can't use integral types without hell in C++
auto mean = &Detail::mean<std::vector<double>::iterator>;
bootstrap_analysis analyse_samples(double confidence_level,
unsigned int n_resamples,
double* first,
double* last) {
auto mean = &Detail::mean;
auto stddev = &standard_deviation;
#if defined(CATCH_CONFIG_USE_ASYNC)
auto Estimate = [=](double(*f)(std::vector<double>::iterator, std::vector<double>::iterator)) {
auto seed = entropy();
auto Estimate = [=](double(*f)(double const*, double const*)) {
std::random_device rd;
auto seed = rd();
return std::async(std::launch::async, [=] {
std::mt19937 rng(seed);
SimplePcg32 rng( seed );
auto resampled = resample(rng, n_resamples, first, last, f);
return bootstrap(confidence_level, first, last, resampled, f);
});
@@ -237,9 +375,10 @@ namespace Catch {
auto mean_estimate = mean_future.get();
auto stddev_estimate = stddev_future.get();
#else
auto Estimate = [=](double(*f)(std::vector<double>::iterator, std::vector<double>::iterator)) {
auto seed = entropy();
std::mt19937 rng(seed);
auto Estimate = [=](double(*f)(double const* , double const*)) {
std::random_device rd;
auto seed = rd();
SimplePcg32 rng( seed );
auto resampled = resample(rng, n_resamples, first, last, f);
return bootstrap(confidence_level, first, last, resampled, f);
};
@@ -248,6 +387,7 @@ namespace Catch {
auto stddev_estimate = Estimate(stddev);
#endif // CATCH_USE_ASYNC
auto n = static_cast<int>(last - first); // seriously, one can't use integral types without hell in C++
double outlier_variance = Detail::outlier_variance(mean_estimate, stddev_estimate, n);
return { mean_estimate, stddev_estimate, outlier_variance };

View File

@@ -13,122 +13,35 @@
#include <catch2/benchmark/catch_estimate.hpp>
#include <catch2/benchmark/catch_outlier_classification.hpp>
#include <algorithm>
#include <vector>
#include <numeric>
#include <tuple>
#include <cmath>
namespace Catch {
namespace Benchmark {
namespace Detail {
using sample = std::vector<double>;
// Used when we know we want == comparison of two doubles
// to centralize warning suppression
bool directCompare( double lhs, double rhs );
double weighted_average_quantile( int k,
int q,
double* first,
double* last );
double weighted_average_quantile(int k, int q, std::vector<double>::iterator first, std::vector<double>::iterator last);
OutlierClassification
classify_outliers( double const* first, double const* last );
template <typename Iterator>
OutlierClassification classify_outliers(Iterator first, Iterator last) {
std::vector<double> copy(first, last);
double mean( double const* first, double const* last );
auto q1 = weighted_average_quantile(1, 4, copy.begin(), copy.end());
auto q3 = weighted_average_quantile(3, 4, copy.begin(), copy.end());
auto iqr = q3 - q1;
auto los = q1 - (iqr * 3.);
auto lom = q1 - (iqr * 1.5);
auto him = q3 + (iqr * 1.5);
auto his = q3 + (iqr * 3.);
OutlierClassification o;
for (; first != last; ++first) {
auto&& t = *first;
if (t < los) ++o.low_severe;
else if (t < lom) ++o.low_mild;
else if (t > his) ++o.high_severe;
else if (t > him) ++o.high_mild;
++o.samples_seen;
}
return o;
}
template <typename Iterator>
double mean(Iterator first, Iterator last) {
auto count = last - first;
double sum = std::accumulate(first, last, 0.);
return sum / static_cast<double>(count);
}
template <typename Estimator, typename Iterator>
sample jackknife(Estimator&& estimator, Iterator first, Iterator last) {
auto n = static_cast<size_t>(last - first);
auto second = first;
++second;
sample results;
results.reserve(n);
for (auto it = first; it != last; ++it) {
std::iter_swap(it, first);
results.push_back(estimator(second, last));
}
return results;
}
inline double normal_cdf(double x) {
return std::erfc(-x / std::sqrt(2.0)) / 2.0;
}
double normal_cdf( double x );
double erfc_inv(double x);
double normal_quantile(double p);
template <typename Iterator, typename Estimator>
Estimate<double> bootstrap(double confidence_level, Iterator first, Iterator last, sample const& resample, Estimator&& estimator) {
auto n_samples = last - first;
double point = estimator(first, last);
// Degenerate case with a single sample
if (n_samples == 1) return { point, point, point, confidence_level };
sample jack = jackknife(estimator, first, last);
double jack_mean = mean(jack.begin(), jack.end());
double sum_squares, sum_cubes;
std::tie(sum_squares, sum_cubes) = std::accumulate(jack.begin(), jack.end(), std::make_pair(0., 0.), [jack_mean](std::pair<double, double> sqcb, double x) -> std::pair<double, double> {
auto d = jack_mean - x;
auto d2 = d * d;
auto d3 = d2 * d;
return { sqcb.first + d2, sqcb.second + d3 };
});
double accel = sum_cubes / (6 * std::pow(sum_squares, 1.5));
long n = static_cast<long>(resample.size());
double prob_n = std::count_if(resample.begin(), resample.end(), [point](double x) { return x < point; }) / static_cast<double>(n);
// degenerate case with uniform samples
if ( directCompare( prob_n, 0. ) ) {
return { point, point, point, confidence_level };
}
double bias = normal_quantile(prob_n);
double z1 = normal_quantile((1. - confidence_level) / 2.);
auto cumn = [n]( double x ) -> long {
return std::lround( normal_cdf( x ) * static_cast<double>(n) );
};
auto a = [bias, accel](double b) { return bias + b / (1. - accel * b); };
double b1 = bias + z1;
double b2 = bias - z1;
double a1 = a(b1);
double a2 = a(b2);
auto lo = static_cast<size_t>((std::max)(cumn(a1), 0l));
auto hi = static_cast<size_t>((std::min)(cumn(a2), n - 1));
return { point, resample[lo], resample[hi], confidence_level };
}
double outlier_variance(Estimate<double> mean, Estimate<double> stddev, int n);
Estimate<double>
bootstrap( double confidence_level,
double* first,
double* last,
sample const& resample,
double ( *estimator )( double const*, double const* ) );
struct bootstrap_analysis {
Estimate<double> mean;
@@ -136,7 +49,10 @@ namespace Catch {
double outlier_variance;
};
bootstrap_analysis analyse_samples(double confidence_level, unsigned int n_resamples, std::vector<double>::iterator first, std::vector<double>::iterator last);
bootstrap_analysis analyse_samples(double confidence_level,
unsigned int n_resamples,
double* first,
double* last);
} // namespace Detail
} // namespace Benchmark
} // namespace Catch

View File

@@ -17,14 +17,14 @@
namespace Catch {
namespace Benchmark {
template <typename Duration, typename Result>
template <typename Result>
struct Timing {
Duration elapsed;
IDuration elapsed;
Result result;
int iterations;
};
template <typename Clock, typename Func, typename... Args>
using TimingOf = Timing<ClockDuration<Clock>, Detail::CompleteType_t<FunctionReturnType<Func, Args...>>>;
template <typename Func, typename... Args>
using TimingOf = Timing<Detail::CompleteType_t<FunctionReturnType<Func, Args...>>>;
} // namespace Benchmark
} // namespace Catch

View File

@@ -54,6 +54,8 @@
#include <catch2/internal/catch_compiler_capabilities.hpp>
#include <catch2/internal/catch_config_android_logwrite.hpp>
#include <catch2/internal/catch_config_counter.hpp>
#include <catch2/internal/catch_config_prefix_messages.hpp>
#include <catch2/internal/catch_config_static_analysis_support.hpp>
#include <catch2/internal/catch_config_uncaught_exceptions.hpp>
#include <catch2/internal/catch_config_wchar.hpp>
#include <catch2/internal/catch_console_colour.hpp>
@@ -72,6 +74,7 @@
#include <catch2/internal/catch_getenv.hpp>
#include <catch2/internal/catch_is_permutation.hpp>
#include <catch2/internal/catch_istream.hpp>
#include <catch2/internal/catch_jsonwriter.hpp>
#include <catch2/internal/catch_lazy_expr.hpp>
#include <catch2/internal/catch_leak_detector.hpp>
#include <catch2/internal/catch_list.hpp>
@@ -86,7 +89,10 @@
#include <catch2/internal/catch_platform.hpp>
#include <catch2/internal/catch_polyfills.hpp>
#include <catch2/internal/catch_preprocessor.hpp>
#include <catch2/internal/catch_preprocessor_internal_stringify.hpp>
#include <catch2/internal/catch_preprocessor_remove_parens.hpp>
#include <catch2/internal/catch_random_floating_point_helpers.hpp>
#include <catch2/internal/catch_random_integer_helpers.hpp>
#include <catch2/internal/catch_random_number_generator.hpp>
#include <catch2/internal/catch_random_seed_generation.hpp>
#include <catch2/internal/catch_reporter_registry.hpp>
@@ -111,10 +117,13 @@
#include <catch2/internal/catch_test_failure_exception.hpp>
#include <catch2/internal/catch_test_macro_impl.hpp>
#include <catch2/internal/catch_test_registry.hpp>
#include <catch2/internal/catch_test_run_info.hpp>
#include <catch2/internal/catch_test_spec_parser.hpp>
#include <catch2/internal/catch_textflow.hpp>
#include <catch2/internal/catch_to_string.hpp>
#include <catch2/internal/catch_uncaught_exceptions.hpp>
#include <catch2/internal/catch_uniform_floating_point_distribution.hpp>
#include <catch2/internal/catch_uniform_integer_distribution.hpp>
#include <catch2/internal/catch_unique_name.hpp>
#include <catch2/internal/catch_unique_ptr.hpp>
#include <catch2/internal/catch_void_type.hpp>

View File

@@ -70,10 +70,10 @@ namespace Catch {
}
namespace literals {
Approx operator "" _a(long double val) {
Approx operator ""_a(long double val) {
return Approx(val);
}
Approx operator "" _a(unsigned long long val) {
Approx operator ""_a(unsigned long long val) {
return Approx(val);
}
} // end namespace literals

View File

@@ -105,7 +105,7 @@ namespace Catch {
elem = trim(elem);
}
// Insert the default reporter if user hasn't asked for a specfic one
// Insert the default reporter if user hasn't asked for a specific one
if ( m_data.reporterSpecifications.empty() ) {
m_data.reporterSpecifications.push_back( {
#if defined( CATCH_CONFIG_DEFAULT_REPORTER )

View File

@@ -37,7 +37,11 @@ namespace Catch {
}
Capturer::Capturer( StringRef macroName, SourceLineInfo const& lineInfo, ResultWas::OfType resultType, StringRef names ) {
Capturer::Capturer( StringRef macroName,
SourceLineInfo const& lineInfo,
ResultWas::OfType resultType,
StringRef names ):
m_resultCapture( getResultCapture() ) {
auto trimmed = [&] (size_t start, size_t end) {
while (names[start] == ',' || isspace(static_cast<unsigned char>(names[start]))) {
++start;

View File

@@ -8,12 +8,13 @@
#ifndef CATCH_MESSAGE_HPP_INCLUDED
#define CATCH_MESSAGE_HPP_INCLUDED
#include <catch2/internal/catch_config_prefix_messages.hpp>
#include <catch2/internal/catch_result_type.hpp>
#include <catch2/internal/catch_reusable_string_stream.hpp>
#include <catch2/internal/catch_stream_end_stop.hpp>
#include <catch2/internal/catch_message_info.hpp>
#include <catch2/interfaces/catch_interfaces_capture.hpp>
#include <catch2/catch_tostring.hpp>
#include <catch2/interfaces/catch_interfaces_capture.hpp>
#include <string>
#include <vector>
@@ -21,6 +22,7 @@
namespace Catch {
struct SourceLineInfo;
class IResultCapture;
struct MessageStream {
@@ -61,7 +63,7 @@ namespace Catch {
class Capturer {
std::vector<MessageInfo> m_messages;
IResultCapture& m_resultCapture = getResultCapture();
IResultCapture& m_resultCapture;
size_t m_captured = 0;
public:
Capturer( StringRef macroName, SourceLineInfo const& lineInfo, ResultWas::OfType resultType, StringRef names );
@@ -112,28 +114,28 @@ namespace Catch {
Catch::getResultCapture().emplaceUnscopedMessage( Catch::MessageBuilder( macroName##_catch_sr, CATCH_INTERNAL_LINEINFO, Catch::ResultWas::Info ) << log )
#if defined(CATCH_CONFIG_PREFIX_ALL) && !defined(CATCH_CONFIG_DISABLE)
#if defined(CATCH_CONFIG_PREFIX_MESSAGES) && !defined(CATCH_CONFIG_DISABLE)
#define CATCH_INFO( msg ) INTERNAL_CATCH_INFO( "CATCH_INFO", msg )
#define CATCH_UNSCOPED_INFO( msg ) INTERNAL_CATCH_UNSCOPED_INFO( "CATCH_UNSCOPED_INFO", msg )
#define CATCH_WARN( msg ) INTERNAL_CATCH_MSG( "CATCH_WARN", Catch::ResultWas::Warning, Catch::ResultDisposition::ContinueOnFailure, msg )
#define CATCH_CAPTURE( ... ) INTERNAL_CATCH_CAPTURE( INTERNAL_CATCH_UNIQUE_NAME(capturer), "CATCH_CAPTURE", __VA_ARGS__ )
#elif defined(CATCH_CONFIG_PREFIX_ALL) && defined(CATCH_CONFIG_DISABLE)
#elif defined(CATCH_CONFIG_PREFIX_MESSAGES) && defined(CATCH_CONFIG_DISABLE)
#define CATCH_INFO( msg ) (void)(0)
#define CATCH_UNSCOPED_INFO( msg ) (void)(0)
#define CATCH_WARN( msg ) (void)(0)
#define CATCH_CAPTURE( ... ) (void)(0)
#elif !defined(CATCH_CONFIG_PREFIX_ALL) && !defined(CATCH_CONFIG_DISABLE)
#elif !defined(CATCH_CONFIG_PREFIX_MESSAGES) && !defined(CATCH_CONFIG_DISABLE)
#define INFO( msg ) INTERNAL_CATCH_INFO( "INFO", msg )
#define UNSCOPED_INFO( msg ) INTERNAL_CATCH_UNSCOPED_INFO( "UNSCOPED_INFO", msg )
#define WARN( msg ) INTERNAL_CATCH_MSG( "WARN", Catch::ResultWas::Warning, Catch::ResultDisposition::ContinueOnFailure, msg )
#define CAPTURE( ... ) INTERNAL_CATCH_CAPTURE( INTERNAL_CATCH_UNIQUE_NAME(capturer), "CAPTURE", __VA_ARGS__ )
#elif !defined(CATCH_CONFIG_PREFIX_ALL) && defined(CATCH_CONFIG_DISABLE)
#elif !defined(CATCH_CONFIG_PREFIX_MESSAGES) && defined(CATCH_CONFIG_DISABLE)
#define INFO( msg ) (void)(0)
#define UNSCOPED_INFO( msg ) (void)(0)

View File

@@ -20,6 +20,9 @@
#include <catch2/internal/catch_noncopyable.hpp>
#include <catch2/interfaces/catch_interfaces_reporter_factory.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <catch2/internal/catch_reporter_registry.hpp>
#include <exception>
namespace Catch {
@@ -31,7 +34,7 @@ namespace Catch {
public: // IRegistryHub
RegistryHub() = default;
IReporterRegistry const& getReporterRegistry() const override {
ReporterRegistry const& getReporterRegistry() const override {
return m_reporterRegistry;
}
ITestCaseRegistry const& getTestCaseRegistry() const override {

View File

@@ -13,13 +13,13 @@
#include <catch2/internal/catch_run_context.hpp>
#include <catch2/catch_test_spec.hpp>
#include <catch2/catch_version.hpp>
#include <catch2/interfaces/catch_interfaces_reporter.hpp>
#include <catch2/internal/catch_startup_exception_registry.hpp>
#include <catch2/internal/catch_sharding.hpp>
#include <catch2/internal/catch_test_case_registry_impl.hpp>
#include <catch2/internal/catch_textflow.hpp>
#include <catch2/internal/catch_windows_h_proxy.hpp>
#include <catch2/reporters/catch_reporter_multi.hpp>
#include <catch2/interfaces/catch_interfaces_reporter_registry.hpp>
#include <catch2/internal/catch_reporter_registry.hpp>
#include <catch2/interfaces/catch_interfaces_reporter_factory.hpp>
#include <catch2/internal/catch_move_and_forward.hpp>
#include <catch2/internal/catch_stdstreams.hpp>
@@ -27,6 +27,7 @@
#include <algorithm>
#include <cassert>
#include <exception>
#include <iomanip>
#include <set>

View File

@@ -9,6 +9,7 @@
#include <catch2/internal/catch_enforce.hpp>
#include <catch2/internal/catch_string_manip.hpp>
#include <catch2/internal/catch_case_insensitive_comparisons.hpp>
#include <catch2/internal/catch_test_registry.hpp>
#include <cassert>
#include <cctype>
@@ -139,12 +140,20 @@ namespace Catch {
for (size_t idx = 0; idx < originalTags.size(); ++idx) {
auto c = originalTags[idx];
if (c == '[') {
assert(!inTag);
CATCH_ENFORCE(
!inTag,
"Found '[' inside a tag while registering test case '"
<< _nameAndTags.name << "' at " << _lineInfo );
inTag = true;
tagStart = idx;
}
if (c == ']') {
assert(inTag);
CATCH_ENFORCE(
inTag,
"Found unmatched ']' while registering test case '"
<< _nameAndTags.name << "' at " << _lineInfo );
inTag = false;
tagEnd = idx;
assert(tagStart < tagEnd);
@@ -153,7 +162,11 @@ namespace Catch {
// it over to backing storage and actually reference the
// backing storage in the saved tags
StringRef tagStr = originalTags.substr(tagStart+1, tagEnd - tagStart - 1);
CATCH_ENFORCE(!tagStr.empty(), "Empty tags are not allowed");
CATCH_ENFORCE( !tagStr.empty(),
"Found an empty tag while registering test case '"
<< _nameAndTags.name << "' at "
<< _lineInfo );
enforceNotReservedTag(tagStr, lineInfo);
properties |= parseSpecialTag(tagStr);
// When copying a tag to the backing storage, we need to
@@ -167,8 +180,12 @@ namespace Catch {
// the tags.
internalAppendTag(tagStr);
}
(void)inTag; // Silence "set-but-unused" warning in release mode.
}
CATCH_ENFORCE( !inTag,
"Found an unclosed tag while registering test case '"
<< _nameAndTags.name << "' at " << _lineInfo );
// Add [.] if relevant
if (isHidden()) {
internalAppendTag("."_sr);

View File

@@ -8,10 +8,10 @@
#ifndef CATCH_TEST_CASE_INFO_HPP_INCLUDED
#define CATCH_TEST_CASE_INFO_HPP_INCLUDED
#include <catch2/interfaces/catch_interfaces_test_invoker.hpp>
#include <catch2/internal/catch_source_line_info.hpp>
#include <catch2/internal/catch_noncopyable.hpp>
#include <catch2/internal/catch_stringref.hpp>
#include <catch2/internal/catch_test_registry.hpp>
#include <catch2/internal/catch_unique_ptr.hpp>
@@ -44,6 +44,7 @@ namespace Catch {
};
class ITestInvoker;
struct NameAndTags;
enum class TestCaseProperties : uint8_t {
None = 0,

View File

@@ -6,6 +6,8 @@
// SPDX-License-Identifier: BSL-1.0
#include <catch2/catch_test_spec.hpp>
#include <catch2/interfaces/catch_interfaces_testcase.hpp>
#include <catch2/internal/catch_test_case_registry_impl.hpp>
#include <catch2/internal/catch_reusable_string_stream.hpp>
#include <catch2/internal/catch_string_manip.hpp>
#include <catch2/catch_test_case_info.hpp>
@@ -106,16 +108,18 @@ namespace Catch {
return std::any_of( m_filters.begin(), m_filters.end(), [&]( Filter const& f ){ return f.matches( testCase ); } );
}
TestSpec::Matches TestSpec::matchesByFilter( std::vector<TestCaseHandle> const& testCases, IConfig const& config ) const
{
Matches matches( m_filters.size() );
std::transform( m_filters.begin(), m_filters.end(), matches.begin(), [&]( Filter const& filter ){
TestSpec::Matches TestSpec::matchesByFilter( std::vector<TestCaseHandle> const& testCases, IConfig const& config ) const {
Matches matches;
matches.reserve( m_filters.size() );
for ( auto const& filter : m_filters ) {
std::vector<TestCaseHandle const*> currentMatches;
for( auto const& test : testCases )
if( isThrowSafe( test, config ) && filter.matches( test.getTestCaseInfo() ) )
for ( auto const& test : testCases )
if ( isThrowSafe( test, config ) &&
filter.matches( test.getTestCaseInfo() ) )
currentMatches.emplace_back( &test );
return FilterMatch{ extractFilterName(filter), currentMatches };
} );
matches.push_back(
FilterMatch{ extractFilterName( filter ), currentMatches } );
}
return matches;
}

View File

@@ -116,7 +116,6 @@ namespace Catch {
} // namespace Detail
// If we decide for C++14, change these to enable_if_ts
template <typename T, typename = void>
struct StringMaker {
template <typename Fake = T>
@@ -399,6 +398,12 @@ namespace Catch {
}
}
};
template <>
struct StringMaker<std::nullopt_t> {
static std::string convert(const std::nullopt_t&) {
return "{ }";
}
};
}
#endif // CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER

View File

@@ -0,0 +1,20 @@
// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)
// SPDX-License-Identifier: BSL-1.0
#include <catch2/catch_translate_exception.hpp>
#include <catch2/interfaces/catch_interfaces_registry_hub.hpp>
namespace Catch {
namespace Detail {
void registerTranslatorImpl(
Detail::unique_ptr<IExceptionTranslator>&& translator ) {
getMutableRegistryHub().registerTranslator(
CATCH_MOVE( translator ) );
}
} // namespace Detail
} // namespace Catch

View File

@@ -15,6 +15,10 @@
#include <exception>
namespace Catch {
namespace Detail {
void registerTranslatorImpl(
Detail::unique_ptr<IExceptionTranslator>&& translator );
}
class ExceptionTranslatorRegistrar {
template<typename T>
@@ -48,9 +52,9 @@ namespace Catch {
public:
template<typename T>
ExceptionTranslatorRegistrar( std::string(*translateFunction)( T const& ) ) {
getMutableRegistryHub().registerTranslator(
Detail::make_unique<ExceptionTranslator<T>>(translateFunction)
);
Detail::registerTranslatorImpl(
Detail::make_unique<ExceptionTranslator<T>>(
translateFunction ) );
}
};

View File

@@ -169,9 +169,18 @@
#endif
#cmakedefine CATCH_CONFIG_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT
#cmakedefine CATCH_CONFIG_NO_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT
#if defined( CATCH_CONFIG_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT ) && \
defined( CATCH_CONFIG_NO_EXPERIMENTAL_STATIC_ANALYSIS_SUPPORT )
# error Cannot force STATIC_ANALYSIS_SUPPORT to both ON and OFF
#endif
// ------
// Simple toggle defines
// their value is never used and they cannot be overriden
// their value is never used and they cannot be overridden
// ------
@@ -189,6 +198,7 @@
#cmakedefine CATCH_CONFIG_FAST_COMPILE
#cmakedefine CATCH_CONFIG_NOSTDOUT
#cmakedefine CATCH_CONFIG_PREFIX_ALL
#cmakedefine CATCH_CONFIG_PREFIX_MESSAGES
#cmakedefine CATCH_CONFIG_WINDOWS_CRTDBG
#cmakedefine CATCH_CONFIG_SHARED_LIBRARY

View File

@@ -36,7 +36,7 @@ namespace Catch {
}
Version const& libraryVersion() {
static Version version( 3, 3, 2, "", 0 );
static Version version( 3, 5, 0, "", 0 );
return version;
}

View File

@@ -9,7 +9,7 @@
#define CATCH_VERSION_MACROS_HPP_INCLUDED
#define CATCH_VERSION_MAJOR 3
#define CATCH_VERSION_MINOR 3
#define CATCH_VERSION_PATCH 2
#define CATCH_VERSION_MINOR 5
#define CATCH_VERSION_PATCH 0
#endif // CATCH_VERSION_MACROS_HPP_INCLUDED

Some files were not shown because too many files have changed in this diff Show More