This requires a bunch of different changes across the reporter
subsystem.
* We need to handle multiple reporters and their differing
preferences in `ListeningReporter`, e.g. what to do when
we mix reporters that capture and don't capture stdout.
* We need to change how the reporter is given output and
how we parse reporter's output destination from CLI.
* Approval tests need to handle multireporter option
* Add buildAndTest.cmd for Windows users
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
A new flag, `--allow-running-no-tests` was added to override this
behaviour if exit code of 0 was desired.
This change also made `-w NoTests` obsolete, and so it has been
removed.
Previously registration was case preserving, but lookup used
lowercased reporter name, so a reporter whose name contained
upper case character could not be requested by the user.
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.
Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.
Closes#2131
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.
Closes#2152
`FetchContent` doesn't include `contrib` directory as part of `CMAKE_MODULE_PATH`. This results into `include(Catch)` to fail. This patch just updates the documentation describing how to do include the path, so the new users don't have to figure this out themselves.
Source: https://github.com/catchorg/Catch2/issues/2103#issuecomment-730626324
Parsing C++ with regex in CMake is error prone and regularly leads to silently
dropped (not run) test cases.
Going forward the function `catch_discover_tests` from `contrib/CMake.cmake`
should be used.
For more information see https://github.com/catchorg/Catch2/issues/2092#issuecomment-747342765
* Update cmake-integration.md
CMake related, mainly more modern and provide an executable to be correct
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
The base was also renamed from `TestEventListenerBase` to
`EventListenerBase`, and modified to derive directly from the
reporter interface, rather than deriving from `StreamingReporterBase`.
A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.
When there are many tests, this produces output spam which makes it hard
to find the test failure output. Nevertheless, it is helpful to be
informed of tests which are unusually slow.
Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed. This allows slow
tests to be visible without mentioning every test.
There are some examples on issue #850 of using this feature, but they
are not easily found from the documentation. Adding them here as an
example makes them more findable and ensures they keep working if the
API changes.
The two changes are
`catch_matchers_templates` -> `catch_matchers_templated` and
`catch_matchers_generic` -> `catch_matchers_predicate`. The former
is mostly cosmetic, but the second was previously significantly
misleading, and as the library is now to be consumed by including
specific headers, this needed to be fixed.
`SizeIs` can accept both `size_t` and a matcher. In the first case,
it checks whether the size of the range is equal to specified size.
In the second case, it checks whether the provided matcher accepts
the size of the range.
In general, for Catch2 v3 we are making virtual types `final`,
unless they were explicitly designed to be derived-from.
`ListeningReporter` is definitely not designed to be derived-from.
This commit also forbids composing lvalues of composed matchers, as
per previous deprecation notice. I do not expect this to be contentious
in practice, because there was a bug in that usage for years, and
nobody complained.
Given that in the 2 or so years that matchers are thing nobody complained,
it seems that people do not actually write this sort of code, and the
possibility will be removed in v3. However, to avoid correctness bugs,
we will have to support this weird code in v2.
Now that Catch2 is a proper library, we can always build the full
library (comparatively minor slowdown) and the user can avoid
including benchmarking headers to avoid the compilation slowdown.
Now that the recommended distribution and usage method is proper
library, users can just avoid including the matcher headers to get
basically the same effect.
Users can still write a description for their sections, but it will
no longer be saved as part of the `SectionInfo` struct. This ability
has also been added to the documentation.
Closes#1319
This also required some refactoring of how the pattern matching
works. This means that the concepts of include and exclude patterns
are no longer unified, with exclusion patterns working as just
negation of an inclusion patterns (which led to including hidden
tags by default, as they did not match the exclusion), but rather
both include and exclude patterns are handled separately.
The new logic is that given a filter and a test case, the test
case must match _all_ include patterns and _no_ exclude patterns
to be included by the filter. Furthermore, if the test case is
hidden, then the filter must have at least one include pattern
for the test case to be used.
Closes#1184
Previously it returned the sum of listed things because ???. This
was completely useless and in many ways actively counterproductive
because of the success/failure conventions around exit codes.
Closes#1410
