Commit Graph

31 Commits

Author SHA1 Message Date
Martin Hořeňovský
c65e5b6514
Add 'uses-python' label to tests using Python 2022-04-24 13:31:28 +02:00
Lukasz Okraszewski
cb551b4f6d
Add default reporter for Bazel integration (#2399)
When the added Bazel configuration flag is enabled,
a default JUnit reporter will be added if the XML
envrioment variable is defined.
Fix include paths for generated config header.
Enable Bazel config by default when building with
Bazel.


Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-04-08 18:01:59 +02:00
Martin Hořeňovský
423e1d2ebb
Integrate the new reporter specs into Catch2
This means that the CLI interface now uses the new key-value oriented
reporter spec, the common reporter base creates the colour implementation
based on the reporter-specific configuration, and it also stores the
custom configuration options for each reporter instance.

Closes #339 as it allows per-reporter forcing of ansi colour codes.
2022-04-07 12:50:08 +02:00
Martin Hořeňovský
e53a75b425
Add integration test for listener-reporter event ordering 2022-01-28 00:00:52 +01:00
Martin Hořeňovský
899554bff2 Add test for handling of reporter's assertion preferences 2022-01-01 14:15:44 +01:00
Martin Hořeňovský
b4efa4751a Add test for handling of Listeners requesting stdout capture 2022-01-01 14:13:48 +01:00
Martin Hořeňovský
6b55f5d780
Don't send full stdout so far to testCasePartialEnded event 2022-01-01 12:21:41 +01:00
Martin Hořeňovský
9f2dca5384
Exit with non-0 return code if no tests were run
A new flag, `--allow-running-no-tests` was added to override this
behaviour if exit code of 0 was desired.

This change also made `-w NoTests` obsolete, and so it has been
removed.
2021-12-18 20:50:00 +01:00
Martin Hořeňovský
d012735c6e
Add test for runtime behaviour of STATIC_CHECK 2021-11-15 20:06:29 +01:00
Martin Hořeňovský
62d8913d67
Cumulative reporter base records benchmark results 2021-11-09 11:52:50 +01:00
Martin Hořeňovský
8780425385
Make reporter lookup case insensitive, registration case preserving
Previously registration was case preserving, but lookup used
lowercased reporter name, so a reporter whose name contained
upper case character could not be requested by the user.
2021-11-09 11:50:03 +01:00
Ben Dunkin
3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský
fb4153e05e
Multiple tests can have same name as long as their tags differ
This change also changes it so that test case macros using a
class name can have same name **and** tags as long as the
used class name differs.

Closes #1915
Closes #1999
2021-09-25 21:37:03 +02:00
Martin Hořeňovský
10fb93cce8
Add new reporter event called for each test case enter/exit
This means that e.g. for `TEST_CASE` with two sibling `SECTION`s
the event will fire twice, because the `TEST_CASE` will be entered
twice.

Closes #2107 (the event mentioned there already exists, but this
is its counterpart that we also want to provide to users)
2021-09-14 13:19:45 +02:00
Martin Hořeňovský
3afea8128a
Increase timing window for min duration tests to 1s
This test tends to be brittle on Mac CI machines, which are
heavily loaded and bursty. Since the tests are only run as part
of the "extra tests" test set, this increase should not have
a significant impact on the total duration of CI runs.
2021-02-20 23:09:02 +01:00
Martin Hořeňovský
19cbdebb0e
Lenghten timing threshold in --min-duration integration test 2020-11-28 21:00:09 +01:00
Martin Hořeňovský
60dfec559f
Provide 1 .hpp + 1 .cpp distribution of Catch2
This commits also adds a script that does the amalgamation of headers
and .cpp files into the distributable version, removes the old
`generateSingleHeader` script, and also adds a very simple compilation
test for the amalgamated distribution.
2020-09-09 13:02:50 +02:00
Martin Hořeňovský
2d4f8ac8e6
Disable problematic test on Windows 2020-09-06 20:41:58 +02:00
Martin Hořeňovský
90d6fd849e
Increase tolerances in --min-duration tests
The underpowered and oversubscribed CI servers are hell.
2020-07-26 10:48:03 +02:00
Martin Hořeňovský
13917c44b4
--min-duration is overriden by -d no 2020-07-26 10:48:01 +02:00
Martin Hořeňovský
e6d947f6d4
Refactor tests for duration reporting threshold 2020-07-26 10:47:58 +02:00
Martin Hořeňovský
c086746cc9
Use main-as-static-lib in ExtraTests 2020-05-03 19:21:58 +02:00
Martin Hořeňovský
6e270958a2
Add development build option to CMake and enable it on CI
Development build enables warnings and and `Werror` or equivalent.
2020-02-26 16:07:26 +01:00
Martin Hořeňovský
a1be19aa1b
Remove the separately compiled test for Benchmarking Macros
Thanks to the changes to compilation model, the tests for benchmarking
macros have been made part of the normal test run. This means that
the only purpose these separately compiled tests served was to waste
CI time.
2020-02-18 13:48:34 +01:00
Martin Hořeňovský
c745adb81c
Temporarily disable the test for DebugBreakMacro
The change to static library means that the new implementation needs
to be somewhat different, and I do not want to fix it right now.
2020-02-18 13:43:37 +01:00
Martin Hořeňovský
89f18f15ca
Add a test for custom debug break macros
See #1846
2020-02-16 15:47:00 +01:00
Martin Hořeňovský
cd7d7a1c67
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle
Now that Catch2 is a proper library, we can always build the full
library (comparatively minor slowdown) and the user can avoid
including benchmarking headers to avoid the compilation slowdown.
2020-02-06 11:36:46 +01:00
Martin Hořeňovský
a3ffc20f57
Provide CTest a hint on test costs (and thus ordering)
When running tests in parallel, CTest runs the tests in decreasing
order of cost (time required), to get the largest speed up from
parallelism. However, the initial cost estimates for all tests are
0, and they are only updated after a test run. This works on a dev
machine, where the tests are ran over and over again, because
eventually the estimates become quite precise, but CI always does
a clean build with 0 estimates.

Because we have 2 slow tests, we want them to run first to avoid
losing parallelism. To do this, we provide them with a cost estimate
manually.
2020-02-03 09:07:23 +01:00
Martin Hořeňovský
5fbf04cd59
Redo how the separate compilation tests are handled 2019-12-15 20:33:39 +01:00
Martin Hořeňovský
8b42acc328
Reintegrate extra tests 2019-12-09 10:04:26 +01:00
Martin Hořeňovský
0fea081ad1
Move tests from projects/ to tests/ 2019-12-05 16:00:20 +01:00