Commit Graph

14 Commits

Author SHA1 Message Date
Martin Hořeňovský
17a04f88d9
Always report rng seed from builtin reporters
Not all reporters use a format that supports this, so TeamCity
and Automake reporters still do not report it. The console
reporter now reports it even on successful runs, where before
it only reported the rng seed in the header, which was showed
either for failed run, or for run with `-s`.

CLoses #2065
2022-04-19 10:32:13 +02:00
Lukasz Okraszewski
cb551b4f6d
Add default reporter for Bazel integration (#2399)
When the added Bazel configuration flag is enabled,
a default JUnit reporter will be added if the XML
envrioment variable is defined.
Fix include paths for generated config header.
Enable Bazel config by default when building with
Bazel.


Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-04-08 18:01:59 +02:00
Martin Hořeňovský
72f0372664
Add tests for CMake configure toggles passing to Catch2 build 2022-02-20 19:32:30 +01:00
Martin Hořeňovský
51888d360a
Use the new licence header in TestScripts 2022-01-29 00:04:49 +01:00
Martin Hořeňovský
d861e73f86
Handle full unique test identifiers in testRandomOrder.py 2022-01-27 21:11:37 +01:00
Martin Hořeňovský
ec2d5013fb Make testSharding.py test script executable 2021-10-27 17:24:30 +02:00
Martin Hořeňovský
342ef5ca7e Cleanup the shard integration test script 2021-10-27 17:24:30 +02:00
Ben Dunkin
3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský
10fb93cce8
Add new reporter event called for each test case enter/exit
This means that e.g. for `TEST_CASE` with two sibling `SECTION`s
the event will fire twice, because the `TEST_CASE` will be entered
twice.

Closes #2107 (the event mentioned there already exists, but this
is its counterpart that we also want to provide to users)
2021-09-14 13:19:45 +02:00
Martin Hořeňovský
250d9b9c72
Fix how testRandomOrder.py builds tag arguments 2020-07-26 10:51:55 +02:00
Martin Hořeňovský
e6d947f6d4
Refactor tests for duration reporting threshold 2020-07-26 10:47:58 +02:00
John Bytheway
80b0d6975c
Add --min-duration option
A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.

When there are many tests, this produces output spam which makes it hard
to find the test failure output.  Nevertheless, it is helpful to be
informed of tests which are unusually slow.

Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed.  This allows slow
tests to be visible without mentioning every test.
2020-07-26 10:47:53 +02:00
Martin Hořeňovský
9b5fc9eaea
Randomize test for subset invariant random ordering of tests
Also removed the iterative checking that seeds 1-100 do not create
the same output, because it used too much runtime.
2020-05-01 14:45:36 +02:00
John Bytheway
630ba26278
Add test for consistent random ordering
-- Fixed up during cherrypicking
2020-05-01 14:30:01 +02:00