Commit Graph

7 Commits

Author SHA1 Message Date
Ben Dunkin
3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský
10fb93cce8
Add new reporter event called for each test case enter/exit
This means that e.g. for `TEST_CASE` with two sibling `SECTION`s
the event will fire twice, because the `TEST_CASE` will be entered
twice.

Closes #2107 (the event mentioned there already exists, but this
is its counterpart that we also want to provide to users)
2021-09-14 13:19:45 +02:00
Martin Hořeňovský
250d9b9c72
Fix how testRandomOrder.py builds tag arguments 2020-07-26 10:51:55 +02:00
Martin Hořeňovský
e6d947f6d4
Refactor tests for duration reporting threshold 2020-07-26 10:47:58 +02:00
John Bytheway
80b0d6975c
Add --min-duration option
A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.

When there are many tests, this produces output spam which makes it hard
to find the test failure output.  Nevertheless, it is helpful to be
informed of tests which are unusually slow.

Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed.  This allows slow
tests to be visible without mentioning every test.
2020-07-26 10:47:53 +02:00
Martin Hořeňovský
9b5fc9eaea
Randomize test for subset invariant random ordering of tests
Also removed the iterative checking that seeds 1-100 do not create
the same output, because it used too much runtime.
2020-05-01 14:45:36 +02:00
John Bytheway
630ba26278
Add test for consistent random ordering
-- Fixed up during cherrypicking
2020-05-01 14:30:01 +02:00