A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.
When there are many tests, this produces output spam which makes it hard
to find the test failure output. Nevertheless, it is helpful to be
informed of tests which are unusually slow.
Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed. This allows slow
tests to be visible without mentioning every test.
This brings our output inline with GTest's. We do not handle skipped
tests properly, but that should be currently less important than
having the attribute exist with proper value for non-skipped tests.
Thanks @joda-01.
Closes#1899
- Overrides added
- usages of push_back() replaced with emplace_back()
- Loop variable made const-refernce
- NULL replaced with nullptr
- Names used in the declaration and definition unified
- size() replaced with empty
- Identical cases merged
The JUnit report is improved in that:
* The message shows the testing condition, not the result
* The actual message has similar output than the console one
* Units from <ratio> are no longer redeclared in our own namespace
* The default clock is `steady_clock`, not `high_resolution_clock`,
because, as HH says "high_resolution_clock is useless. If you want
measure the passing of time, use steady_clock. If you want user
friendly time, use system_clock".
* Benchmarking support is opt-in, not opt-out, to avoid the large
(~10%) compile time penalty.
* Benchmarking-related options in CLI are always present, to decrease
the amount of code that is only compiled conditionally and making
the whole shebang more maintainble.
Changes done to Nonius:
* Moved things into "Catch::Benchmark" namespace
* Benchmarks were integrated with `TEST_CASE`/`SECTION`/`GENERATE` macros
* Removed Nonius's parameters for benchmarks, Generators should be used instead
* Added relevant methods to the reporter interface (default-implemented, to avoid
breaking existing 3rd party reporters)
* Async processing is guarded with `_REENTRANT` macro for GCC/Clang, used by default
on MSVC
* Added a macro `CATCH_CONFIG_DISABLE_BENCHMARKING` that removes all traces of
benchmarking from Catch
This is a temporarily workaround until we can nuke the current
verbosities system from the orbit and replace it with something
actually sane.
Fixes#1426
Xml result of reported will now contain value of rng-seed in case it
is not zero.
The value will be stored in element Randomness and it's attribute seed.
Relates to #1402
By opting the JUnit and XML reporters into it, we no longer run
into problem where they underreport the results without `-s` flag.
Related to #1264, #1267, #1310
Catch2's documentation promises that listeners are called _before_
reporters, but because of the previous implementation, they were
called _after_ reporters. This commit fixes that.
Closes#1234
ReusableStringStream holds a std::ostringstream internally, but only exposes the ostream interface.
It caches a pool of ostringstreams in a vector which is currently global, but will be made thread-local.
Altogether this should enable both runtime and compile-time benefits. although more work is needed to realise the compile time opportunities.
