We used to use whatever precision we ended up having from C++'s
stdlib. However, some relatively popular tools, like Jenkins,
use Maven SureFire XML schema to validate JUnit test reports, and
Maven SureFire schema requires the duration to have at most 3
decimal places.
For compatibility, the JUnit reporter will now respect this
limitation.
Closes#2221
TAP format requires all results to be reported.
Removed extraneous preferences function (handled by parent)
Incorporated fix from 3d9e7db2e0
Simplified total printing
A test runner already has a --durations option to print durations.
However, this isn't entirely satisfactory.
When there are many tests, this produces output spam which makes it hard
to find the test failure output. Nevertheless, it is helpful to be
informed of tests which are unusually slow.
Therefore, introduce a new option --min-duration that causes all
durations above a certain threshold to be printed. This allows slow
tests to be visible without mentioning every test.
This brings our output inline with GTest's. We do not handle skipped
tests properly, but that should be currently less important than
having the attribute exist with proper value for non-skipped tests.
Thanks @joda-01.
Closes#1899
- Overrides added
- usages of push_back() replaced with emplace_back()
- Loop variable made const-refernce
- NULL replaced with nullptr
- Names used in the declaration and definition unified
- size() replaced with empty
- Identical cases merged
The JUnit report is improved in that:
* The message shows the testing condition, not the result
* The actual message has similar output than the console one
* Units from <ratio> are no longer redeclared in our own namespace
* The default clock is `steady_clock`, not `high_resolution_clock`,
because, as HH says "high_resolution_clock is useless. If you want
measure the passing of time, use steady_clock. If you want user
friendly time, use system_clock".
* Benchmarking support is opt-in, not opt-out, to avoid the large
(~10%) compile time penalty.
* Benchmarking-related options in CLI are always present, to decrease
the amount of code that is only compiled conditionally and making
the whole shebang more maintainble.
Changes done to Nonius:
* Moved things into "Catch::Benchmark" namespace
* Benchmarks were integrated with `TEST_CASE`/`SECTION`/`GENERATE` macros
* Removed Nonius's parameters for benchmarks, Generators should be used instead
* Added relevant methods to the reporter interface (default-implemented, to avoid
breaking existing 3rd party reporters)
* Async processing is guarded with `_REENTRANT` macro for GCC/Clang, used by default
on MSVC
* Added a macro `CATCH_CONFIG_DISABLE_BENCHMARKING` that removes all traces of
benchmarking from Catch
This is a temporarily workaround until we can nuke the current
verbosities system from the orbit and replace it with something
actually sane.
Fixes#1426
Xml result of reported will now contain value of rng-seed in case it
is not zero.
The value will be stored in element Randomness and it's attribute seed.
Relates to #1402
By opting the JUnit and XML reporters into it, we no longer run
into problem where they underreport the results without `-s` flag.
Related to #1264, #1267, #1310
Catch2's documentation promises that listeners are called _before_
reporters, but because of the previous implementation, they were
called _after_ reporters. This commit fixes that.
Closes#1234
