Specifically we turned `mean`, `classify_outliers`, `jackknife`,
into concrete functions that take only `const_iterator` from vecs,
instead of generic iterators over anything.
I also changed `resample` to take `const_iterator` instead of
plain `iterator`, and similar for `standard_deviation`, and
`analyse_samples`.
Introducing a new DISCOVERY_MODE mode option, which provides greater
control over when catch_discover_tests perforsm test discovery.
It has two supported modes:
* POST_BUILD: The default behavior, which adds a POST_BUILD command
to perform test discovery after the test has been built as was
always done so far.
* PRE_TEST: New mode, which delays test discovery until test execution.
The generated include file generates the appropriate CTest files at
runtime and regenerates the CTest files when the executable is
updated.
This mode can be used in build-environments that don't allow for
executing the linked binaries at build-time (like in a
cross-compilation environment).
DISCOVERY_MODE can be controlled in two ways:
1. Setting the DISCOVERY_MODE when calling catch_discover_tests.
2. Setting the global CMAKE_CATCH_DISCOVER_TESTS_DISCOVERY_MODE prior
to calling gtest_discover_tests.
Closes#2493
Move test discovery logic into new catch_discover_tests_impl method
and make CatchAddTests aware of whether it is being launched in
CMake's script mode.
When launched in script mode, catch_discover_tests_impl is called
passing arguments obtained from the definitions passed into the call
to cmake. This preserves the existing behavior assumed by Catch.cmake.
Looking ahead, it also allows CatchAddTests to be included in
generated files and call catch_discover_tests_impl to perform test
discovery at test runtime with the new PRE_TEST discovery mode
introduced later.
* Split out BenchmarkInfo and BenchmarkStats to their own header
* Outline BenchmarkStats<> declaration to separate header
* Split out TestRunInfo into its own header
These changes let us remove the large `interfaces_reporter.hpp`
include from `benchmark.hpp`, and replace it with
`interfaces_capture.hpp` in `run_context.hpp`.
I also cleaned out `interfaces_repoter.hpp` from reporter headers
that depend on `reporter_common_base.hpp`. This will not change
anything in the actual inclusion set, but makes it logically
more consistent.
To keep the compilation firewall effect, the implementations
are hidden behind a PIMPL. In this case it is probably not
worth it, but we can inline it later if needed.
We now show the more modern `-S {source}` instead of the old and
undocumented `-H{source}` CMake flag, and also show the available
presets, instead of individually specifying the different testing
options.
Closes#2593
I do not know if checking the tracker name or the tracker's file
part of the location first would provide better results, but
in the common case, the line part of the location check should be
rather unique, because different `SECTION`s will have different
source lines where they are defined.
I also propagated this same check into `ITracker::findChild`,
because this significantly improves performance of section tracking
in Debug builds -> 10% in macro benchmark heavily focused on section
tracking. In Release build there is usually no difference, because
the inliner will inline `NameAndLoc::operator==` into `findChild`,
and then eliminate the redundant check. (If the inliner decides
against, then this still improves the performance on average).
`NameAndLocationRef` is pretty large type, so even in release build,
it is unlikely to be passed in registers. In addition to the fact
that some platforms currently do not allow passing even small types
in register (Windows ABI!!), it is better to pass it as a ref,
effectively passing around a pointer.
