Previously a random test ordering was obtained by applying std::shuffle
to the tests in declaration order. This has two problems:
- It depends on the declaration order, so the order in which the tests
will be run will be platform-specific.
- When trying to debug accidental inter-test dependencies, it is helpful
to be able to find a minimal subset of tests which exhibits the issue.
However, any change to the set of tests being run will completely
change the test ordering, making it difficult or impossible to reduce
the set of tests being run in any reasonably efficient manner.
Therefore, change the randomization approach to resolve both these
issues.
Generate a random value based on the user-provided RNG seed. Convert
every test case to an integer by hashing a combination of that value
with the test name. Sort the test cases by this integer.
The test names and RNG are platform-independent, so this should be
consistent across platforms. Also, removing one test does not change
the integer value associated with the remaining tests, so they remain in
the same order.
To hash, use the FNV-1a hash, except with the basis being our randomly
selected value rather than the fixed basis set in the algorithm. Cannot
use std::hash, because it is important that the result be
platform-independent.
It did not clear out all of its internal state when switching from
one pattern to another, so when it should've escaped `,`, it took
its position from its position in the original user-provided string,
rather than its position in the current pattern.
Fixes#1905
CATCH_INTERNAL_IGNORE_BUT_WARN() introduced with b7b346c triggers
clang-tidy warning 'cppcoreguidelines-pro-type-vararg' for every usage
of assertion macros like CHECK() and REQUIRE(). Silence it via NOLINT
in the '#if defined(__clang__)' block only, as clang-tidy honors those.
The old code caused warnings to fire under MSVC, and Clang <3.8.
I could not find a GCC version where it worked, but I assume that it
did at some point.
This new code causes all of MSVC, GCC, Clang, in current versions,
to emit signed/unsigned comparison warning in test like this:
```cpp
TEST_CASE() {
int32_t i = -1;
uint32_t j = 1;
REQUIRE(i != j);
}
```
Where previously only MSVC would emit the warning.
Fixes#1880
These files are not included by the default
`#include <catch2/catch_test_macros.hpp>` path, so that users do
not have to pay for them if they do not use them. Follow up is to
split out the small part of `catch_preprocessor.hpp` used by the
default test macros (AFAIK, it is just `INTERNAL_CATCH_REMOVE_PARENS`
macro), so that it is not included by the default path either.
Also fixes#1892 by providing the missing macros.
This is both a really big and a really small commit. It is small in
that it only contains renaming, moving and modification of include
directives caused by this.
It is really big in the obvious way of touching something like 200
files.
The new rules for naming files is simple: headers use the `.hpp`
extension. The rules for physical file layout is still kinda in
progress, but the basics are also simple:
* Significant parts of functionality get their own subfolder
* Benchmarking is in `catch2/benchmark`
* Matchers are in `catch2/matchers`
* Generators are in `catch2/generators`
* Reporters are in `catch2/reporters`
* Baseline testing facilities are in `catch2/`
* Various top level folders also contain `internal` subfolder,
with files that users probably do not want to include directly,
at least not until they have to write something like their own
reporter.
* The exact files in these subfolders is likely to change later
on
Note that while some includes were cleaned up in this commit, it
is only the low hanging fruit and further cleanup using automatic
tooling will happen later.
Also note that various include guards, copyright notices and file
headers will also be standardized later, rather than in this commit.
It used to be a file that would collect interfaces we always wanted
to provide to users, so that the single header stitching script
would place them in the common part of the single header version.
As v3 is moving to separate headers model, the file is no longer
useful.
This was an old "include all" header, that we no longer want to be
usable, to make the include differences in new versions explicit.
We will introduce new "include all" headers later, in the form of
`catch_all.hpp`, `catch_matchers_all.hpp` and so on...
Removed nested `StdString` namespace and added Doxygen comments.
Also renamed some matchers to avoid colisions now that there are
less separate namespaces for matchers to go to. Since this is a
breaking release anyway, it shouldn't matter, and the factory
functions that the users should use remain the same anyway.
Removed the `generic` nested namespace, so PredicateMatcher now
lives in `Catch::Matchers` namespace, just like other matchers.
Also cleaned up and doxygenized comments on the `Predicate` factory
function for `PredicateMatcher`.
The two changes are
`catch_matchers_templates` -> `catch_matchers_templated` and
`catch_matchers_generic` -> `catch_matchers_predicate`. The former
is mostly cosmetic, but the second was previously significantly
misleading, and as the library is now to be consumed by including
specific headers, this needed to be fixed.
`SizeIs` can accept both `size_t` and a matcher. In the first case,
it checks whether the size of the range is equal to specified size.
In the second case, it checks whether the provided matcher accepts
the size of the range.
Outside of `MatcherBase` and `GenericMatcherBase`, matchers are not
designed to be overriden. This means that doing so can easily lead
to errors, and matchers are generally fairly simple functionality-wise.
so there is not much code reuse to be gained anyway.
Thus, Catch2-provided concrete matchers are now final.
In general, for Catch2 v3 we are making virtual types `final`,
unless they were explicitly designed to be derived-from.
`ListeningReporter` is definitely not designed to be derived-from.
This was previously used to avoid `dynamic_cast` inside our code,
when we were creating more than one reporter, or a reporter
together with listeners. However, since then the offending code
was refactored to be smarter instead, and this query member function
is no longer needed nor used.
The only way to stream those is to use the `bool` overload of `op<<`.
However, to convert a function to bool, GCC creates AST equivalent
of `A? true : false`. Then, because `A` is a function, it warns that
it will never be `false`. 🤦
As a bonus, newer GCC versions issue _two_ different warnings about
this, but older GCC versions do not know both of them, so we also
have to suppress warning about unknown warning suppression.
The variable initialization has test registration as a side-effect,
but as far as GCC is concerned, the variable itself is unused.
Because the macro substitution always happens at global scope, we
cannot use cast to `void` as is usually done.
Sadly most versions still cannot properly handle the suppression
via `_Pragma`, so it has to leak to the users when they use older
GCC versions to compile their code
