Previously a random test ordering was obtained by applying std::shuffle
to the tests in declaration order. This has two problems:
- It depends on the declaration order, so the order in which the tests
will be run will be platform-specific.
- When trying to debug accidental inter-test dependencies, it is helpful
to be able to find a minimal subset of tests which exhibits the issue.
However, any change to the set of tests being run will completely
change the test ordering, making it difficult or impossible to reduce
the set of tests being run in any reasonably efficient manner.
Therefore, change the randomization approach to resolve both these
issues.
Generate a random value based on the user-provided RNG seed. Convert
every test case to an integer by hashing a combination of that value
with the test name. Sort the test cases by this integer.
The test names and RNG are platform-independent, so this should be
consistent across platforms. Also, removing one test does not change
the integer value associated with the remaining tests, so they remain in
the same order.
To hash, use the FNV-1a hash, except with the basis being our randomly
selected value rather than the fixed basis set in the algorithm. Cannot
use std::hash, because it is important that the result be
platform-independent.
It did not clear out all of its internal state when switching from
one pattern to another, so when it should've escaped `,`, it took
its position from its position in the original user-provided string,
rather than its position in the current pattern.
Fixes#1905
CATCH_INTERNAL_IGNORE_BUT_WARN() introduced with b7b346c triggers
clang-tidy warning 'cppcoreguidelines-pro-type-vararg' for every usage
of assertion macros like CHECK() and REQUIRE(). Silence it via NOLINT
in the '#if defined(__clang__)' block only, as clang-tidy honors those.
The old code caused warnings to fire under MSVC, and Clang <3.8.
I could not find a GCC version where it worked, but I assume that it
did at some point.
This new code causes all of MSVC, GCC, Clang, in current versions,
to emit signed/unsigned comparison warning in test like this:
```cpp
TEST_CASE() {
int32_t i = -1;
uint32_t j = 1;
REQUIRE(i != j);
}
```
Where previously only MSVC would emit the warning.
Fixes#1880
These files are not included by the default
`#include <catch2/catch_test_macros.hpp>` path, so that users do
not have to pay for them if they do not use them. Follow up is to
split out the small part of `catch_preprocessor.hpp` used by the
default test macros (AFAIK, it is just `INTERNAL_CATCH_REMOVE_PARENS`
macro), so that it is not included by the default path either.
Also fixes#1892 by providing the missing macros.
This is both a really big and a really small commit. It is small in
that it only contains renaming, moving and modification of include
directives caused by this.
It is really big in the obvious way of touching something like 200
files.
The new rules for naming files is simple: headers use the `.hpp`
extension. The rules for physical file layout is still kinda in
progress, but the basics are also simple:
* Significant parts of functionality get their own subfolder
* Benchmarking is in `catch2/benchmark`
* Matchers are in `catch2/matchers`
* Generators are in `catch2/generators`
* Reporters are in `catch2/reporters`
* Baseline testing facilities are in `catch2/`
* Various top level folders also contain `internal` subfolder,
with files that users probably do not want to include directly,
at least not until they have to write something like their own
reporter.
* The exact files in these subfolders is likely to change later
on
Note that while some includes were cleaned up in this commit, it
is only the low hanging fruit and further cleanup using automatic
tooling will happen later.
Also note that various include guards, copyright notices and file
headers will also be standardized later, rather than in this commit.
It used to be a file that would collect interfaces we always wanted
to provide to users, so that the single header stitching script
would place them in the common part of the single header version.
As v3 is moving to separate headers model, the file is no longer
useful.
This was an old "include all" header, that we no longer want to be
usable, to make the include differences in new versions explicit.
We will introduce new "include all" headers later, in the form of
`catch_all.hpp`, `catch_matchers_all.hpp` and so on...
Removed nested `StdString` namespace and added Doxygen comments.
Also renamed some matchers to avoid colisions now that there are
less separate namespaces for matchers to go to. Since this is a
breaking release anyway, it shouldn't matter, and the factory
functions that the users should use remain the same anyway.
Martin Hořeňovský