Commit Graph

254 Commits

Author SHA1 Message Date
Martin Hořeňovský
7f931d6df4
Add tests for scaled ULP distance between +/- FLT/DBL_MAX 2024-01-14 21:15:02 +01:00
Martin Hořeňovský
cb07ff9a7e
Fix uniform_floating_point_distribution for unit ranges 2023-12-10 19:53:46 +01:00
Martin Hořeňovský
28c66fdc5a
Make uniform_floating_point_distribution reproducible
By moving to use our `uniform_integer_distribution`, which is
reproducible across different platforms, instead of the stdlib
one which is not, we can provide reproducible results for `float`s
and `double`s. Still no reproducibility for `long double`s, because
those are too different across different platforms.
2023-12-10 19:53:41 +01:00
Martin Hořeňovský
ed9d672b5c
Add uniform_integer_distribution 2023-12-10 19:53:39 +01:00
Martin Hořeňovský
04a829b0e1
Add helpers for implementing uniform integer distribution
* Utility for extended mult n x n bits -> 2n bits
* Utility to adapt output from URBG to target (unsigned) integral
  type
* Utility to reorder signed values into unsigned type while keeping
  the order.
2023-12-10 19:53:36 +01:00
Martin Hořeňovský
ab1b079e4d
Add uniform_floating_point_distribution 2023-12-10 18:44:12 +01:00
Martin Hořeňovský
d139b4ff7c
Add implementation of helpers for uniform float distribution
Specifically we add
 * `gamma(a, b)`, which returns the magnitude of largest 1-ULP
   step in range [a, b].
 * `count_equidistant_float(a, b, distance)`, which returns the
   number of equi-distant floats in range [a, b].
2023-12-10 18:44:10 +01:00
Martin Hořeňovský
9a1e73568c
Add test showing literals and complex generators in one GENERATE 2023-12-10 18:32:45 +01:00
Martin Hořeňovský
733b901dd2
Fix special character escaping in JsonWriter 2023-11-14 23:35:22 +01:00
Beartama
7bf136b501
Add JSON reporter (#2706)
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-11-14 22:52:15 +01:00
Martin Hořeňovský
79d39a1954
Fix tests for C++23's multi-arg index operator
Closes #2744
2023-10-28 21:49:58 +02:00
Martin Hořeňovský
6ebc013b8c
Fix UDL definitions for C++23
Technically, the declaration should not have a space between
the quotes and the underscore, because `_foo` is a reserved
identifier, but `""_foo` is not. In general it works, but newer
Clang versions warn about this, because WG21 wants to deprecate
and later remove this form completely.
2023-10-28 21:35:03 +02:00
Martin Hořeňovský
9c541ca72e
Add test for multiple streaming parts in UNSCOPED_INFO 2023-09-17 10:44:31 +02:00
Martin Hořeňovský
47a2c96938
Reduce the number of templates in Benchmarking
The basic idea was to reduce the number of things dependent on the `Clock`
type. To that end, I replaced `Duration<Clock>` with `IDuration` typedef
for `std::nanoseconds`, and `FloatDuration<Clock>` with `FDuration`
typedef for `Duration<double, std::nano>`. We can generally assume that
any clock's duration can be expressed in nanoseconds, as long as we insert
`duration_cast`s into the right places.

Note that we cannot remove all dependence on `Clock` as a template
arguments, because functions that actually measure the elapsed time have
to use the Clock.

We also changed some template function arguments to pass plain function
pointers, so that the actual implementation can be placed into a cpp file.
2023-09-08 10:04:31 +02:00
Martin Hořeňovský
9bba07cb87
Replace vector iterator args in benchmarks with ptr args 2023-09-08 10:04:19 +02:00
Martin Jeřábek
3a5cde55b7 implement stringify for std::nullopt_t 2023-08-30 16:21:02 +02:00
Martin Hořeňovský
f24d39e42b
Support C arrays and ADL ranges in from_range generator
Closes #2737
2023-08-29 15:38:13 +02:00
rosstang
cd60a0301c
Assert Info reset need to also reset result disposition to normal to handle uncaught exception correctly (#2723)
* AssertionEnd does not reset the assertion info yet. That is done after populateReaction. And reset assertion info would also reset the result disposition to normal, so that any uncaught exception would be reported as failure

* Approving test output changes due to added unit tests

* Unit tests to throw std::runtime_error instead of std::exception

* Add a unit test to test incomplete assertion handler

---------

Co-authored-by: Ross <ross.tang@gfo-x.com>
2023-08-07 22:07:31 +02:00
Martin Hořeňovský
0631b607ee
Test & document SKIP in generator constructor
Closes #1593
2023-05-31 15:12:43 +02:00
Martin Hořeňovský
dff7513b28
Static analysis cleanup in tests 2023-05-29 21:45:30 +02:00
Martin Hořeňovský
b3cf1bfb5d
Avoid unused variable warning in GeneratorsImpl tests 2023-05-28 21:05:01 +02:00
Martin Hořeňovský
689fdcd7dc
Fix some tests never being run 2023-05-20 22:03:48 +02:00
Martin Hořeňovský
a153fce724
Improve error messages for TEST_CASE tag parsing errors
Also removes a duplicated test case checking for empty tag error.

Related to #2650
2023-05-20 21:13:48 +02:00
Martin Hořeňovský
c57b5cdf43
Move-enable Catch::optional
This avoids copies in couple places through Catch2, e.g. reporter
spec handling, and moving around `AssertionResult` in `RunContext`.
2023-05-06 15:22:02 +02:00
Martin Hořeňovský
fe64c28925
Reduce compilation costs of benchmarks
We replaced some simple std::algorithm usage by loops, and reduced
header inclusion.
2023-05-01 13:21:41 +02:00
Vertexwahn
46539b6d9b Fix spelling 2023-04-29 12:55:51 +02:00
Holger Kaelberer
10596b2278 Fix unreachable-code-return warnings 2023-04-20 14:53:03 +02:00
Martin Hořeňovský
9a2a4eadc0
Bump xml-format-version in XML reporter 2023-04-10 21:59:50 +02:00
Martin Hořeňovský
fb806da76f
Add lineinfo to XML reporter output for INFO/WARN
Closes #1251
2023-04-10 16:32:07 +02:00
Martin Hořeňovský
d0f70fdfd6
Unify IReporterRegistry and ReporterRegistry
To keep the compilation firewall effect, the implementations
are hidden behind a PIMPL. In this case it is probably not
worth it, but we can inline it later if needed.
2023-03-22 23:58:44 +01:00
Martin Hořeňovský
0a0ebf5003
Support elements without op!= in VectorEquals
Closes #2648
2023-03-05 00:11:38 +01:00
Martin Hořeňovský
6fbb3f0723
Add IsNaN matcher 2023-02-26 00:14:32 +01:00
Martin Hořeňovský
e8ba329b6c
Add support for iterator+sentinel pairs in Contains matcher 2023-02-10 23:25:45 +01:00
Martin Hořeňovský
cefa8fcf32
Enable use of UnorderedRangeEquals with iterator+sentinel pairs 2023-02-06 15:34:38 +01:00
Martin Hořeňovský
772fa3f790
Add Catch::Detail::is_permutation that supports sentinels
Also split out helpers for testing matcher ranges (types whose
begin/end/empty/etc require ADL lookup, types whose iteration
uses iterator + sentinel pair, etc) into their own file.
2023-02-06 15:29:01 +01:00
Martin Hořeňovský
f3c0a3cd09
Fix RangeEquals matcher to handle iterator + sentinel ranges
Also added tests for types that require ADL lookup for their
`begin` and `end`.
2023-02-03 18:22:41 +01:00
Martin Hořeňovský
42d9d4533e
Add test for empty result of filter generator 2023-02-01 18:27:41 +01:00
Martin Hořeňovský
43f02027e4
Avoid allocations when looking for trackers
Now we delay allocating owning `NameAndLocation` instances until
we construct a new tracker (because a tracker's lifetime can be
significantly different from the underlying tracked-thing's name).

This saves 4239 allocations (436948 -> 432709) when running
`./tests/SelfTest -o /dev/null`, at some cost to code clarity
due to introducing a new ref type, `NameAndLocationRef`.
2023-01-29 10:14:20 +01:00
Martin Hořeňovský
8359a6b244
Stop exceptions in generator constructors from aborting the binary
Fixes #2615
2023-01-22 16:04:16 +01:00
John Beard
efca9a0f18
Added ElementsAre and UnorderedElementsAre (#2377)
Co-authored-by: Garz4 <fancygarz4@gmail.com>
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-22 00:33:04 +01:00
Philip Salzmann
d548be26e3
Add new SKIP macro for skipping tests at runtime (#2360)
* Add new SKIP macro for skipping tests at runtime

This adds a new `SKIP` macro for dynamically skipping tests at runtime.
The "skipped" status of a test case is treated as a first-class citizen,
like "succeeded" or "failed", and is reported with a new color on the
console.

* Don't show "skipped assertions" in console/compact reporters

Also extend skip tests to cover a few more use cases.

* Return exit code 4 if all test cases are skipped

* Use LightGrey for the skip colour

This isn't great, but is better than the deep blue that was borderline
invisible on dark backgrounds. The fix is to redo the colouring
a bit, including introducing light-blue that is actually visible.

* Add support for explicit skips in all reporters

* --allow-running-no-tests also allows all tests to be skipped

* Add docs for SKIP macro, deprecate IEventListener::skipTest

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
Martin Hořeňovský
ac93f19437
Improved path normalization in approvalTests.py
Instead of redoing the whole line where path was found, only the
directory part of the path is removed, instead of removing all
of the line before the path starts.

This results in slight change in how junit and sonarqube approvals
come out, and significant change in how TeamCity reporter approvals
come out. This latter difference is the reason for the change,
as now the lines with `testFailed` and `testIgnored` messages
are not completely butchered.
2023-01-01 20:27:59 +01:00
lbckmnn
9c0533a905
Add MessageMatches matcher for exception (#2570) 2022-12-12 00:40:47 +01:00
Martin Hořeňovský
28e651f152
Move SFINAE in decomposer into return type
This is needed so that we can use conjunction and other logical
type traits to workaround issue with older GCC versions (8 and
below), when they run into types that have ambiguous constructor
from `0`, see e.g. #2571.

However, using conjunction and friends in the SFINAE constraint
in the template parameter breaks for C++20 and up, due to the new
comparison operator rewriting rules. With C++20, when the compiler
see `a == b`, it also tries `b == a` and collects overload set
for both of these expressions.

In Catch2, this means that e.g. `REQUIRE( 1 == 2 )` would lead
the compiler to check overloads for both `ExprLhs<int> == int`
and `int == ExprLhs<int>`. Since the overload set and SFINAE
constraints assume that `ExprLhs<T>` is always on the left side,
when the compiler tries to resolve the template parameters, all
hell breaks loose and the compilation fails.

By moving the SFINAE constraints to the return type, the compiler
can discard the switched expression without having to resolve
the complex SFINAE constraints, and thus everything works the
way it is supposed to.

Fixes #2571
2022-12-09 00:40:01 +01:00
Martin Hořeňovský
a20200be7e
Revert "Fix old GCC + types with ambiguous constructor from 0"
This reverts commit 291c502f66.

The issue is that it breaks under C++20 for some reason.
2022-11-22 15:23:03 +01:00
Martin Hořeňovský
291c502f66
Fix old GCC + types with ambiguous constructor from 0
Closes #2571
2022-11-20 17:07:32 +01:00
Martin Hořeňovský
943c6e3dee
Add version field to the XML reporter output
We start at version 2, with version 1 being taken up by the output
from Catch2 v2.
2022-11-10 12:04:51 +01:00
Martin Hořeňovský
3b139ae51a
Decomposer checks for 0 when assuming an int arg was 0 literal 2022-11-05 20:43:52 +01:00
Martin Hořeňovský
ec59cd8736
Support decomposing types that only compare with literal 0
This is primarily done to support new `std::*_ordering` types,
but the refactoring also supports any other type with this
property.

The compilation overhead is surprisingly low. Testing it with
clang on a Linux machine, compiling our SelfTest project takes
only 2-3% longer with these changes than it takes otherwise.

Closes #2555
2022-11-04 19:24:44 +01:00
Martin Hořeňovský
d7f8c36e4c
Add traits for checking whether types are comparable 2022-11-04 19:24:42 +01:00