Commit Graph

291 Commits

Author SHA1 Message Date
Martin Hořeňovský d1ef461471
Update basic cost estimates for approvals/benchmarks 2022-11-06 21:23:03 +01:00
Martin Hořeňovský 0c75caf77b
Don't write ApprovalTests' temporary files into the source tree
This has two effects:
 1) This significantly improves the performance of ApprovalTests
    run in WSL, because running the massively multi-reporter
    test would cause lots of small and parallel writes across the
    WSL-Windows FS boundary, completely mudering performance.

 2) ApprovalTests can be run from multiple build dirs in parallel,
    as long as they do not fail. However, if they do fail,
    the multiple runs will still step on each other toes when
    writing the unapproved files for user.
2022-11-06 00:11:38 +01:00
Martin Hořeňovský 3b139ae51a
Decomposer checks for 0 when assuming an int arg was 0 literal 2022-11-05 20:43:52 +01:00
Martin Hořeňovský ec59cd8736
Support decomposing types that only compare with literal 0
This is primarily done to support new `std::*_ordering` types,
but the refactoring also supports any other type with this
property.

The compilation overhead is surprisingly low. Testing it with
clang on a Linux machine, compiling our SelfTest project takes
only 2-3% longer with these changes than it takes otherwise.

Closes #2555
2022-11-04 19:24:44 +01:00
Martin Hořeňovský d7f8c36e4c
Add traits for checking whether types are comparable 2022-11-04 19:24:42 +01:00
Martin Hořeňovský b3dbd83da2
Add test for comparing pointers to NULL with != 2022-11-04 19:24:39 +01:00
Martin Hořeňovský 0a1b0ae9d1
Meson build fixes
* Add missing files to `tests/meson.build`
* Rename meson build workflows to be more easily identifiable
2022-11-04 19:24:36 +01:00
Martin Hořeňovský 82cec69e93
Don't set C++ standard in CMakeLists for tests 2022-10-29 21:04:58 +02:00
Martin Hořeňovský 12b4390169
Fix license rebase error in test helpers 2022-10-28 14:39:56 +02:00
Martin Hořeňovský 3b40cf13eb
Split out parseTestSpec into test-only helpers 2022-10-28 13:27:46 +02:00
Lars Toenning f1084fb309
Fix references to license file
The license file was renamed with 6a502cc2f5
2022-10-28 11:30:15 +02:00
Martin Hořeňovský d41da10c54
Change reporters to report filters in round-trippable format 2022-10-27 20:46:59 +02:00
Martin Hořeňovský d2294ad9b6
Add test for round-tripping serialization of unmatched test specs 2022-10-27 19:11:20 +02:00
Martin Hořeňovský e19ed221bd
Provide round-tripping serialization for TestSpec 2022-10-27 18:23:59 +02:00
Martin Hořeňovský c6dfeb5e7d
Split tests for TestSpec into its own file
They also got slapped with the `[approvals]` tag in the process,
because we have too many approval tests and want less of them,
and these particular tests don't bring much value.

Related to #2090
2022-10-26 21:03:56 +02:00
Martin Hořeňovský 17fac854ae
Add test checking that reporters report filters 2022-10-26 11:26:10 +02:00
Martin Hořeňovský ffa152095c
Report used filters in the SonarQube reporter 2022-10-26 00:05:00 +02:00
Martin Hořeňovský 0ce8c25566
Report used filters in the TAP reporter 2022-10-26 00:04:49 +02:00
Philip Salzmann 6185d0cc0a Use console reporter's totals summary for compact reporter
This changes the compact reporter's summary of test run totals to use
the same format as the console reporter. This means that while output is
no longer on a single line (two instead), it now includes totals for
`failedButOk` test cases and assertions, which were previously missing.
2022-10-25 20:02:26 +02:00
Martin Hořeňovský a43f67962e
Refactor parsing of shard index/count in cmdline handling
This worsens the message for negative numbers a bit, but
simplifies the code enough that this is still a win.
2022-10-23 00:10:05 +02:00
Martin Hořeňovský f1361ef624
Refactor how the RNG seed is parsed when handling cmdline 2022-10-22 22:18:30 +02:00
Martin Hořeňovský 80d58a791d
Add support for Bazel's sharding env variables
Closes #2491
2022-10-21 10:53:53 +02:00
Martin Hořeňovský d7341b5dc1
Add parseUInt utility function
There is an increasing number of places where Catch2 wants to parse
strings into numbers, but being stuck in C++14 world, we do not
have good stdlib facilities to do this (`strtoul` and `stoul`
are both bad).
2022-10-21 10:53:50 +02:00
Martin Hořeňovský 38d926090a
Tiny fix for testBazelReporter.py 2022-10-20 21:34:35 +02:00
Martin Hořeňovský 9d08689845
Add support for Bazel's TESTBRIDGE_TEST_ONLY env var
Closes #2490
2022-10-20 21:34:32 +02:00
Martin Hořeňovský 77f7c0104d
Fix fibonacci impl in benchmark tests
The previous implementation was always 1 number ahead. Fixing this
is not important at all, but it was bugging me.
2022-10-18 10:44:18 +02:00
Rosen Penev 6a9bf2e0af
meson cleanups with muon (#2539)
* meson: run through muon's fmt to fix formatting

* meson: switch arrays to files

Allows muon to alphabetically sort files. switch headers back to arrays
as split() can only be used on strings.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2022-10-03 17:23:38 +02:00
Mike Crowe 4db8b50aab
Add support for building with Meson (#2530)
The Meson[1] build system makes it easier incorporate third-party
libaries into a project if they also build using Meson.

Let's add a minimal Meson build that's compatible with the CMake build,
along with a GitHub workflow to check that it builds and that at least
the simplest SelfTest runs.

The handling of catch_user_config.hpp is inspired by BUILD.bazel and
doesn't attempt to support any configuratons options. Such features
could be added later.

Meson strongly discourages using wildcards to specify sources, so the
source and header lists are copied from CMakeLists.txt.

Add a new test workflow to test the Meson builds. I was unable to get
these tests to pass with Ubuntu 20.04, so they use Ubuntu 22.04.

I'm neither a CMake nor a Meson expert, but the results seem to work for
me.

[1] https://mesonbuild.com/

Co-authored-by: Mike Crowe <mcrowe@brightsign.biz>
2022-10-01 23:28:30 +02:00
Dimitrij Mijoski 98d37da03e
Raise the minimum CMake version to 3.10 (#2523) 2022-09-16 17:56:00 +02:00
Dimitrij Mijoski 997a7d4165 Fix running the tests with shared library on Windows.
Without this fix, the test executable fails because it can not find
the dll of Catch2.
2022-07-25 21:07:49 +02:00
Martin Hořeňovský 9c9f35068e
Normalize C++ namespace in JUnit's reporter classname field
Closes #2468
2022-07-17 19:15:20 +02:00
Raphael Schaller 1bd233866c Add AllTrue, AnyTrue, NoneTrue matchers 2022-07-16 16:16:05 +02:00
Martin Hořeňovský caf1264588 Fixes for matcher testing helpers when testing bools 2022-07-16 16:16:05 +02:00
Martin Hořeňovský bdfa920f93
Use binary path in testBazelReporter reporter output path 2022-06-21 14:21:12 +02:00
Brandon Jones 7e4ec432d0 Change Bazel XML support to depend upon BAZEL_TEST 2022-06-17 16:36:14 +02:00
Martin Hořeňovský 338e4ec1f8
Force disable console colours for tests that regex check output
Fixes #2458
2022-06-14 23:24:06 +02:00
Martin Hořeňovský ff9506cedd
Only test SEH handling with MSVC
MinGW doesn't support `__try` and friends at all, while Clang
only supports it partially, and the test would require some
changes to make it work there. Since this is only a test, we can
afford to keep it MSVC-only.

Closes #2447
2022-06-06 00:29:01 +02:00
Sergio Losilla 5efd327dd4
Fix crash when running with --list-listeners and no registered listeners (#2442)
Co-authored-by: Sergio Losilla <sergio.losilla@nt-bnct.com>
2022-06-02 16:11:12 +02:00
Martin Hořeňovský 7a2a6c632f
Make Random*Generators be seedable
This is a necessary step to fix issue with combining multiple
random generators in one test case, with different section nesting.
2022-05-17 17:57:21 +02:00
Martin Hořeňovský 5d7883b551
Remove leftover comment in tests 2022-05-16 21:50:52 +02:00
Martin Hořeňovský 48f3226974
Add support for retrieving generator's element as string 2022-05-16 16:34:06 +02:00
Martin Hořeňovský af8b54ecd5
Generators now keep track of the index of current element
This might become potentially useful in the future, when we want
to provide the ability to forward jump generators, to be able to
simply reproduce specific input to a test.

I am not yet sure how it would work, but the basic idea is that
it could be similar to the `-c` switch for selecting specific
`SECTION` in a test case.
2022-05-16 11:56:21 +02:00
Martin Hořeňovský 316025a0d8
Properly handle startup reporter registration errors 2022-05-15 10:35:12 +02:00
Martin Hořeňovský 33aeb603fe
Add check for registering multiple reporters under same name 2022-05-15 09:54:27 +02:00
Martin Hořeňovský fc3d11b1d1
Add Wfloat-equal to default-enabled warnings
This is kinda messy, because there is no good way to signal to
the compiler that some code uses direct comparison of floating
point numbers intentionally, so instead we have to use diagnostic
pragmas.

We also have to over-suppress the test files, because Clang (and
possibly GCC) still issue warnings from template instantiation
even if the instantion site is under warning suppression, so the
template definition has to be under warning suppression as well.

Closes #2406
2022-05-14 15:47:20 +02:00
Martin Hořeňovský 1ef65d60f1
Tiny fixup in Misc.tests.cpp 2022-05-11 22:56:30 +02:00
Martin Hořeňovský ed6b38b096
Specialize XML reporter's listener listing 2022-05-10 20:58:02 +02:00
Martin Hořeňovský 5a49285e9c
Add --list-listeners option 2022-05-10 20:57:59 +02:00
Martin Hořeňovský d60fbe49be
Slight improvement for reporter test 2022-05-10 20:57:50 +02:00
Martin Hořeňovský d9b0a38f81
Report Catch2's version in the xml reporter output 2022-05-03 19:46:35 +02:00