3rd-party/catch2
1
0
Fork 0
mirror of https://github.com/catchorg/Catch2.git synced 2025-08-03 05:45:39 +02:00
Code Issues Packages Projects Releases Wiki Activity

Add new SKIP macro for skipping tests at runtime (#2360)

Browse Source 
* Add new SKIP macro for skipping tests at runtime

This adds a new `SKIP` macro for dynamically skipping tests at runtime.
The "skipped" status of a test case is treated as a first-class citizen,
like "succeeded" or "failed", and is reported with a new color on the
console.

* Don't show "skipped assertions" in console/compact reporters

Also extend skip tests to cover a few more use cases.

* Return exit code 4 if all test cases are skipped

* Use LightGrey for the skip colour

This isn't great, but is better than the deep blue that was borderline
invisible on dark backgrounds. The fix is to redo the colouring
a bit, including introducing light-blue that is actually visible.

* Add support for explicit skips in all reporters

* --allow-running-no-tests also allows all tests to be skipped

* Add docs for SKIP macro, deprecate IEventListener::skipTest

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
This commit is contained in:
Philip Salzmann
2023-01-12 15:01:47 +01:00
committed by GitHub
parent 52066dbc2a
commit d548be26e3
47 changed files with 3722 additions and 2171 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
tests/SelfTest/Baselines/automake.sw.multi.approved.txt
View File

@@ -290,6 +290,7 @@
:test-result: PASS X/level/1/b
:test-result: PASS XmlEncode
:test-result: PASS XmlWriter writes boolean attributes as true/false
:test-result: SKIP a succeeding test can still be skipped
:test-result: PASS analyse no analysis
:test-result: PASS array<int, N> -> toString
:test-result: PASS benchmark function call
@@ -302,10 +303,14 @@
:test-result: PASS comparisons between const int variables
:test-result: PASS comparisons between int variables
:test-result: PASS convertToBits
:test-result: SKIP dynamic skipping works with generators
:test-result: PASS empty tags are not allowed
:test-result: PASS erfc_inv
:test-result: PASS estimate_clock_resolution
:test-result: PASS even more nested SECTION tests
:test-result: XFAIL failed assertions before SKIP cause test case to fail
:test-result: XFAIL failing for some generator values causes entire test case to fail
:test-result: XFAIL failing in some unskipped sections causes entire test case to fail
:test-result: FAIL first tag
:test-result: FAIL has printf
:test-result: PASS is_unary_function
@@ -323,6 +328,7 @@
:test-result: FAIL mix info, unscoped info and warning
:test-result: FAIL more nested SECTION tests
:test-result: PASS nested SECTION tests
:test-result: FAIL nested sections can be skipped dynamically at runtime
:test-result: PASS non streamable - with conv. op
:test-result: PASS non-copyable objects
:test-result: PASS normal_cdf
@@ -344,9 +350,11 @@
:test-result: PASS run_for_at_least, chronometer
:test-result: PASS run_for_at_least, int
:test-result: FAIL second tag
:test-result: SKIP sections can be skipped dynamically at runtime
:test-result: FAIL send a single char to INFO
:test-result: FAIL sends information to INFO
:test-result: PASS shortened hide tags are split apart
:test-result: SKIP skipped tests can optionally provide a reason
:test-result: PASS splitString
:test-result: FAIL stacks unscoped info in loops
:test-result: PASS startsWith
@@ -368,6 +376,7 @@
:test-result: PASS strlen3
:test-result: PASS tables
:test-result: PASS tags with dots in later positions are not parsed as hidden
:test-result: SKIP tests can be skipped dynamically at runtime
:test-result: FAIL thrown std::strings are translated
:test-result: PASS toString on const wchar_t const pointer returns the string contents
:test-result: PASS toString on const wchar_t pointer returns the string contents