3rd-party/catch2
1
0
Fork 0
mirror of https://github.com/catchorg/Catch2.git synced 2025-08-03 13:55:39 +02:00
Code Issues Packages Projects Releases Wiki Activity

Add new SKIP macro for skipping tests at runtime (#2360)

Browse Source 
* Add new SKIP macro for skipping tests at runtime

This adds a new `SKIP` macro for dynamically skipping tests at runtime.
The "skipped" status of a test case is treated as a first-class citizen,
like "succeeded" or "failed", and is reported with a new color on the
console.

* Don't show "skipped assertions" in console/compact reporters

Also extend skip tests to cover a few more use cases.

* Return exit code 4 if all test cases are skipped

* Use LightGrey for the skip colour

This isn't great, but is better than the deep blue that was borderline
invisible on dark backgrounds. The fix is to redo the colouring
a bit, including introducing light-blue that is actually visible.

* Add support for explicit skips in all reporters

* --allow-running-no-tests also allows all tests to be skipped

* Add docs for SKIP macro, deprecate IEventListener::skipTest

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
This commit is contained in:
Philip Salzmann
2023-01-12 15:01:47 +01:00
committed by GitHub
parent 52066dbc2a
commit d548be26e3
47 changed files with 3722 additions and 2171 deletions
Download Patch File Download Diff File Expand all files Collapse all files

89
tests/SelfTest/Baselines/sonarqube.sw.approved.txt
View File

@@ -1832,6 +1832,95 @@ at Misc.tests.cpp:<line number>
<testCase name="xmlentitycheck/embedded xml: &lt;test>it should be possible to embed xml characters, such as &lt;, &quot; or &amp;, or even whole &lt;xml>documents&lt;/xml> within an attribute&lt;/test>" duration="{duration}"/>
<testCase name="xmlentitycheck/encoded chars: these should all be encoded: &amp;&amp;&amp;&quot;&quot;&quot;&lt;&lt;&lt;&amp;&quot;&lt;&lt;&amp;&quot;" duration="{duration}"/>
</file>
<file path="tests/<exe-name>/UsageTests/Skip.tests.cpp">
<testCase name="a succeeding test can still be skipped" duration="{duration}">
<skipped message="SKIP()">
SKIPPED
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="dynamic skipping works with generators" duration="{duration}">
<skipped message="SKIP()">
SKIPPED
skipping because answer = 41
at Skip.tests.cpp:<line number>
</skipped>
<skipped message="SKIP()">
SKIPPED
skipping because answer = 43
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="failed assertions before SKIP cause test case to fail" duration="{duration}">
<skipped message="CHECK(3 == 4)">
FAILED:
CHECK( 3 == 4 )
at Skip.tests.cpp:<line number>
</skipped>
<skipped message="SKIP()">
SKIPPED
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="failing for some generator values causes entire test case to fail" duration="{duration}">
<skipped message="FAIL()">
FAILED:
at Skip.tests.cpp:<line number>
</skipped>
<skipped message="SKIP()">
SKIPPED
at Skip.tests.cpp:<line number>
</skipped>
<skipped message="FAIL()">
FAILED:
at Skip.tests.cpp:<line number>
</skipped>
<skipped message="SKIP()">
SKIPPED
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="failing in some unskipped sections causes entire test case to fail/skipped" duration="{duration}">
<skipped message="SKIP()">
SKIPPED
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="failing in some unskipped sections causes entire test case to fail/not skipped" duration="{duration}">
<skipped message="FAIL()">
FAILED:
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="nested sections can be skipped dynamically at runtime/B2/B" duration="{duration}">
<skipped message="SKIP()">
SKIPPED
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="nested sections can be skipped dynamically at runtime/B" duration="{duration}"/>
<testCase name="sections can be skipped dynamically at runtime/not skipped" duration="{duration}"/>
<testCase name="sections can be skipped dynamically at runtime/skipped" duration="{duration}">
<skipped message="SKIP()">
SKIPPED
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="sections can be skipped dynamically at runtime/also not skipped" duration="{duration}"/>
<testCase name="skipped tests can optionally provide a reason" duration="{duration}">
<skipped message="SKIP()">
SKIPPED
skipping because answer = 43
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="tests can be skipped dynamically at runtime" duration="{duration}">
<skipped message="SKIP()">
SKIPPED
at Skip.tests.cpp:<line number>
</skipped>
</testCase>
</file>
<file path="tests/<exe-name>/UsageTests/ToStringChrono.tests.cpp">
<testCase name="Stringifying std::chrono::duration helpers" duration="{duration}"/>
<testCase name="Stringifying std::chrono::duration with weird ratios" duration="{duration}"/>