Add new SKIP macro for skipping tests at runtime (#2360)

* Add new SKIP macro for skipping tests at runtime

This adds a new `SKIP` macro for dynamically skipping tests at runtime.
The "skipped" status of a test case is treated as a first-class citizen,
like "succeeded" or "failed", and is reported with a new color on the
console.

* Don't show "skipped assertions" in console/compact reporters

Also extend skip tests to cover a few more use cases.

* Return exit code 4 if all test cases are skipped

* Use LightGrey for the skip colour

This isn't great, but is better than the deep blue that was borderline
invisible on dark backgrounds. The fix is to redo the colouring
a bit, including introducing light-blue that is actually visible.

* Add support for explicit skips in all reporters

* --allow-running-no-tests also allows all tests to be skipped

* Add docs for SKIP macro, deprecate IEventListener::skipTest

Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
This commit is contained in:
Philip Salzmann
2023-01-12 15:01:47 +01:00
committed by GitHub
parent 52066dbc2a
commit d548be26e3
47 changed files with 3722 additions and 2171 deletions

View File

@@ -116,6 +116,7 @@ set(TEST_SOURCES
${SELF_TEST_DIR}/UsageTests/Generators.tests.cpp
${SELF_TEST_DIR}/UsageTests/Message.tests.cpp
${SELF_TEST_DIR}/UsageTests/Misc.tests.cpp
${SELF_TEST_DIR}/UsageTests/Skip.tests.cpp
${SELF_TEST_DIR}/UsageTests/ToStringByte.tests.cpp
${SELF_TEST_DIR}/UsageTests/ToStringChrono.tests.cpp
${SELF_TEST_DIR}/UsageTests/ToStringGeneral.tests.cpp
@@ -272,6 +273,10 @@ add_test(NAME TestSpecs::OverrideFailureWithNoMatchedTests
COMMAND $<TARGET_FILE:SelfTest> "___nonexistent_test___" --allow-running-no-tests
)
add_test(NAME TestSpecs::OverrideAllSkipFailure
COMMAND $<TARGET_FILE:SelfTest> "tests can be skipped dynamically at runtime" --allow-running-no-tests
)
add_test(NAME TestSpecs::NonMatchingTestSpecIsRoundTrippable
COMMAND $<TARGET_FILE:SelfTest> Tracker, "this test does not exist" "[nor does this tag]"
)