Fix bad error reporting for nested exceptions in default configuration

Bad handling of `TestFailureException` when translating unexpected
exceptions inside assertion macros led to the unexpected exceptions
handling erroring out through throwing the same exception again.

This was then backstopped by the machinery for handling uncaught
exceptions from assertions, which is normally used by the
`CATCH_CONFIG_FAST_COMPILE` machinery, where we assume that it can
only be invoked because the assertion macros are not configured to
catch assertions.

Closes #1292
This commit is contained in:
Martin Hořeňovský
2025-07-27 20:28:51 +02:00
parent fee81626d2
commit d547cae549
21 changed files with 595 additions and 23 deletions

View File

@@ -52,6 +52,20 @@ TEST_CASE("failing test") {
}
```
Same applies for a `SKIP` nested inside an assertion:
```cpp
static bool do_skip() {
SKIP();
return true;
}
TEST_CASE("Another failing test") {
CHECK(do_skip());
}
```
### Interaction with Sections and Generators
Sections, nested sections as well as specific outputs from [generators](generators.md#top)

View File

@@ -86,8 +86,6 @@ namespace Catch {
auto analysis = Detail::analyse(*cfg, samples.data(), samples.data() + samples.size());
BenchmarkStats<> stats{ CATCH_MOVE(info), CATCH_MOVE(analysis.samples), analysis.mean, analysis.standard_deviation, analysis.outliers, analysis.outlier_variance };
getResultCapture().benchmarkEnded(stats);
} CATCH_CATCH_ANON (TestFailureException const&) {
getResultCapture().benchmarkFailed("Benchmark failed due to failed assertion"_sr);
} CATCH_CATCH_ALL {
getResultCapture().benchmarkFailed(translateActiveException());
// We let the exception go further up so that the

View File

@@ -59,10 +59,10 @@ namespace Catch {
// To avoid having to handle TFE explicitly everywhere, we just
// rethrow it so that it goes back up the caller.
catch( TestFailureException& ) {
std::rethrow_exception(std::current_exception());
return "{ nested assertion failed }";
}
catch( TestSkipException& ) {
std::rethrow_exception(std::current_exception());
return "{ nested SKIP() called }";
}
catch( std::exception const& ex ) {
return ex.what();

View File

@@ -95,6 +95,8 @@ Nor would this
:test-result: PASS Approximate comparisons with mixed numeric types
:test-result: PASS Arbitrary predicate matcher
:test-result: PASS Assertion macros support bit operators and bool conversions
:test-result: XFAIL Assertions can be nested - CHECK
:test-result: XFAIL Assertions can be nested - REQUIRE
:test-result: PASS Assertions then sections
:test-result: PASS Basic use of the Contains range matcher
:test-result: PASS Basic use of the Empty range matcher
@@ -153,6 +155,7 @@ Nor would this
:test-result: PASS Exceptions matchers
:test-result: FAIL Expected exceptions that don't throw or unexpected exceptions fail the test
:test-result: FAIL FAIL aborts the test
:test-result: XFAIL FAIL can be nested in assertion
:test-result: FAIL FAIL does not require an argument
:test-result: FAIL FAIL_CHECK does not abort the test
:test-result: PASS Factorials are computed

View File

@@ -93,6 +93,8 @@
:test-result: PASS Approximate comparisons with mixed numeric types
:test-result: PASS Arbitrary predicate matcher
:test-result: PASS Assertion macros support bit operators and bool conversions
:test-result: XFAIL Assertions can be nested - CHECK
:test-result: XFAIL Assertions can be nested - REQUIRE
:test-result: PASS Assertions then sections
:test-result: PASS Basic use of the Contains range matcher
:test-result: PASS Basic use of the Empty range matcher
@@ -151,6 +153,7 @@
:test-result: PASS Exceptions matchers
:test-result: FAIL Expected exceptions that don't throw or unexpected exceptions fail the test
:test-result: FAIL FAIL aborts the test
:test-result: XFAIL FAIL can be nested in assertion
:test-result: FAIL FAIL does not require an argument
:test-result: FAIL FAIL_CHECK does not abort the test
:test-result: PASS Factorials are computed

View File

@@ -332,6 +332,11 @@ Compilation.tests.cpp:<line number>: passed: !(lhs & rhs) for: !(Val: 1 & Val: 2
Compilation.tests.cpp:<line number>: passed: HasBitOperators{ 1 } & HasBitOperators{ 1 } for: Val: 1 & Val: 1
Compilation.tests.cpp:<line number>: passed: lhs ^ rhs for: Val: 1 ^ Val: 2
Compilation.tests.cpp:<line number>: passed: !(lhs ^ lhs) for: !(Val: 1 ^ Val: 1)
AssertionHandler.tests.cpp:<line number>: failed: i > 10 for: 2 > 10
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
AssertionHandler.tests.cpp:<line number>: passed: true
AssertionHandler.tests.cpp:<line number>: failed: i > 10 for: 2 > 10
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
Tricky.tests.cpp:<line number>: passed: true
Tricky.tests.cpp:<line number>: passed: true
Tricky.tests.cpp:<line number>: passed: true
@@ -636,6 +641,8 @@ Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'e
Exception.tests.cpp:<line number>: failed: expected exception, got none; expression was: thisDoesntThrow(), std::domain_error
Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'expected exception'; expression was: thisThrows()
Message.tests.cpp:<line number>: failed: explicitly with 1 message: 'This is a failure'
AssertionHandler.tests.cpp:<line number>: failed: explicitly with 1 message: 'Throw a Catch::TestFailureException'
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: do_fail()
Message.tests.cpp:<line number>: failed: explicitly
Message.tests.cpp:<line number>: failed: explicitly with 1 message: 'This is a failure'
Message.tests.cpp:<line number>: warning: 'This message appears in the output'
@@ -2862,7 +2869,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
Misc.tests.cpp:<line number>: passed:
Misc.tests.cpp:<line number>: passed:
test cases: 428 | 313 passed | 95 failed | 6 skipped | 14 failed as expected
assertions: 2281 | 2089 passed | 157 failed | 35 failed as expected
test cases: 431 | 313 passed | 95 failed | 6 skipped | 17 failed as expected
assertions: 2288 | 2090 passed | 157 failed | 41 failed as expected

View File

@@ -330,6 +330,11 @@ Compilation.tests.cpp:<line number>: passed: !(lhs & rhs) for: !(Val: 1 & Val: 2
Compilation.tests.cpp:<line number>: passed: HasBitOperators{ 1 } & HasBitOperators{ 1 } for: Val: 1 & Val: 1
Compilation.tests.cpp:<line number>: passed: lhs ^ rhs for: Val: 1 ^ Val: 2
Compilation.tests.cpp:<line number>: passed: !(lhs ^ lhs) for: !(Val: 1 ^ Val: 1)
AssertionHandler.tests.cpp:<line number>: failed: i > 10 for: 2 > 10
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
AssertionHandler.tests.cpp:<line number>: passed: true
AssertionHandler.tests.cpp:<line number>: failed: i > 10 for: 2 > 10
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
Tricky.tests.cpp:<line number>: passed: true
Tricky.tests.cpp:<line number>: passed: true
Tricky.tests.cpp:<line number>: passed: true
@@ -634,6 +639,8 @@ Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'e
Exception.tests.cpp:<line number>: failed: expected exception, got none; expression was: thisDoesntThrow(), std::domain_error
Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'expected exception'; expression was: thisThrows()
Message.tests.cpp:<line number>: failed: explicitly with 1 message: 'This is a failure'
AssertionHandler.tests.cpp:<line number>: failed: explicitly with 1 message: 'Throw a Catch::TestFailureException'
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: do_fail()
Message.tests.cpp:<line number>: failed: explicitly
Message.tests.cpp:<line number>: failed: explicitly with 1 message: 'This is a failure'
Message.tests.cpp:<line number>: warning: 'This message appears in the output'
@@ -2851,7 +2858,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
Misc.tests.cpp:<line number>: passed:
Misc.tests.cpp:<line number>: passed:
test cases: 428 | 313 passed | 95 failed | 6 skipped | 14 failed as expected
assertions: 2281 | 2089 passed | 157 failed | 35 failed as expected
test cases: 431 | 313 passed | 95 failed | 6 skipped | 17 failed as expected
assertions: 2288 | 2090 passed | 157 failed | 41 failed as expected

View File

@@ -347,6 +347,38 @@ Exception.tests.cpp:<line number>: FAILED:
due to unexpected exception with message:
unexpected exception
-------------------------------------------------------------------------------
Assertions can be nested - CHECK
-------------------------------------------------------------------------------
AssertionHandler.tests.cpp:<line number>
...............................................................................
AssertionHandler.tests.cpp:<line number>: FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
AssertionHandler.tests.cpp:<line number>: FAILED:
CHECK( foo( 2 ) == 2 )
due to unexpected exception with message:
{ nested assertion failed }
-------------------------------------------------------------------------------
Assertions can be nested - REQUIRE
-------------------------------------------------------------------------------
AssertionHandler.tests.cpp:<line number>
...............................................................................
AssertionHandler.tests.cpp:<line number>: FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
AssertionHandler.tests.cpp:<line number>: FAILED:
REQUIRE( foo( 2 ) == 2 )
due to unexpected exception with message:
{ nested assertion failed }
-------------------------------------------------------------------------------
Captures do not leave block with an exception
-------------------------------------------------------------------------------
@@ -619,6 +651,21 @@ Message.tests.cpp:<line number>: FAILED:
explicitly with message:
This is a failure
-------------------------------------------------------------------------------
FAIL can be nested in assertion
-------------------------------------------------------------------------------
AssertionHandler.tests.cpp:<line number>
...............................................................................
AssertionHandler.tests.cpp:<line number>: FAILED:
explicitly with message:
Throw a Catch::TestFailureException
AssertionHandler.tests.cpp:<line number>: FAILED:
CHECK_NOTHROW( do_fail() )
due to unexpected exception with message:
{ nested assertion failed }
-------------------------------------------------------------------------------
FAIL does not require an argument
-------------------------------------------------------------------------------
@@ -1672,6 +1719,6 @@ due to unexpected exception with message:
Why would you throw a std::string?
===============================================================================
test cases: 428 | 331 passed | 76 failed | 7 skipped | 14 failed as expected
assertions: 2260 | 2089 passed | 136 failed | 35 failed as expected
test cases: 431 | 331 passed | 76 failed | 7 skipped | 17 failed as expected
assertions: 2267 | 2090 passed | 136 failed | 41 failed as expected

View File

@@ -2566,6 +2566,41 @@ Compilation.tests.cpp:<line number>: PASSED:
with expansion:
!(Val: 1 ^ Val: 1)
-------------------------------------------------------------------------------
Assertions can be nested - CHECK
-------------------------------------------------------------------------------
AssertionHandler.tests.cpp:<line number>
...............................................................................
AssertionHandler.tests.cpp:<line number>: FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
AssertionHandler.tests.cpp:<line number>: FAILED:
CHECK( foo( 2 ) == 2 )
due to unexpected exception with message:
{ nested assertion failed }
AssertionHandler.tests.cpp:<line number>: PASSED:
CHECK( true )
-------------------------------------------------------------------------------
Assertions can be nested - REQUIRE
-------------------------------------------------------------------------------
AssertionHandler.tests.cpp:<line number>
...............................................................................
AssertionHandler.tests.cpp:<line number>: FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
AssertionHandler.tests.cpp:<line number>: FAILED:
REQUIRE( foo( 2 ) == 2 )
due to unexpected exception with message:
{ nested assertion failed }
-------------------------------------------------------------------------------
Assertions then sections
-------------------------------------------------------------------------------
@@ -4561,6 +4596,21 @@ Message.tests.cpp:<line number>: FAILED:
explicitly with message:
This is a failure
-------------------------------------------------------------------------------
FAIL can be nested in assertion
-------------------------------------------------------------------------------
AssertionHandler.tests.cpp:<line number>
...............................................................................
AssertionHandler.tests.cpp:<line number>: FAILED:
explicitly with message:
Throw a Catch::TestFailureException
AssertionHandler.tests.cpp:<line number>: FAILED:
CHECK_NOTHROW( do_fail() )
due to unexpected exception with message:
{ nested assertion failed }
-------------------------------------------------------------------------------
FAIL does not require an argument
-------------------------------------------------------------------------------
@@ -19134,6 +19184,6 @@ Misc.tests.cpp:<line number>
Misc.tests.cpp:<line number>: PASSED:
===============================================================================
test cases: 428 | 313 passed | 95 failed | 6 skipped | 14 failed as expected
assertions: 2281 | 2089 passed | 157 failed | 35 failed as expected
test cases: 431 | 313 passed | 95 failed | 6 skipped | 17 failed as expected
assertions: 2288 | 2090 passed | 157 failed | 41 failed as expected

View File

@@ -2564,6 +2564,41 @@ Compilation.tests.cpp:<line number>: PASSED:
with expansion:
!(Val: 1 ^ Val: 1)
-------------------------------------------------------------------------------
Assertions can be nested - CHECK
-------------------------------------------------------------------------------
AssertionHandler.tests.cpp:<line number>
...............................................................................
AssertionHandler.tests.cpp:<line number>: FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
AssertionHandler.tests.cpp:<line number>: FAILED:
CHECK( foo( 2 ) == 2 )
due to unexpected exception with message:
{ nested assertion failed }
AssertionHandler.tests.cpp:<line number>: PASSED:
CHECK( true )
-------------------------------------------------------------------------------
Assertions can be nested - REQUIRE
-------------------------------------------------------------------------------
AssertionHandler.tests.cpp:<line number>
...............................................................................
AssertionHandler.tests.cpp:<line number>: FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
AssertionHandler.tests.cpp:<line number>: FAILED:
REQUIRE( foo( 2 ) == 2 )
due to unexpected exception with message:
{ nested assertion failed }
-------------------------------------------------------------------------------
Assertions then sections
-------------------------------------------------------------------------------
@@ -4559,6 +4594,21 @@ Message.tests.cpp:<line number>: FAILED:
explicitly with message:
This is a failure
-------------------------------------------------------------------------------
FAIL can be nested in assertion
-------------------------------------------------------------------------------
AssertionHandler.tests.cpp:<line number>
...............................................................................
AssertionHandler.tests.cpp:<line number>: FAILED:
explicitly with message:
Throw a Catch::TestFailureException
AssertionHandler.tests.cpp:<line number>: FAILED:
CHECK_NOTHROW( do_fail() )
due to unexpected exception with message:
{ nested assertion failed }
-------------------------------------------------------------------------------
FAIL does not require an argument
-------------------------------------------------------------------------------
@@ -19123,6 +19173,6 @@ Misc.tests.cpp:<line number>
Misc.tests.cpp:<line number>: PASSED:
===============================================================================
test cases: 428 | 313 passed | 95 failed | 6 skipped | 14 failed as expected
assertions: 2281 | 2089 passed | 157 failed | 35 failed as expected
test cases: 431 | 313 passed | 95 failed | 6 skipped | 17 failed as expected
assertions: 2288 | 2090 passed | 157 failed | 41 failed as expected

View File

@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuitesloose text artifact
>
<testsuite name="<exe-name>" errors="17" failures="140" skipped="12" tests="2293" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<testsuite name="<exe-name>" errors="17" failures="140" skipped="12" tests="2300" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<properties>
<property name="random-seed" value="1"/>
<property name="filters" value="&quot;*&quot; ~[!nonportable] ~[!benchmark] ~[approvals]"/>
@@ -390,6 +390,38 @@ at Exception.tests.cpp:<line number>
<testcase classname="<exe-name>.global" name="Arbitrary predicate matcher/Function pointer" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Arbitrary predicate matcher/Lambdas + different type" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Assertion macros support bit operators and bool conversions" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Assertions can be nested - CHECK" time="{duration}" status="run">
<skipped message="TEST_CASE tagged with !mayfail"/>
<failure message="i > 10" type="REQUIRE">
FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
at AssertionHandler.tests.cpp:<line number>
</failure>
<error message="foo( 2 ) == 2" type="CHECK">
FAILED:
CHECK( foo( 2 ) == 2 )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</error>
</testcase>
<testcase classname="<exe-name>.global" name="Assertions can be nested - REQUIRE" time="{duration}" status="run">
<skipped message="TEST_CASE tagged with !mayfail"/>
<failure message="i > 10" type="REQUIRE">
FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
at AssertionHandler.tests.cpp:<line number>
</failure>
<error message="foo( 2 ) == 2" type="REQUIRE">
FAILED:
REQUIRE( foo( 2 ) == 2 )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</error>
</testcase>
<testcase classname="<exe-name>.global" name="Assertions then sections" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Assertions then sections/A section" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Assertions then sections/A section/Another section" time="{duration}" status="run"/>
@@ -738,6 +770,20 @@ This is a failure
at Message.tests.cpp:<line number>
</failure>
</testcase>
<testcase classname="<exe-name>.global" name="FAIL can be nested in assertion" time="{duration}" status="run">
<skipped message="TEST_CASE tagged with !mayfail"/>
<failure type="FAIL">
FAILED:
Throw a Catch::TestFailureException
at AssertionHandler.tests.cpp:<line number>
</failure>
<error message="do_fail()" type="CHECK_NOTHROW">
FAILED:
CHECK_NOTHROW( do_fail() )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</error>
</testcase>
<testcase classname="<exe-name>.global" name="FAIL does not require an argument" time="{duration}" status="run">
<failure type="FAIL">
FAILED:

View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="<exe-name>" errors="17" failures="140" skipped="12" tests="2293" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<testsuite name="<exe-name>" errors="17" failures="140" skipped="12" tests="2300" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<properties>
<property name="random-seed" value="1"/>
<property name="filters" value="&quot;*&quot; ~[!nonportable] ~[!benchmark] ~[approvals]"/>
@@ -389,6 +389,38 @@ at Exception.tests.cpp:<line number>
<testcase classname="<exe-name>.global" name="Arbitrary predicate matcher/Function pointer" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Arbitrary predicate matcher/Lambdas + different type" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Assertion macros support bit operators and bool conversions" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Assertions can be nested - CHECK" time="{duration}" status="run">
<skipped message="TEST_CASE tagged with !mayfail"/>
<failure message="i > 10" type="REQUIRE">
FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
at AssertionHandler.tests.cpp:<line number>
</failure>
<error message="foo( 2 ) == 2" type="CHECK">
FAILED:
CHECK( foo( 2 ) == 2 )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</error>
</testcase>
<testcase classname="<exe-name>.global" name="Assertions can be nested - REQUIRE" time="{duration}" status="run">
<skipped message="TEST_CASE tagged with !mayfail"/>
<failure message="i > 10" type="REQUIRE">
FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
at AssertionHandler.tests.cpp:<line number>
</failure>
<error message="foo( 2 ) == 2" type="REQUIRE">
FAILED:
REQUIRE( foo( 2 ) == 2 )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</error>
</testcase>
<testcase classname="<exe-name>.global" name="Assertions then sections" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Assertions then sections/A section" time="{duration}" status="run"/>
<testcase classname="<exe-name>.global" name="Assertions then sections/A section/Another section" time="{duration}" status="run"/>
@@ -737,6 +769,20 @@ This is a failure
at Message.tests.cpp:<line number>
</failure>
</testcase>
<testcase classname="<exe-name>.global" name="FAIL can be nested in assertion" time="{duration}" status="run">
<skipped message="TEST_CASE tagged with !mayfail"/>
<failure type="FAIL">
FAILED:
Throw a Catch::TestFailureException
at AssertionHandler.tests.cpp:<line number>
</failure>
<error message="do_fail()" type="CHECK_NOTHROW">
FAILED:
CHECK_NOTHROW( do_fail() )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</error>
</testcase>
<testcase classname="<exe-name>.global" name="FAIL does not require an argument" time="{duration}" status="run">
<failure type="FAIL">
FAILED:

View File

@@ -3,6 +3,49 @@
<testExecutions version="1"loose text artifact
>
<file path="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp">
<testCase name="Assertions can be nested - CHECK" duration="{duration}">
<skipped message="REQUIRE(i > 10)">
FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
at AssertionHandler.tests.cpp:<line number>
</skipped>
<skipped message="CHECK(foo( 2 ) == 2)">
FAILED:
CHECK( foo( 2 ) == 2 )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="Assertions can be nested - REQUIRE" duration="{duration}">
<skipped message="REQUIRE(i > 10)">
FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
at AssertionHandler.tests.cpp:<line number>
</skipped>
<skipped message="REQUIRE(foo( 2 ) == 2)">
FAILED:
REQUIRE( foo( 2 ) == 2 )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="FAIL can be nested in assertion" duration="{duration}">
<skipped message="FAIL()">
FAILED:
Throw a Catch::TestFailureException
at AssertionHandler.tests.cpp:<line number>
</skipped>
<skipped message="CHECK_NOTHROW(do_fail())">
FAILED:
CHECK_NOTHROW( do_fail() )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="Incomplete AssertionHandler" duration="{duration}">
<skipped message="REQUIRE(Dummy)">
FAILED:

View File

@@ -2,6 +2,49 @@
<!-- filters='"*" ~[!nonportable] ~[!benchmark] ~[approvals]' rng-seed=1 -->
<testExecutions version="1">
<file path="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp">
<testCase name="Assertions can be nested - CHECK" duration="{duration}">
<skipped message="REQUIRE(i > 10)">
FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
at AssertionHandler.tests.cpp:<line number>
</skipped>
<skipped message="CHECK(foo( 2 ) == 2)">
FAILED:
CHECK( foo( 2 ) == 2 )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="Assertions can be nested - REQUIRE" duration="{duration}">
<skipped message="REQUIRE(i > 10)">
FAILED:
REQUIRE( i > 10 )
with expansion:
2 > 10
at AssertionHandler.tests.cpp:<line number>
</skipped>
<skipped message="REQUIRE(foo( 2 ) == 2)">
FAILED:
REQUIRE( foo( 2 ) == 2 )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="FAIL can be nested in assertion" duration="{duration}">
<skipped message="FAIL()">
FAILED:
Throw a Catch::TestFailureException
at AssertionHandler.tests.cpp:<line number>
</skipped>
<skipped message="CHECK_NOTHROW(do_fail())">
FAILED:
CHECK_NOTHROW( do_fail() )
{ nested assertion failed }
at AssertionHandler.tests.cpp:<line number>
</skipped>
</testCase>
<testCase name="Incomplete AssertionHandler" duration="{duration}">
<skipped message="REQUIRE(Dummy)">
FAILED:

View File

@@ -612,6 +612,16 @@ ok {test-number} - HasBitOperators{ 1 } & HasBitOperators{ 1 } for: Val: 1 & Val
ok {test-number} - lhs ^ rhs for: Val: 1 ^ Val: 2
# Assertion macros support bit operators and bool conversions
ok {test-number} - !(lhs ^ lhs) for: !(Val: 1 ^ Val: 1)
# Assertions can be nested - CHECK
not ok {test-number} - i > 10 for: 2 > 10
# Assertions can be nested - CHECK
not ok {test-number} - unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
# Assertions can be nested - CHECK
ok {test-number} - true
# Assertions can be nested - REQUIRE
not ok {test-number} - i > 10 for: 2 > 10
# Assertions can be nested - REQUIRE
not ok {test-number} - unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
# Assertions then sections
ok {test-number} - true
# Assertions then sections
@@ -1132,6 +1142,10 @@ not ok {test-number} - expected exception, got none; expression was: thisDoesntT
not ok {test-number} - unexpected exception with message: 'expected exception'; expression was: thisThrows()
# FAIL aborts the test
not ok {test-number} - explicitly with 1 message: 'This is a failure'
# FAIL can be nested in assertion
not ok {test-number} - explicitly with 1 message: 'Throw a Catch::TestFailureException'
# FAIL can be nested in assertion
not ok {test-number} - unexpected exception with message: '{ nested assertion failed }'; expression was: do_fail()
# FAIL does not require an argument
not ok {test-number} - explicitly
# FAIL_CHECK does not abort the test
@@ -4583,5 +4597,5 @@ ok {test-number} - q3 == 23. for: 23.0 == 23.0
ok {test-number} -
# xmlentitycheck
ok {test-number} -
1..2293
1..2300

View File

@@ -610,6 +610,16 @@ ok {test-number} - HasBitOperators{ 1 } & HasBitOperators{ 1 } for: Val: 1 & Val
ok {test-number} - lhs ^ rhs for: Val: 1 ^ Val: 2
# Assertion macros support bit operators and bool conversions
ok {test-number} - !(lhs ^ lhs) for: !(Val: 1 ^ Val: 1)
# Assertions can be nested - CHECK
not ok {test-number} - i > 10 for: 2 > 10
# Assertions can be nested - CHECK
not ok {test-number} - unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
# Assertions can be nested - CHECK
ok {test-number} - true
# Assertions can be nested - REQUIRE
not ok {test-number} - i > 10 for: 2 > 10
# Assertions can be nested - REQUIRE
not ok {test-number} - unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
# Assertions then sections
ok {test-number} - true
# Assertions then sections
@@ -1130,6 +1140,10 @@ not ok {test-number} - expected exception, got none; expression was: thisDoesntT
not ok {test-number} - unexpected exception with message: 'expected exception'; expression was: thisThrows()
# FAIL aborts the test
not ok {test-number} - explicitly with 1 message: 'This is a failure'
# FAIL can be nested in assertion
not ok {test-number} - explicitly with 1 message: 'Throw a Catch::TestFailureException'
# FAIL can be nested in assertion
not ok {test-number} - unexpected exception with message: '{ nested assertion failed }'; expression was: do_fail()
# FAIL does not require an argument
not ok {test-number} - explicitly
# FAIL_CHECK does not abort the test
@@ -4572,5 +4586,5 @@ ok {test-number} - q3 == 23. for: 23.0 == 23.0
ok {test-number} -
# xmlentitycheck
ok {test-number} -
1..2293
1..2300

View File

@@ -225,6 +225,14 @@
##teamcity[testFinished name='Arbitrary predicate matcher' duration="{duration}"]
##teamcity[testStarted name='Assertion macros support bit operators and bool conversions']
##teamcity[testFinished name='Assertion macros support bit operators and bool conversions' duration="{duration}"]
##teamcity[testStarted name='Assertions can be nested - CHECK']
##teamcity[testIgnored name='Assertions can be nested - CHECK' message='AssertionHandler.tests.cpp:<line number>|n...............................................................................|n|nAssertionHandler.tests.cpp:<line number>|nexpression failed|n REQUIRE( i > 10 )|nwith expansion:|n 2 > 10|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Assertions can be nested - CHECK' message='AssertionHandler.tests.cpp:<line number>|nunexpected exception with message:|n "{ nested assertion failed }"|n CHECK( foo( 2 ) == 2 )|nwith expansion:|n foo( 2 ) == 2|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Assertions can be nested - CHECK' duration="{duration}"]
##teamcity[testStarted name='Assertions can be nested - REQUIRE']
##teamcity[testIgnored name='Assertions can be nested - REQUIRE' message='AssertionHandler.tests.cpp:<line number>|n...............................................................................|n|nAssertionHandler.tests.cpp:<line number>|nexpression failed|n REQUIRE( i > 10 )|nwith expansion:|n 2 > 10|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Assertions can be nested - REQUIRE' message='AssertionHandler.tests.cpp:<line number>|nunexpected exception with message:|n "{ nested assertion failed }"|n REQUIRE( foo( 2 ) == 2 )|nwith expansion:|n foo( 2 ) == 2|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Assertions can be nested - REQUIRE' duration="{duration}"]
##teamcity[testStarted name='Assertions then sections']
##teamcity[testFinished name='Assertions then sections' duration="{duration}"]
##teamcity[testStarted name='Basic use of the Contains range matcher']
@@ -376,6 +384,10 @@
##teamcity[testStarted name='FAIL aborts the test']
##teamcity[testFailed name='FAIL aborts the test' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "This is a failure"']
##teamcity[testFinished name='FAIL aborts the test' duration="{duration}"]
##teamcity[testStarted name='FAIL can be nested in assertion']
##teamcity[testIgnored name='FAIL can be nested in assertion' message='AssertionHandler.tests.cpp:<line number>|n...............................................................................|n|nAssertionHandler.tests.cpp:<line number>|nexplicit failure with message:|n "Throw a Catch::TestFailureException"- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='FAIL can be nested in assertion' message='AssertionHandler.tests.cpp:<line number>|nunexpected exception with message:|n "{ nested assertion failed }"|n CHECK_NOTHROW( do_fail() )|nwith expansion:|n do_fail()|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='FAIL can be nested in assertion' duration="{duration}"]
##teamcity[testStarted name='FAIL does not require an argument']
##teamcity[testFailed name='FAIL does not require an argument' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure']
##teamcity[testFinished name='FAIL does not require an argument' duration="{duration}"]

View File

@@ -225,6 +225,14 @@
##teamcity[testFinished name='Arbitrary predicate matcher' duration="{duration}"]
##teamcity[testStarted name='Assertion macros support bit operators and bool conversions']
##teamcity[testFinished name='Assertion macros support bit operators and bool conversions' duration="{duration}"]
##teamcity[testStarted name='Assertions can be nested - CHECK']
##teamcity[testIgnored name='Assertions can be nested - CHECK' message='AssertionHandler.tests.cpp:<line number>|n...............................................................................|n|nAssertionHandler.tests.cpp:<line number>|nexpression failed|n REQUIRE( i > 10 )|nwith expansion:|n 2 > 10|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Assertions can be nested - CHECK' message='AssertionHandler.tests.cpp:<line number>|nunexpected exception with message:|n "{ nested assertion failed }"|n CHECK( foo( 2 ) == 2 )|nwith expansion:|n foo( 2 ) == 2|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Assertions can be nested - CHECK' duration="{duration}"]
##teamcity[testStarted name='Assertions can be nested - REQUIRE']
##teamcity[testIgnored name='Assertions can be nested - REQUIRE' message='AssertionHandler.tests.cpp:<line number>|n...............................................................................|n|nAssertionHandler.tests.cpp:<line number>|nexpression failed|n REQUIRE( i > 10 )|nwith expansion:|n 2 > 10|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Assertions can be nested - REQUIRE' message='AssertionHandler.tests.cpp:<line number>|nunexpected exception with message:|n "{ nested assertion failed }"|n REQUIRE( foo( 2 ) == 2 )|nwith expansion:|n foo( 2 ) == 2|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Assertions can be nested - REQUIRE' duration="{duration}"]
##teamcity[testStarted name='Assertions then sections']
##teamcity[testFinished name='Assertions then sections' duration="{duration}"]
##teamcity[testStarted name='Basic use of the Contains range matcher']
@@ -376,6 +384,10 @@
##teamcity[testStarted name='FAIL aborts the test']
##teamcity[testFailed name='FAIL aborts the test' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "This is a failure"']
##teamcity[testFinished name='FAIL aborts the test' duration="{duration}"]
##teamcity[testStarted name='FAIL can be nested in assertion']
##teamcity[testIgnored name='FAIL can be nested in assertion' message='AssertionHandler.tests.cpp:<line number>|n...............................................................................|n|nAssertionHandler.tests.cpp:<line number>|nexplicit failure with message:|n "Throw a Catch::TestFailureException"- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='FAIL can be nested in assertion' message='AssertionHandler.tests.cpp:<line number>|nunexpected exception with message:|n "{ nested assertion failed }"|n CHECK_NOTHROW( do_fail() )|nwith expansion:|n do_fail()|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='FAIL can be nested in assertion' duration="{duration}"]
##teamcity[testStarted name='FAIL does not require an argument']
##teamcity[testFailed name='FAIL does not require an argument' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure']
##teamcity[testFinished name='FAIL does not require an argument' duration="{duration}"]

View File

@@ -2716,6 +2716,58 @@ Approx( 1.23399996757507324 )
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="Assertions can be nested - CHECK" tags="[!shouldfail][assertions]" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Expression success="false" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
i > 10
</Original>
<Expanded>
2 > 10
</Expanded>
</Expression>
<Expression success="false" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
foo( 2 ) == 2
</Original>
<Expanded>
foo( 2 ) == 2
</Expanded>
<Exception filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
{ nested assertion failed }
</Exception>
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
true
</Original>
<Expanded>
true
</Expanded>
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="Assertions can be nested - REQUIRE" tags="[!shouldfail][assertions]" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Expression success="false" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
i > 10
</Original>
<Expanded>
2 > 10
</Expanded>
</Expression>
<Expression success="false" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
foo( 2 ) == 2
</Original>
<Expanded>
foo( 2 ) == 2
</Expanded>
<Exception filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
{ nested assertion failed }
</Exception>
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="Assertions then sections" tags="[Tricky]" filename="tests/<exe-name>/UsageTests/Tricky.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/UsageTests/Tricky.tests.cpp" >
<Original>
@@ -5124,6 +5176,23 @@ Approx( 1.30000000000000004 )
</Failure>
<OverallResult success="false" skips="0"/>
</TestCase>
<TestCase name="FAIL can be nested in assertion" tags="[!shouldfail][assertions]" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Failure filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
Throw a Catch::TestFailureException
</Failure>
<Expression success="false" type="CHECK_NOTHROW" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
do_fail()
</Original>
<Expanded>
do_fail()
</Expanded>
<Exception filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
{ nested assertion failed }
</Exception>
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="FAIL does not require an argument" tags="[.][failing][messages]" filename="tests/<exe-name>/UsageTests/Message.tests.cpp" >
<Failure filename="tests/<exe-name>/UsageTests/Message.tests.cpp" />
<OverallResult success="false" skips="0"/>
@@ -22113,6 +22182,6 @@ Approx( -1.95996398454005449 )
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<OverallResults successes="2089" failures="157" expectedFailures="35" skips="12"/>
<OverallResultsCases successes="313" failures="95" expectedFailures="14" skips="6"/>
<OverallResults successes="2090" failures="157" expectedFailures="41" skips="12"/>
<OverallResultsCases successes="313" failures="95" expectedFailures="17" skips="6"/>
</Catch2TestRun>

View File

@@ -2716,6 +2716,58 @@ Approx( 1.23399996757507324 )
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="Assertions can be nested - CHECK" tags="[!shouldfail][assertions]" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Expression success="false" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
i > 10
</Original>
<Expanded>
2 > 10
</Expanded>
</Expression>
<Expression success="false" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
foo( 2 ) == 2
</Original>
<Expanded>
foo( 2 ) == 2
</Expanded>
<Exception filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
{ nested assertion failed }
</Exception>
</Expression>
<Expression success="true" type="CHECK" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
true
</Original>
<Expanded>
true
</Expanded>
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="Assertions can be nested - REQUIRE" tags="[!shouldfail][assertions]" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Expression success="false" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
i > 10
</Original>
<Expanded>
2 > 10
</Expanded>
</Expression>
<Expression success="false" type="REQUIRE" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
foo( 2 ) == 2
</Original>
<Expanded>
foo( 2 ) == 2
</Expanded>
<Exception filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
{ nested assertion failed }
</Exception>
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="Assertions then sections" tags="[Tricky]" filename="tests/<exe-name>/UsageTests/Tricky.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="tests/<exe-name>/UsageTests/Tricky.tests.cpp" >
<Original>
@@ -5124,6 +5176,23 @@ Approx( 1.30000000000000004 )
</Failure>
<OverallResult success="false" skips="0"/>
</TestCase>
<TestCase name="FAIL can be nested in assertion" tags="[!shouldfail][assertions]" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Failure filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
Throw a Catch::TestFailureException
</Failure>
<Expression success="false" type="CHECK_NOTHROW" filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
<Original>
do_fail()
</Original>
<Expanded>
do_fail()
</Expanded>
<Exception filename="tests/<exe-name>/IntrospectiveTests/AssertionHandler.tests.cpp" >
{ nested assertion failed }
</Exception>
</Expression>
<OverallResult success="true" skips="0"/>
</TestCase>
<TestCase name="FAIL does not require an argument" tags="[.][failing][messages]" filename="tests/<exe-name>/UsageTests/Message.tests.cpp" >
<Failure filename="tests/<exe-name>/UsageTests/Message.tests.cpp" />
<OverallResult success="false" skips="0"/>
@@ -22112,6 +22181,6 @@ Approx( -1.95996398454005449 )
</Section>
<OverallResult success="true" skips="0"/>
</TestCase>
<OverallResults successes="2089" failures="157" expectedFailures="35" skips="12"/>
<OverallResultsCases successes="313" failures="95" expectedFailures="14" skips="6"/>
<OverallResults successes="2090" failures="157" expectedFailures="41" skips="12"/>
<OverallResultsCases successes="313" failures="95" expectedFailures="17" skips="6"/>
</Catch2TestRun>

View File

@@ -15,3 +15,28 @@ TEST_CASE( "Incomplete AssertionHandler", "[assertion-handler][!shouldfail]" ) {
"Dummy",
Catch::ResultDisposition::Normal );
}
static int foo( int i ) {
REQUIRE( i > 10 );
return 42;
}
TEST_CASE( "Assertions can be nested - CHECK", "[assertions][!shouldfail]" ) {
CHECK( foo( 2 ) == 2 );
// We should hit this, because CHECK continues on failure
CHECK( true );
}
TEST_CASE( "Assertions can be nested - REQUIRE", "[assertions][!shouldfail]" ) {
REQUIRE( foo( 2 ) == 2 );
// We should not hit this, because REQUIRE does not continue on failure
CHECK( true );
}
static void do_fail() { FAIL( "Throw a Catch::TestFailureException" ); }
TEST_CASE( "FAIL can be nested in assertion", "[assertions][!shouldfail]" ) {
// Fails, but the error message makes sense.
CHECK_NOTHROW( do_fail() );
}