approved tests that were correct the first time (previous approvals were erroneous)

This commit is contained in:
Phil Nash 2017-04-09 20:40:01 +01:00
parent c6990cdf91
commit 714d01c07c
5 changed files with 245 additions and 17 deletions

View File

@ -3,6 +3,18 @@
<exe-name> is a <version> host application.
Run with -? for options
-------------------------------------------------------------------------------
#748 - captures with unexpected exceptions
outside assertions
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with messages:
answer := 42
expected exception
-------------------------------------------------------------------------------
#748 - captures with unexpected exceptions
inside REQUIRE_NOTHROW
@ -115,6 +127,17 @@ TrickyTests.cpp:<line number>: FAILED:
with expansion:
{?} == {?}
-------------------------------------------------------------------------------
An unchecked exception reports the line of the last assertion
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
{Unknown expression after the reported line}
due to unexpected exception with message:
unexpected exception
-------------------------------------------------------------------------------
Contains string matcher
-------------------------------------------------------------------------------
@ -148,6 +171,16 @@ ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom exception - not std
-------------------------------------------------------------------------------
Custom std-exceptions can be custom translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom std exception
-------------------------------------------------------------------------------
EndsWith string matcher
-------------------------------------------------------------------------------
@ -413,6 +446,16 @@ MiscTests.cpp:<line number>:
warning:
This one ran
-------------------------------------------------------------------------------
Non-std exceptions can be translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom exception
-------------------------------------------------------------------------------
Ordering comparison checks that should fail
-------------------------------------------------------------------------------
@ -599,6 +642,16 @@ with expansion:
}
"
-------------------------------------------------------------------------------
Unexpected exceptions can be translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
3.14
-------------------------------------------------------------------------------
Vector matchers that fail
Contains (element)
@ -660,6 +713,16 @@ MatchersTests.cpp:<line number>: FAILED:
with expansion:
{ 1, 2, 3 } Equals: { }
-------------------------------------------------------------------------------
When unchecked exceptions are thrown directly they are always failures
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
unexpected exception
-------------------------------------------------------------------------------
When unchecked exceptions are thrown during a CHECK the test should continue
-------------------------------------------------------------------------------
@ -694,6 +757,17 @@ ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
expected exception
-------------------------------------------------------------------------------
When unchecked exceptions are thrown from sections they are always failures
section name
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
unexpected exception
-------------------------------------------------------------------------------
Where the LHS is not a simple value
-------------------------------------------------------------------------------
@ -866,6 +940,6 @@ with expansion:
"first" == "second"
===============================================================================
test cases: 167 | 125 passed | 38 failed | 4 failed as expected
assertions: 960 | 859 passed | 81 failed | 20 failed as expected
test cases: 167 | 119 passed | 44 failed | 4 failed as expected
assertions: 967 | 859 passed | 87 failed | 21 failed as expected

View File

@ -14,6 +14,18 @@ PASSED:
with message:
yay
-------------------------------------------------------------------------------
#748 - captures with unexpected exceptions
outside assertions
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with messages:
answer := 42
expected exception
-------------------------------------------------------------------------------
#748 - captures with unexpected exceptions
inside REQUIRE_NOTHROW
@ -444,6 +456,11 @@ ExceptionTests.cpp:<line number>:
PASSED:
CHECK( 1 == 1 )
ExceptionTests.cpp:<line number>: FAILED:
{Unknown expression after the reported line}
due to unexpected exception with message:
unexpected exception
-------------------------------------------------------------------------------
Anonymous test case 1
-------------------------------------------------------------------------------
@ -998,6 +1015,16 @@ ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom exception - not std
-------------------------------------------------------------------------------
Custom std-exceptions can be custom translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom std exception
-------------------------------------------------------------------------------
Demonstrate that a non-const == is not used
-------------------------------------------------------------------------------
@ -4508,6 +4535,16 @@ MiscTests.cpp:<line number>:
warning:
This one ran
-------------------------------------------------------------------------------
Non-std exceptions can be translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom exception
-------------------------------------------------------------------------------
NotImplemented exception
-------------------------------------------------------------------------------
@ -7945,6 +7982,16 @@ PASSED:
with expansion:
true
-------------------------------------------------------------------------------
Unexpected exceptions can be translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
3.14
-------------------------------------------------------------------------------
Use a custom approx
-------------------------------------------------------------------------------
@ -8165,6 +8212,16 @@ ExceptionTests.cpp:<line number>:
PASSED:
REQUIRE_THROWS( thisThrows() )
-------------------------------------------------------------------------------
When unchecked exceptions are thrown directly they are always failures
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
unexpected exception
-------------------------------------------------------------------------------
When unchecked exceptions are thrown during a CHECK the test should continue
-------------------------------------------------------------------------------
@ -8199,6 +8256,17 @@ ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
expected exception
-------------------------------------------------------------------------------
When unchecked exceptions are thrown from sections they are always failures
section name
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
unexpected exception
-------------------------------------------------------------------------------
Where the LHS is not a simple value
-------------------------------------------------------------------------------
@ -9398,6 +9466,6 @@ MiscTests.cpp:<line number>:
PASSED:
===============================================================================
test cases: 167 | 124 passed | 39 failed | 4 failed as expected
assertions: 962 | 859 passed | 83 failed | 20 failed as expected
test cases: 167 | 118 passed | 45 failed | 4 failed as expected
assertions: 969 | 859 passed | 89 failed | 21 failed as expected

View File

@ -14,6 +14,18 @@ PASSED:
with message:
yay
-------------------------------------------------------------------------------
#748 - captures with unexpected exceptions
outside assertions
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................
ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with messages:
answer := 42
expected exception
-------------------------------------------------------------------------------
#748 - captures with unexpected exceptions
inside REQUIRE_NOTHROW
@ -133,5 +145,5 @@ ConditionTests.cpp:<line number>: FAILED:
===============================================================================
test cases: 6 | 3 passed | 1 failed | 2 failed as expected
assertions: 17 | 11 passed | 4 failed | 2 failed as expected
assertions: 18 | 11 passed | 4 failed | 3 failed as expected

View File

@ -1,7 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuitesspanner>
<testsuite name="<exe-name>" errors="8" failures="76" tests="963" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<testsuite name="<exe-name>" errors="15" failures="75" tests="970" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<testcase classname="global" name="# A test name that starts with a #" time="{duration}"/>
<testcase classname="#748 - captures with unexpected exceptions" name="outside assertions" time="{duration}">
<error type="TEST_CASE">
expected exception
answer := 42
ExceptionTests.cpp:<line number>
</error>
</testcase>
<testcase classname="#748 - captures with unexpected exceptions" name="inside REQUIRE_NOTHROW" time="{duration}">
<error message="thisThrows()" type="REQUIRE_NOTHROW">
expected exception
@ -79,7 +86,12 @@ TrickyTests.cpp:<line number>
<testcase classname="global" name="Absolute margin" time="{duration}"/>
<testcase classname="global" name="AllOf matcher" time="{duration}"/>
<testcase classname="global" name="An expression with side-effects should only be evaluated once" time="{duration}"/>
<testcase classname="global" name="An unchecked exception reports the line of the last assertion" time="{duration}"/>
<testcase classname="global" name="An unchecked exception reports the line of the last assertion" time="{duration}">
<error message="{Unknown expression after the reported line}">
unexpected exception
ExceptionTests.cpp:<line number>
</error>
</testcase>
<testcase classname="global" name="Anonymous test case 1" time="{duration}"/>
<testcase classname="global" name="AnyOf matcher" time="{duration}"/>
<testcase classname="global" name="Approximate PI" time="{duration}"/>
@ -115,6 +127,12 @@ ExceptionTests.cpp:<line number>
<testcase classname="global" name="Custom exceptions can be translated when testing for throwing as something else" time="{duration}">
<error message="throwCustom(), std::exception" type="REQUIRE_THROWS_AS">
custom exception - not std
ExceptionTests.cpp:<line number>
</error>
</testcase>
<testcase classname="global" name="Custom std-exceptions can be custom translated" time="{duration}">
<error type="TEST_CASE">
custom std exception
ExceptionTests.cpp:<line number>
</error>
</testcase>
@ -282,6 +300,12 @@ ExceptionTests.cpp:<line number>
</failure>
</testcase>
<testcase classname="global" name="Nice descriptive name" time="{duration}"/>
<testcase classname="global" name="Non-std exceptions can be translated" time="{duration}">
<error type="TEST_CASE">
custom exception
ExceptionTests.cpp:<line number>
</error>
</testcase>
<testcase classname="global" name="NotImplemented exception" time="{duration}"/>
<testcase classname="global" name="Objects that evaluated in boolean contexts can be checked" time="{duration}"/>
<testcase classname="global" name="Operators at different namespace levels not hijacked by Koenig lookup" time="{duration}"/>
@ -497,6 +521,12 @@ MiscTests.cpp:<line number>
<testcase classname="Tracker" name="start a generator/Start a new inner section/Re-enter for second generation" time="{duration}"/>
<testcase classname="Tracker" name="start a generator/Fail an inner section" time="{duration}"/>
<testcase classname="Tracker" name="start a generator/Fail an inner section/Re-enter for second generation" time="{duration}"/>
<testcase classname="global" name="Unexpected exceptions can be translated" time="{duration}">
<error type="TEST_CASE">
3.14
ExceptionTests.cpp:<line number>
</error>
</testcase>
<testcase classname="global" name="Use a custom approx" time="{duration}"/>
<testcase classname="Variadic macros" name="Section with one argument" time="{duration}"/>
<testcase classname="Vector matchers" name="Contains (element)" time="{duration}"/>
@ -533,6 +563,12 @@ MatchersTests.cpp:<line number>
</failure>
</testcase>
<testcase classname="global" name="When checked exceptions are thrown they can be expected or unexpected" time="{duration}"/>
<testcase classname="global" name="When unchecked exceptions are thrown directly they are always failures" time="{duration}">
<error type="TEST_CASE">
unexpected exception
ExceptionTests.cpp:<line number>
</error>
</testcase>
<testcase classname="global" name="When unchecked exceptions are thrown during a CHECK the test should continue" time="{duration}">
<error message="thisThrows() == 0" type="CHECK">
expected exception
@ -548,6 +584,12 @@ ExceptionTests.cpp:<line number>
<testcase classname="global" name="When unchecked exceptions are thrown from functions they are always failures" time="{duration}">
<error message="thisThrows() == 0" type="CHECK">
expected exception
ExceptionTests.cpp:<line number>
</error>
</testcase>
<testcase classname="When unchecked exceptions are thrown from sections they are always failures" name="section name" time="{duration}">
<error type="TEST_CASE">
unexpected exception
ExceptionTests.cpp:<line number>
</error>
</testcase>

View File

@ -6,7 +6,13 @@
</TestCase>
<TestCase name="#748 - captures with unexpected exceptions" tags="[!shouldfail][!throws]" filename="projects/<exe-name>/ExceptionTests.cpp" >
<Section name="outside assertions" filename="projects/<exe-name>/ExceptionTests.cpp" >
<OverallResults successes="0" failures="0" expectedFailures="0"/>
<Info>
answer := 42
</Info>
<Exception filename="projects/<exe-name>/ExceptionTests.cpp" >
expected exception
</Exception>
<OverallResults successes="0" failures="1" expectedFailures="0"/>
</Section>
<Section name="inside REQUIRE_NOTHROW" filename="projects/<exe-name>/ExceptionTests.cpp" >
<Info>
@ -506,7 +512,18 @@
1 == 1
</Expanded>
</Expression>
<OverallResult success="true"/>
<Expression success="false" filename="projects/<exe-name>/ExceptionTests.cpp" >
<Original>
{Unknown expression after the reported line}
</Original>
<Expanded>
{Unknown expression after the reported line}
</Expanded>
<Exception filename="projects/<exe-name>/ExceptionTests.cpp" >
unexpected exception
</Exception>
</Expression>
<OverallResult success="false"/>
</TestCase>
<TestCase name="Anonymous test case 1" filename="projects/<exe-name>/VariadicMacrosTests.cpp" >
<OverallResult success="true"/>
@ -1110,7 +1127,10 @@
<OverallResult success="false"/>
</TestCase>
<TestCase name="Custom std-exceptions can be custom translated" tags="[!throws][.][failing][hide]" filename="projects/<exe-name>/ExceptionTests.cpp" >
<OverallResult success="true"/>
<Exception filename="projects/<exe-name>/ExceptionTests.cpp" >
custom std exception
</Exception>
<OverallResult success="false"/>
</TestCase>
<TestCase name="Demonstrate that a non-const == is not used" tags="[Tricky]" filename="projects/<exe-name>/TrickyTests.cpp" >
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/TrickyTests.cpp" >
@ -4635,7 +4655,10 @@ re>"
<OverallResult success="true"/>
</TestCase>
<TestCase name="Non-std exceptions can be translated" tags="[!throws][.][failing][hide]" filename="projects/<exe-name>/ExceptionTests.cpp" >
<OverallResult success="true"/>
<Exception filename="projects/<exe-name>/ExceptionTests.cpp" >
custom exception
</Exception>
<OverallResult success="false"/>
</TestCase>
<TestCase name="NotImplemented exception" tags="[!throws]" filename="projects/<exe-name>/ExceptionTests.cpp" >
<Expression success="true" type="REQUIRE_THROWS" filename="projects/<exe-name>/ExceptionTests.cpp" >
@ -8586,7 +8609,10 @@ there"
<OverallResult success="true"/>
</TestCase>
<TestCase name="Unexpected exceptions can be translated" tags="[!throws][.][failing][hide]" filename="projects/<exe-name>/ExceptionTests.cpp" >
<OverallResult success="true"/>
<Exception filename="projects/<exe-name>/ExceptionTests.cpp" >
3.14
</Exception>
<OverallResult success="false"/>
</TestCase>
<TestCase name="Use a custom approx" tags="[Approx][custom]" filename="projects/<exe-name>/ApproxTests.cpp" >
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/ApproxTests.cpp" >
@ -8849,7 +8875,10 @@ there"
<OverallResult success="true"/>
</TestCase>
<TestCase name="When unchecked exceptions are thrown directly they are always failures" tags="[!throws][.][failing][hide]" filename="projects/<exe-name>/ExceptionTests.cpp" >
<OverallResult success="true"/>
<Exception filename="projects/<exe-name>/ExceptionTests.cpp" >
unexpected exception
</Exception>
<OverallResult success="false"/>
</TestCase>
<TestCase name="When unchecked exceptions are thrown during a CHECK the test should continue" tags="[!throws][.][failing][hide]" filename="projects/<exe-name>/ExceptionTests.cpp" >
<Expression success="false" type="CHECK" filename="projects/<exe-name>/ExceptionTests.cpp" >
@ -8895,9 +8924,12 @@ there"
</TestCase>
<TestCase name="When unchecked exceptions are thrown from sections they are always failures" tags="[!throws][.][failing][hide]" filename="projects/<exe-name>/ExceptionTests.cpp" >
<Section name="section name" filename="projects/<exe-name>/ExceptionTests.cpp" >
<OverallResults successes="0" failures="0" expectedFailures="0"/>
<Exception filename="projects/<exe-name>/ExceptionTests.cpp" >
unexpected exception
</Exception>
<OverallResults successes="0" failures="1" expectedFailures="0"/>
</Section>
<OverallResult success="true"/>
<OverallResult success="false"/>
</TestCase>
<TestCase name="When unchecked exceptions are thrown, but caught, they do not affect the test" tags="[!throws]" filename="projects/<exe-name>/ExceptionTests.cpp" >
<OverallResult success="true"/>
@ -10082,7 +10114,7 @@ spanner <OverallResult success="true"/>
</Section>
<OverallResult success="true"/>
</TestCase>
<OverallResults successes="859" failures="84" expectedFailures="20"/>
<OverallResults successes="859" failures="90" expectedFailures="21"/>
</Group>
<OverallResults successes="859" failures="83" expectedFailures="20"/>
<OverallResults successes="859" failures="89" expectedFailures="21"/>
</Catch>