Add a Relative Comparison matcher for floating point numbers

It checks Knuth's _close enough with tolerance_ relationship, that
is `|lhs - rhs| <= epsilon * max(|lhs|, |rhs|)`, rather then the
_very close with tolerance_ relationship that can be written down as
`|lhs - rhs| <= epsilon * min(|lhs|, |rhs|)`.

This is because it is the more common model around the internet, and
as such is likely to be less surprising to the users. In the future
we might want to provide the other model as well.

Closes #1746
This commit is contained in:
Martin Hořeňovský
2019-10-13 11:56:50 +02:00
parent c8db4e77c4
commit d2d418a9cb
8 changed files with 410 additions and 10 deletions

View File

@@ -407,6 +407,11 @@ Misc.tests.cpp:<line number>: passed: Factorial(1) == 1 for: 1 == 1
Misc.tests.cpp:<line number>: passed: Factorial(2) == 2 for: 2 == 2
Misc.tests.cpp:<line number>: passed: Factorial(3) == 6 for: 6 == 6
Misc.tests.cpp:<line number>: passed: Factorial(10) == 3628800 for: 3628800 (0x<hex digits>) == 3628800 (0x<hex digits>)
Matchers.tests.cpp:<line number>: passed: 10., WithinRel(11.1, 0.1) for: 10.0 and 11.1 are within 10% of each other
Matchers.tests.cpp:<line number>: passed: 10., !WithinRel(11.2, 0.1) for: 10.0 not and 11.2 are within 10% of each other
Matchers.tests.cpp:<line number>: passed: 1., !WithinRel(0., 0.99) for: 1.0 not and 0 are within 99% of each other
Matchers.tests.cpp:<line number>: passed: -0., WithinRel(0.) for: -0.0 and 0 are within 2.22045e-12% of each other
Matchers.tests.cpp:<line number>: passed: v1, WithinRel(v2) for: 0.0 and 2.22507e-308 are within 2.22045e-12% of each other
Matchers.tests.cpp:<line number>: passed: 1., WithinAbs(1., 0) for: 1.0 is within 0.0 of 1.0
Matchers.tests.cpp:<line number>: passed: 0., WithinAbs(1., 1) for: 0.0 is within 1.0 of 1.0
Matchers.tests.cpp:<line number>: passed: 0., !WithinAbs(1., 0.99) for: 0.0 not is within 0.99 of 1.0
@@ -423,9 +428,18 @@ Matchers.tests.cpp:<line number>: passed: 1., WithinULP(1., 0) for: 1.0 is withi
Matchers.tests.cpp:<line number>: passed: -0., WithinULP(0., 0) for: -0.0 is within 0 ULPs of 0.0 ([0.00000000000000000, 0.00000000000000000])
Matchers.tests.cpp:<line number>: passed: 1., WithinAbs(1., 0.5) || WithinULP(2., 1) for: 1.0 ( is within 0.5 of 1.0 or is within 1 ULPs of 2.0 ([1.99999999999999978, 2.00000000000000044]) )
Matchers.tests.cpp:<line number>: passed: 1., WithinAbs(2., 0.5) || WithinULP(1., 0) for: 1.0 ( is within 0.5 of 2.0 or is within 0 ULPs of 1.0 ([1.00000000000000000, 1.00000000000000000]) )
Matchers.tests.cpp:<line number>: passed: 0.0001, WithinAbs(0., 0.001) || WithinRel(0., 0.1) for: 0.0001 ( is within 0.001 of 0.0 or and 0 are within 10% of each other )
Matchers.tests.cpp:<line number>: passed: WithinAbs(1., 0.)
Matchers.tests.cpp:<line number>: passed: WithinAbs(1., -1.), std::domain_error
Matchers.tests.cpp:<line number>: passed: WithinULP(1., 0)
Matchers.tests.cpp:<line number>: passed: WithinRel(1., 0.)
Matchers.tests.cpp:<line number>: passed: WithinRel(1., -0.2), std::domain_error
Matchers.tests.cpp:<line number>: passed: WithinRel(1., 1.), std::domain_error
Matchers.tests.cpp:<line number>: passed: 10.f, WithinRel(11.1f, 0.1f) for: 10.0f and 11.1 are within 10% of each other
Matchers.tests.cpp:<line number>: passed: 10.f, !WithinRel(11.2f, 0.1f) for: 10.0f not and 11.2 are within 10% of each other
Matchers.tests.cpp:<line number>: passed: 1.f, !WithinRel(0.f, 0.99f) for: 1.0f not and 0 are within 99% of each other
Matchers.tests.cpp:<line number>: passed: -0.f, WithinRel(0.f) for: -0.0f and 0 are within 0.00119209% of each other
Matchers.tests.cpp:<line number>: passed: v1, WithinRel(v2) for: 0.0f and 1.17549e-38 are within 0.00119209% of each other
Matchers.tests.cpp:<line number>: passed: 1.f, WithinAbs(1.f, 0) for: 1.0f is within 0.0 of 1.0
Matchers.tests.cpp:<line number>: passed: 0.f, WithinAbs(1.f, 1) for: 0.0f is within 1.0 of 1.0
Matchers.tests.cpp:<line number>: passed: 0.f, !WithinAbs(1.f, 0.99f) for: 0.0f not is within 0.9900000095 of 1.0
@@ -443,10 +457,14 @@ Matchers.tests.cpp:<line number>: passed: 1.f, WithinULP(1.f, 0) for: 1.0f is wi
Matchers.tests.cpp:<line number>: passed: -0.f, WithinULP(0.f, 0) for: -0.0f is within 0 ULPs of 0.0f ([0.00000000000000000, 0.00000000000000000])
Matchers.tests.cpp:<line number>: passed: 1.f, WithinAbs(1.f, 0.5) || WithinULP(1.f, 1) for: 1.0f ( is within 0.5 of 1.0 or is within 1 ULPs of 1.0f ([0.99999994039535522, 1.00000011920928955]) )
Matchers.tests.cpp:<line number>: passed: 1.f, WithinAbs(2.f, 0.5) || WithinULP(1.f, 0) for: 1.0f ( is within 0.5 of 2.0 or is within 0 ULPs of 1.0f ([1.00000000000000000, 1.00000000000000000]) )
Matchers.tests.cpp:<line number>: passed: 0.0001f, WithinAbs(0.f, 0.001f) || WithinRel(0.f, 0.1f) for: 0.0001f ( is within 0.001 of 0.0 or and 0 are within 10% of each other )
Matchers.tests.cpp:<line number>: passed: WithinAbs(1.f, 0.f)
Matchers.tests.cpp:<line number>: passed: WithinAbs(1.f, -1.f), std::domain_error
Matchers.tests.cpp:<line number>: passed: WithinULP(1.f, 0)
Matchers.tests.cpp:<line number>: passed: WithinULP(1.f, static_cast<uint64_t>(-1)), std::domain_error
Matchers.tests.cpp:<line number>: passed: WithinRel(1.f, 0.f)
Matchers.tests.cpp:<line number>: passed: WithinRel(1.f, -0.2f), std::domain_error
Matchers.tests.cpp:<line number>: passed: WithinRel(1.f, 1.f), std::domain_error
Generators.tests.cpp:<line number>: passed: i % 2 == 0 for: 0 == 0
Generators.tests.cpp:<line number>: passed: i % 2 == 0 for: 0 == 0
Generators.tests.cpp:<line number>: passed: i % 2 == 0 for: 0 == 0

View File

@@ -1381,5 +1381,5 @@ due to unexpected exception with message:
===============================================================================
test cases: 303 | 229 passed | 70 failed | 4 failed as expected
assertions: 1597 | 1445 passed | 131 failed | 21 failed as expected
assertions: 1615 | 1463 passed | 131 failed | 21 failed as expected

View File

@@ -3088,6 +3088,46 @@ Misc.tests.cpp:<line number>: PASSED:
with expansion:
3628800 (0x<hex digits>) == 3628800 (0x<hex digits>)
-------------------------------------------------------------------------------
Floating point matchers: double
Relative
-------------------------------------------------------------------------------
Matchers.tests.cpp:<line number>
...............................................................................
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( 10., WithinRel(11.1, 0.1) )
with expansion:
10.0 and 11.1 are within 10% of each other
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( 10., !WithinRel(11.2, 0.1) )
with expansion:
10.0 not and 11.2 are within 10% of each other
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( 1., !WithinRel(0., 0.99) )
with expansion:
1.0 not and 0 are within 99% of each other
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( -0., WithinRel(0.) )
with expansion:
-0.0 and 0 are within 2.22045e-12% of each other
-------------------------------------------------------------------------------
Floating point matchers: double
Relative
Some subnormal values
-------------------------------------------------------------------------------
Matchers.tests.cpp:<line number>
...............................................................................
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( v1, WithinRel(v2) )
with expansion:
0.0 and 2.22507e-308 are within 2.22045e-12% of each other
-------------------------------------------------------------------------------
Floating point matchers: double
Margin
@@ -3191,6 +3231,11 @@ with expansion:
1.0 ( is within 0.5 of 2.0 or is within 0 ULPs of 1.0 ([1.00000000000000000,
1.00000000000000000]) )
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( 0.0001, WithinAbs(0., 0.001) || WithinRel(0., 0.1) )
with expansion:
0.0001 ( is within 0.001 of 0.0 or and 0 are within 10% of each other )
-------------------------------------------------------------------------------
Floating point matchers: double
Constructor validation
@@ -3207,6 +3252,55 @@ Matchers.tests.cpp:<line number>: PASSED:
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_NOTHROW( WithinULP(1., 0) )
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_NOTHROW( WithinRel(1., 0.) )
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THROWS_AS( WithinRel(1., -0.2), std::domain_error )
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THROWS_AS( WithinRel(1., 1.), std::domain_error )
-------------------------------------------------------------------------------
Floating point matchers: float
Relative
-------------------------------------------------------------------------------
Matchers.tests.cpp:<line number>
...............................................................................
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( 10.f, WithinRel(11.1f, 0.1f) )
with expansion:
10.0f and 11.1 are within 10% of each other
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( 10.f, !WithinRel(11.2f, 0.1f) )
with expansion:
10.0f not and 11.2 are within 10% of each other
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( 1.f, !WithinRel(0.f, 0.99f) )
with expansion:
1.0f not and 0 are within 99% of each other
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( -0.f, WithinRel(0.f) )
with expansion:
-0.0f and 0 are within 0.00119209% of each other
-------------------------------------------------------------------------------
Floating point matchers: float
Relative
Some subnormal values
-------------------------------------------------------------------------------
Matchers.tests.cpp:<line number>
...............................................................................
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( v1, WithinRel(v2) )
with expansion:
0.0f and 1.17549e-38 are within 0.00119209% of each other
-------------------------------------------------------------------------------
Floating point matchers: float
Margin
@@ -3316,6 +3410,11 @@ with expansion:
1.0f ( is within 0.5 of 2.0 or is within 0 ULPs of 1.0f ([1.
00000000000000000, 1.00000000000000000]) )
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THAT( 0.0001f, WithinAbs(0.f, 0.001f) || WithinRel(0.f, 0.1f) )
with expansion:
0.0001f ( is within 0.001 of 0.0 or and 0 are within 10% of each other )
-------------------------------------------------------------------------------
Floating point matchers: float
Constructor validation
@@ -3335,6 +3434,15 @@ Matchers.tests.cpp:<line number>: PASSED:
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THROWS_AS( WithinULP(1.f, static_cast<uint64_t>(-1)), std::domain_error )
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_NOTHROW( WithinRel(1.f, 0.f) )
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THROWS_AS( WithinRel(1.f, -0.2f), std::domain_error )
Matchers.tests.cpp:<line number>: PASSED:
REQUIRE_THROWS_AS( WithinRel(1.f, 1.f), std::domain_error )
-------------------------------------------------------------------------------
Generators -- adapters
Filtering by predicate
@@ -12806,5 +12914,5 @@ Misc.tests.cpp:<line number>: PASSED:
===============================================================================
test cases: 303 | 213 passed | 86 failed | 4 failed as expected
assertions: 1614 | 1445 passed | 148 failed | 21 failed as expected
assertions: 1632 | 1463 passed | 148 failed | 21 failed as expected

View File

@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuitesloose text artifact
>
<testsuite name="<exe-name>" errors="17" failures="132" tests="1615" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<testsuite name="<exe-name>" errors="17" failures="132" tests="1633" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<properties>
<property name="filters" value="~[!nonportable]~[!benchmark]~[approvals]"/>
<property name="random-seed" value="1"/>
@@ -402,10 +402,14 @@ Message.tests.cpp:<line number>
</failure>
</testcase>
<testcase classname="<exe-name>.global" name="Factorials are computed" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: double/Relative" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: double/Relative/Some subnormal values" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: double/Margin" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: double/ULPs" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: double/Composed" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: double/Constructor validation" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: float/Relative" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: float/Relative/Some subnormal values" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: float/Margin" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: float/ULPs" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Floating point matchers: float/Composed" time="{duration}"/>

View File

@@ -3657,6 +3657,52 @@ Nor would this
<OverallResult success="true"/>
</TestCase>
<TestCase name="Floating point matchers: double" tags="[floating-point][matchers]" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Section name="Relative" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
10., WithinRel(11.1, 0.1)
</Original>
<Expanded>
10.0 and 11.1 are within 10% of each other
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
10., !WithinRel(11.2, 0.1)
</Original>
<Expanded>
10.0 not and 11.2 are within 10% of each other
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
1., !WithinRel(0., 0.99)
</Original>
<Expanded>
1.0 not and 0 are within 99% of each other
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
-0., WithinRel(0.)
</Original>
<Expanded>
-0.0 and 0 are within 2.22045e-12% of each other
</Expanded>
</Expression>
<Section name="Some subnormal values" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
v1, WithinRel(v2)
</Original>
<Expanded>
0.0 and 2.22507e-308 are within 2.22045e-12% of each other
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0"/>
</Section>
<OverallResults successes="5" failures="0" expectedFailures="0"/>
</Section>
<Section name="Margin" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
@@ -3792,7 +3838,15 @@ Nor would this
1.0 ( is within 0.5 of 2.0 or is within 0 ULPs of 1.0 ([1.00000000000000000, 1.00000000000000000]) )
</Expanded>
</Expression>
<OverallResults successes="2" failures="0" expectedFailures="0"/>
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
0.0001, WithinAbs(0., 0.001) || WithinRel(0., 0.1)
</Original>
<Expanded>
0.0001 ( is within 0.001 of 0.0 or and 0 are within 10% of each other )
</Expanded>
</Expression>
<OverallResults successes="3" failures="0" expectedFailures="0"/>
</Section>
<Section name="Constructor validation" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Expression success="true" type="REQUIRE_NOTHROW" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
@@ -3819,11 +3873,81 @@ Nor would this
WithinULP(1., 0)
</Expanded>
</Expression>
<OverallResults successes="3" failures="0" expectedFailures="0"/>
<Expression success="true" type="REQUIRE_NOTHROW" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
WithinRel(1., 0.)
</Original>
<Expanded>
WithinRel(1., 0.)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THROWS_AS" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
WithinRel(1., -0.2), std::domain_error
</Original>
<Expanded>
WithinRel(1., -0.2), std::domain_error
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THROWS_AS" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
WithinRel(1., 1.), std::domain_error
</Original>
<Expanded>
WithinRel(1., 1.), std::domain_error
</Expanded>
</Expression>
<OverallResults successes="6" failures="0" expectedFailures="0"/>
</Section>
<OverallResult success="true"/>
</TestCase>
<TestCase name="Floating point matchers: float" tags="[floating-point][matchers]" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Section name="Relative" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
10.f, WithinRel(11.1f, 0.1f)
</Original>
<Expanded>
10.0f and 11.1 are within 10% of each other
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
10.f, !WithinRel(11.2f, 0.1f)
</Original>
<Expanded>
10.0f not and 11.2 are within 10% of each other
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
1.f, !WithinRel(0.f, 0.99f)
</Original>
<Expanded>
1.0f not and 0 are within 99% of each other
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
-0.f, WithinRel(0.f)
</Original>
<Expanded>
-0.0f and 0 are within 0.00119209% of each other
</Expanded>
</Expression>
<Section name="Some subnormal values" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
v1, WithinRel(v2)
</Original>
<Expanded>
0.0f and 1.17549e-38 are within 0.00119209% of each other
</Expanded>
</Expression>
<OverallResults successes="1" failures="0" expectedFailures="0"/>
</Section>
<OverallResults successes="5" failures="0" expectedFailures="0"/>
</Section>
<Section name="Margin" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
@@ -3967,7 +4091,15 @@ Nor would this
1.0f ( is within 0.5 of 2.0 or is within 0 ULPs of 1.0f ([1.00000000000000000, 1.00000000000000000]) )
</Expanded>
</Expression>
<OverallResults successes="2" failures="0" expectedFailures="0"/>
<Expression success="true" type="REQUIRE_THAT" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
0.0001f, WithinAbs(0.f, 0.001f) || WithinRel(0.f, 0.1f)
</Original>
<Expanded>
0.0001f ( is within 0.001 of 0.0 or and 0 are within 10% of each other )
</Expanded>
</Expression>
<OverallResults successes="3" failures="0" expectedFailures="0"/>
</Section>
<Section name="Constructor validation" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Expression success="true" type="REQUIRE_NOTHROW" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
@@ -4002,7 +4134,31 @@ Nor would this
WithinULP(1.f, static_cast&lt;uint64_t>(-1)), std::domain_error
</Expanded>
</Expression>
<OverallResults successes="4" failures="0" expectedFailures="0"/>
<Expression success="true" type="REQUIRE_NOTHROW" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
WithinRel(1.f, 0.f)
</Original>
<Expanded>
WithinRel(1.f, 0.f)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THROWS_AS" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
WithinRel(1.f, -0.2f), std::domain_error
</Original>
<Expanded>
WithinRel(1.f, -0.2f), std::domain_error
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_THROWS_AS" filename="projects/<exe-name>/UsageTests/Matchers.tests.cpp" >
<Original>
WithinRel(1.f, 1.f), std::domain_error
</Original>
<Expanded>
WithinRel(1.f, 1.f), std::domain_error
</Expanded>
</Expression>
<OverallResults successes="7" failures="0" expectedFailures="0"/>
</Section>
<OverallResult success="true"/>
</TestCase>
@@ -15220,7 +15376,7 @@ loose text artifact
</Section>
<OverallResult success="true"/>
</TestCase>
<OverallResults successes="1445" failures="149" expectedFailures="21"/>
<OverallResults successes="1463" failures="149" expectedFailures="21"/>
</Group>
<OverallResults successes="1445" failures="148" expectedFailures="21"/>
<OverallResults successes="1463" failures="148" expectedFailures="21"/>
</Catch>