3rd-party/catch2
1
0
Fork 0
mirror of https://github.com/catchorg/Catch2.git synced 2025-08-01 21:05:39 +02:00
Code Issues Packages Projects Releases Wiki Activity

Introduce Catch's own RNG based on the PCG family of RNGs

Browse Source 
In the future, we will also want to introduce our own
`uniform_int_distribution` and `uniform_real_distribution` to get
repeatable test runs across different platforms.
This commit is contained in:
Martin Hořeňovský
2019-10-06 21:47:54 +02:00
parent 319cb9e1da
commit 535da5c513
14 changed files with 565 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

49
projects/SelfTest/Baselines/compact.sw.approved.txt
View File

@@ -276,6 +276,10 @@ Tricky.tests.cpp:<line number>: passed: true
Tricky.tests.cpp:<line number>: passed: std::vector<int>{1, 2} == std::vector<int>{1, 2} for: { 1, 2 } == { 1, 2 }
Tricky.tests.cpp:<line number>: passed: a for: 0x<hex digits>
Tricky.tests.cpp:<line number>: passed: a == &foo for: 0x<hex digits> == 0x<hex digits>
RandomNumberGeneration.tests.cpp:<line number>: passed: SimplePcg32{} == SimplePcg32{} for: {?} == {?}
RandomNumberGeneration.tests.cpp:<line number>: passed: SimplePcg32{ 0 } != SimplePcg32{} for: {?} != {?}
RandomNumberGeneration.tests.cpp:<line number>: passed: !(SimplePcg32{ 1 } == SimplePcg32{ 2 }) for: !({?} == {?})
RandomNumberGeneration.tests.cpp:<line number>: passed: !(SimplePcg32{ 1 } != SimplePcg32{ 1 }) for: !({?} != {?})
Approx.tests.cpp:<line number>: passed: td == Approx(10.0) for: StrongDoubleTypedef(10) == Approx( 10.0 )
Approx.tests.cpp:<line number>: passed: Approx(10.0) == td for: Approx( 10.0 ) == StrongDoubleTypedef(10)
Approx.tests.cpp:<line number>: passed: td != Approx(11.0) for: StrongDoubleTypedef(10) != Approx( 11.0 )
@@ -731,6 +735,51 @@ Condition.tests.cpp:<line number>: passed: data.str_hello < "hellp" for: "hello"
Condition.tests.cpp:<line number>: passed: data.str_hello < "zebra" for: "hello" < "zebra"
Condition.tests.cpp:<line number>: passed: data.str_hello > "hellm" for: "hello" > "hellm"
Condition.tests.cpp:<line number>: passed: data.str_hello > "a" for: "hello" > "a"
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 4242248763 (0x<hex digits>)
==
4242248763 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 1867888929 (0x<hex digits>)
==
1867888929 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 1276619030 (0x<hex digits>)
==
1276619030 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 1911218783 (0x<hex digits>)
==
1911218783 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 1827115164 (0x<hex digits>)
==
1827115164 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 1472234645 (0x<hex digits>)
==
1472234645 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 868832940 (0x<hex digits>)
==
868832940 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 570883446 (0x<hex digits>)
==
570883446 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 889299803 (0x<hex digits>)
==
889299803 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 4261393167 (0x<hex digits>)
==
4261393167 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 1472234645 (0x<hex digits>)
==
1472234645 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 868832940 (0x<hex digits>)
==
868832940 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 570883446 (0x<hex digits>)
==
570883446 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 889299803 (0x<hex digits>)
==
889299803 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: passed: rng() == 0x<hex digits> for: 4261393167 (0x<hex digits>)
==
4261393167 (0x<hex digits>)
Message.tests.cpp:<line number>: failed: explicitly with 1 message: 'Message from section one'
Message.tests.cpp:<line number>: failed: explicitly with 1 message: 'Message from section two'
CmdLine.tests.cpp:<line number>: passed: spec.hasFilters() == false for: false == false

4
projects/SelfTest/Baselines/console.std.approved.txt
View File

@@ -1380,6 +1380,6 @@ due to unexpected exception with message:
Why would you throw a std::string?
===============================================================================
test cases: 301 | 227 passed | 70 failed | 4 failed as expected
assertions: 1570 | 1418 passed | 131 failed | 21 failed as expected
test cases: 303 | 229 passed | 70 failed | 4 failed as expected
assertions: 1589 | 1437 passed | 131 failed | 21 failed as expected

149
projects/SelfTest/Baselines/console.sw.approved.txt
View File

@@ -2179,6 +2179,32 @@ Tricky.tests.cpp:<line number>: PASSED:
with expansion:
0x<hex digits> == 0x<hex digits>
-------------------------------------------------------------------------------
Comparison ops
-------------------------------------------------------------------------------
RandomNumberGeneration.tests.cpp:<line number>
...............................................................................
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( SimplePcg32{} == SimplePcg32{} )
with expansion:
{?} == {?}
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( SimplePcg32{ 0 } != SimplePcg32{} )
with expansion:
{?} != {?}
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE_FALSE( SimplePcg32{ 1 } == SimplePcg32{ 2 } )
with expansion:
!({?} == {?})
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE_FALSE( SimplePcg32{ 1 } != SimplePcg32{ 1 } )
with expansion:
!({?} != {?})
-------------------------------------------------------------------------------
Comparison with explicitly convertible types
-------------------------------------------------------------------------------
@@ -5494,6 +5520,125 @@ Condition.tests.cpp:<line number>: PASSED:
with expansion:
"hello" > "a"
-------------------------------------------------------------------------------
Our PCG implementation provides expected results for known seeds
Default seeded
-------------------------------------------------------------------------------
RandomNumberGeneration.tests.cpp:<line number>
...............................................................................
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
4242248763 (0x<hex digits>)
==
4242248763 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
1867888929 (0x<hex digits>)
==
1867888929 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
1276619030 (0x<hex digits>)
==
1276619030 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
1911218783 (0x<hex digits>)
==
1911218783 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
1827115164 (0x<hex digits>)
==
1827115164 (0x<hex digits>)
-------------------------------------------------------------------------------
Our PCG implementation provides expected results for known seeds
Specific seed
-------------------------------------------------------------------------------
RandomNumberGeneration.tests.cpp:<line number>
...............................................................................
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
1472234645 (0x<hex digits>)
==
1472234645 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
868832940 (0x<hex digits>)
==
868832940 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
570883446 (0x<hex digits>)
==
570883446 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
889299803 (0x<hex digits>)
==
889299803 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
4261393167 (0x<hex digits>)
==
4261393167 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
1472234645 (0x<hex digits>)
==
1472234645 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
868832940 (0x<hex digits>)
==
868832940 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
570883446 (0x<hex digits>)
==
570883446 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
889299803 (0x<hex digits>)
==
889299803 (0x<hex digits>)
RandomNumberGeneration.tests.cpp:<line number>: PASSED:
REQUIRE( rng() == 0x<hex digits> )
with expansion:
4261393167 (0x<hex digits>)
==
4261393167 (0x<hex digits>)
-------------------------------------------------------------------------------
Output from all sections is reported
one
@@ -12565,6 +12710,6 @@ Misc.tests.cpp:<line number>
Misc.tests.cpp:<line number>: PASSED:
===============================================================================
test cases: 301 | 211 passed | 86 failed | 4 failed as expected
assertions: 1587 | 1418 passed | 148 failed | 21 failed as expected
test cases: 303 | 213 passed | 86 failed | 4 failed as expected
assertions: 1606 | 1437 passed | 148 failed | 21 failed as expected

5
projects/SelfTest/Baselines/junit.sw.approved.txt
View File

@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<testsuitesloose text artifact
>
<testsuite name="<exe-name>" errors="17" failures="132" tests="1588" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<testsuite name="<exe-name>" errors="17" failures="132" tests="1607" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
<properties>
<property name="filters" value="~[!nonportable]~[!benchmark]~[approvals]"/>
<property name="random-seed" value="1"/>
@@ -242,6 +242,7 @@ Exception.tests.cpp:<line number>
<testcase classname="<exe-name>.global" name="Character pretty printing/Low ASCII" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Commas in various macros are allowed" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Comparing function pointers" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Comparison ops" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Comparison with explicitly convertible types" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Comparisons between ints where one side is computed" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Comparisons between unsigned ints and negative signed ints match c++ standard behaviour" time="{duration}"/>
@@ -577,6 +578,8 @@ Condition.tests.cpp:<line number>
</failure>
</testcase>
<testcase classname="<exe-name>.global" name="Ordering comparison checks that should succeed" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Our PCG implementation provides expected results for known seeds/Default seeded" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Our PCG implementation provides expected results for known seeds/Specific seed" time="{duration}"/>
<testcase classname="<exe-name>.global" name="Output from all sections is reported/one" time="{duration}">
<failure type="FAIL">
Message from section one

198
projects/SelfTest/Baselines/xml.sw.approved.txt
View File

@@ -2529,6 +2529,41 @@ Nor would this
</Expression>
<OverallResult success="true"/>
</TestCase>
<TestCase name="Comparison ops" tags="[rng]" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
SimplePcg32{} == SimplePcg32{}
</Original>
<Expanded>
{?} == {?}
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
SimplePcg32{ 0 } != SimplePcg32{}
</Original>
<Expanded>
{?} != {?}
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_FALSE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
!(SimplePcg32{ 1 } == SimplePcg32{ 2 })
</Original>
<Expanded>
!({?} == {?})
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE_FALSE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
!(SimplePcg32{ 1 } != SimplePcg32{ 1 })
</Original>
<Expanded>
!({?} != {?})
</Expanded>
</Expression>
<OverallResult success="true"/>
</TestCase>
<TestCase name="Comparison with explicitly convertible types" tags="[Approx]" filename="projects/<exe-name>/UsageTests/Approx.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/UsageTests/Approx.tests.cpp" >
<Original>
@@ -6728,6 +6763,165 @@ Nor would this
</Expression>
<OverallResult success="true"/>
</TestCase>
<TestCase name="Our PCG implementation provides expected results for known seeds" tags="[rng]" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Section name="Default seeded" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
4242248763 (0x<hex digits>)
==
4242248763 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
1867888929 (0x<hex digits>)
==
1867888929 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
1276619030 (0x<hex digits>)
==
1276619030 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
1911218783 (0x<hex digits>)
==
1911218783 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
1827115164 (0x<hex digits>)
==
1827115164 (0x<hex digits>)
</Expanded>
</Expression>
<OverallResults successes="5" failures="0" expectedFailures="0"/>
</Section>
<Section name="Specific seed" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
1472234645 (0x<hex digits>)
==
1472234645 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
868832940 (0x<hex digits>)
==
868832940 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
570883446 (0x<hex digits>)
==
570883446 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
889299803 (0x<hex digits>)
==
889299803 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
4261393167 (0x<hex digits>)
==
4261393167 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
1472234645 (0x<hex digits>)
==
1472234645 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
868832940 (0x<hex digits>)
==
868832940 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
570883446 (0x<hex digits>)
==
570883446 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
889299803 (0x<hex digits>)
==
889299803 (0x<hex digits>)
</Expanded>
</Expression>
<Expression success="true" type="REQUIRE" filename="projects/<exe-name>/IntrospectiveTests/RandomNumberGeneration.tests.cpp" >
<Original>
rng() == 0x<hex digits>
</Original>
<Expanded>
4261393167 (0x<hex digits>)
==
4261393167 (0x<hex digits>)
</Expanded>
</Expression>
<OverallResults successes="10" failures="0" expectedFailures="0"/>
</Section>
<OverallResult success="true"/>
</TestCase>
<TestCase name="Output from all sections is reported" tags="[.][failing][messages]" filename="projects/<exe-name>/UsageTests/Message.tests.cpp" >
<Section name="one" filename="projects/<exe-name>/UsageTests/Message.tests.cpp" >
<Failure filename="projects/<exe-name>/UsageTests/Message.tests.cpp" >
@@ -14923,7 +15117,7 @@ loose text artifact
</Section>
<OverallResult success="true"/>
</TestCase>
<OverallResults successes="1418" failures="149" expectedFailures="21"/>
<OverallResults successes="1437" failures="149" expectedFailures="21"/>
</Group>
<OverallResults successes="1418" failures="148" expectedFailures="21"/>
<OverallResults successes="1437" failures="148" expectedFailures="21"/>
</Catch>