3rd-party/catch2
1
0
Fork 0
mirror of https://github.com/catchorg/Catch2.git synced 2024-11-05 05:39:53 +01:00
Code Issues Packages Projects Releases Wiki Activity
c43947eb47
catch2/tests/SelfTest/Baselines/teamcity.sw.multi.approved.txt

943 lines
88 KiB
Plaintext
Raw Normal View History

Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testSuiteStarted name='<exe-name>']
##teamcity[testStarted name='# A test name that starts with a #']
##teamcity[testFinished name='# A test name that starts with a #' duration="{duration}"]
##teamcity[testStarted name='#1027: Bitfields can be captured']
##teamcity[testFinished name='#1027: Bitfields can be captured' duration="{duration}"]
##teamcity[testStarted name='#1147']
##teamcity[testFinished name='#1147' duration="{duration}"]
##teamcity[testStarted name='#1175 - Hidden Test']
##teamcity[testFinished name='#1175 - Hidden Test' duration="{duration}"]
##teamcity[testStarted name='#1238']
##teamcity[testFinished name='#1238' duration="{duration}"]
##teamcity[testStarted name='#1245']
##teamcity[testFinished name='#1245' duration="{duration}"]
##teamcity[testStarted name='#1319: Sections can have description (even if it is not saved']
##teamcity[testFinished name='#1319: Sections can have description (even if it is not saved' duration="{duration}"]
##teamcity[testStarted name='#1403']
##teamcity[testFinished name='#1403' duration="{duration}"]
##teamcity[testStarted name='#1455 - INFO and WARN can start with a linebreak']
##teamcity[testFinished name='#1455 - INFO and WARN can start with a linebreak' duration="{duration}"]
##teamcity[testStarted name='#1514: stderr/stdout is not captured in tests aborted by an exception']
Tricky.tests.cpp:<line number>|nexplicit failure with message:|n "1514"']
##teamcity[testStdOut name='#1514: stderr/stdout is not captured in tests aborted by an exception' out='This would not be caught previously|n']
##teamcity[testStdErr name='#1514: stderr/stdout is not captured in tests aborted by an exception' out='Nor would this|n']
##teamcity[testFinished name='#1514: stderr/stdout is not captured in tests aborted by an exception' duration="{duration}"]
##teamcity[testStarted name='#1548']
##teamcity[testFinished name='#1548' duration="{duration}"]
##teamcity[testStarted name='#1905 -- test spec parser properly clears internal state between compound tests']
##teamcity[testFinished name='#1905 -- test spec parser properly clears internal state between compound tests' duration="{duration}"]
##teamcity[testStarted name='#1912 -- test spec parser handles escaping']
##teamcity[testFinished name='#1912 -- test spec parser handles escaping' duration="{duration}"]
##teamcity[testStarted name='#1913 - GENERATE inside a for loop should not keep recreating the generator']
##teamcity[testFinished name='#1913 - GENERATE inside a for loop should not keep recreating the generator' duration="{duration}"]
##teamcity[testStarted name='#1913 - GENERATEs can share a line']
##teamcity[testFinished name='#1913 - GENERATEs can share a line' duration="{duration}"]
##teamcity[testStarted name='#1938 - GENERATE after a section']
##teamcity[testFinished name='#1938 - GENERATE after a section' duration="{duration}"]
##teamcity[testStarted name='#1938 - Section followed by flat generate']
##teamcity[testFinished name='#1938 - Section followed by flat generate' duration="{duration}"]
##teamcity[testStarted name='#1938 - flat generate']
##teamcity[testFinished name='#1938 - flat generate' duration="{duration}"]
##teamcity[testStarted name='#1938 - mixed sections and generates']
##teamcity[testFinished name='#1938 - mixed sections and generates' duration="{duration}"]
##teamcity[testStarted name='#1938 - nested generate']
##teamcity[testFinished name='#1938 - nested generate' duration="{duration}"]
##teamcity[testStarted name='#1954 - 7 arg template test case sig compiles - 1, 1, 1, 1, 1, 0, 0']
##teamcity[testFinished name='#1954 - 7 arg template test case sig compiles - 1, 1, 1, 1, 1, 0, 0' duration="{duration}"]
##teamcity[testStarted name='#1954 - 7 arg template test case sig compiles - 5, 1, 1, 1, 1, 0, 0']
##teamcity[testFinished name='#1954 - 7 arg template test case sig compiles - 5, 1, 1, 1, 1, 0, 0' duration="{duration}"]
##teamcity[testStarted name='#1954 - 7 arg template test case sig compiles - 5, 3, 1, 1, 1, 0, 0']
##teamcity[testFinished name='#1954 - 7 arg template test case sig compiles - 5, 3, 1, 1, 1, 0, 0' duration="{duration}"]
##teamcity[testStarted name='#2152 - ULP checks between differently signed values were wrong - double']
##teamcity[testFinished name='#2152 - ULP checks between differently signed values were wrong - double' duration="{duration}"]
##teamcity[testStarted name='#2152 - ULP checks between differently signed values were wrong - float']
##teamcity[testFinished name='#2152 - ULP checks between differently signed values were wrong - float' duration="{duration}"]
##teamcity[testStarted name='#748 - captures with unexpected exceptions']
Exception.tests.cpp:<line number>|nunexpected exception with messages:|n "answer := 42"|n "expected exception"- failure ignore as test marked as |'ok to fail|'|n']
Exception.tests.cpp:<line number>|nunexpected exception with messages:|n "answer := 42"|n "expected exception"|n REQUIRE_NOTHROW( thisThrows() )|nwith expansion:|n thisThrows()|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='#748 - captures with unexpected exceptions' duration="{duration}"]
##teamcity[testStarted name='#809']
##teamcity[testFinished name='#809' duration="{duration}"]
##teamcity[testStarted name='#833']
##teamcity[testFinished name='#833' duration="{duration}"]
Cleanups output normalization in ApprovalTests Removed: * NaN normalization * INFINITY normalization * errno normalization * Completely unused duration regex Tests using these macros should be tagged `[approvals]` so they are not run as part of approval tests. Also simplified regex for the test's executable filename, and hidden some tests relying on nullptr normalization.
2022-01-26 23:47:40 +01:00
##teamcity[testStarted name='#835 -- errno should not be touched by Catch2']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
Misc.tests.cpp:<line number>|nexpression failed|n CHECK( f() == 0 )|nwith expansion:|n 1 == 0|n- failure ignore as test marked as |'ok to fail|'|n']
Cleanups output normalization in ApprovalTests Removed: * NaN normalization * INFINITY normalization * errno normalization * Completely unused duration regex Tests using these macros should be tagged `[approvals]` so they are not run as part of approval tests. Also simplified regex for the test's executable filename, and hidden some tests relying on nullptr normalization.
2022-01-26 23:47:40 +01:00
##teamcity[testFinished name='#835 -- errno should not be touched by Catch2' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='#872']
##teamcity[testFinished name='#872' duration="{duration}"]
##teamcity[testStarted name='#961 -- Dynamically created sections should all be reported']
##teamcity[testFinished name='#961 -- Dynamically created sections should all be reported' duration="{duration}"]
##teamcity[testStarted name='|'Not|' checks that should fail']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( false != false )|nwith expansion:|n false != false|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( true != true )|nwith expansion:|n true != true|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( !true )|nwith expansion:|n false|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK_FALSE( true )|nwith expansion:|n !true|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( !trueValue )|nwith expansion:|n false|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK_FALSE( trueValue )|nwith expansion:|n !true|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( !(1 == 1) )|nwith expansion:|n false|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK_FALSE( 1 == 1 )|nwith expansion:|n !(1 == 1)|n']
##teamcity[testFinished name='|'Not|' checks that should fail' duration="{duration}"]
##teamcity[testStarted name='|'Not|' checks that should succeed']
##teamcity[testFinished name='|'Not|' checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='(unimplemented) static bools can be evaluated']
##teamcity[testFinished name='(unimplemented) static bools can be evaluated' duration="{duration}"]
##teamcity[testStarted name='3x3x3 ints']
##teamcity[testFinished name='3x3x3 ints' duration="{duration}"]
##teamcity[testStarted name='A METHOD_AS_TEST_CASE based test run that fails']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( s == "world" )|nwith expansion:|n "hello" == "world"|n']
##teamcity[testFinished name='A METHOD_AS_TEST_CASE based test run that fails' duration="{duration}"]
##teamcity[testStarted name='A METHOD_AS_TEST_CASE based test run that succeeds']
##teamcity[testFinished name='A METHOD_AS_TEST_CASE based test run that succeeds' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<float>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<int>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<float>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<int>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 6 < 2|n']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 2 < 2|n']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 6 < 2|n']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 2 < 2|n']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<float,6>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<float,6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<int,2>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<int,2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<float,6>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<float,6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<int,2>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<int,2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - double']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture<TestType>::m_a == 2 )|nwith expansion:|n 1.0 == 2|n']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - double' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - float']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture<TestType>::m_a == 2 )|nwith expansion:|n 1.0f == 2|n']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - float' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - int']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture<TestType>::m_a == 2 )|nwith expansion:|n 1 == 2|n']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - int' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - double']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - double' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - float']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - float' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - int']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - int' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 1']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Nttp_Fixture<V>::value == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 1' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 3']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Nttp_Fixture<V>::value == 0 )|nwith expansion:|n 3 == 0|n']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 3' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 6']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( Nttp_Fixture<V>::value == 0 )|nwith expansion:|n 6 == 0|n']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 6' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 1']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 1' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 3']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 3' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 6']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 6' duration="{duration}"]
##teamcity[testStarted name='A TEST_CASE_METHOD based test run that fails']
Class.tests.cpp:<line number>|nexpression failed|n REQUIRE( m_a == 2 )|nwith expansion:|n 1 == 2|n']
##teamcity[testFinished name='A TEST_CASE_METHOD based test run that fails' duration="{duration}"]
##teamcity[testStarted name='A TEST_CASE_METHOD based test run that succeeds']
##teamcity[testFinished name='A TEST_CASE_METHOD based test run that succeeds' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - Foo<float>']
##teamcity[testFinished name='A Template product test case - Foo<float>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - Foo<int>']
##teamcity[testFinished name='A Template product test case - Foo<int>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - std::vector<float>']
##teamcity[testFinished name='A Template product test case - std::vector<float>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - std::vector<int>']
##teamcity[testFinished name='A Template product test case - std::vector<int>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - Bar<float, 42>']
##teamcity[testFinished name='A Template product test case with array signature - Bar<float, 42>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - Bar<int, 9>']
##teamcity[testFinished name='A Template product test case with array signature - Bar<int, 9>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - std::array<float, 42>']
##teamcity[testFinished name='A Template product test case with array signature - std::array<float, 42>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - std::array<int, 9>']
##teamcity[testFinished name='A Template product test case with array signature - std::array<int, 9>' duration="{duration}"]
##teamcity[testStarted name='A comparison that uses literals instead of the normal constructor']
##teamcity[testFinished name='A comparison that uses literals instead of the normal constructor' duration="{duration}"]
##teamcity[testStarted name='A couple of nested sections followed by a failure']
Misc.tests.cpp:<line number>|nexplicit failure with message:|n "to infinity and beyond"']
##teamcity[testFinished name='A couple of nested sections followed by a failure' duration="{duration}"]
##teamcity[testStarted name='A failing expression with a non streamable type is still captured']
Tricky.tests.cpp:<line number>|nexpression failed|n CHECK( &o1 == &o2 )|nwith expansion:|n 0x<hex digits> == 0x<hex digits>|n']
Tricky.tests.cpp:<line number>|nexpression failed|n CHECK( o1 == o2 )|nwith expansion:|n {?} == {?}|n']
##teamcity[testFinished name='A failing expression with a non streamable type is still captured' duration="{duration}"]
##teamcity[testStarted name='Absolute margin']
##teamcity[testFinished name='Absolute margin' duration="{duration}"]
##teamcity[testStarted name='An empty test with no assertions']
##teamcity[testFinished name='An empty test with no assertions' duration="{duration}"]
##teamcity[testStarted name='An expression with side-effects should only be evaluated once']
##teamcity[testFinished name='An expression with side-effects should only be evaluated once' duration="{duration}"]
##teamcity[testStarted name='An unchecked exception reports the line of the last assertion']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "unexpected exception"|n {Unknown expression after the reported line}|nwith expansion:|n {Unknown expression after the reported line}|n']
##teamcity[testFinished name='An unchecked exception reports the line of the last assertion' duration="{duration}"]
##teamcity[testStarted name='Anonymous test case 1']
##teamcity[testFinished name='Anonymous test case 1' duration="{duration}"]
##teamcity[testStarted name='Approx setters validate their arguments']
##teamcity[testFinished name='Approx setters validate their arguments' duration="{duration}"]
##teamcity[testStarted name='Approx with exactly-representable margin']
##teamcity[testFinished name='Approx with exactly-representable margin' duration="{duration}"]
##teamcity[testStarted name='Approximate PI']
##teamcity[testFinished name='Approximate PI' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with different epsilons']
##teamcity[testFinished name='Approximate comparisons with different epsilons' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with floats']
##teamcity[testFinished name='Approximate comparisons with floats' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with ints']
##teamcity[testFinished name='Approximate comparisons with ints' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with mixed numeric types']
##teamcity[testFinished name='Approximate comparisons with mixed numeric types' duration="{duration}"]
##teamcity[testStarted name='Arbitrary predicate matcher']
##teamcity[testFinished name='Arbitrary predicate matcher' duration="{duration}"]
##teamcity[testStarted name='Assertion macros support bit operators and bool conversions']
##teamcity[testFinished name='Assertion macros support bit operators and bool conversions' duration="{duration}"]
##teamcity[testStarted name='Assertions then sections']
##teamcity[testFinished name='Assertions then sections' duration="{duration}"]
##teamcity[testStarted name='Basic use of the Contains range matcher']
##teamcity[testFinished name='Basic use of the Contains range matcher' duration="{duration}"]
##teamcity[testStarted name='Basic use of the Empty range matcher']
##teamcity[testFinished name='Basic use of the Empty range matcher' duration="{duration}"]
##teamcity[testStarted name='CAPTURE can deal with complex expressions']
##teamcity[testFinished name='CAPTURE can deal with complex expressions' duration="{duration}"]
##teamcity[testStarted name='CAPTURE can deal with complex expressions involving commas']
##teamcity[testFinished name='CAPTURE can deal with complex expressions involving commas' duration="{duration}"]
##teamcity[testStarted name='CAPTURE parses string and character constants']
##teamcity[testFinished name='CAPTURE parses string and character constants' duration="{duration}"]
##teamcity[testStarted name='Capture and info messages']
##teamcity[testFinished name='Capture and info messages' duration="{duration}"]
##teamcity[testStarted name='CaseInsensitiveEqualsTo is case insensitive']
##teamcity[testFinished name='CaseInsensitiveEqualsTo is case insensitive' duration="{duration}"]
##teamcity[testStarted name='CaseInsensitiveLess is case insensitive']
##teamcity[testFinished name='CaseInsensitiveLess is case insensitive' duration="{duration}"]
##teamcity[testStarted name='Character pretty printing']
##teamcity[testFinished name='Character pretty printing' duration="{duration}"]
##teamcity[testStarted name='Clara::Arg supports single-arg parse the way Opt does']
##teamcity[testFinished name='Clara::Arg supports single-arg parse the way Opt does' duration="{duration}"]
##teamcity[testStarted name='Clara::Opt supports accept-many lambdas']
##teamcity[testFinished name='Clara::Opt supports accept-many lambdas' duration="{duration}"]
Add tests for ColourGuard
2022-03-24 00:09:27 +01:00
##teamcity[testStarted name='ColourGuard behaviour']
##teamcity[testFinished name='ColourGuard behaviour' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Combining MatchAllOfGeneric does not nest']
##teamcity[testFinished name='Combining MatchAllOfGeneric does not nest' duration="{duration}"]
##teamcity[testStarted name='Combining MatchAnyOfGeneric does not nest']
##teamcity[testFinished name='Combining MatchAnyOfGeneric does not nest' duration="{duration}"]
##teamcity[testStarted name='Combining MatchNotOfGeneric does not nest']
##teamcity[testFinished name='Combining MatchNotOfGeneric does not nest' duration="{duration}"]
##teamcity[testStarted name='Combining concrete matchers does not use templated matchers']
##teamcity[testFinished name='Combining concrete matchers does not use templated matchers' duration="{duration}"]
##teamcity[testStarted name='Combining only templated matchers']
##teamcity[testFinished name='Combining only templated matchers' duration="{duration}"]
##teamcity[testStarted name='Combining templated and concrete matchers']
##teamcity[testFinished name='Combining templated and concrete matchers' duration="{duration}"]
##teamcity[testStarted name='Combining templated matchers']
##teamcity[testFinished name='Combining templated matchers' duration="{duration}"]
##teamcity[testStarted name='Commas in various macros are allowed']
##teamcity[testFinished name='Commas in various macros are allowed' duration="{duration}"]
##teamcity[testStarted name='Comparing function pointers']
##teamcity[testFinished name='Comparing function pointers' duration="{duration}"]
##teamcity[testStarted name='Comparison ops']
##teamcity[testFinished name='Comparison ops' duration="{duration}"]
##teamcity[testStarted name='Comparison with explicitly convertible types']
##teamcity[testFinished name='Comparison with explicitly convertible types' duration="{duration}"]
##teamcity[testStarted name='Comparisons between ints where one side is computed']
##teamcity[testFinished name='Comparisons between ints where one side is computed' duration="{duration}"]
##teamcity[testStarted name='Comparisons between unsigned ints and negative signed ints match c++ standard behaviour']
##teamcity[testFinished name='Comparisons between unsigned ints and negative signed ints match c++ standard behaviour' duration="{duration}"]
##teamcity[testStarted name='Comparisons with int literals don|'t warn when mixing signed/ unsigned']
##teamcity[testFinished name='Comparisons with int literals don|'t warn when mixing signed/ unsigned' duration="{duration}"]
##teamcity[testStarted name='Composed generic matchers shortcircuit']
##teamcity[testFinished name='Composed generic matchers shortcircuit' duration="{duration}"]
##teamcity[testStarted name='Composed matchers shortcircuit']
##teamcity[testFinished name='Composed matchers shortcircuit' duration="{duration}"]
##teamcity[testStarted name='Contains string matcher']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), ContainsSubstring( "not there", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" contains: "not there" (case insensitive)|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), ContainsSubstring( "STRING" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" contains: "STRING"|n']
##teamcity[testFinished name='Contains string matcher' duration="{duration}"]
##teamcity[testStarted name='Copy and then generate a range']
##teamcity[testFinished name='Copy and then generate a range' duration="{duration}"]
Add IStream::isStdOut
2022-02-21 23:48:15 +01:00
##teamcity[testStarted name='Cout stream properly declares it writes to stdout']
##teamcity[testFinished name='Cout stream properly declares it writes to stdout' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Custom exceptions can be translated when testing for nothrow']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "custom exception - not std"|n REQUIRE_NOTHROW( throwCustom() )|nwith expansion:|n throwCustom()|n']
##teamcity[testFinished name='Custom exceptions can be translated when testing for nothrow' duration="{duration}"]
##teamcity[testStarted name='Custom exceptions can be translated when testing for throwing as something else']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "custom exception - not std"|n REQUIRE_THROWS_AS( throwCustom(), std::exception )|nwith expansion:|n throwCustom(), std::exception|n']
##teamcity[testFinished name='Custom exceptions can be translated when testing for throwing as something else' duration="{duration}"]
##teamcity[testStarted name='Custom std-exceptions can be custom translated']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "custom std exception"']
##teamcity[testFinished name='Custom std-exceptions can be custom translated' duration="{duration}"]
##teamcity[testStarted name='Default scale is invisible to comparison']
##teamcity[testFinished name='Default scale is invisible to comparison' duration="{duration}"]
##teamcity[testStarted name='Directly creating an EnumInfo']
##teamcity[testFinished name='Directly creating an EnumInfo' duration="{duration}"]
Add IStream::isStdOut
2022-02-21 23:48:15 +01:00
##teamcity[testStarted name='Empty stream name opens cout stream']
##teamcity[testFinished name='Empty stream name opens cout stream' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Empty tag is not allowed']
##teamcity[testFinished name='Empty tag is not allowed' duration="{duration}"]
##teamcity[testStarted name='EndsWith string matcher']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), EndsWith( "Substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ends with: "Substring"|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), EndsWith( "this", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ends with: "this" (case insensitive)|n']
##teamcity[testFinished name='EndsWith string matcher' duration="{duration}"]
##teamcity[testStarted name='Enums can quickly have stringification enabled using REGISTER_ENUM']
##teamcity[testFinished name='Enums can quickly have stringification enabled using REGISTER_ENUM' duration="{duration}"]
##teamcity[testStarted name='Enums in namespaces can quickly have stringification enabled using REGISTER_ENUM']
##teamcity[testFinished name='Enums in namespaces can quickly have stringification enabled using REGISTER_ENUM' duration="{duration}"]
##teamcity[testStarted name='Epsilon only applies to Approx|'s value']
##teamcity[testFinished name='Epsilon only applies to Approx|'s value' duration="{duration}"]
##teamcity[testStarted name='Equality checks that should fail']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven == 6 )|nwith expansion:|n 7 == 6|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven == 8 )|nwith expansion:|n 7 == 8|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven == 0 )|nwith expansion:|n 7 == 0|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 9.11f ) )|nwith expansion:|n 9.1f == Approx( 9.1099996567 )|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 9.0f ) )|nwith expansion:|n 9.1f == Approx( 9.0 )|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 1 ) )|nwith expansion:|n 9.1f == Approx( 1.0 )|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 0 ) )|nwith expansion:|n 9.1f == Approx( 0.0 )|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.double_pi == Approx( 3.1415 ) )|nwith expansion:|n 3.1415926535 == Approx( 3.1415 )|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello == "goodbye" )|nwith expansion:|n "hello" == "goodbye"|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello == "hell" )|nwith expansion:|n "hello" == "hell"|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello == "hello1" )|nwith expansion:|n "hello" == "hello1"|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello.size() == 6 )|nwith expansion:|n 5 == 6|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( x == Approx( 1.301 ) )|nwith expansion:|n 1.3 == Approx( 1.301 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Equality checks that should fail' duration="{duration}"]
##teamcity[testStarted name='Equality checks that should succeed']
##teamcity[testFinished name='Equality checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='Equals']
##teamcity[testFinished name='Equals' duration="{duration}"]
##teamcity[testStarted name='Equals string matcher']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Equals( "this string contains |'ABC|' as a substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" equals: "this string contains |'ABC|' as a substring"|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Equals( "something else", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" equals: "something else" (case insensitive)|n']
##teamcity[testFinished name='Equals string matcher' duration="{duration}"]
##teamcity[testStarted name='Exception as a value (e.g. in REQUIRE_THROWS_MATCHES) can be stringified']
##teamcity[testFinished name='Exception as a value (e.g. in REQUIRE_THROWS_MATCHES) can be stringified' duration="{duration}"]
##teamcity[testStarted name='Exception matchers that fail']
Matchers.tests.cpp:<line number>|nno exception was thrown where one was expected|n CHECK_THROWS_MATCHES( doesNotThrow(), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n doesNotThrow(), SpecialException, ExceptionMatcher{ 1 }|n']
Matchers.tests.cpp:<line number>|nno exception was thrown where one was expected|n REQUIRE_THROWS_MATCHES( doesNotThrow(), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n doesNotThrow(), SpecialException, ExceptionMatcher{ 1 }|n']
Matchers.tests.cpp:<line number>|nunexpected exception with message:|n "Unknown exception"|n CHECK_THROWS_MATCHES( throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 }|n']
Matchers.tests.cpp:<line number>|nunexpected exception with message:|n "Unknown exception"|n REQUIRE_THROWS_MATCHES( throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THROWS_MATCHES( throwsSpecialException( 3 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n SpecialException::what special exception has value of 1|n']
Matchers.tests.cpp:<line number>|nexpression failed|n REQUIRE_THROWS_MATCHES( throwsSpecialException( 4 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n SpecialException::what special exception has value of 1|n']
##teamcity[testFinished name='Exception matchers that fail' duration="{duration}"]
##teamcity[testStarted name='Exception matchers that succeed']
##teamcity[testFinished name='Exception matchers that succeed' duration="{duration}"]
##teamcity[testStarted name='Exception messages can be tested for']
##teamcity[testFinished name='Exception messages can be tested for' duration="{duration}"]
##teamcity[testStarted name='Exceptions matchers']
##teamcity[testFinished name='Exceptions matchers' duration="{duration}"]
##teamcity[testStarted name='Expected exceptions that don|'t throw or unexpected exceptions fail the test']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK_THROWS_AS( thisThrows(), std::string )|nwith expansion:|n thisThrows(), std::string|n']
Exception.tests.cpp:<line number>|nno exception was thrown where one was expected|n CHECK_THROWS_AS( thisDoesntThrow(), std::domain_error )|nwith expansion:|n thisDoesntThrow(), std::domain_error|n']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK_NOTHROW( thisThrows() )|nwith expansion:|n thisThrows()|n']
##teamcity[testFinished name='Expected exceptions that don|'t throw or unexpected exceptions fail the test' duration="{duration}"]
##teamcity[testStarted name='FAIL aborts the test']
Message.tests.cpp:<line number>|nexplicit failure with message:|n "This is a failure"']
##teamcity[testFinished name='FAIL aborts the test' duration="{duration}"]
##teamcity[testStarted name='FAIL does not require an argument']
Message.tests.cpp:<line number>|nexplicit failure']
##teamcity[testFinished name='FAIL does not require an argument' duration="{duration}"]
##teamcity[testStarted name='FAIL_CHECK does not abort the test']
Message.tests.cpp:<line number>|nexplicit failure with message:|n "This is a failure"']
##teamcity[testFinished name='FAIL_CHECK does not abort the test' duration="{duration}"]
##teamcity[testStarted name='Factorials are computed']
##teamcity[testFinished name='Factorials are computed' duration="{duration}"]
##teamcity[testStarted name='Floating point matchers: double']
##teamcity[testFinished name='Floating point matchers: double' duration="{duration}"]
##teamcity[testStarted name='Floating point matchers: float']
##teamcity[testFinished name='Floating point matchers: float' duration="{duration}"]
##teamcity[testStarted name='Generators -- adapters']
##teamcity[testFinished name='Generators -- adapters' duration="{duration}"]
##teamcity[testStarted name='Generators -- simple']
##teamcity[testFinished name='Generators -- simple' duration="{duration}"]
##teamcity[testStarted name='Generators internals']
##teamcity[testFinished name='Generators internals' duration="{duration}"]
##teamcity[testStarted name='Greater-than inequalities with different epsilons']
##teamcity[testFinished name='Greater-than inequalities with different epsilons' duration="{duration}"]
Refactor tests
2022-03-31 22:30:38 +02:00
##teamcity[testStarted name='Hashers with different seed produce different hash with same test case']
##teamcity[testFinished name='Hashers with different seed produce different hash with same test case' duration="{duration}"]
##teamcity[testStarted name='Hashers with same seed produce same hash']
##teamcity[testFinished name='Hashers with same seed produce same hash' duration="{duration}"]
##teamcity[testStarted name='Hashing different test cases produces different result']
##teamcity[testFinished name='Hashing different test cases produces different result' duration="{duration}"]
##teamcity[testStarted name='Hashing test case produces same hash across multiple calls']
##teamcity[testFinished name='Hashing test case produces same hash across multiple calls' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='INFO and WARN do not abort tests']
##teamcity[testFinished name='INFO and WARN do not abort tests' duration="{duration}"]
##teamcity[testStarted name='INFO gets logged on failure']
Message.tests.cpp:<line number>|nexpression failed with messages:|n "this message should be logged"|n "so should this"|n REQUIRE( a == 1 )|nwith expansion:|n 2 == 1|n']
##teamcity[testFinished name='INFO gets logged on failure' duration="{duration}"]
##teamcity[testStarted name='INFO gets logged on failure, even if captured before successful assertions']
Message.tests.cpp:<line number>|nexpression failed with messages:|n "this message may be logged later"|n "this message should be logged"|n CHECK( a == 1 )|nwith expansion:|n 2 == 1|n']
Message.tests.cpp:<line number>|nexpression failed with messages:|n "this message may be logged later"|n "this message should be logged"|n "and this, but later"|n CHECK( a == 0 )|nwith expansion:|n 2 == 0|n']
##teamcity[testFinished name='INFO gets logged on failure, even if captured before successful assertions' duration="{duration}"]
##teamcity[testStarted name='INFO is reset for each loop']
Message.tests.cpp:<line number>|nexpression failed with messages:|n "current counter 10"|n "i := 10"|n REQUIRE( i < 10 )|nwith expansion:|n 10 < 10|n']
##teamcity[testFinished name='INFO is reset for each loop' duration="{duration}"]
##teamcity[testStarted name='Inequality checks that should fail']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven != 7 )|nwith expansion:|n 7 != 7|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one != Approx( 9.1f ) )|nwith expansion:|n 9.1f != Approx( 9.1000003815 )|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.double_pi != Approx( 3.1415926535 ) )|nwith expansion:|n 3.1415926535 != Approx( 3.1415926535 )|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello != "hello" )|nwith expansion:|n "hello" != "hello"|n- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello.size() != 5 )|nwith expansion:|n 5 != 5|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Inequality checks that should fail' duration="{duration}"]
##teamcity[testStarted name='Inequality checks that should succeed']
##teamcity[testFinished name='Inequality checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='Lambdas in assertions']
##teamcity[testFinished name='Lambdas in assertions' duration="{duration}"]
##teamcity[testStarted name='Less-than inequalities with different epsilons']
##teamcity[testFinished name='Less-than inequalities with different epsilons' duration="{duration}"]
##teamcity[testStarted name='ManuallyRegistered']
##teamcity[testFinished name='ManuallyRegistered' duration="{duration}"]
##teamcity[testStarted name='Matchers can be (AllOf) composed with the && operator']
##teamcity[testFinished name='Matchers can be (AllOf) composed with the && operator' duration="{duration}"]
##teamcity[testStarted name='Matchers can be (AnyOf) composed with the |||| operator']
##teamcity[testFinished name='Matchers can be (AnyOf) composed with the |||| operator' duration="{duration}"]
##teamcity[testStarted name='Matchers can be composed with both && and ||||']
##teamcity[testFinished name='Matchers can be composed with both && and ||||' duration="{duration}"]
##teamcity[testStarted name='Matchers can be composed with both && and |||| - failing']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), ( ContainsSubstring( "string" ) |||| ContainsSubstring( "different" ) ) && ContainsSubstring( "random" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ( ( contains: "string" or contains: "different" ) and contains: "random" )|n']
##teamcity[testFinished name='Matchers can be composed with both && and |||| - failing' duration="{duration}"]
##teamcity[testStarted name='Matchers can be negated (Not) with the ! operator']
##teamcity[testFinished name='Matchers can be negated (Not) with the ! operator' duration="{duration}"]
##teamcity[testStarted name='Matchers can be negated (Not) with the ! operator - failing']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), !ContainsSubstring( "substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" not contains: "substring"|n']
##teamcity[testFinished name='Matchers can be negated (Not) with the ! operator - failing' duration="{duration}"]
##teamcity[testStarted name='Mayfail test case with nested sections']
Condition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
Condition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Mayfail test case with nested sections' duration="{duration}"]
##teamcity[testStarted name='Mismatching exception messages failing the test']
Exception.tests.cpp:<line number>|nexpression failed|n REQUIRE_THROWS_WITH( thisThrows(), "should fail" )|nwith expansion:|n "expected exception" equals: "should fail"|n']
##teamcity[testFinished name='Mismatching exception messages failing the test' duration="{duration}"]
Add more unit tests * Call order of listeners/reporters in multireporter * Adding listeners/reporters properly updates reporter preferences
2021-12-19 15:16:56 +01:00
##teamcity[testStarted name='Multireporter calls reporters and listeners in correct order']
##teamcity[testFinished name='Multireporter calls reporters and listeners in correct order' duration="{duration}"]
##teamcity[testStarted name='Multireporter updates ReporterPreferences properly']
##teamcity[testFinished name='Multireporter updates ReporterPreferences properly' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Nested generators and captured variables']
##teamcity[testFinished name='Nested generators and captured variables' duration="{duration}"]
##teamcity[testStarted name='Nice descriptive name']
##teamcity[testFinished name='Nice descriptive name' duration="{duration}"]
##teamcity[testStarted name='Non-std exceptions can be translated']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "custom exception"']
##teamcity[testFinished name='Non-std exceptions can be translated' duration="{duration}"]
##teamcity[testStarted name='Objects that evaluated in boolean contexts can be checked']
##teamcity[testFinished name='Objects that evaluated in boolean contexts can be checked' duration="{duration}"]
##teamcity[testStarted name='Optionally static assertions']
##teamcity[testFinished name='Optionally static assertions' duration="{duration}"]
##teamcity[testStarted name='Ordering comparison checks that should fail']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven > 7 )|nwith expansion:|n 7 > 7|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < 7 )|nwith expansion:|n 7 < 7|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven > 8 )|nwith expansion:|n 7 > 8|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < 6 )|nwith expansion:|n 7 < 6|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < 0 )|nwith expansion:|n 7 < 0|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < -1 )|nwith expansion:|n 7 < -1|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven >= 8 )|nwith expansion:|n 7 >= 8|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven <= 6 )|nwith expansion:|n 7 <= 6|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one < 9 )|nwith expansion:|n 9.1f < 9|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one > 10 )|nwith expansion:|n 9.1f > 10|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one > 9.2 )|nwith expansion:|n 9.1f > 9.2|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello > "hello" )|nwith expansion:|n "hello" > "hello"|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello < "hello" )|nwith expansion:|n "hello" < "hello"|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello > "hellp" )|nwith expansion:|n "hello" > "hellp"|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello > "z" )|nwith expansion:|n "hello" > "z"|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello < "hellm" )|nwith expansion:|n "hello" < "hellm"|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello < "a" )|nwith expansion:|n "hello" < "a"|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello >= "z" )|nwith expansion:|n "hello" >= "z"|n']
Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello <= "a" )|nwith expansion:|n "hello" <= "a"|n']
##teamcity[testFinished name='Ordering comparison checks that should fail' duration="{duration}"]
##teamcity[testStarted name='Ordering comparison checks that should succeed']
##teamcity[testFinished name='Ordering comparison checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='Our PCG implementation provides expected results for known seeds']
##teamcity[testFinished name='Our PCG implementation provides expected results for known seeds' duration="{duration}"]
##teamcity[testStarted name='Output from all sections is reported']
Message.tests.cpp:<line number>|nexplicit failure with message:|n "Message from section one"']
Message.tests.cpp:<line number>|nexplicit failure with message:|n "Message from section two"']
##teamcity[testFinished name='Output from all sections is reported' duration="{duration}"]
##teamcity[testStarted name='Overloaded comma or address-of operators are not used']
##teamcity[testFinished name='Overloaded comma or address-of operators are not used' duration="{duration}"]
##teamcity[testStarted name='Parse test names and tags']
##teamcity[testFinished name='Parse test names and tags' duration="{duration}"]
##teamcity[testStarted name='Parsed tags are matched case insensitive']
##teamcity[testFinished name='Parsed tags are matched case insensitive' duration="{duration}"]
##teamcity[testStarted name='Parsing sharding-related cli flags']
##teamcity[testFinished name='Parsing sharding-related cli flags' duration="{duration}"]
##teamcity[testStarted name='Parsing tags with non-alphabetical characters is pass-through']
##teamcity[testFinished name='Parsing tags with non-alphabetical characters is pass-through' duration="{duration}"]
##teamcity[testStarted name='Parsing warnings']
##teamcity[testFinished name='Parsing warnings' duration="{duration}"]
##teamcity[testStarted name='Pointers can be compared to null']
##teamcity[testFinished name='Pointers can be compared to null' duration="{duration}"]
##teamcity[testStarted name='Precision of floating point stringification can be set']
##teamcity[testFinished name='Precision of floating point stringification can be set' duration="{duration}"]
##teamcity[testStarted name='Predicate matcher can accept const char*']
##teamcity[testFinished name='Predicate matcher can accept const char*' duration="{duration}"]
##teamcity[testStarted name='Process can be configured on command line']
##teamcity[testFinished name='Process can be configured on command line' duration="{duration}"]
##teamcity[testStarted name='Product with differing arities - std::tuple<int, double, float>']
##teamcity[testFinished name='Product with differing arities - std::tuple<int, double, float>' duration="{duration}"]
##teamcity[testStarted name='Product with differing arities - std::tuple<int, double>']
##teamcity[testFinished name='Product with differing arities - std::tuple<int, double>' duration="{duration}"]
##teamcity[testStarted name='Product with differing arities - std::tuple<int>']
##teamcity[testFinished name='Product with differing arities - std::tuple<int>' duration="{duration}"]
##teamcity[testStarted name='Random seed generation accepts known methods']
##teamcity[testFinished name='Random seed generation accepts known methods' duration="{duration}"]
##teamcity[testStarted name='Random seed generation reports unknown methods']
##teamcity[testFinished name='Random seed generation reports unknown methods' duration="{duration}"]
##teamcity[testStarted name='Range type with sentinel']
##teamcity[testFinished name='Range type with sentinel' duration="{duration}"]
##teamcity[testStarted name='Reconstruction should be based on stringification: #914']
Decomposition.tests.cpp:<line number>|nexpression failed|n CHECK( truthy(false) )|nwith expansion:|n Hey, its truthy!|n']
##teamcity[testFinished name='Reconstruction should be based on stringification: #914' duration="{duration}"]
##teamcity[testStarted name='Regex string matcher']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Matches( "this STRING contains |'abc|' as a substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" matches "this STRING contains |'abc|' as a substring" case sensitively|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Matches( "contains |'abc|' as a substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" matches "contains |'abc|' as a substring" case sensitively|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Matches( "this string contains |'abc|' as a" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" matches "this string contains |'abc|' as a" case sensitively|n']
##teamcity[testFinished name='Regex string matcher' duration="{duration}"]
Error out reporter registration for reporter names with '::'
2022-01-09 19:01:14 +01:00
##teamcity[testStarted name='Registering reporter with |'::|' in name fails']
##teamcity[testFinished name='Registering reporter with |'::|' in name fails' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Regression test #1']
##teamcity[testFinished name='Regression test #1' duration="{duration}"]
##teamcity[testStarted name='Reporter|'s write listings to provided stream']
##teamcity[testFinished name='Reporter|'s write listings to provided stream' duration="{duration}"]
##teamcity[testStarted name='Reproducer for #2309 - a very long description past 80 chars (default console width) with a late colon : blablabla']
##teamcity[testFinished name='Reproducer for #2309 - a very long description past 80 chars (default console width) with a late colon : blablabla' duration="{duration}"]
##teamcity[testStarted name='SUCCEED counts as a test pass']
##teamcity[testFinished name='SUCCEED counts as a test pass' duration="{duration}"]
##teamcity[testStarted name='SUCCEED does not require an argument']
##teamcity[testFinished name='SUCCEED does not require an argument' duration="{duration}"]
##teamcity[testStarted name='Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or methods']
##teamcity[testFinished name='Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or methods' duration="{duration}"]
##teamcity[testStarted name='Scenario: Do that thing with the thing']
##teamcity[testFinished name='Scenario: Do that thing with the thing' duration="{duration}"]
##teamcity[testStarted name='Scenario: This is a really long scenario name to see how the list command deals with wrapping']
##teamcity[testFinished name='Scenario: This is a really long scenario name to see how the list command deals with wrapping' duration="{duration}"]
##teamcity[testStarted name='Scenario: Vector resizing affects size and capacity']
##teamcity[testFinished name='Scenario: Vector resizing affects size and capacity' duration="{duration}"]
##teamcity[testStarted name='Sends stuff to stdout and stderr']
##teamcity[testStdOut name='Sends stuff to stdout and stderr' out='A string sent directly to stdout|n']
##teamcity[testStdErr name='Sends stuff to stdout and stderr' out='A string sent directly to stderr|nA string sent to stderr via clog|n']
##teamcity[testFinished name='Sends stuff to stdout and stderr' duration="{duration}"]
##teamcity[testStarted name='Some simple comparisons between doubles']
##teamcity[testFinished name='Some simple comparisons between doubles' duration="{duration}"]
##teamcity[testStarted name='Standard output from all sections is reported']
##teamcity[testStdOut name='Standard output from all sections is reported' out='Message from section one|nMessage from section two|n']
##teamcity[testFinished name='Standard output from all sections is reported' duration="{duration}"]
##teamcity[testStarted name='StartsWith string matcher']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), StartsWith( "This String" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" starts with: "This String"|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), StartsWith( "string", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" starts with: "string" (case insensitive)|n']
##teamcity[testFinished name='StartsWith string matcher' duration="{duration}"]
##teamcity[testStarted name='Static arrays are convertible to string']
##teamcity[testFinished name='Static arrays are convertible to string' duration="{duration}"]
##teamcity[testStarted name='String matchers']
##teamcity[testFinished name='String matchers' duration="{duration}"]
##teamcity[testStarted name='StringRef']
##teamcity[testFinished name='StringRef' duration="{duration}"]
##teamcity[testStarted name='StringRef at compilation time']
##teamcity[testFinished name='StringRef at compilation time' duration="{duration}"]
##teamcity[testStarted name='Stringifying char arrays with statically known sizes - char']
##teamcity[testFinished name='Stringifying char arrays with statically known sizes - char' duration="{duration}"]
##teamcity[testStarted name='Stringifying char arrays with statically known sizes - signed char']
##teamcity[testFinished name='Stringifying char arrays with statically known sizes - signed char' duration="{duration}"]
##teamcity[testStarted name='Stringifying char arrays with statically known sizes - unsigned char']
##teamcity[testFinished name='Stringifying char arrays with statically known sizes - unsigned char' duration="{duration}"]
##teamcity[testStarted name='Stringifying std::chrono::duration helpers']
##teamcity[testFinished name='Stringifying std::chrono::duration helpers' duration="{duration}"]
##teamcity[testStarted name='Stringifying std::chrono::duration with weird ratios']
##teamcity[testFinished name='Stringifying std::chrono::duration with weird ratios' duration="{duration}"]
##teamcity[testStarted name='Stringifying std::chrono::time_point<system_clock>']
##teamcity[testFinished name='Stringifying std::chrono::time_point<system_clock>' duration="{duration}"]
##teamcity[testStarted name='Tabs and newlines show in output']
Misc.tests.cpp:<line number>|nexpression failed|n CHECK( s1 == s2 )|nwith expansion:|n "if ($b == 10) {|n $a = 20;|n}"|n==|n"if ($b == 10) {|n $a = 20;|n}|n"|n']
##teamcity[testFinished name='Tabs and newlines show in output' duration="{duration}"]
##teamcity[testStarted name='Tag alias can be registered against tag patterns']
##teamcity[testFinished name='Tag alias can be registered against tag patterns' duration="{duration}"]
##teamcity[testStarted name='Tags with spaces and non-alphanumerical characters are accepted']
##teamcity[testFinished name='Tags with spaces and non-alphanumerical characters are accepted' duration="{duration}"]
##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 0']
##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 1']
##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 2']
##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 2' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 0']
##teamcity[testFinished name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 1']
##teamcity[testFinished name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 0']
##teamcity[testFinished name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 1']
##teamcity[testFinished name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside std::tuple - MyTypes - 0']
##teamcity[testFinished name='Template test case with test types specified inside std::tuple - MyTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside std::tuple - MyTypes - 1']
##teamcity[testFinished name='Template test case with test types specified inside std::tuple - MyTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside std::tuple - MyTypes - 2']
##teamcity[testFinished name='Template test case with test types specified inside std::tuple - MyTypes - 2' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - float']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - float' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - int']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - int' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - std::string']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - std::string' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - std::tuple<int,float>']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - std::tuple<int,float>' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - (std::tuple<int, float>), 6']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - (std::tuple<int, float>), 6' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - float,4']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - float,4' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - int,5']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - int,5' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - std::string,15']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - std::string,15' duration="{duration}"]
##teamcity[testStarted name='Test case with identical tags keeps just one']
##teamcity[testFinished name='Test case with identical tags keeps just one' duration="{duration}"]
##teamcity[testStarted name='Test case with one argument']
##teamcity[testFinished name='Test case with one argument' duration="{duration}"]
##teamcity[testStarted name='Test enum bit values']
##teamcity[testFinished name='Test enum bit values' duration="{duration}"]
##teamcity[testStarted name='Test with special, characters "in name']
##teamcity[testFinished name='Test with special, characters "in name' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if']
##teamcity[testFinished name='Testing checked-if' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if 2']
Misc.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Testing checked-if 2' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if 3']
Misc.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Testing checked-if 3' duration="{duration}"]
##teamcity[testStarted name='The NO_FAIL macro reports a failure but does not fail the test']
##teamcity[testFinished name='The NO_FAIL macro reports a failure but does not fail the test' duration="{duration}"]
##teamcity[testStarted name='The default listing implementation write to provided stream']
##teamcity[testFinished name='The default listing implementation write to provided stream' duration="{duration}"]
##teamcity[testStarted name='This test |'should|' fail but doesn|'t']
##teamcity[testFinished name='This test |'should|' fail but doesn|'t' duration="{duration}"]
##teamcity[testStarted name='Thrown string literals are translated']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "For some reason someone is throwing a string literal!"']
##teamcity[testFinished name='Thrown string literals are translated' duration="{duration}"]
##teamcity[testStarted name='Tracker']
##teamcity[testFinished name='Tracker' duration="{duration}"]
##teamcity[testStarted name='Trim strings']
##teamcity[testFinished name='Trim strings' duration="{duration}"]
##teamcity[testStarted name='Unexpected exceptions can be translated']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "3.14"']
##teamcity[testFinished name='Unexpected exceptions can be translated' duration="{duration}"]
##teamcity[testStarted name='Upcasting special member functions']
##teamcity[testFinished name='Upcasting special member functions' duration="{duration}"]
##teamcity[testStarted name='Usage of AllMatch range matcher']
##teamcity[testFinished name='Usage of AllMatch range matcher' duration="{duration}"]
##teamcity[testStarted name='Usage of AnyMatch range matcher']
##teamcity[testFinished name='Usage of AnyMatch range matcher' duration="{duration}"]
##teamcity[testStarted name='Usage of NoneMatch range matcher']
##teamcity[testFinished name='Usage of NoneMatch range matcher' duration="{duration}"]
##teamcity[testStarted name='Usage of the SizeIs range matcher']
##teamcity[testFinished name='Usage of the SizeIs range matcher' duration="{duration}"]
##teamcity[testStarted name='Use a custom approx']
##teamcity[testFinished name='Use a custom approx' duration="{duration}"]
##teamcity[testStarted name='Variadic macros']
##teamcity[testFinished name='Variadic macros' duration="{duration}"]
##teamcity[testStarted name='Vector Approx matcher']
##teamcity[testFinished name='Vector Approx matcher' duration="{duration}"]
##teamcity[testStarted name='Vector Approx matcher -- failing']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, Approx( t1 ) )|nwith expansion:|n { } is approx: { 1.0, 2.0 }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v1, Approx( v2 ) )|nwith expansion:|n { 2.0, 4.0, 6.0 } is approx: { 1.0, 3.0, 5.0 }|n']
##teamcity[testFinished name='Vector Approx matcher -- failing' duration="{duration}"]
##teamcity[testStarted name='Vector matchers']
##teamcity[testFinished name='Vector matchers' duration="{duration}"]
##teamcity[testStarted name='Vector matchers that fail']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, VectorContains( -1 ) )|nwith expansion:|n { 1, 2, 3 } Contains: -1|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, VectorContains( 1 ) )|nwith expansion:|n { } Contains: 1|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, Contains( v ) )|nwith expansion:|n { } Contains: { 1, 2, 3 }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, Contains( v2 ) )|nwith expansion:|n { 1, 2, 3 } Contains: { 1, 2, 4 }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, Equals( v2 ) )|nwith expansion:|n { 1, 2, 3 } Equals: { 1, 2 }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v2, Equals( v ) )|nwith expansion:|n { 1, 2 } Equals: { 1, 2, 3 }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, Equals( v ) )|nwith expansion:|n { } Equals: { 1, 2, 3 }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, Equals( empty ) )|nwith expansion:|n { 1, 2, 3 } Equals: { }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, UnorderedEquals( empty ) )|nwith expansion:|n { 1, 2, 3 } UnorderedEquals: { }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, UnorderedEquals( v ) )|nwith expansion:|n { } UnorderedEquals: { 1, 2, 3 }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( permuted, UnorderedEquals( v ) )|nwith expansion:|n { 1, 3 } UnorderedEquals: { 1, 2, 3 }|n']
Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( permuted, UnorderedEquals( v ) )|nwith expansion:|n { 3, 1 } UnorderedEquals: { 1, 2, 3 }|n']
##teamcity[testFinished name='Vector matchers that fail' duration="{duration}"]
##teamcity[testStarted name='When checked exceptions are thrown they can be expected or unexpected']
##teamcity[testFinished name='When checked exceptions are thrown they can be expected or unexpected' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown directly they are always failures']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "unexpected exception"']
##teamcity[testFinished name='When unchecked exceptions are thrown directly they are always failures' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown during a CHECK the test should continue']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK( thisThrows() == 0 )|nwith expansion:|n thisThrows() == 0|n']
##teamcity[testFinished name='When unchecked exceptions are thrown during a CHECK the test should continue' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown during a REQUIRE the test should abort fail']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n REQUIRE( thisThrows() == 0 )|nwith expansion:|n thisThrows() == 0|n']
##teamcity[testFinished name='When unchecked exceptions are thrown during a REQUIRE the test should abort fail' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown from functions they are always failures']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK( thisThrows() == 0 )|nwith expansion:|n thisThrows() == 0|n']
##teamcity[testFinished name='When unchecked exceptions are thrown from functions they are always failures' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown from sections they are always failures']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "unexpected exception"']
##teamcity[testFinished name='When unchecked exceptions are thrown from sections they are always failures' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown, but caught, they do not affect the test']
##teamcity[testFinished name='When unchecked exceptions are thrown, but caught, they do not affect the test' duration="{duration}"]
##teamcity[testStarted name='X/level/0/a']
##teamcity[testFinished name='X/level/0/a' duration="{duration}"]
##teamcity[testStarted name='X/level/0/b']
##teamcity[testFinished name='X/level/0/b' duration="{duration}"]
##teamcity[testStarted name='X/level/1/a']
##teamcity[testFinished name='X/level/1/a' duration="{duration}"]
##teamcity[testStarted name='X/level/1/b']
##teamcity[testFinished name='X/level/1/b' duration="{duration}"]
##teamcity[testStarted name='XmlEncode']
##teamcity[testFinished name='XmlEncode' duration="{duration}"]
##teamcity[testStarted name='XmlWriter writes boolean attributes as true/false']
##teamcity[testFinished name='XmlWriter writes boolean attributes as true/false' duration="{duration}"]
##teamcity[testStarted name='analyse no analysis']
##teamcity[testFinished name='analyse no analysis' duration="{duration}"]
##teamcity[testStarted name='array<int, N> -> toString']
##teamcity[testFinished name='array<int, N> -> toString' duration="{duration}"]
##teamcity[testStarted name='benchmark function call']
##teamcity[testFinished name='benchmark function call' duration="{duration}"]
##teamcity[testStarted name='boolean member']
##teamcity[testFinished name='boolean member' duration="{duration}"]
##teamcity[testStarted name='checkedElse']
##teamcity[testFinished name='checkedElse' duration="{duration}"]
##teamcity[testStarted name='checkedElse, failing']
Misc.tests.cpp:<line number>|nexpression failed|n REQUIRE( testCheckedElse( false ) )|nwith expansion:|n false|n']
##teamcity[testFinished name='checkedElse, failing' duration="{duration}"]
##teamcity[testStarted name='checkedIf']
##teamcity[testFinished name='checkedIf' duration="{duration}"]
##teamcity[testStarted name='checkedIf, failing']
Misc.tests.cpp:<line number>|nexpression failed|n REQUIRE( testCheckedIf( false ) )|nwith expansion:|n false|n']
##teamcity[testFinished name='checkedIf, failing' duration="{duration}"]
##teamcity[testStarted name='classify_outliers']
##teamcity[testFinished name='classify_outliers' duration="{duration}"]
##teamcity[testStarted name='comparisons between const int variables']
##teamcity[testFinished name='comparisons between const int variables' duration="{duration}"]
##teamcity[testStarted name='comparisons between int variables']
##teamcity[testFinished name='comparisons between int variables' duration="{duration}"]
##teamcity[testStarted name='convertToBits']
##teamcity[testFinished name='convertToBits' duration="{duration}"]
##teamcity[testStarted name='empty tags are not allowed']
##teamcity[testFinished name='empty tags are not allowed' duration="{duration}"]
##teamcity[testStarted name='erfc_inv']
##teamcity[testFinished name='erfc_inv' duration="{duration}"]
##teamcity[testStarted name='estimate_clock_resolution']
##teamcity[testFinished name='estimate_clock_resolution' duration="{duration}"]
##teamcity[testStarted name='even more nested SECTION tests']
##teamcity[testFinished name='even more nested SECTION tests' duration="{duration}"]
##teamcity[testStarted name='first tag']
##teamcity[testFinished name='first tag' duration="{duration}"]
##teamcity[testStarted name='has printf']
##teamcity[testFinished name='has printf' duration="{duration}"]
##teamcity[testStarted name='is_unary_function']
##teamcity[testFinished name='is_unary_function' duration="{duration}"]
##teamcity[testStarted name='just failure']
Message.tests.cpp:<line number>|nexplicit failure with message:|n "Previous info should not be seen"']
##teamcity[testFinished name='just failure' duration="{duration}"]
##teamcity[testStarted name='just failure after unscoped info']
Message.tests.cpp:<line number>|nexplicit failure with message:|n "previous unscoped info SHOULD not be seen"']
##teamcity[testFinished name='just failure after unscoped info' duration="{duration}"]
##teamcity[testStarted name='just info']
##teamcity[testFinished name='just info' duration="{duration}"]
##teamcity[testStarted name='just unscoped info']
##teamcity[testFinished name='just unscoped info' duration="{duration}"]
##teamcity[testStarted name='long long']
##teamcity[testFinished name='long long' duration="{duration}"]
##teamcity[testStarted name='looped SECTION tests']
Misc.tests.cpp:<line number>|nexpression failed|n CHECK( b > a )|nwith expansion:|n 0 > 1|n']
Misc.tests.cpp:<line number>|nexpression failed|n CHECK( b > a )|nwith expansion:|n 1 > 1|n']
##teamcity[testFinished name='looped SECTION tests' duration="{duration}"]
##teamcity[testStarted name='looped tests']
Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[0|] (1) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[1|] (1) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[3|] (3) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[4|] (5) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[6|] (13) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[7|] (21) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFinished name='looped tests' duration="{duration}"]
##teamcity[testStarted name='make_unique reimplementation']
##teamcity[testFinished name='make_unique reimplementation' duration="{duration}"]
##teamcity[testStarted name='mean']
##teamcity[testFinished name='mean' duration="{duration}"]
##teamcity[testStarted name='measure']
##teamcity[testFinished name='measure' duration="{duration}"]
##teamcity[testStarted name='mix info, unscoped info and warning']
##teamcity[testFinished name='mix info, unscoped info and warning' duration="{duration}"]
##teamcity[testStarted name='more nested SECTION tests']
Misc.tests.cpp:<line number>|nexpression failed|n REQUIRE( a == b )|nwith expansion:|n 1 == 2|n']
##teamcity[testFinished name='more nested SECTION tests' duration="{duration}"]
##teamcity[testStarted name='nested SECTION tests']
##teamcity[testFinished name='nested SECTION tests' duration="{duration}"]
##teamcity[testStarted name='non streamable - with conv. op']
##teamcity[testFinished name='non streamable - with conv. op' duration="{duration}"]
##teamcity[testStarted name='non-copyable objects']
##teamcity[testFinished name='non-copyable objects' duration="{duration}"]
##teamcity[testStarted name='normal_cdf']
##teamcity[testFinished name='normal_cdf' duration="{duration}"]
##teamcity[testStarted name='normal_quantile']
##teamcity[testFinished name='normal_quantile' duration="{duration}"]
##teamcity[testStarted name='not allowed']
##teamcity[testFinished name='not allowed' duration="{duration}"]
##teamcity[testStarted name='not prints unscoped info from previous failures']
Message.tests.cpp:<line number>|nexpression failed with message:|n "this SHOULD be seen"|n REQUIRE( false )|nwith expansion:|n false|n']
##teamcity[testFinished name='not prints unscoped info from previous failures' duration="{duration}"]
##teamcity[testStarted name='null strings']
##teamcity[testFinished name='null strings' duration="{duration}"]
##teamcity[testStarted name='null_ptr']
##teamcity[testFinished name='null_ptr' duration="{duration}"]
##teamcity[testStarted name='pair<pair<int,const char *,pair<std::string,int> > -> toString']
##teamcity[testFinished name='pair<pair<int,const char *,pair<std::string,int> > -> toString' duration="{duration}"]
##teamcity[testStarted name='parseEnums']
##teamcity[testFinished name='parseEnums' duration="{duration}"]
##teamcity[testStarted name='pointer to class']
##teamcity[testFinished name='pointer to class' duration="{duration}"]
##teamcity[testStarted name='print unscoped info if passing unscoped info is printed']
##teamcity[testFinished name='print unscoped info if passing unscoped info is printed' duration="{duration}"]
##teamcity[testStarted name='prints unscoped info on failure']
Message.tests.cpp:<line number>|nexpression failed with messages:|n "this SHOULD be seen"|n "this SHOULD also be seen"|n REQUIRE( false )|nwith expansion:|n false|n']
##teamcity[testFinished name='prints unscoped info on failure' duration="{duration}"]
##teamcity[testStarted name='prints unscoped info only for the first assertion']
Message.tests.cpp:<line number>|nexpression failed with message:|n "this SHOULD be seen only ONCE"|n CHECK( false )|nwith expansion:|n false|n']
##teamcity[testFinished name='prints unscoped info only for the first assertion' duration="{duration}"]
##teamcity[testStarted name='random SECTION tests']
##teamcity[testFinished name='random SECTION tests' duration="{duration}"]
##teamcity[testStarted name='replaceInPlace']
##teamcity[testFinished name='replaceInPlace' duration="{duration}"]
Add %stderr and %stdout as recognized magic stream names
2022-03-07 10:42:26 +01:00
##teamcity[testStarted name='request an unknown %-starting stream fails']
##teamcity[testFinished name='request an unknown %-starting stream fails' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='resolution']
##teamcity[testFinished name='resolution' duration="{duration}"]
##teamcity[testStarted name='run_for_at_least, chronometer']
##teamcity[testFinished name='run_for_at_least, chronometer' duration="{duration}"]
##teamcity[testStarted name='run_for_at_least, int']
##teamcity[testFinished name='run_for_at_least, int' duration="{duration}"]
##teamcity[testStarted name='second tag']
##teamcity[testFinished name='second tag' duration="{duration}"]
##teamcity[testStarted name='send a single char to INFO']
Misc.tests.cpp:<line number>|nexpression failed with message:|n "3"|n REQUIRE( false )|nwith expansion:|n false|n']
##teamcity[testFinished name='send a single char to INFO' duration="{duration}"]
##teamcity[testStarted name='sends information to INFO']
Message.tests.cpp:<line number>|nexpression failed with messages:|n "hi"|n "i := 7"|n REQUIRE( false )|nwith expansion:|n false|n']
##teamcity[testFinished name='sends information to INFO' duration="{duration}"]
##teamcity[testStarted name='shortened hide tags are split apart']
##teamcity[testFinished name='shortened hide tags are split apart' duration="{duration}"]
##teamcity[testStarted name='splitString']
##teamcity[testFinished name='splitString' duration="{duration}"]
##teamcity[testStarted name='stacks unscoped info in loops']
Message.tests.cpp:<line number>|nexpression failed with messages:|n "Count 1 to 3..."|n "1"|n "2"|n "3"|n CHECK( false )|nwith expansion:|n false|n']
Message.tests.cpp:<line number>|nexpression failed with messages:|n "Count 4 to 6..."|n "4"|n "5"|n "6"|n CHECK( false )|nwith expansion:|n false|n']
##teamcity[testFinished name='stacks unscoped info in loops' duration="{duration}"]
##teamcity[testStarted name='startsWith']
##teamcity[testFinished name='startsWith' duration="{duration}"]
##teamcity[testStarted name='std::map is convertible string']
##teamcity[testFinished name='std::map is convertible string' duration="{duration}"]
##teamcity[testStarted name='std::pair<int,const std::string> -> toString']
##teamcity[testFinished name='std::pair<int,const std::string> -> toString' duration="{duration}"]
##teamcity[testStarted name='std::pair<int,std::string> -> toString']
##teamcity[testFinished name='std::pair<int,std::string> -> toString' duration="{duration}"]
##teamcity[testStarted name='std::set is convertible string']
##teamcity[testFinished name='std::set is convertible string' duration="{duration}"]
##teamcity[testStarted name='std::vector<std::pair<std::string,int> > -> toString']
##teamcity[testFinished name='std::vector<std::pair<std::string,int> > -> toString' duration="{duration}"]
Add %stderr and %stdout as recognized magic stream names
2022-03-07 10:42:26 +01:00
##teamcity[testStarted name='stdout and stderr streams have %-starting name']
##teamcity[testFinished name='stdout and stderr streams have %-starting name' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='stringify ranges']
##teamcity[testFinished name='stringify ranges' duration="{duration}"]
##teamcity[testStarted name='stringify( has_maker )']
##teamcity[testFinished name='stringify( has_maker )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_maker_and_operator )']
##teamcity[testFinished name='stringify( has_maker_and_operator )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_neither )']
##teamcity[testFinished name='stringify( has_neither )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_operator )']
##teamcity[testFinished name='stringify( has_operator )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_template_operator )']
##teamcity[testFinished name='stringify( has_template_operator )' duration="{duration}"]
##teamcity[testStarted name='stringify( vectors<has_maker> )']
##teamcity[testFinished name='stringify( vectors<has_maker> )' duration="{duration}"]
##teamcity[testStarted name='stringify( vectors<has_maker_and_operator> )']
##teamcity[testFinished name='stringify( vectors<has_maker_and_operator> )' duration="{duration}"]
##teamcity[testStarted name='stringify( vectors<has_operator> )']
##teamcity[testFinished name='stringify( vectors<has_operator> )' duration="{duration}"]
##teamcity[testStarted name='strlen3']
##teamcity[testFinished name='strlen3' duration="{duration}"]
##teamcity[testStarted name='tables']
##teamcity[testFinished name='tables' duration="{duration}"]
##teamcity[testStarted name='tags with dots in later positions are not parsed as hidden']
##teamcity[testFinished name='tags with dots in later positions are not parsed as hidden' duration="{duration}"]
##teamcity[testStarted name='thrown std::strings are translated']
Exception.tests.cpp:<line number>|nunexpected exception with message:|n "Why would you throw a std::string?"']
##teamcity[testFinished name='thrown std::strings are translated' duration="{duration}"]
##teamcity[testStarted name='toString on const wchar_t const pointer returns the string contents']
##teamcity[testFinished name='toString on const wchar_t const pointer returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString on const wchar_t pointer returns the string contents']
##teamcity[testFinished name='toString on const wchar_t pointer returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString on wchar_t const pointer returns the string contents']
##teamcity[testFinished name='toString on wchar_t const pointer returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString on wchar_t returns the string contents']
##teamcity[testFinished name='toString on wchar_t returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString(enum class w/operator<<)']
##teamcity[testFinished name='toString(enum class w/operator<<)' duration="{duration}"]
##teamcity[testStarted name='toString(enum class)']
##teamcity[testFinished name='toString(enum class)' duration="{duration}"]
##teamcity[testStarted name='toString(enum w/operator<<)']
##teamcity[testFinished name='toString(enum w/operator<<)' duration="{duration}"]
##teamcity[testStarted name='toString(enum)']
##teamcity[testFinished name='toString(enum)' duration="{duration}"]
##teamcity[testStarted name='tuple<>']
##teamcity[testFinished name='tuple<>' duration="{duration}"]
##teamcity[testStarted name='tuple<float,int>']
##teamcity[testFinished name='tuple<float,int>' duration="{duration}"]
##teamcity[testStarted name='tuple<int>']
##teamcity[testFinished name='tuple<int>' duration="{duration}"]
##teamcity[testStarted name='tuple<string,string>']
##teamcity[testFinished name='tuple<string,string>' duration="{duration}"]
##teamcity[testStarted name='tuple<tuple<int>,tuple<>,float>']
##teamcity[testFinished name='tuple<tuple<int>,tuple<>,float>' duration="{duration}"]
##teamcity[testStarted name='uniform samples']
##teamcity[testFinished name='uniform samples' duration="{duration}"]
##teamcity[testStarted name='unique_ptr reimplementation: basic functionality']
##teamcity[testFinished name='unique_ptr reimplementation: basic functionality' duration="{duration}"]
##teamcity[testStarted name='vec<vec<string,alloc>> -> toString']
##teamcity[testFinished name='vec<vec<string,alloc>> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<bool> -> toString']
##teamcity[testFinished name='vector<bool> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<int,allocator> -> toString']
##teamcity[testFinished name='vector<int,allocator> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<int> -> toString']
##teamcity[testFinished name='vector<int> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<string> -> toString']
##teamcity[testFinished name='vector<string> -> toString' duration="{duration}"]
##teamcity[testStarted name='vectors can be sized and resized']
##teamcity[testFinished name='vectors can be sized and resized' duration="{duration}"]
##teamcity[testStarted name='warmup']
##teamcity[testFinished name='warmup' duration="{duration}"]
##teamcity[testStarted name='weighted_average_quantile']
##teamcity[testFinished name='weighted_average_quantile' duration="{duration}"]
##teamcity[testStarted name='xmlentitycheck']
##teamcity[testFinished name='xmlentitycheck' duration="{duration}"]
##teamcity[testSuiteFinished name='<exe-name>']
Reference in New Issue Copy Permalink