3rd-party/catch2
1
0
Fork 0
mirror of https://github.com/catchorg/Catch2.git synced 2024-11-16 18:52:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
codespell
catch2/tests/SelfTest/Baselines/teamcity.sw.multi.approved.txt

1022 lines
126 KiB
Plaintext
Raw Permalink Normal View History

Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testSuiteStarted name='<exe-name>']
##teamcity[testStarted name='# A test name that starts with a #']
##teamcity[testFinished name='# A test name that starts with a #' duration="{duration}"]
##teamcity[testStarted name='#1027: Bitfields can be captured']
##teamcity[testFinished name='#1027: Bitfields can be captured' duration="{duration}"]
##teamcity[testStarted name='#1147']
##teamcity[testFinished name='#1147' duration="{duration}"]
##teamcity[testStarted name='#1175 - Hidden Test']
##teamcity[testFinished name='#1175 - Hidden Test' duration="{duration}"]
##teamcity[testStarted name='#1238']
##teamcity[testFinished name='#1238' duration="{duration}"]
##teamcity[testStarted name='#1245']
##teamcity[testFinished name='#1245' duration="{duration}"]
##teamcity[testStarted name='#1319: Sections can have description (even if it is not saved']
##teamcity[testFinished name='#1319: Sections can have description (even if it is not saved' duration="{duration}"]
##teamcity[testStarted name='#1403']
##teamcity[testFinished name='#1403' duration="{duration}"]
##teamcity[testStarted name='#1455 - INFO and WARN can start with a linebreak']
##teamcity[testFinished name='#1455 - INFO and WARN can start with a linebreak' duration="{duration}"]
##teamcity[testStarted name='#1514: stderr/stdout is not captured in tests aborted by an exception']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='#1514: stderr/stdout is not captured in tests aborted by an exception' message='Tricky.tests.cpp:<line number>|n...............................................................................|n|nTricky.tests.cpp:<line number>|nexplicit failure with message:|n "1514"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStdOut name='#1514: stderr/stdout is not captured in tests aborted by an exception' out='This would not be caught previously|n']
##teamcity[testStdErr name='#1514: stderr/stdout is not captured in tests aborted by an exception' out='Nor would this|n']
##teamcity[testFinished name='#1514: stderr/stdout is not captured in tests aborted by an exception' duration="{duration}"]
##teamcity[testStarted name='#1548']
##teamcity[testFinished name='#1548' duration="{duration}"]
##teamcity[testStarted name='#1905 -- test spec parser properly clears internal state between compound tests']
##teamcity[testFinished name='#1905 -- test spec parser properly clears internal state between compound tests' duration="{duration}"]
##teamcity[testStarted name='#1912 -- test spec parser handles escaping']
##teamcity[testFinished name='#1912 -- test spec parser handles escaping' duration="{duration}"]
##teamcity[testStarted name='#1913 - GENERATE inside a for loop should not keep recreating the generator']
##teamcity[testFinished name='#1913 - GENERATE inside a for loop should not keep recreating the generator' duration="{duration}"]
##teamcity[testStarted name='#1913 - GENERATEs can share a line']
##teamcity[testFinished name='#1913 - GENERATEs can share a line' duration="{duration}"]
##teamcity[testStarted name='#1938 - GENERATE after a section']
##teamcity[testFinished name='#1938 - GENERATE after a section' duration="{duration}"]
##teamcity[testStarted name='#1938 - Section followed by flat generate']
##teamcity[testFinished name='#1938 - Section followed by flat generate' duration="{duration}"]
##teamcity[testStarted name='#1938 - flat generate']
##teamcity[testFinished name='#1938 - flat generate' duration="{duration}"]
##teamcity[testStarted name='#1938 - mixed sections and generates']
##teamcity[testFinished name='#1938 - mixed sections and generates' duration="{duration}"]
##teamcity[testStarted name='#1938 - nested generate']
##teamcity[testFinished name='#1938 - nested generate' duration="{duration}"]
##teamcity[testStarted name='#1954 - 7 arg template test case sig compiles - 1, 1, 1, 1, 1, 0, 0']
##teamcity[testFinished name='#1954 - 7 arg template test case sig compiles - 1, 1, 1, 1, 1, 0, 0' duration="{duration}"]
##teamcity[testStarted name='#1954 - 7 arg template test case sig compiles - 5, 1, 1, 1, 1, 0, 0']
##teamcity[testFinished name='#1954 - 7 arg template test case sig compiles - 5, 1, 1, 1, 1, 0, 0' duration="{duration}"]
##teamcity[testStarted name='#1954 - 7 arg template test case sig compiles - 5, 3, 1, 1, 1, 0, 0']
##teamcity[testFinished name='#1954 - 7 arg template test case sig compiles - 5, 3, 1, 1, 1, 0, 0' duration="{duration}"]
##teamcity[testStarted name='#2152 - ULP checks between differently signed values were wrong - double']
##teamcity[testFinished name='#2152 - ULP checks between differently signed values were wrong - double' duration="{duration}"]
##teamcity[testStarted name='#2152 - ULP checks between differently signed values were wrong - float']
##teamcity[testFinished name='#2152 - ULP checks between differently signed values were wrong - float' duration="{duration}"]
Stop exceptions in generator constructors from aborting the binary Fixes #2615
2023-01-17 11:04:49 +01:00
##teamcity[testStarted name='#2615 - Throwing in constructor generator fails test case but does not abort']
##teamcity[testIgnored name='#2615 - Throwing in constructor generator fails test case but does not abort' message='Generators.tests.cpp:<line number>|n...............................................................................|n|nGenerators.tests.cpp:<line number>|nunexpected exception with message:|n "failure to init"- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='#2615 - Throwing in constructor generator fails test case but does not abort' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='#748 - captures with unexpected exceptions']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='#748 - captures with unexpected exceptions' message='-------------------------------------------------------------------------------|noutside assertions|n-------------------------------------------------------------------------------|nException.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with messages:|n "answer := 42"|n "expected exception"- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='#748 - captures with unexpected exceptions' message='-------------------------------------------------------------------------------|ninside REQUIRE_NOTHROW|n-------------------------------------------------------------------------------|nException.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with messages:|n "answer := 42"|n "expected exception"|n REQUIRE_NOTHROW( thisThrows() )|nwith expansion:|n thisThrows()|n- failure ignore as test marked as |'ok to fail|'|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='#748 - captures with unexpected exceptions' duration="{duration}"]
##teamcity[testStarted name='#809']
##teamcity[testFinished name='#809' duration="{duration}"]
##teamcity[testStarted name='#833']
##teamcity[testFinished name='#833' duration="{duration}"]
Cleanups output normalization in ApprovalTests Removed: * NaN normalization * INFINITY normalization * errno normalization * Completely unused duration regex Tests using these macros should be tagged `[approvals]` so they are not run as part of approval tests. Also simplified regex for the test's executable filename, and hidden some tests relying on nullptr normalization.
2022-01-26 23:47:40 +01:00
##teamcity[testStarted name='#835 -- errno should not be touched by Catch2']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='#835 -- errno should not be touched by Catch2' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n CHECK( f() == 0 )|nwith expansion:|n 1 == 0|n- failure ignore as test marked as |'ok to fail|'|n']
Cleanups output normalization in ApprovalTests Removed: * NaN normalization * INFINITY normalization * errno normalization * Completely unused duration regex Tests using these macros should be tagged `[approvals]` so they are not run as part of approval tests. Also simplified regex for the test's executable filename, and hidden some tests relying on nullptr normalization.
2022-01-26 23:47:40 +01:00
##teamcity[testFinished name='#835 -- errno should not be touched by Catch2' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='#872']
##teamcity[testFinished name='#872' duration="{duration}"]
##teamcity[testStarted name='#961 -- Dynamically created sections should all be reported']
##teamcity[testFinished name='#961 -- Dynamically created sections should all be reported' duration="{duration}"]
##teamcity[testStarted name='|'Not|' checks that should fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexpression failed|n CHECK( false != false )|nwith expansion:|n false != false|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( true != true )|nwith expansion:|n true != true|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( !true )|nwith expansion:|n false|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK_FALSE( true )|nwith expansion:|n !true|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( !trueValue )|nwith expansion:|n false|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK_FALSE( trueValue )|nwith expansion:|n !true|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( !(1 == 1) )|nwith expansion:|n false|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK_FALSE( 1 == 1 )|nwith expansion:|n !(1 == 1)|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='|'Not|' checks that should fail' duration="{duration}"]
##teamcity[testStarted name='|'Not|' checks that should succeed']
##teamcity[testFinished name='|'Not|' checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='(unimplemented) static bools can be evaluated']
##teamcity[testFinished name='(unimplemented) static bools can be evaluated' duration="{duration}"]
##teamcity[testStarted name='3x3x3 ints']
##teamcity[testFinished name='3x3x3 ints' duration="{duration}"]
##teamcity[testStarted name='A METHOD_AS_TEST_CASE based test run that fails']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A METHOD_AS_TEST_CASE based test run that fails' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( s == "world" )|nwith expansion:|n "hello" == "world"|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A METHOD_AS_TEST_CASE based test run that fails' duration="{duration}"]
##teamcity[testStarted name='A METHOD_AS_TEST_CASE based test run that succeeds']
##teamcity[testFinished name='A METHOD_AS_TEST_CASE based test run that succeeds' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<float>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<int>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<float>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<int>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 6 < 2|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 2 < 2|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 6 < 2|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 2 < 2|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<float,6>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<float,6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<int,2>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<int,2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<float,6>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<float,6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<int,2>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<int,2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - double']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - double' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture<TestType>::m_a == 2 )|nwith expansion:|n 1.0 == 2|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - double' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - float']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - float' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture<TestType>::m_a == 2 )|nwith expansion:|n 1.0f == 2|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - float' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - int']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - int' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture<TestType>::m_a == 2 )|nwith expansion:|n 1 == 2|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - int' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - double']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - double' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - float']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - float' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - int']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - int' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 1']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 1' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Nttp_Fixture<V>::value == 0 )|nwith expansion:|n 1 == 0|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 1' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 3']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 3' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Nttp_Fixture<V>::value == 0 )|nwith expansion:|n 3 == 0|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 3' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 6']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 6' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Nttp_Fixture<V>::value == 0 )|nwith expansion:|n 6 == 0|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 6' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 1']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 1' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 3']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 3' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 6']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 6' duration="{duration}"]
##teamcity[testStarted name='A TEST_CASE_METHOD based test run that fails']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEST_CASE_METHOD based test run that fails' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( m_a == 2 )|nwith expansion:|n 1 == 2|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A TEST_CASE_METHOD based test run that fails' duration="{duration}"]
##teamcity[testStarted name='A TEST_CASE_METHOD based test run that succeeds']
##teamcity[testFinished name='A TEST_CASE_METHOD based test run that succeeds' duration="{duration}"]
TEST_CASE_PERSISTENT_FIXTURE: A new fixture macro for allowing persistent fixtures throughout a TEST_CASE (#2885) This PR introduces a new `TEST_CASE` macro called `TEST_CASE_PERSISTENT_FIXTURE`. `TEST_CASE_PERSISTENT_FIXTURE` offers the same functionality as `TEST_CASE_METHOD` except for one difference. The object on which the test method is invoked is only created once for all invocations of the test case. The object is created just after the `testCaseStarting` event is broadcast and the object is destroyed just before the `testCaseEnding` event is broadcast. The main motivation for this new functionality is to allow `TEST_CASE`s to do expensive setup and teardown once per `TEST_CASE`, without having to resort to abusing event listeners or static function variables with manual initialization. Implements #1602 --------- Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2024-08-05 17:01:41 +02:00
##teamcity[testStarted name='A TEST_CASE_PERSISTENT_FIXTURE based test run that fails']
##teamcity[testFailed name='A TEST_CASE_PERSISTENT_FIXTURE based test run that fails' message='-------------------------------------------------------------------------------|nSecond partial run|n-------------------------------------------------------------------------------|nClass.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( m_a == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFinished name='A TEST_CASE_PERSISTENT_FIXTURE based test run that fails' duration="{duration}"]
##teamcity[testStarted name='A TEST_CASE_PERSISTENT_FIXTURE based test run that succeeds']
##teamcity[testFinished name='A TEST_CASE_PERSISTENT_FIXTURE based test run that succeeds' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='A Template product test case - Foo<float>']
##teamcity[testFinished name='A Template product test case - Foo<float>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - Foo<int>']
##teamcity[testFinished name='A Template product test case - Foo<int>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - std::vector<float>']
##teamcity[testFinished name='A Template product test case - std::vector<float>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - std::vector<int>']
##teamcity[testFinished name='A Template product test case - std::vector<int>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - Bar<float, 42>']
##teamcity[testFinished name='A Template product test case with array signature - Bar<float, 42>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - Bar<int, 9>']
##teamcity[testFinished name='A Template product test case with array signature - Bar<int, 9>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - std::array<float, 42>']
##teamcity[testFinished name='A Template product test case with array signature - std::array<float, 42>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - std::array<int, 9>']
##teamcity[testFinished name='A Template product test case with array signature - std::array<int, 9>' duration="{duration}"]
##teamcity[testStarted name='A comparison that uses literals instead of the normal constructor']
##teamcity[testFinished name='A comparison that uses literals instead of the normal constructor' duration="{duration}"]
##teamcity[testStarted name='A couple of nested sections followed by a failure']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A couple of nested sections followed by a failure' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexplicit failure with message:|n "to infinity and beyond"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A couple of nested sections followed by a failure' duration="{duration}"]
##teamcity[testStarted name='A failing expression with a non streamable type is still captured']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A failing expression with a non streamable type is still captured' message='Tricky.tests.cpp:<line number>|n...............................................................................|n|nTricky.tests.cpp:<line number>|nexpression failed|n CHECK( &o1 == &o2 )|nwith expansion:|n 0x<hex digits> == 0x<hex digits>|n']
##teamcity[testFailed name='A failing expression with a non streamable type is still captured' message='Tricky.tests.cpp:<line number>|nexpression failed|n CHECK( o1 == o2 )|nwith expansion:|n {?} == {?}|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='A failing expression with a non streamable type is still captured' duration="{duration}"]
##teamcity[testStarted name='Absolute margin']
##teamcity[testFinished name='Absolute margin' duration="{duration}"]
##teamcity[testStarted name='An empty test with no assertions']
##teamcity[testFinished name='An empty test with no assertions' duration="{duration}"]
##teamcity[testStarted name='An expression with side-effects should only be evaluated once']
##teamcity[testFinished name='An expression with side-effects should only be evaluated once' duration="{duration}"]
##teamcity[testStarted name='An unchecked exception reports the line of the last assertion']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='An unchecked exception reports the line of the last assertion' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "unexpected exception"|n {Unknown expression after the reported line}|nwith expansion:|n {Unknown expression after the reported line}|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='An unchecked exception reports the line of the last assertion' duration="{duration}"]
##teamcity[testStarted name='Anonymous test case 1']
##teamcity[testFinished name='Anonymous test case 1' duration="{duration}"]
##teamcity[testStarted name='Approx setters validate their arguments']
##teamcity[testFinished name='Approx setters validate their arguments' duration="{duration}"]
##teamcity[testStarted name='Approx with exactly-representable margin']
##teamcity[testFinished name='Approx with exactly-representable margin' duration="{duration}"]
##teamcity[testStarted name='Approximate PI']
##teamcity[testFinished name='Approximate PI' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with different epsilons']
##teamcity[testFinished name='Approximate comparisons with different epsilons' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with floats']
##teamcity[testFinished name='Approximate comparisons with floats' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with ints']
##teamcity[testFinished name='Approximate comparisons with ints' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with mixed numeric types']
##teamcity[testFinished name='Approximate comparisons with mixed numeric types' duration="{duration}"]
##teamcity[testStarted name='Arbitrary predicate matcher']
##teamcity[testFinished name='Arbitrary predicate matcher' duration="{duration}"]
##teamcity[testStarted name='Assertion macros support bit operators and bool conversions']
##teamcity[testFinished name='Assertion macros support bit operators and bool conversions' duration="{duration}"]
##teamcity[testStarted name='Assertions then sections']
##teamcity[testFinished name='Assertions then sections' duration="{duration}"]
##teamcity[testStarted name='Basic use of the Contains range matcher']
##teamcity[testFinished name='Basic use of the Contains range matcher' duration="{duration}"]
##teamcity[testStarted name='Basic use of the Empty range matcher']
##teamcity[testFinished name='Basic use of the Empty range matcher' duration="{duration}"]
##teamcity[testStarted name='CAPTURE can deal with complex expressions']
##teamcity[testFinished name='CAPTURE can deal with complex expressions' duration="{duration}"]
##teamcity[testStarted name='CAPTURE can deal with complex expressions involving commas']
##teamcity[testFinished name='CAPTURE can deal with complex expressions involving commas' duration="{duration}"]
##teamcity[testStarted name='CAPTURE parses string and character constants']
##teamcity[testFinished name='CAPTURE parses string and character constants' duration="{duration}"]
##teamcity[testStarted name='Capture and info messages']
##teamcity[testFinished name='Capture and info messages' duration="{duration}"]
##teamcity[testStarted name='CaseInsensitiveEqualsTo is case insensitive']
##teamcity[testFinished name='CaseInsensitiveEqualsTo is case insensitive' duration="{duration}"]
##teamcity[testStarted name='CaseInsensitiveLess is case insensitive']
##teamcity[testFinished name='CaseInsensitiveLess is case insensitive' duration="{duration}"]
##teamcity[testStarted name='Character pretty printing']
##teamcity[testFinished name='Character pretty printing' duration="{duration}"]
##teamcity[testStarted name='Clara::Arg supports single-arg parse the way Opt does']
##teamcity[testFinished name='Clara::Arg supports single-arg parse the way Opt does' duration="{duration}"]
##teamcity[testStarted name='Clara::Opt supports accept-many lambdas']
##teamcity[testFinished name='Clara::Opt supports accept-many lambdas' duration="{duration}"]
Add tests for ColourGuard
2022-03-24 00:09:27 +01:00
##teamcity[testStarted name='ColourGuard behaviour']
##teamcity[testFinished name='ColourGuard behaviour' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Combining MatchAllOfGeneric does not nest']
##teamcity[testFinished name='Combining MatchAllOfGeneric does not nest' duration="{duration}"]
##teamcity[testStarted name='Combining MatchAnyOfGeneric does not nest']
##teamcity[testFinished name='Combining MatchAnyOfGeneric does not nest' duration="{duration}"]
##teamcity[testStarted name='Combining MatchNotOfGeneric does not nest']
##teamcity[testFinished name='Combining MatchNotOfGeneric does not nest' duration="{duration}"]
##teamcity[testStarted name='Combining concrete matchers does not use templated matchers']
##teamcity[testFinished name='Combining concrete matchers does not use templated matchers' duration="{duration}"]
##teamcity[testStarted name='Combining only templated matchers']
##teamcity[testFinished name='Combining only templated matchers' duration="{duration}"]
##teamcity[testStarted name='Combining templated and concrete matchers']
##teamcity[testFinished name='Combining templated and concrete matchers' duration="{duration}"]
##teamcity[testStarted name='Combining templated matchers']
##teamcity[testFinished name='Combining templated matchers' duration="{duration}"]
##teamcity[testStarted name='Commas in various macros are allowed']
##teamcity[testFinished name='Commas in various macros are allowed' duration="{duration}"]
##teamcity[testStarted name='Comparing function pointers']
##teamcity[testFinished name='Comparing function pointers' duration="{duration}"]
##teamcity[testStarted name='Comparison ops']
##teamcity[testFinished name='Comparison ops' duration="{duration}"]
##teamcity[testStarted name='Comparison with explicitly convertible types']
##teamcity[testFinished name='Comparison with explicitly convertible types' duration="{duration}"]
##teamcity[testStarted name='Comparisons between ints where one side is computed']
##teamcity[testFinished name='Comparisons between ints where one side is computed' duration="{duration}"]
##teamcity[testStarted name='Comparisons between unsigned ints and negative signed ints match c++ standard behaviour']
##teamcity[testFinished name='Comparisons between unsigned ints and negative signed ints match c++ standard behaviour' duration="{duration}"]
##teamcity[testStarted name='Comparisons with int literals don|'t warn when mixing signed/ unsigned']
##teamcity[testFinished name='Comparisons with int literals don|'t warn when mixing signed/ unsigned' duration="{duration}"]
##teamcity[testStarted name='Composed generic matchers shortcircuit']
##teamcity[testFinished name='Composed generic matchers shortcircuit' duration="{duration}"]
##teamcity[testStarted name='Composed matchers shortcircuit']
##teamcity[testFinished name='Composed matchers shortcircuit' duration="{duration}"]
##teamcity[testStarted name='Contains string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Contains string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), ContainsSubstring( "not there", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" contains: "not there" (case insensitive)|n']
##teamcity[testFailed name='Contains string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), ContainsSubstring( "STRING" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" contains: "STRING"|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Contains string matcher' duration="{duration}"]
##teamcity[testStarted name='Copy and then generate a range']
##teamcity[testFinished name='Copy and then generate a range' duration="{duration}"]
Add IStream::isStdOut
2022-02-21 23:48:15 +01:00
##teamcity[testStarted name='Cout stream properly declares it writes to stdout']
##teamcity[testFinished name='Cout stream properly declares it writes to stdout' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Custom exceptions can be translated when testing for nothrow']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Custom exceptions can be translated when testing for nothrow' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "custom exception - not std"|n REQUIRE_NOTHROW( throwCustom() )|nwith expansion:|n throwCustom()|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Custom exceptions can be translated when testing for nothrow' duration="{duration}"]
##teamcity[testStarted name='Custom exceptions can be translated when testing for throwing as something else']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Custom exceptions can be translated when testing for throwing as something else' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "custom exception - not std"|n REQUIRE_THROWS_AS( throwCustom(), std::exception )|nwith expansion:|n throwCustom(), std::exception|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Custom exceptions can be translated when testing for throwing as something else' duration="{duration}"]
##teamcity[testStarted name='Custom std-exceptions can be custom translated']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Custom std-exceptions can be custom translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "custom std exception"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Custom std-exceptions can be custom translated' duration="{duration}"]
##teamcity[testStarted name='Default scale is invisible to comparison']
##teamcity[testFinished name='Default scale is invisible to comparison' duration="{duration}"]
##teamcity[testStarted name='Directly creating an EnumInfo']
##teamcity[testFinished name='Directly creating an EnumInfo' duration="{duration}"]
Test & document SKIP in generator constructor Closes #1593
2023-05-31 15:12:23 +02:00
##teamcity[testStarted name='Empty generators can SKIP in constructor']
##teamcity[testIgnored name='Empty generators can SKIP in constructor' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "This generator is empty"']
##teamcity[testFinished name='Empty generators can SKIP in constructor' duration="{duration}"]
Add IStream::isStdOut
2022-02-21 23:48:15 +01:00
##teamcity[testStarted name='Empty stream name opens cout stream']
##teamcity[testFinished name='Empty stream name opens cout stream' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='EndsWith string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='EndsWith string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), EndsWith( "Substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ends with: "Substring"|n']
##teamcity[testFailed name='EndsWith string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), EndsWith( "this", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ends with: "this" (case insensitive)|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='EndsWith string matcher' duration="{duration}"]
##teamcity[testStarted name='Enums can quickly have stringification enabled using REGISTER_ENUM']
##teamcity[testFinished name='Enums can quickly have stringification enabled using REGISTER_ENUM' duration="{duration}"]
##teamcity[testStarted name='Enums in namespaces can quickly have stringification enabled using REGISTER_ENUM']
##teamcity[testFinished name='Enums in namespaces can quickly have stringification enabled using REGISTER_ENUM' duration="{duration}"]
##teamcity[testStarted name='Epsilon only applies to Approx|'s value']
##teamcity[testFinished name='Epsilon only applies to Approx|'s value' duration="{duration}"]
##teamcity[testStarted name='Equality checks that should fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven == 6 )|nwith expansion:|n 7 == 6|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven == 8 )|nwith expansion:|n 7 == 8|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven == 0 )|nwith expansion:|n 7 == 0|n- failure ignore as test marked as |'ok to fail|'|n']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 9.11f ) )|nwith expansion:|n 9.100000381f|n==|nApprox( 9.10999965667724609 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 9.0f ) )|nwith expansion:|n 9.100000381f == Approx( 9.0 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 1 ) )|nwith expansion:|n 9.100000381f == Approx( 1.0 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 0 ) )|nwith expansion:|n 9.100000381f == Approx( 0.0 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.double_pi == Approx( 3.1415 ) )|nwith expansion:|n 3.14159265350000005|n==|nApprox( 3.14150000000000018 )|n- failure ignore as test marked as |'ok to fail|'|n']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello == "goodbye" )|nwith expansion:|n "hello" == "goodbye"|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello == "hell" )|nwith expansion:|n "hello" == "hell"|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello == "hello1" )|nwith expansion:|n "hello" == "hello1"|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello.size() == 6 )|nwith expansion:|n 5 == 6|n- failure ignore as test marked as |'ok to fail|'|n']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( x == Approx( 1.301 ) )|nwith expansion:|n 1.30000000000000027|n==|nApprox( 1.30099999999999993 )|n- failure ignore as test marked as |'ok to fail|'|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Equality checks that should fail' duration="{duration}"]
##teamcity[testStarted name='Equality checks that should succeed']
##teamcity[testFinished name='Equality checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='Equals']
##teamcity[testFinished name='Equals' duration="{duration}"]
##teamcity[testStarted name='Equals string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Equals string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Equals( "this string contains |'ABC|' as a substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" equals: "this string contains |'ABC|' as a substring"|n']
##teamcity[testFailed name='Equals string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Equals( "something else", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" equals: "something else" (case insensitive)|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Equals string matcher' duration="{duration}"]
##teamcity[testStarted name='Exception as a value (e.g. in REQUIRE_THROWS_MATCHES) can be stringified']
##teamcity[testFinished name='Exception as a value (e.g. in REQUIRE_THROWS_MATCHES) can be stringified' duration="{duration}"]
##teamcity[testStarted name='Exception matchers that fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Exception matchers that fail' message='-------------------------------------------------------------------------------|nNo exception|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nno exception was thrown where one was expected|n CHECK_THROWS_MATCHES( doesNotThrow(), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n doesNotThrow(), SpecialException, ExceptionMatcher{ 1 }|n']
##teamcity[testFailed name='Exception matchers that fail' message='Matchers.tests.cpp:<line number>|nno exception was thrown where one was expected|n REQUIRE_THROWS_MATCHES( doesNotThrow(), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n doesNotThrow(), SpecialException, ExceptionMatcher{ 1 }|n']
##teamcity[testFailed name='Exception matchers that fail' message='-------------------------------------------------------------------------------|nType mismatch|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nunexpected exception with message:|n "Unknown exception"|n CHECK_THROWS_MATCHES( throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 }|n']
##teamcity[testFailed name='Exception matchers that fail' message='Matchers.tests.cpp:<line number>|nunexpected exception with message:|n "Unknown exception"|n REQUIRE_THROWS_MATCHES( throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 }|n']
##teamcity[testFailed name='Exception matchers that fail' message='-------------------------------------------------------------------------------|nContents are wrong|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THROWS_MATCHES( throwsSpecialException( 3 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n SpecialException::what special exception has value of 1|n']
##teamcity[testFailed name='Exception matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n REQUIRE_THROWS_MATCHES( throwsSpecialException( 4 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n SpecialException::what special exception has value of 1|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Exception matchers that fail' duration="{duration}"]
##teamcity[testStarted name='Exception matchers that succeed']
##teamcity[testFinished name='Exception matchers that succeed' duration="{duration}"]
Add MessageMatches matcher for exception (#2570)
2022-12-12 00:40:47 +01:00
##teamcity[testStarted name='Exception message can be matched']
##teamcity[testFinished name='Exception message can be matched' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Exception messages can be tested for']
##teamcity[testFinished name='Exception messages can be tested for' duration="{duration}"]
##teamcity[testStarted name='Exceptions matchers']
##teamcity[testFinished name='Exceptions matchers' duration="{duration}"]
##teamcity[testStarted name='Expected exceptions that don|'t throw or unexpected exceptions fail the test']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Expected exceptions that don|'t throw or unexpected exceptions fail the test' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK_THROWS_AS( thisThrows(), std::string )|nwith expansion:|n thisThrows(), std::string|n']
##teamcity[testFailed name='Expected exceptions that don|'t throw or unexpected exceptions fail the test' message='Exception.tests.cpp:<line number>|nno exception was thrown where one was expected|n CHECK_THROWS_AS( thisDoesntThrow(), std::domain_error )|nwith expansion:|n thisDoesntThrow(), std::domain_error|n']
##teamcity[testFailed name='Expected exceptions that don|'t throw or unexpected exceptions fail the test' message='Exception.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK_NOTHROW( thisThrows() )|nwith expansion:|n thisThrows()|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Expected exceptions that don|'t throw or unexpected exceptions fail the test' duration="{duration}"]
##teamcity[testStarted name='FAIL aborts the test']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='FAIL aborts the test' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "This is a failure"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='FAIL aborts the test' duration="{duration}"]
##teamcity[testStarted name='FAIL does not require an argument']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='FAIL does not require an argument' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='FAIL does not require an argument' duration="{duration}"]
##teamcity[testStarted name='FAIL_CHECK does not abort the test']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='FAIL_CHECK does not abort the test' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "This is a failure"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='FAIL_CHECK does not abort the test' duration="{duration}"]
##teamcity[testStarted name='Factorials are computed']
##teamcity[testFinished name='Factorials are computed' duration="{duration}"]
Add test for empty result of filter generator
2023-02-01 18:27:41 +01:00
##teamcity[testStarted name='Filter generator throws exception for empty generator']
##teamcity[testFinished name='Filter generator throws exception for empty generator' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Floating point matchers: double']
##teamcity[testFinished name='Floating point matchers: double' duration="{duration}"]
##teamcity[testStarted name='Floating point matchers: float']
##teamcity[testFinished name='Floating point matchers: float' duration="{duration}"]
Add test showing literals and complex generators in one GENERATE
2023-12-10 16:38:48 +01:00
##teamcity[testStarted name='GENERATE can combine literals and generators']
##teamcity[testFinished name='GENERATE can combine literals and generators' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Generators -- adapters']
##teamcity[testFinished name='Generators -- adapters' duration="{duration}"]
##teamcity[testStarted name='Generators -- simple']
##teamcity[testFinished name='Generators -- simple' duration="{duration}"]
##teamcity[testStarted name='Generators internals']
##teamcity[testFinished name='Generators internals' duration="{duration}"]
##teamcity[testStarted name='Greater-than inequalities with different epsilons']
##teamcity[testFinished name='Greater-than inequalities with different epsilons' duration="{duration}"]
Refactor tests
2022-03-31 22:30:38 +02:00
##teamcity[testStarted name='Hashers with different seed produce different hash with same test case']
##teamcity[testFinished name='Hashers with different seed produce different hash with same test case' duration="{duration}"]
##teamcity[testStarted name='Hashers with same seed produce same hash']
##teamcity[testFinished name='Hashers with same seed produce same hash' duration="{duration}"]
##teamcity[testStarted name='Hashing different test cases produces different result']
##teamcity[testFinished name='Hashing different test cases produces different result' duration="{duration}"]
##teamcity[testStarted name='Hashing test case produces same hash across multiple calls']
##teamcity[testFinished name='Hashing test case produces same hash across multiple calls' duration="{duration}"]
Add test for multiple streaming parts in UNSCOPED_INFO
2023-09-17 10:44:31 +02:00
##teamcity[testStarted name='INFO and UNSCOPED_INFO can stream multiple arguments']
##teamcity[testFailed name='INFO and UNSCOPED_INFO can stream multiple arguments' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with messages:|n "This info has multiple parts."|n "This unscoped info has multiple parts."|n "Show infos!"']
##teamcity[testFinished name='INFO and UNSCOPED_INFO can stream multiple arguments' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='INFO and WARN do not abort tests']
##teamcity[testFinished name='INFO and WARN do not abort tests' duration="{duration}"]
##teamcity[testStarted name='INFO gets logged on failure']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='INFO gets logged on failure' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "this message should be logged"|n "so should this"|n REQUIRE( a == 1 )|nwith expansion:|n 2 == 1|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='INFO gets logged on failure' duration="{duration}"]
##teamcity[testStarted name='INFO gets logged on failure, even if captured before successful assertions']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='INFO gets logged on failure, even if captured before successful assertions' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "this message may be logged later"|n "this message should be logged"|n CHECK( a == 1 )|nwith expansion:|n 2 == 1|n']
##teamcity[testFailed name='INFO gets logged on failure, even if captured before successful assertions' message='Message.tests.cpp:<line number>|nexpression failed with messages:|n "this message may be logged later"|n "this message should be logged"|n "and this, but later"|n CHECK( a == 0 )|nwith expansion:|n 2 == 0|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='INFO gets logged on failure, even if captured before successful assertions' duration="{duration}"]
##teamcity[testStarted name='INFO is reset for each loop']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='INFO is reset for each loop' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "current counter 10"|n "i := 10"|n REQUIRE( i < 10 )|nwith expansion:|n 10 < 10|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='INFO is reset for each loop' duration="{duration}"]
Assert Info reset need to also reset result disposition to normal to handle uncaught exception correctly (#2723) * AssertionEnd does not reset the assertion info yet. That is done after populateReaction. And reset assertion info would also reset the result disposition to normal, so that any uncaught exception would be reported as failure * Approving test output changes due to added unit tests * Unit tests to throw std::runtime_error instead of std::exception * Add a unit test to test incomplete assertion handler --------- Co-authored-by: Ross <ross.tang@gfo-x.com>
2023-08-07 22:07:31 +02:00
##teamcity[testStarted name='Incomplete AssertionHandler']
##teamcity[testIgnored name='Incomplete AssertionHandler' message='AssertionHandler.tests.cpp:<line number>|n...............................................................................|n|nAssertionHandler.tests.cpp:<line number>|nunexpected exception with message:|n "Exception translation was disabled by CATCH_CONFIG_FAST_COMPILE"|n REQUIRE( Dummy )|nwith expansion:|n Dummy|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Incomplete AssertionHandler' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Inequality checks that should fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven != 7 )|nwith expansion:|n 7 != 7|n- failure ignore as test marked as |'ok to fail|'|n']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one != Approx( 9.1f ) )|nwith expansion:|n 9.100000381f|n!=|nApprox( 9.10000038146972656 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.double_pi != Approx( 3.1415926535 ) )|nwith expansion:|n 3.14159265350000005|n!=|nApprox( 3.14159265350000005 )|n- failure ignore as test marked as |'ok to fail|'|n']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello != "hello" )|nwith expansion:|n "hello" != "hello"|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello.size() != 5 )|nwith expansion:|n 5 != 5|n- failure ignore as test marked as |'ok to fail|'|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Inequality checks that should fail' duration="{duration}"]
##teamcity[testStarted name='Inequality checks that should succeed']
##teamcity[testFinished name='Inequality checks that should succeed' duration="{duration}"]
Add JSON reporter (#2706) Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-11-14 22:52:15 +01:00
##teamcity[testStarted name='JsonWriter']
##teamcity[testFinished name='JsonWriter' duration="{duration}"]
Fix typos
2024-08-26 01:07:28 +02:00
##teamcity[testStarted name='JsonWriter escapes characters in strings properly']
##teamcity[testFinished name='JsonWriter escapes characters in strings properly' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Lambdas in assertions']
##teamcity[testFinished name='Lambdas in assertions' duration="{duration}"]
##teamcity[testStarted name='Less-than inequalities with different epsilons']
##teamcity[testFinished name='Less-than inequalities with different epsilons' duration="{duration}"]
##teamcity[testStarted name='ManuallyRegistered']
##teamcity[testFinished name='ManuallyRegistered' duration="{duration}"]
##teamcity[testStarted name='Matchers can be (AllOf) composed with the && operator']
##teamcity[testFinished name='Matchers can be (AllOf) composed with the && operator' duration="{duration}"]
##teamcity[testStarted name='Matchers can be (AnyOf) composed with the |||| operator']
##teamcity[testFinished name='Matchers can be (AnyOf) composed with the |||| operator' duration="{duration}"]
##teamcity[testStarted name='Matchers can be composed with both && and ||||']
##teamcity[testFinished name='Matchers can be composed with both && and ||||' duration="{duration}"]
##teamcity[testStarted name='Matchers can be composed with both && and |||| - failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Matchers can be composed with both && and |||| - failing' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), ( ContainsSubstring( "string" ) |||| ContainsSubstring( "different" ) ) && ContainsSubstring( "random" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ( ( contains: "string" or contains: "different" ) and contains: "random" )|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Matchers can be composed with both && and |||| - failing' duration="{duration}"]
##teamcity[testStarted name='Matchers can be negated (Not) with the ! operator']
##teamcity[testFinished name='Matchers can be negated (Not) with the ! operator' duration="{duration}"]
##teamcity[testStarted name='Matchers can be negated (Not) with the ! operator - failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Matchers can be negated (Not) with the ! operator - failing' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), !ContainsSubstring( "substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" not contains: "substring"|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Matchers can be negated (Not) with the ! operator - failing' duration="{duration}"]
##teamcity[testStarted name='Mayfail test case with nested sections']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Mayfail test case with nested sections' message='-------------------------------------------------------------------------------|nA|n1|n-------------------------------------------------------------------------------|nCondition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Mayfail test case with nested sections' message='-------------------------------------------------------------------------------|nA|n2|n-------------------------------------------------------------------------------|nCondition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Mayfail test case with nested sections' message='-------------------------------------------------------------------------------|nB|n1|n-------------------------------------------------------------------------------|nCondition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Mayfail test case with nested sections' message='-------------------------------------------------------------------------------|nB|n2|n-------------------------------------------------------------------------------|nCondition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Mayfail test case with nested sections' duration="{duration}"]
##teamcity[testStarted name='Mismatching exception messages failing the test']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Mismatching exception messages failing the test' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nexpression failed|n REQUIRE_THROWS_WITH( thisThrows(), "should fail" )|nwith expansion:|n "expected exception" equals: "should fail"|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Mismatching exception messages failing the test' duration="{duration}"]
Add more unit tests * Call order of listeners/reporters in multireporter * Adding listeners/reporters properly updates reporter preferences
2021-12-19 15:16:56 +01:00
##teamcity[testStarted name='Multireporter calls reporters and listeners in correct order']
##teamcity[testFinished name='Multireporter calls reporters and listeners in correct order' duration="{duration}"]
##teamcity[testStarted name='Multireporter updates ReporterPreferences properly']
##teamcity[testFinished name='Multireporter updates ReporterPreferences properly' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Nested generators and captured variables']
##teamcity[testFinished name='Nested generators and captured variables' duration="{duration}"]
##teamcity[testStarted name='Nice descriptive name']
##teamcity[testFinished name='Nice descriptive name' duration="{duration}"]
##teamcity[testStarted name='Non-std exceptions can be translated']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Non-std exceptions can be translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "custom exception"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Non-std exceptions can be translated' duration="{duration}"]
##teamcity[testStarted name='Objects that evaluated in boolean contexts can be checked']
##teamcity[testFinished name='Objects that evaluated in boolean contexts can be checked' duration="{duration}"]
##teamcity[testStarted name='Optionally static assertions']
##teamcity[testFinished name='Optionally static assertions' duration="{duration}"]
##teamcity[testStarted name='Ordering comparison checks that should fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven > 7 )|nwith expansion:|n 7 > 7|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < 7 )|nwith expansion:|n 7 < 7|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven > 8 )|nwith expansion:|n 7 > 8|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < 6 )|nwith expansion:|n 7 < 6|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < 0 )|nwith expansion:|n 7 < 0|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < -1 )|nwith expansion:|n 7 < -1|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven >= 8 )|nwith expansion:|n 7 >= 8|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven <= 6 )|nwith expansion:|n 7 <= 6|n']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one < 9 )|nwith expansion:|n 9.100000381f < 9|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one > 10 )|nwith expansion:|n 9.100000381f > 10|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one > 9.2 )|nwith expansion:|n 9.100000381f > 9.19999999999999929|n']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello > "hello" )|nwith expansion:|n "hello" > "hello"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello < "hello" )|nwith expansion:|n "hello" < "hello"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello > "hellp" )|nwith expansion:|n "hello" > "hellp"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello > "z" )|nwith expansion:|n "hello" > "z"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello < "hellm" )|nwith expansion:|n "hello" < "hellm"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello < "a" )|nwith expansion:|n "hello" < "a"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello >= "z" )|nwith expansion:|n "hello" >= "z"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello <= "a" )|nwith expansion:|n "hello" <= "a"|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Ordering comparison checks that should fail' duration="{duration}"]
##teamcity[testStarted name='Ordering comparison checks that should succeed']
##teamcity[testFinished name='Ordering comparison checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='Our PCG implementation provides expected results for known seeds']
##teamcity[testFinished name='Our PCG implementation provides expected results for known seeds' duration="{duration}"]
##teamcity[testStarted name='Output from all sections is reported']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Output from all sections is reported' message='-------------------------------------------------------------------------------|none|n-------------------------------------------------------------------------------|nMessage.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "Message from section one"']
##teamcity[testFailed name='Output from all sections is reported' message='-------------------------------------------------------------------------------|ntwo|n-------------------------------------------------------------------------------|nMessage.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "Message from section two"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Output from all sections is reported' duration="{duration}"]
##teamcity[testStarted name='Overloaded comma or address-of operators are not used']
##teamcity[testFinished name='Overloaded comma or address-of operators are not used' duration="{duration}"]
Add parseUInt utility function There is an increasing number of places where Catch2 wants to parse strings into numbers, but being stuck in C++14 world, we do not have good stdlib facilities to do this (`strtoul` and `stoul` are both bad).
2022-10-20 15:06:26 +02:00
##teamcity[testStarted name='Parse uints']
##teamcity[testFinished name='Parse uints' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Parsed tags are matched case insensitive']
##teamcity[testFinished name='Parsed tags are matched case insensitive' duration="{duration}"]
##teamcity[testStarted name='Parsing sharding-related cli flags']
##teamcity[testFinished name='Parsing sharding-related cli flags' duration="{duration}"]
##teamcity[testStarted name='Parsing tags with non-alphabetical characters is pass-through']
##teamcity[testFinished name='Parsing tags with non-alphabetical characters is pass-through' duration="{duration}"]
##teamcity[testStarted name='Parsing warnings']
##teamcity[testFinished name='Parsing warnings' duration="{duration}"]
##teamcity[testStarted name='Pointers can be compared to null']
##teamcity[testFinished name='Pointers can be compared to null' duration="{duration}"]
##teamcity[testStarted name='Precision of floating point stringification can be set']
##teamcity[testFinished name='Precision of floating point stringification can be set' duration="{duration}"]
##teamcity[testStarted name='Predicate matcher can accept const char*']
##teamcity[testFinished name='Predicate matcher can accept const char*' duration="{duration}"]
##teamcity[testStarted name='Process can be configured on command line']
##teamcity[testFinished name='Process can be configured on command line' duration="{duration}"]
##teamcity[testStarted name='Product with differing arities - std::tuple<int, double, float>']
##teamcity[testFinished name='Product with differing arities - std::tuple<int, double, float>' duration="{duration}"]
##teamcity[testStarted name='Product with differing arities - std::tuple<int, double>']
##teamcity[testFinished name='Product with differing arities - std::tuple<int, double>' duration="{duration}"]
##teamcity[testStarted name='Product with differing arities - std::tuple<int>']
##teamcity[testFinished name='Product with differing arities - std::tuple<int>' duration="{duration}"]
##teamcity[testStarted name='Random seed generation accepts known methods']
##teamcity[testFinished name='Random seed generation accepts known methods' duration="{duration}"]
##teamcity[testStarted name='Random seed generation reports unknown methods']
##teamcity[testFinished name='Random seed generation reports unknown methods' duration="{duration}"]
##teamcity[testStarted name='Range type with sentinel']
##teamcity[testFinished name='Range type with sentinel' duration="{duration}"]
##teamcity[testStarted name='Reconstruction should be based on stringification: #914']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Reconstruction should be based on stringification: #914' message='Decomposition.tests.cpp:<line number>|n...............................................................................|n|nDecomposition.tests.cpp:<line number>|nexpression failed|n CHECK( truthy(false) )|nwith expansion:|n Hey, its truthy!|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Reconstruction should be based on stringification: #914' duration="{duration}"]
##teamcity[testStarted name='Regex string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Regex string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Matches( "this STRING contains |'abc|' as a substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" matches "this STRING contains |'abc|' as a substring" case sensitively|n']
##teamcity[testFailed name='Regex string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Matches( "contains |'abc|' as a substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" matches "contains |'abc|' as a substring" case sensitively|n']
##teamcity[testFailed name='Regex string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Matches( "this string contains |'abc|' as a" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" matches "this string contains |'abc|' as a" case sensitively|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Regex string matcher' duration="{duration}"]
Error out reporter registration for reporter names with '::'
2022-01-09 19:01:14 +01:00
##teamcity[testStarted name='Registering reporter with |'::|' in name fails']
##teamcity[testFinished name='Registering reporter with |'::|' in name fails' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Regression test #1']
##teamcity[testFinished name='Regression test #1' duration="{duration}"]
##teamcity[testStarted name='Reporter|'s write listings to provided stream']
##teamcity[testFinished name='Reporter|'s write listings to provided stream' duration="{duration}"]
##teamcity[testStarted name='Reproducer for #2309 - a very long description past 80 chars (default console width) with a late colon : blablabla']
##teamcity[testFinished name='Reproducer for #2309 - a very long description past 80 chars (default console width) with a late colon : blablabla' duration="{duration}"]
##teamcity[testStarted name='SUCCEED counts as a test pass']
##teamcity[testFinished name='SUCCEED counts as a test pass' duration="{duration}"]
##teamcity[testStarted name='SUCCEED does not require an argument']
##teamcity[testFinished name='SUCCEED does not require an argument' duration="{duration}"]
##teamcity[testStarted name='Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or methods']
##teamcity[testFinished name='Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or methods' duration="{duration}"]
##teamcity[testStarted name='Scenario: Do that thing with the thing']
##teamcity[testFinished name='Scenario: Do that thing with the thing' duration="{duration}"]
##teamcity[testStarted name='Scenario: This is a really long scenario name to see how the list command deals with wrapping']
##teamcity[testFinished name='Scenario: This is a really long scenario name to see how the list command deals with wrapping' duration="{duration}"]
##teamcity[testStarted name='Scenario: Vector resizing affects size and capacity']
##teamcity[testFinished name='Scenario: Vector resizing affects size and capacity' duration="{duration}"]
##teamcity[testStarted name='Sends stuff to stdout and stderr']
##teamcity[testStdOut name='Sends stuff to stdout and stderr' out='A string sent directly to stdout|n']
##teamcity[testStdErr name='Sends stuff to stdout and stderr' out='A string sent directly to stderr|nA string sent to stderr via clog|n']
##teamcity[testFinished name='Sends stuff to stdout and stderr' duration="{duration}"]
##teamcity[testStarted name='Some simple comparisons between doubles']
##teamcity[testFinished name='Some simple comparisons between doubles' duration="{duration}"]
##teamcity[testStarted name='Standard output from all sections is reported']
##teamcity[testStdOut name='Standard output from all sections is reported' out='Message from section one|nMessage from section two|n']
##teamcity[testFinished name='Standard output from all sections is reported' duration="{duration}"]
##teamcity[testStarted name='StartsWith string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='StartsWith string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), StartsWith( "This String" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" starts with: "This String"|n']
##teamcity[testFailed name='StartsWith string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), StartsWith( "string", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" starts with: "string" (case insensitive)|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='StartsWith string matcher' duration="{duration}"]
##teamcity[testStarted name='Static arrays are convertible to string']
##teamcity[testFinished name='Static arrays are convertible to string' duration="{duration}"]
##teamcity[testStarted name='String matchers']
##teamcity[testFinished name='String matchers' duration="{duration}"]
##teamcity[testStarted name='StringRef']
##teamcity[testFinished name='StringRef' duration="{duration}"]
##teamcity[testStarted name='StringRef at compilation time']
##teamcity[testFinished name='StringRef at compilation time' duration="{duration}"]
##teamcity[testStarted name='Stringifying char arrays with statically known sizes - char']
##teamcity[testFinished name='Stringifying char arrays with statically known sizes - char' duration="{duration}"]
##teamcity[testStarted name='Stringifying char arrays with statically known sizes - signed char']
##teamcity[testFinished name='Stringifying char arrays with statically known sizes - signed char' duration="{duration}"]
##teamcity[testStarted name='Stringifying char arrays with statically known sizes - unsigned char']
##teamcity[testFinished name='Stringifying char arrays with statically known sizes - unsigned char' duration="{duration}"]
##teamcity[testStarted name='Stringifying std::chrono::duration helpers']
##teamcity[testFinished name='Stringifying std::chrono::duration helpers' duration="{duration}"]
##teamcity[testStarted name='Stringifying std::chrono::duration with weird ratios']
##teamcity[testFinished name='Stringifying std::chrono::duration with weird ratios' duration="{duration}"]
##teamcity[testStarted name='Stringifying std::chrono::time_point<system_clock>']
##teamcity[testFinished name='Stringifying std::chrono::time_point<system_clock>' duration="{duration}"]
##teamcity[testStarted name='Tabs and newlines show in output']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Tabs and newlines show in output' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n CHECK( s1 == s2 )|nwith expansion:|n "if ($b == 10) {|n $a = 20;|n}"|n==|n"if ($b == 10) {|n $a = 20;|n}|n"|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Tabs and newlines show in output' duration="{duration}"]
##teamcity[testStarted name='Tag alias can be registered against tag patterns']
##teamcity[testFinished name='Tag alias can be registered against tag patterns' duration="{duration}"]
##teamcity[testStarted name='Tags with spaces and non-alphanumerical characters are accepted']
##teamcity[testFinished name='Tags with spaces and non-alphanumerical characters are accepted' duration="{duration}"]
##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 0']
##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 1']
##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 2']
##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 2' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 0']
##teamcity[testFinished name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 1']
##teamcity[testFinished name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 0']
##teamcity[testFinished name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 1']
##teamcity[testFinished name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside std::tuple - MyTypes - 0']
##teamcity[testFinished name='Template test case with test types specified inside std::tuple - MyTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside std::tuple - MyTypes - 1']
##teamcity[testFinished name='Template test case with test types specified inside std::tuple - MyTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside std::tuple - MyTypes - 2']
##teamcity[testFinished name='Template test case with test types specified inside std::tuple - MyTypes - 2' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - float']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - float' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - int']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - int' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - std::string']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - std::string' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - std::tuple<int,float>']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - std::tuple<int,float>' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - (std::tuple<int, float>), 6']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - (std::tuple<int, float>), 6' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - float,4']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - float,4' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - int,5']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - int,5' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - std::string,15']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - std::string,15' duration="{duration}"]
##teamcity[testStarted name='Test case with identical tags keeps just one']
##teamcity[testFinished name='Test case with identical tags keeps just one' duration="{duration}"]
##teamcity[testStarted name='Test case with one argument']
##teamcity[testFinished name='Test case with one argument' duration="{duration}"]
##teamcity[testStarted name='Test enum bit values']
##teamcity[testFinished name='Test enum bit values' duration="{duration}"]
##teamcity[testStarted name='Test with special, characters "in name']
##teamcity[testFinished name='Test with special, characters "in name' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if']
##teamcity[testFinished name='Testing checked-if' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if 2']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Testing checked-if 2' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Testing checked-if 2' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if 3']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Testing checked-if 3' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Testing checked-if 3' duration="{duration}"]
Assert Info reset need to also reset result disposition to normal to handle uncaught exception correctly (#2723) * AssertionEnd does not reset the assertion info yet. That is done after populateReaction. And reset assertion info would also reset the result disposition to normal, so that any uncaught exception would be reported as failure * Approving test output changes due to added unit tests * Unit tests to throw std::runtime_error instead of std::exception * Add a unit test to test incomplete assertion handler --------- Co-authored-by: Ross <ross.tang@gfo-x.com>
2023-08-07 22:07:31 +02:00
##teamcity[testStarted name='Testing checked-if 4']
##teamcity[testIgnored name='Testing checked-if 4' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nunexpected exception with message:|n "Uncaught exception should fail!"|n {Unknown expression after the reported line}|nwith expansion:|n {Unknown expression after the reported line}|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Testing checked-if 4' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if 5']
##teamcity[testIgnored name='Testing checked-if 5' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nunexpected exception with message:|n "Uncaught exception should fail!"|n {Unknown expression after the reported line}|nwith expansion:|n {Unknown expression after the reported line}|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Testing checked-if 5' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='The NO_FAIL macro reports a failure but does not fail the test']
##teamcity[testFinished name='The NO_FAIL macro reports a failure but does not fail the test' duration="{duration}"]
##teamcity[testStarted name='The default listing implementation write to provided stream']
##teamcity[testFinished name='The default listing implementation write to provided stream' duration="{duration}"]
##teamcity[testStarted name='This test |'should|' fail but doesn|'t']
##teamcity[testFinished name='This test |'should|' fail but doesn|'t' duration="{duration}"]
##teamcity[testStarted name='Thrown string literals are translated']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Thrown string literals are translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "For some reason someone is throwing a string literal!"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Thrown string literals are translated' duration="{duration}"]
##teamcity[testStarted name='Tracker']
##teamcity[testFinished name='Tracker' duration="{duration}"]
##teamcity[testStarted name='Trim strings']
##teamcity[testFinished name='Trim strings' duration="{duration}"]
Added ElementsAre and UnorderedElementsAre (#2377) Co-authored-by: Garz4 <fancygarz4@gmail.com> Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-22 00:33:04 +01:00
##teamcity[testStarted name='Type conversions of RangeEquals and similar']
##teamcity[testFinished name='Type conversions of RangeEquals and similar' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Unexpected exceptions can be translated']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testFailed name='Unexpected exceptions can be translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "3.14000000000000012"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Unexpected exceptions can be translated' duration="{duration}"]
##teamcity[testStarted name='Upcasting special member functions']
##teamcity[testFinished name='Upcasting special member functions' duration="{duration}"]
##teamcity[testStarted name='Usage of AllMatch range matcher']
##teamcity[testFinished name='Usage of AllMatch range matcher' duration="{duration}"]
Add AllTrue, AnyTrue, NoneTrue matchers
2021-11-14 17:05:31 +01:00
##teamcity[testStarted name='Usage of AllTrue range matcher']
##teamcity[testFinished name='Usage of AllTrue range matcher' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Usage of AnyMatch range matcher']
##teamcity[testFinished name='Usage of AnyMatch range matcher' duration="{duration}"]
Add AllTrue, AnyTrue, NoneTrue matchers
2021-11-14 17:05:31 +01:00
##teamcity[testStarted name='Usage of AnyTrue range matcher']
##teamcity[testFinished name='Usage of AnyTrue range matcher' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Usage of NoneMatch range matcher']
##teamcity[testFinished name='Usage of NoneMatch range matcher' duration="{duration}"]
Add AllTrue, AnyTrue, NoneTrue matchers
2021-11-14 17:05:31 +01:00
##teamcity[testStarted name='Usage of NoneTrue range matcher']
##teamcity[testFinished name='Usage of NoneTrue range matcher' duration="{duration}"]
Added ElementsAre and UnorderedElementsAre (#2377) Co-authored-by: Garz4 <fancygarz4@gmail.com> Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-22 00:33:04 +01:00
##teamcity[testStarted name='Usage of RangeEquals range matcher']
##teamcity[testFinished name='Usage of RangeEquals range matcher' duration="{duration}"]
##teamcity[testStarted name='Usage of UnorderedRangeEquals range matcher']
##teamcity[testFinished name='Usage of UnorderedRangeEquals range matcher' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='Usage of the SizeIs range matcher']
##teamcity[testFinished name='Usage of the SizeIs range matcher' duration="{duration}"]
##teamcity[testStarted name='Use a custom approx']
##teamcity[testFinished name='Use a custom approx' duration="{duration}"]
##teamcity[testStarted name='Variadic macros']
##teamcity[testFinished name='Variadic macros' duration="{duration}"]
##teamcity[testStarted name='Vector Approx matcher']
##teamcity[testFinished name='Vector Approx matcher' duration="{duration}"]
##teamcity[testStarted name='Vector Approx matcher -- failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Vector Approx matcher -- failing' message='-------------------------------------------------------------------------------|nEmpty and non empty vectors are not approx equal|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, Approx( t1 ) )|nwith expansion:|n { } is approx: { 1.0, 2.0 }|n']
##teamcity[testFailed name='Vector Approx matcher -- failing' message='-------------------------------------------------------------------------------|nJust different vectors|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v1, Approx( v2 ) )|nwith expansion:|n { 2.0, 4.0, 6.0 } is approx: { 1.0, 3.0, 5.0 }|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Vector Approx matcher -- failing' duration="{duration}"]
##teamcity[testStarted name='Vector matchers']
##teamcity[testFinished name='Vector matchers' duration="{duration}"]
##teamcity[testStarted name='Vector matchers that fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Vector matchers that fail' message='-------------------------------------------------------------------------------|nContains (element)|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, VectorContains( -1 ) )|nwith expansion:|n { 1, 2, 3 } Contains: -1|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, VectorContains( 1 ) )|nwith expansion:|n { } Contains: 1|n']
##teamcity[testFailed name='Vector matchers that fail' message='-------------------------------------------------------------------------------|nContains (vector)|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, Contains( v ) )|nwith expansion:|n { } Contains: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, Contains( v2 ) )|nwith expansion:|n { 1, 2, 3 } Contains: { 1, 2, 4 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='-------------------------------------------------------------------------------|nEquals|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, Equals( v2 ) )|nwith expansion:|n { 1, 2, 3 } Equals: { 1, 2 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v2, Equals( v ) )|nwith expansion:|n { 1, 2 } Equals: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, Equals( v ) )|nwith expansion:|n { } Equals: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, Equals( empty ) )|nwith expansion:|n { 1, 2, 3 } Equals: { }|n']
##teamcity[testFailed name='Vector matchers that fail' message='-------------------------------------------------------------------------------|nUnorderedEquals|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, UnorderedEquals( empty ) )|nwith expansion:|n { 1, 2, 3 } UnorderedEquals: { }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, UnorderedEquals( v ) )|nwith expansion:|n { } UnorderedEquals: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( permuted, UnorderedEquals( v ) )|nwith expansion:|n { 1, 3 } UnorderedEquals: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( permuted, UnorderedEquals( v ) )|nwith expansion:|n { 3, 1 } UnorderedEquals: { 1, 2, 3 }|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='Vector matchers that fail' duration="{duration}"]
##teamcity[testStarted name='When checked exceptions are thrown they can be expected or unexpected']
##teamcity[testFinished name='When checked exceptions are thrown they can be expected or unexpected' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown directly they are always failures']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown directly they are always failures' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "unexpected exception"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown directly they are always failures' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown during a CHECK the test should continue']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown during a CHECK the test should continue' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK( thisThrows() == 0 )|nwith expansion:|n thisThrows() == 0|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown during a CHECK the test should continue' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown during a REQUIRE the test should abort fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown during a REQUIRE the test should abort fail' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n REQUIRE( thisThrows() == 0 )|nwith expansion:|n thisThrows() == 0|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown during a REQUIRE the test should abort fail' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown from functions they are always failures']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown from functions they are always failures' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK( thisThrows() == 0 )|nwith expansion:|n thisThrows() == 0|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown from functions they are always failures' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown from sections they are always failures']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown from sections they are always failures' message='-------------------------------------------------------------------------------|nsection name|n-------------------------------------------------------------------------------|nException.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "unexpected exception"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown from sections they are always failures' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown, but caught, they do not affect the test']
##teamcity[testFinished name='When unchecked exceptions are thrown, but caught, they do not affect the test' duration="{duration}"]
##teamcity[testStarted name='X/level/0/a']
##teamcity[testFinished name='X/level/0/a' duration="{duration}"]
##teamcity[testStarted name='X/level/0/b']
##teamcity[testFinished name='X/level/0/b' duration="{duration}"]
##teamcity[testStarted name='X/level/1/a']
##teamcity[testFinished name='X/level/1/a' duration="{duration}"]
##teamcity[testStarted name='X/level/1/b']
##teamcity[testFinished name='X/level/1/b' duration="{duration}"]
##teamcity[testStarted name='XmlEncode']
##teamcity[testFinished name='XmlEncode' duration="{duration}"]
##teamcity[testStarted name='XmlWriter writes boolean attributes as true/false']
##teamcity[testFinished name='XmlWriter writes boolean attributes as true/false' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='a succeeding test can still be skipped']
##teamcity[testIgnored name='a succeeding test can still be skipped' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='a succeeding test can still be skipped' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='analyse no analysis']
##teamcity[testFinished name='analyse no analysis' duration="{duration}"]
##teamcity[testStarted name='array<int, N> -> toString']
##teamcity[testFinished name='array<int, N> -> toString' duration="{duration}"]
##teamcity[testStarted name='benchmark function call']
##teamcity[testFinished name='benchmark function call' duration="{duration}"]
##teamcity[testStarted name='boolean member']
##teamcity[testFinished name='boolean member' duration="{duration}"]
##teamcity[testStarted name='checkedElse']
##teamcity[testFinished name='checkedElse' duration="{duration}"]
##teamcity[testStarted name='checkedElse, failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='checkedElse, failing' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n REQUIRE( testCheckedElse( false ) )|nwith expansion:|n false|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='checkedElse, failing' duration="{duration}"]
##teamcity[testStarted name='checkedIf']
##teamcity[testFinished name='checkedIf' duration="{duration}"]
##teamcity[testStarted name='checkedIf, failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='checkedIf, failing' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n REQUIRE( testCheckedIf( false ) )|nwith expansion:|n false|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='checkedIf, failing' duration="{duration}"]
##teamcity[testStarted name='classify_outliers']
##teamcity[testFinished name='classify_outliers' duration="{duration}"]
##teamcity[testStarted name='comparisons between const int variables']
##teamcity[testFinished name='comparisons between const int variables' duration="{duration}"]
##teamcity[testStarted name='comparisons between int variables']
##teamcity[testFinished name='comparisons between int variables' duration="{duration}"]
##teamcity[testStarted name='convertToBits']
##teamcity[testFinished name='convertToBits' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='dynamic skipping works with generators']
##teamcity[testIgnored name='dynamic skipping works with generators' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 41"']
##teamcity[testIgnored name='dynamic skipping works with generators' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 43"']
##teamcity[testFinished name='dynamic skipping works with generators' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='empty tags are not allowed']
##teamcity[testFinished name='empty tags are not allowed' duration="{duration}"]
##teamcity[testStarted name='erfc_inv']
##teamcity[testFinished name='erfc_inv' duration="{duration}"]
##teamcity[testStarted name='estimate_clock_resolution']
##teamcity[testFinished name='estimate_clock_resolution' duration="{duration}"]
##teamcity[testStarted name='even more nested SECTION tests']
##teamcity[testFinished name='even more nested SECTION tests' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='failed assertions before SKIP cause test case to fail']
##teamcity[testIgnored name='failed assertions before SKIP cause test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexpression failed|n CHECK( 3 == 4 )|nwith expansion:|n 3 == 4|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='failed assertions before SKIP cause test case to fail' message='Skip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='failed assertions before SKIP cause test case to fail' duration="{duration}"]
##teamcity[testStarted name='failing for some generator values causes entire test case to fail']
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='failing for some generator values causes entire test case to fail' duration="{duration}"]
##teamcity[testStarted name='failing in some unskipped sections causes entire test case to fail']
##teamcity[testIgnored name='failing in some unskipped sections causes entire test case to fail' message='-------------------------------------------------------------------------------|nskipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testIgnored name='failing in some unskipped sections causes entire test case to fail' message='-------------------------------------------------------------------------------|nnot skipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='failing in some unskipped sections causes entire test case to fail' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='first tag']
##teamcity[testFinished name='first tag' duration="{duration}"]
##teamcity[testStarted name='has printf']
##teamcity[testFinished name='has printf' duration="{duration}"]
##teamcity[testStarted name='is_unary_function']
##teamcity[testFinished name='is_unary_function' duration="{duration}"]
##teamcity[testStarted name='just failure']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='just failure' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "Previous info should not be seen"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='just failure' duration="{duration}"]
##teamcity[testStarted name='just failure after unscoped info']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='just failure after unscoped info' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "previous unscoped info SHOULD not be seen"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='just failure after unscoped info' duration="{duration}"]
##teamcity[testStarted name='just info']
##teamcity[testFinished name='just info' duration="{duration}"]
##teamcity[testStarted name='just unscoped info']
##teamcity[testFinished name='just unscoped info' duration="{duration}"]
##teamcity[testStarted name='long long']
##teamcity[testFinished name='long long' duration="{duration}"]
##teamcity[testStarted name='looped SECTION tests']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='looped SECTION tests' message='-------------------------------------------------------------------------------|nb is currently: 0|n-------------------------------------------------------------------------------|nMisc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n CHECK( b > a )|nwith expansion:|n 0 > 1|n']
##teamcity[testFailed name='looped SECTION tests' message='-------------------------------------------------------------------------------|nb is currently: 1|n-------------------------------------------------------------------------------|nMisc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n CHECK( b > a )|nwith expansion:|n 1 > 1|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='looped SECTION tests' duration="{duration}"]
##teamcity[testStarted name='looped tests']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[0|] (1) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[1|] (1) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[3|] (3) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[4|] (5) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[6|] (13) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[7|] (21) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='looped tests' duration="{duration}"]
Add test for %debug stream name
2022-04-23 23:34:27 +02:00
##teamcity[testStarted name='makeStream recognizes %debug stream name']
##teamcity[testFinished name='makeStream recognizes %debug stream name' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='make_unique reimplementation']
##teamcity[testFinished name='make_unique reimplementation' duration="{duration}"]
##teamcity[testStarted name='mean']
##teamcity[testFinished name='mean' duration="{duration}"]
##teamcity[testStarted name='measure']
##teamcity[testFinished name='measure' duration="{duration}"]
##teamcity[testStarted name='mix info, unscoped info and warning']
##teamcity[testFinished name='mix info, unscoped info and warning' duration="{duration}"]
##teamcity[testStarted name='more nested SECTION tests']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='more nested SECTION tests' message='-------------------------------------------------------------------------------|ndoesn|'t equal|nequal|n-------------------------------------------------------------------------------|nMisc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n REQUIRE( a == b )|nwith expansion:|n 1 == 2|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='more nested SECTION tests' duration="{duration}"]
##teamcity[testStarted name='nested SECTION tests']
##teamcity[testFinished name='nested SECTION tests' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='nested sections can be skipped dynamically at runtime']
##teamcity[testIgnored name='nested sections can be skipped dynamically at runtime' message='-------------------------------------------------------------------------------|nB|nB2|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testStdOut name='nested sections can be skipped dynamically at runtime' out='a!|nb1!|n!|n']
##teamcity[testFinished name='nested sections can be skipped dynamically at runtime' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='non streamable - with conv. op']
##teamcity[testFinished name='non streamable - with conv. op' duration="{duration}"]
##teamcity[testStarted name='non-copyable objects']
##teamcity[testFinished name='non-copyable objects' duration="{duration}"]
##teamcity[testStarted name='normal_quantile']
##teamcity[testFinished name='normal_quantile' duration="{duration}"]
##teamcity[testStarted name='not allowed']
##teamcity[testFinished name='not allowed' duration="{duration}"]
##teamcity[testStarted name='not prints unscoped info from previous failures']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='not prints unscoped info from previous failures' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with message:|n "this SHOULD be seen"|n REQUIRE( false )|nwith expansion:|n false|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='not prints unscoped info from previous failures' duration="{duration}"]
##teamcity[testStarted name='null strings']
##teamcity[testFinished name='null strings' duration="{duration}"]
##teamcity[testStarted name='null_ptr']
##teamcity[testFinished name='null_ptr' duration="{duration}"]
##teamcity[testStarted name='pair<pair<int,const char *,pair<std::string,int> > -> toString']
##teamcity[testFinished name='pair<pair<int,const char *,pair<std::string,int> > -> toString' duration="{duration}"]
##teamcity[testStarted name='parseEnums']
##teamcity[testFinished name='parseEnums' duration="{duration}"]
##teamcity[testStarted name='pointer to class']
##teamcity[testFinished name='pointer to class' duration="{duration}"]
##teamcity[testStarted name='print unscoped info if passing unscoped info is printed']
##teamcity[testFinished name='print unscoped info if passing unscoped info is printed' duration="{duration}"]
##teamcity[testStarted name='prints unscoped info on failure']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='prints unscoped info on failure' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "this SHOULD be seen"|n "this SHOULD also be seen"|n REQUIRE( false )|nwith expansion:|n false|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='prints unscoped info on failure' duration="{duration}"]
##teamcity[testStarted name='prints unscoped info only for the first assertion']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='prints unscoped info only for the first assertion' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with message:|n "this SHOULD be seen only ONCE"|n CHECK( false )|nwith expansion:|n false|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='prints unscoped info only for the first assertion' duration="{duration}"]
##teamcity[testStarted name='random SECTION tests']
##teamcity[testFinished name='random SECTION tests' duration="{duration}"]
##teamcity[testStarted name='replaceInPlace']
##teamcity[testFinished name='replaceInPlace' duration="{duration}"]
Add %stderr and %stdout as recognized magic stream names
2022-03-07 10:42:26 +01:00
##teamcity[testStarted name='request an unknown %-starting stream fails']
##teamcity[testFinished name='request an unknown %-starting stream fails' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='resolution']
##teamcity[testFinished name='resolution' duration="{duration}"]
##teamcity[testStarted name='run_for_at_least, chronometer']
##teamcity[testFinished name='run_for_at_least, chronometer' duration="{duration}"]
##teamcity[testStarted name='run_for_at_least, int']
##teamcity[testFinished name='run_for_at_least, int' duration="{duration}"]
##teamcity[testStarted name='second tag']
##teamcity[testFinished name='second tag' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='sections can be skipped dynamically at runtime']
##teamcity[testIgnored name='sections can be skipped dynamically at runtime' message='-------------------------------------------------------------------------------|nskipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='sections can be skipped dynamically at runtime' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='send a single char to INFO']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='send a single char to INFO' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed with message:|n "3"|n REQUIRE( false )|nwith expansion:|n false|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='send a single char to INFO' duration="{duration}"]
##teamcity[testStarted name='sends information to INFO']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='sends information to INFO' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "hi"|n "i := 7"|n REQUIRE( false )|nwith expansion:|n false|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='sends information to INFO' duration="{duration}"]
##teamcity[testStarted name='shortened hide tags are split apart']
##teamcity[testFinished name='shortened hide tags are split apart' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='skipped tests can optionally provide a reason']
##teamcity[testIgnored name='skipped tests can optionally provide a reason' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 43"']
##teamcity[testFinished name='skipped tests can optionally provide a reason' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='splitString']
##teamcity[testFinished name='splitString' duration="{duration}"]
##teamcity[testStarted name='stacks unscoped info in loops']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='stacks unscoped info in loops' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "Count 1 to 3..."|n "1"|n "2"|n "3"|n CHECK( false )|nwith expansion:|n false|n']
##teamcity[testFailed name='stacks unscoped info in loops' message='Message.tests.cpp:<line number>|nexpression failed with messages:|n "Count 4 to 6..."|n "4"|n "5"|n "6"|n CHECK( false )|nwith expansion:|n false|n']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='stacks unscoped info in loops' duration="{duration}"]
##teamcity[testStarted name='startsWith']
##teamcity[testFinished name='startsWith' duration="{duration}"]
##teamcity[testStarted name='std::map is convertible string']
##teamcity[testFinished name='std::map is convertible string' duration="{duration}"]
##teamcity[testStarted name='std::pair<int,const std::string> -> toString']
##teamcity[testFinished name='std::pair<int,const std::string> -> toString' duration="{duration}"]
##teamcity[testStarted name='std::pair<int,std::string> -> toString']
##teamcity[testFinished name='std::pair<int,std::string> -> toString' duration="{duration}"]
##teamcity[testStarted name='std::set is convertible string']
##teamcity[testFinished name='std::set is convertible string' duration="{duration}"]
##teamcity[testStarted name='std::vector<std::pair<std::string,int> > -> toString']
##teamcity[testFinished name='std::vector<std::pair<std::string,int> > -> toString' duration="{duration}"]
Add %stderr and %stdout as recognized magic stream names
2022-03-07 10:42:26 +01:00
##teamcity[testStarted name='stdout and stderr streams have %-starting name']
##teamcity[testFinished name='stdout and stderr streams have %-starting name' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='stringify ranges']
##teamcity[testFinished name='stringify ranges' duration="{duration}"]
##teamcity[testStarted name='stringify( has_maker )']
##teamcity[testFinished name='stringify( has_maker )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_maker_and_operator )']
##teamcity[testFinished name='stringify( has_maker_and_operator )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_neither )']
##teamcity[testFinished name='stringify( has_neither )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_operator )']
##teamcity[testFinished name='stringify( has_operator )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_template_operator )']
##teamcity[testFinished name='stringify( has_template_operator )' duration="{duration}"]
##teamcity[testStarted name='stringify( vectors<has_maker> )']
##teamcity[testFinished name='stringify( vectors<has_maker> )' duration="{duration}"]
##teamcity[testStarted name='stringify( vectors<has_maker_and_operator> )']
##teamcity[testFinished name='stringify( vectors<has_maker_and_operator> )' duration="{duration}"]
##teamcity[testStarted name='stringify( vectors<has_operator> )']
##teamcity[testFinished name='stringify( vectors<has_operator> )' duration="{duration}"]
##teamcity[testStarted name='strlen3']
##teamcity[testFinished name='strlen3' duration="{duration}"]
##teamcity[testStarted name='tables']
##teamcity[testFinished name='tables' duration="{duration}"]
##teamcity[testStarted name='tags with dots in later positions are not parsed as hidden']
##teamcity[testFinished name='tags with dots in later positions are not parsed as hidden' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='tests can be skipped dynamically at runtime']
##teamcity[testIgnored name='tests can be skipped dynamically at runtime' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='tests can be skipped dynamically at runtime' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='thrown std::strings are translated']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='thrown std::strings are translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "Why would you throw a std::string?"']
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testFinished name='thrown std::strings are translated' duration="{duration}"]
##teamcity[testStarted name='toString on const wchar_t const pointer returns the string contents']
##teamcity[testFinished name='toString on const wchar_t const pointer returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString on const wchar_t pointer returns the string contents']
##teamcity[testFinished name='toString on const wchar_t pointer returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString on wchar_t const pointer returns the string contents']
##teamcity[testFinished name='toString on wchar_t const pointer returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString on wchar_t returns the string contents']
##teamcity[testFinished name='toString on wchar_t returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString(enum class w/operator<<)']
##teamcity[testFinished name='toString(enum class w/operator<<)' duration="{duration}"]
##teamcity[testStarted name='toString(enum class)']
##teamcity[testFinished name='toString(enum class)' duration="{duration}"]
##teamcity[testStarted name='toString(enum w/operator<<)']
##teamcity[testFinished name='toString(enum w/operator<<)' duration="{duration}"]
##teamcity[testStarted name='toString(enum)']
##teamcity[testFinished name='toString(enum)' duration="{duration}"]
##teamcity[testStarted name='tuple<>']
##teamcity[testFinished name='tuple<>' duration="{duration}"]
##teamcity[testStarted name='tuple<float,int>']
##teamcity[testFinished name='tuple<float,int>' duration="{duration}"]
##teamcity[testStarted name='tuple<int>']
##teamcity[testFinished name='tuple<int>' duration="{duration}"]
##teamcity[testStarted name='tuple<string,string>']
##teamcity[testFinished name='tuple<string,string>' duration="{duration}"]
##teamcity[testStarted name='tuple<tuple<int>,tuple<>,float>']
##teamcity[testFinished name='tuple<tuple<int>,tuple<>,float>' duration="{duration}"]
##teamcity[testStarted name='uniform samples']
##teamcity[testFinished name='uniform samples' duration="{duration}"]
Add uniform_integer_distribution
2023-12-08 22:01:36 +01:00
##teamcity[testStarted name='uniform_integer_distribution can return the bounds']
##teamcity[testFinished name='uniform_integer_distribution can return the bounds' duration="{duration}"]
Add support for multiple parallel reporters This requires a bunch of different changes across the reporter subsystem. * We need to handle multiple reporters and their differing preferences in `ListeningReporter`, e.g. what to do when we mix reporters that capture and don't capture stdout. * We need to change how the reporter is given output and how we parse reporter's output destination from CLI. * Approval tests need to handle multireporter option
2021-02-06 20:12:07 +01:00
##teamcity[testStarted name='unique_ptr reimplementation: basic functionality']
##teamcity[testFinished name='unique_ptr reimplementation: basic functionality' duration="{duration}"]
##teamcity[testStarted name='vec<vec<string,alloc>> -> toString']
##teamcity[testFinished name='vec<vec<string,alloc>> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<bool> -> toString']
##teamcity[testFinished name='vector<bool> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<int,allocator> -> toString']
##teamcity[testFinished name='vector<int,allocator> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<int> -> toString']
##teamcity[testFinished name='vector<int> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<string> -> toString']
##teamcity[testFinished name='vector<string> -> toString' duration="{duration}"]
##teamcity[testStarted name='vectors can be sized and resized']
##teamcity[testFinished name='vectors can be sized and resized' duration="{duration}"]
##teamcity[testStarted name='warmup']
##teamcity[testFinished name='warmup' duration="{duration}"]
##teamcity[testStarted name='weighted_average_quantile']
##teamcity[testFinished name='weighted_average_quantile' duration="{duration}"]
##teamcity[testStarted name='xmlentitycheck']
##teamcity[testFinished name='xmlentitycheck' duration="{duration}"]
##teamcity[testSuiteFinished name='<exe-name>']
Reference in New Issue Copy Permalink