3rd-party/catch2
1
0
Fork 0
mirror of https://github.com/catchorg/Catch2.git synced 2024-11-16 18:52:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
constexpr_approx
catch2/tests/SelfTest/Baselines/teamcity.sw.approved.txt

1018 lines
126 KiB
Plaintext
Raw Permalink Normal View History

Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testSuiteStarted name='<exe-name>']
##teamcity[testStarted name='# A test name that starts with a #']
##teamcity[testFinished name='# A test name that starts with a #' duration="{duration}"]
##teamcity[testStarted name='#1027: Bitfields can be captured']
##teamcity[testFinished name='#1027: Bitfields can be captured' duration="{duration}"]
##teamcity[testStarted name='#1147']
##teamcity[testFinished name='#1147' duration="{duration}"]
##teamcity[testStarted name='#1175 - Hidden Test']
##teamcity[testFinished name='#1175 - Hidden Test' duration="{duration}"]
##teamcity[testStarted name='#1238']
##teamcity[testFinished name='#1238' duration="{duration}"]
##teamcity[testStarted name='#1245']
##teamcity[testFinished name='#1245' duration="{duration}"]
##teamcity[testStarted name='#1319: Sections can have description (even if it is not saved']
##teamcity[testFinished name='#1319: Sections can have description (even if it is not saved' duration="{duration}"]
##teamcity[testStarted name='#1403']
##teamcity[testFinished name='#1403' duration="{duration}"]
##teamcity[testStarted name='#1455 - INFO and WARN can start with a linebreak']
##teamcity[testFinished name='#1455 - INFO and WARN can start with a linebreak' duration="{duration}"]
##teamcity[testStarted name='#1514: stderr/stdout is not captured in tests aborted by an exception']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='#1514: stderr/stdout is not captured in tests aborted by an exception' message='Tricky.tests.cpp:<line number>|n...............................................................................|n|nTricky.tests.cpp:<line number>|nexplicit failure with message:|n "1514"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStdOut name='#1514: stderr/stdout is not captured in tests aborted by an exception' out='This would not be caught previously|n']
##teamcity[testStdErr name='#1514: stderr/stdout is not captured in tests aborted by an exception' out='Nor would this|n']
##teamcity[testFinished name='#1514: stderr/stdout is not captured in tests aborted by an exception' duration="{duration}"]
##teamcity[testStarted name='#1548']
##teamcity[testFinished name='#1548' duration="{duration}"]
Fix bug in test spec parser handling of escaping in ORed patterns It did not clear out all of its internal state when switching from one pattern to another, so when it should've escaped `,`, it took its position from its position in the original user-provided string, rather than its position in the current pattern. Fixes #1905
2020-04-12 18:44:42 +02:00
##teamcity[testStarted name='#1905 -- test spec parser properly clears internal state between compound tests']
##teamcity[testFinished name='#1905 -- test spec parser properly clears internal state between compound tests' duration="{duration}"]
Add more tests for test spec parser Originally the tests were from #1912, but as it turned out, the issue was somewhere else. Still, the inputs provided were interesting, so they are now part of our test suite.
2020-04-17 21:16:18 +02:00
##teamcity[testStarted name='#1912 -- test spec parser handles escaping']
##teamcity[testFinished name='#1912 -- test spec parser handles escaping' duration="{duration}"]
Improved generator tracking * Successive executions of the same `GENERATE` macro (e.g. because of a for loop) no longer lead to multiple nested generators. * The same line can now contain multiple `GENERATE` macros without issues. Fixes #1913
2020-06-01 19:04:23 +02:00
##teamcity[testStarted name='#1913 - GENERATE inside a for loop should not keep recreating the generator']
##teamcity[testFinished name='#1913 - GENERATE inside a for loop should not keep recreating the generator' duration="{duration}"]
##teamcity[testStarted name='#1913 - GENERATEs can share a line']
##teamcity[testFinished name='#1913 - GENERATEs can share a line' duration="{duration}"]
Modify generator tracking to allow GENERATEs between SECTIONs This means that code such as ```cpp TEST_CASE() { SECTION("first") { SUCCEED(); } auto _ = GENERATE(1, 2); SECTION("second") { SUCCEED(); } } ``` will run and report 3 assertions, 1 from section "first" and 2 from section "second". This also applies for greater and potentially more confusing nesting, but fundamentally it is up to the user to avoid overly complex and confusing nestings, just as with `SECTION`s. The old behaviour of `GENERATE` as first thing in a `TEST_CASE`, `GENERATE` not followed by a `SECTION`, etc etc should be unchanged. Closes #1938
2020-07-04 17:45:30 +02:00
##teamcity[testStarted name='#1938 - GENERATE after a section']
##teamcity[testFinished name='#1938 - GENERATE after a section' duration="{duration}"]
##teamcity[testStarted name='#1938 - Section followed by flat generate']
##teamcity[testFinished name='#1938 - Section followed by flat generate' duration="{duration}"]
##teamcity[testStarted name='#1938 - flat generate']
##teamcity[testFinished name='#1938 - flat generate' duration="{duration}"]
##teamcity[testStarted name='#1938 - mixed sections and generates']
##teamcity[testFinished name='#1938 - mixed sections and generates' duration="{duration}"]
##teamcity[testStarted name='#1938 - nested generate']
##teamcity[testFinished name='#1938 - nested generate' duration="{duration}"]
Fix copy paste error in 7-arg TEMPLATE_TEST_CASE_SIG implementation Closes #1954
2020-06-13 15:41:25 +02:00
##teamcity[testStarted name='#1954 - 7 arg template test case sig compiles - 1, 1, 1, 1, 1, 0, 0']
##teamcity[testFinished name='#1954 - 7 arg template test case sig compiles - 1, 1, 1, 1, 1, 0, 0' duration="{duration}"]
##teamcity[testStarted name='#1954 - 7 arg template test case sig compiles - 5, 1, 1, 1, 1, 0, 0']
##teamcity[testFinished name='#1954 - 7 arg template test case sig compiles - 5, 1, 1, 1, 1, 0, 0' duration="{duration}"]
##teamcity[testStarted name='#1954 - 7 arg template test case sig compiles - 5, 3, 1, 1, 1, 0, 0']
##teamcity[testFinished name='#1954 - 7 arg template test case sig compiles - 5, 3, 1, 1, 1, 0, 0' duration="{duration}"]
Fix ulp distance calculation for numbers with different signs This is a simplification of the fix proposed in #2152, with the critical function split out so that it can be tested directly, without having to go through the ULP matcher. Closes #2152
2021-07-26 22:01:04 +02:00
##teamcity[testStarted name='#2152 - ULP checks between differently signed values were wrong - double']
##teamcity[testFinished name='#2152 - ULP checks between differently signed values were wrong - double' duration="{duration}"]
##teamcity[testStarted name='#2152 - ULP checks between differently signed values were wrong - float']
##teamcity[testFinished name='#2152 - ULP checks between differently signed values were wrong - float' duration="{duration}"]
Stop exceptions in generator constructors from aborting the binary Fixes #2615
2023-01-17 11:04:49 +01:00
##teamcity[testStarted name='#2615 - Throwing in constructor generator fails test case but does not abort']
##teamcity[testIgnored name='#2615 - Throwing in constructor generator fails test case but does not abort' message='Generators.tests.cpp:<line number>|n...............................................................................|n|nGenerators.tests.cpp:<line number>|nunexpected exception with message:|n "failure to init"- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='#2615 - Throwing in constructor generator fails test case but does not abort' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='#748 - captures with unexpected exceptions']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='#748 - captures with unexpected exceptions' message='-------------------------------------------------------------------------------|noutside assertions|n-------------------------------------------------------------------------------|nException.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with messages:|n "answer := 42"|n "expected exception"- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='#748 - captures with unexpected exceptions' message='-------------------------------------------------------------------------------|ninside REQUIRE_NOTHROW|n-------------------------------------------------------------------------------|nException.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with messages:|n "answer := 42"|n "expected exception"|n REQUIRE_NOTHROW( thisThrows() )|nwith expansion:|n thisThrows()|n- failure ignore as test marked as |'ok to fail|'|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='#748 - captures with unexpected exceptions' duration="{duration}"]
##teamcity[testStarted name='#809']
##teamcity[testFinished name='#809' duration="{duration}"]
##teamcity[testStarted name='#833']
##teamcity[testFinished name='#833' duration="{duration}"]
Cleanups output normalization in ApprovalTests Removed: * NaN normalization * INFINITY normalization * errno normalization * Completely unused duration regex Tests using these macros should be tagged `[approvals]` so they are not run as part of approval tests. Also simplified regex for the test's executable filename, and hidden some tests relying on nullptr normalization.
2022-01-26 23:47:40 +01:00
##teamcity[testStarted name='#835 -- errno should not be touched by Catch2']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='#835 -- errno should not be touched by Catch2' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n CHECK( f() == 0 )|nwith expansion:|n 1 == 0|n- failure ignore as test marked as |'ok to fail|'|n']
Cleanups output normalization in ApprovalTests Removed: * NaN normalization * INFINITY normalization * errno normalization * Completely unused duration regex Tests using these macros should be tagged `[approvals]` so they are not run as part of approval tests. Also simplified regex for the test's executable filename, and hidden some tests relying on nullptr normalization.
2022-01-26 23:47:40 +01:00
##teamcity[testFinished name='#835 -- errno should not be touched by Catch2' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='#872']
##teamcity[testFinished name='#872' duration="{duration}"]
##teamcity[testStarted name='#961 -- Dynamically created sections should all be reported']
##teamcity[testFinished name='#961 -- Dynamically created sections should all be reported' duration="{duration}"]
##teamcity[testStarted name='|'Not|' checks that should fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexpression failed|n CHECK( false != false )|nwith expansion:|n false != false|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( true != true )|nwith expansion:|n true != true|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( !true )|nwith expansion:|n false|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK_FALSE( true )|nwith expansion:|n !true|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( !trueValue )|nwith expansion:|n false|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK_FALSE( trueValue )|nwith expansion:|n !true|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( !(1 == 1) )|nwith expansion:|n false|n']
##teamcity[testFailed name='|'Not|' checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK_FALSE( 1 == 1 )|nwith expansion:|n !(1 == 1)|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='|'Not|' checks that should fail' duration="{duration}"]
##teamcity[testStarted name='|'Not|' checks that should succeed']
##teamcity[testFinished name='|'Not|' checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='(unimplemented) static bools can be evaluated']
##teamcity[testFinished name='(unimplemented) static bools can be evaluated' duration="{duration}"]
##teamcity[testStarted name='3x3x3 ints']
##teamcity[testFinished name='3x3x3 ints' duration="{duration}"]
##teamcity[testStarted name='A METHOD_AS_TEST_CASE based test run that fails']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A METHOD_AS_TEST_CASE based test run that fails' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( s == "world" )|nwith expansion:|n "hello" == "world"|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A METHOD_AS_TEST_CASE based test run that fails' duration="{duration}"]
##teamcity[testStarted name='A METHOD_AS_TEST_CASE based test run that succeeds']
##teamcity[testFinished name='A METHOD_AS_TEST_CASE based test run that succeeds' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>::m_a.size() == 1 )|nwith expansion:|n 0 == 1|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<float>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<int>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<float>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<float>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<int>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<int>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 6 < 2|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 2 < 2|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 6 < 2|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture_2<TestType>{}.m_a.size() < 2 )|nwith expansion:|n 2 < 2|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<float,6>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<float,6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<int,2>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<int,2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<float,6>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<float,6>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<int,2>']
##teamcity[testFinished name='A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<int,2>' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - double']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - double' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture<TestType>::m_a == 2 )|nwith expansion:|n 1.0 == 2|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - double' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - float']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - float' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture<TestType>::m_a == 2 )|nwith expansion:|n 1.0f == 2|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - float' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - int']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - int' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Template_Fixture<TestType>::m_a == 2 )|nwith expansion:|n 1 == 2|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that fails - int' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - double']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - double' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - float']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - float' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - int']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - int' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 1']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 1' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Nttp_Fixture<V>::value == 0 )|nwith expansion:|n 1 == 0|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 1' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 3']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 3' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Nttp_Fixture<V>::value == 0 )|nwith expansion:|n 3 == 0|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 3' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 6']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 6' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( Nttp_Fixture<V>::value == 0 )|nwith expansion:|n 6 == 0|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 6' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 1']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 1' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 3']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 3' duration="{duration}"]
##teamcity[testStarted name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 6']
##teamcity[testFinished name='A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 6' duration="{duration}"]
##teamcity[testStarted name='A TEST_CASE_METHOD based test run that fails']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A TEST_CASE_METHOD based test run that fails' message='Class.tests.cpp:<line number>|n...............................................................................|n|nClass.tests.cpp:<line number>|nexpression failed|n REQUIRE( m_a == 2 )|nwith expansion:|n 1 == 2|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A TEST_CASE_METHOD based test run that fails' duration="{duration}"]
##teamcity[testStarted name='A TEST_CASE_METHOD based test run that succeeds']
##teamcity[testFinished name='A TEST_CASE_METHOD based test run that succeeds' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - Foo<float>']
##teamcity[testFinished name='A Template product test case - Foo<float>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - Foo<int>']
##teamcity[testFinished name='A Template product test case - Foo<int>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - std::vector<float>']
##teamcity[testFinished name='A Template product test case - std::vector<float>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case - std::vector<int>']
##teamcity[testFinished name='A Template product test case - std::vector<int>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - Bar<float, 42>']
##teamcity[testFinished name='A Template product test case with array signature - Bar<float, 42>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - Bar<int, 9>']
##teamcity[testFinished name='A Template product test case with array signature - Bar<int, 9>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - std::array<float, 42>']
##teamcity[testFinished name='A Template product test case with array signature - std::array<float, 42>' duration="{duration}"]
##teamcity[testStarted name='A Template product test case with array signature - std::array<int, 9>']
##teamcity[testFinished name='A Template product test case with array signature - std::array<int, 9>' duration="{duration}"]
##teamcity[testStarted name='A comparison that uses literals instead of the normal constructor']
##teamcity[testFinished name='A comparison that uses literals instead of the normal constructor' duration="{duration}"]
##teamcity[testStarted name='A couple of nested sections followed by a failure']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A couple of nested sections followed by a failure' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexplicit failure with message:|n "to infinity and beyond"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A couple of nested sections followed by a failure' duration="{duration}"]
##teamcity[testStarted name='A failing expression with a non streamable type is still captured']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='A failing expression with a non streamable type is still captured' message='Tricky.tests.cpp:<line number>|n...............................................................................|n|nTricky.tests.cpp:<line number>|nexpression failed|n CHECK( &o1 == &o2 )|nwith expansion:|n 0x<hex digits> == 0x<hex digits>|n']
##teamcity[testFailed name='A failing expression with a non streamable type is still captured' message='Tricky.tests.cpp:<line number>|nexpression failed|n CHECK( o1 == o2 )|nwith expansion:|n {?} == {?}|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='A failing expression with a non streamable type is still captured' duration="{duration}"]
##teamcity[testStarted name='Absolute margin']
##teamcity[testFinished name='Absolute margin' duration="{duration}"]
##teamcity[testStarted name='An empty test with no assertions']
##teamcity[testFinished name='An empty test with no assertions' duration="{duration}"]
##teamcity[testStarted name='An expression with side-effects should only be evaluated once']
##teamcity[testFinished name='An expression with side-effects should only be evaluated once' duration="{duration}"]
##teamcity[testStarted name='An unchecked exception reports the line of the last assertion']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='An unchecked exception reports the line of the last assertion' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "unexpected exception"|n {Unknown expression after the reported line}|nwith expansion:|n {Unknown expression after the reported line}|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='An unchecked exception reports the line of the last assertion' duration="{duration}"]
##teamcity[testStarted name='Anonymous test case 1']
##teamcity[testFinished name='Anonymous test case 1' duration="{duration}"]
##teamcity[testStarted name='Approx setters validate their arguments']
##teamcity[testFinished name='Approx setters validate their arguments' duration="{duration}"]
##teamcity[testStarted name='Approx with exactly-representable margin']
##teamcity[testFinished name='Approx with exactly-representable margin' duration="{duration}"]
##teamcity[testStarted name='Approximate PI']
##teamcity[testFinished name='Approximate PI' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with different epsilons']
##teamcity[testFinished name='Approximate comparisons with different epsilons' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with floats']
##teamcity[testFinished name='Approximate comparisons with floats' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with ints']
##teamcity[testFinished name='Approximate comparisons with ints' duration="{duration}"]
##teamcity[testStarted name='Approximate comparisons with mixed numeric types']
##teamcity[testFinished name='Approximate comparisons with mixed numeric types' duration="{duration}"]
##teamcity[testStarted name='Arbitrary predicate matcher']
##teamcity[testFinished name='Arbitrary predicate matcher' duration="{duration}"]
Support bitand and bitor in REQUIRE/CHECK This means that bit-flag-like types with conversion to bool can be asserted on, like so `REQUIRE(var & Flags::AddNewline)`.
2020-04-21 11:00:08 +02:00
##teamcity[testStarted name='Assertion macros support bit operators and bool conversions']
##teamcity[testFinished name='Assertion macros support bit operators and bool conversions' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Assertions then sections']
##teamcity[testFinished name='Assertions then sections' duration="{duration}"]
Add generic Contains matcher It matches a range iff the range contains a specific element, or an element in the range matches the provided matcher.
2020-03-02 14:25:43 +01:00
##teamcity[testStarted name='Basic use of the Contains range matcher']
##teamcity[testFinished name='Basic use of the Contains range matcher' duration="{duration}"]
Add IsEmpty and SizeIs matchers for ranges/containers `SizeIs` can accept both `size_t` and a matcher. In the first case, it checks whether the size of the range is equal to specified size. In the second case, it checks whether the provided matcher accepts the size of the range.
2020-03-09 11:13:07 +01:00
##teamcity[testStarted name='Basic use of the Empty range matcher']
##teamcity[testFinished name='Basic use of the Empty range matcher' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='CAPTURE can deal with complex expressions']
##teamcity[testFinished name='CAPTURE can deal with complex expressions' duration="{duration}"]
##teamcity[testStarted name='CAPTURE can deal with complex expressions involving commas']
##teamcity[testFinished name='CAPTURE can deal with complex expressions involving commas' duration="{duration}"]
##teamcity[testStarted name='CAPTURE parses string and character constants']
##teamcity[testFinished name='CAPTURE parses string and character constants' duration="{duration}"]
##teamcity[testStarted name='Capture and info messages']
##teamcity[testFinished name='Capture and info messages' duration="{duration}"]
Add CaseInsensitiveEqualTo comparison type
2021-12-26 18:54:47 +01:00
##teamcity[testStarted name='CaseInsensitiveEqualsTo is case insensitive']
##teamcity[testFinished name='CaseInsensitiveEqualsTo is case insensitive' duration="{duration}"]
##teamcity[testStarted name='CaseInsensitiveLess is case insensitive']
##teamcity[testFinished name='CaseInsensitiveLess is case insensitive' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Character pretty printing']
##teamcity[testFinished name='Character pretty printing' duration="{duration}"]
Arg::parse accepts plain Args as argument
2020-11-07 17:51:50 +01:00
##teamcity[testStarted name='Clara::Arg supports single-arg parse the way Opt does']
##teamcity[testFinished name='Clara::Arg supports single-arg parse the way Opt does' duration="{duration}"]
Add support for multiply calling lambda parsers in Clara Previously a lambda parser in Clara could only be invoked once, even if it internally was ok with being invoked multiple times. With this change, a lambda parser can mark itself as `accept_many`, in which case it will be invoked multiple times if the appropriate flag was supplied multiple times by the user.
2021-10-27 18:57:56 +02:00
##teamcity[testStarted name='Clara::Opt supports accept-many lambdas']
##teamcity[testFinished name='Clara::Opt supports accept-many lambdas' duration="{duration}"]
Add tests for ColourGuard
2022-03-24 00:09:27 +01:00
##teamcity[testStarted name='ColourGuard behaviour']
##teamcity[testFinished name='ColourGuard behaviour' duration="{duration}"]
Feature: generic matchers (#1843) This commit extends the Matchers feature with the ability to have type-independent (e.g. templated) matchers. This is done by adding a new base type that Matchers can extend, `MatcherGenericBase`, and overloads of operators `!`, `&&` and `||` that handle matchers extending `MatcherGenericBase` in a special manner. These new matchers can also take their arguments as values and non-const references. Closes #1307 Closes #1553 Closes #1554 Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-02-16 11:19:10 +01:00
##teamcity[testStarted name='Combining MatchAllOfGeneric does not nest']
##teamcity[testFinished name='Combining MatchAllOfGeneric does not nest' duration="{duration}"]
##teamcity[testStarted name='Combining MatchAnyOfGeneric does not nest']
##teamcity[testFinished name='Combining MatchAnyOfGeneric does not nest' duration="{duration}"]
##teamcity[testStarted name='Combining MatchNotOfGeneric does not nest']
##teamcity[testFinished name='Combining MatchNotOfGeneric does not nest' duration="{duration}"]
##teamcity[testStarted name='Combining concrete matchers does not use templated matchers']
##teamcity[testFinished name='Combining concrete matchers does not use templated matchers' duration="{duration}"]
##teamcity[testStarted name='Combining only templated matchers']
##teamcity[testFinished name='Combining only templated matchers' duration="{duration}"]
##teamcity[testStarted name='Combining templated and concrete matchers']
##teamcity[testFinished name='Combining templated and concrete matchers' duration="{duration}"]
##teamcity[testStarted name='Combining templated matchers']
##teamcity[testFinished name='Combining templated matchers' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Commas in various macros are allowed']
##teamcity[testFinished name='Commas in various macros are allowed' duration="{duration}"]
##teamcity[testStarted name='Comparing function pointers']
##teamcity[testFinished name='Comparing function pointers' duration="{duration}"]
##teamcity[testStarted name='Comparison ops']
##teamcity[testFinished name='Comparison ops' duration="{duration}"]
##teamcity[testStarted name='Comparison with explicitly convertible types']
##teamcity[testFinished name='Comparison with explicitly convertible types' duration="{duration}"]
##teamcity[testStarted name='Comparisons between ints where one side is computed']
##teamcity[testFinished name='Comparisons between ints where one side is computed' duration="{duration}"]
##teamcity[testStarted name='Comparisons between unsigned ints and negative signed ints match c++ standard behaviour']
##teamcity[testFinished name='Comparisons between unsigned ints and negative signed ints match c++ standard behaviour' duration="{duration}"]
##teamcity[testStarted name='Comparisons with int literals don|'t warn when mixing signed/ unsigned']
##teamcity[testFinished name='Comparisons with int literals don|'t warn when mixing signed/ unsigned' duration="{duration}"]
Add test for shortcircuiting behaviour of generic matcher combinators
2020-07-26 21:22:13 +02:00
##teamcity[testStarted name='Composed generic matchers shortcircuit']
##teamcity[testFinished name='Composed generic matchers shortcircuit' duration="{duration}"]
Add explicit test for shortcircuiting behaviour of combined matchers
2020-06-14 21:48:08 +02:00
##teamcity[testStarted name='Composed matchers shortcircuit']
##teamcity[testFinished name='Composed matchers shortcircuit' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Contains string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Contains string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), ContainsSubstring( "not there", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" contains: "not there" (case insensitive)|n']
##teamcity[testFailed name='Contains string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), ContainsSubstring( "STRING" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" contains: "STRING"|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Contains string matcher' duration="{duration}"]
##teamcity[testStarted name='Copy and then generate a range']
##teamcity[testFinished name='Copy and then generate a range' duration="{duration}"]
Add IStream::isStdOut
2022-02-21 23:48:15 +01:00
##teamcity[testStarted name='Cout stream properly declares it writes to stdout']
##teamcity[testFinished name='Cout stream properly declares it writes to stdout' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Custom exceptions can be translated when testing for nothrow']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Custom exceptions can be translated when testing for nothrow' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "custom exception - not std"|n REQUIRE_NOTHROW( throwCustom() )|nwith expansion:|n throwCustom()|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Custom exceptions can be translated when testing for nothrow' duration="{duration}"]
##teamcity[testStarted name='Custom exceptions can be translated when testing for throwing as something else']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Custom exceptions can be translated when testing for throwing as something else' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "custom exception - not std"|n REQUIRE_THROWS_AS( throwCustom(), std::exception )|nwith expansion:|n throwCustom(), std::exception|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Custom exceptions can be translated when testing for throwing as something else' duration="{duration}"]
##teamcity[testStarted name='Custom std-exceptions can be custom translated']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Custom std-exceptions can be custom translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "custom std exception"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Custom std-exceptions can be custom translated' duration="{duration}"]
##teamcity[testStarted name='Default scale is invisible to comparison']
##teamcity[testFinished name='Default scale is invisible to comparison' duration="{duration}"]
##teamcity[testStarted name='Directly creating an EnumInfo']
##teamcity[testFinished name='Directly creating an EnumInfo' duration="{duration}"]
Test & document SKIP in generator constructor Closes #1593
2023-05-31 15:12:23 +02:00
##teamcity[testStarted name='Empty generators can SKIP in constructor']
##teamcity[testIgnored name='Empty generators can SKIP in constructor' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "This generator is empty"']
##teamcity[testFinished name='Empty generators can SKIP in constructor' duration="{duration}"]
Add IStream::isStdOut
2022-02-21 23:48:15 +01:00
##teamcity[testStarted name='Empty stream name opens cout stream']
##teamcity[testFinished name='Empty stream name opens cout stream' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='EndsWith string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='EndsWith string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), EndsWith( "Substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ends with: "Substring"|n']
##teamcity[testFailed name='EndsWith string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), EndsWith( "this", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ends with: "this" (case insensitive)|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='EndsWith string matcher' duration="{duration}"]
##teamcity[testStarted name='Enums can quickly have stringification enabled using REGISTER_ENUM']
##teamcity[testFinished name='Enums can quickly have stringification enabled using REGISTER_ENUM' duration="{duration}"]
##teamcity[testStarted name='Enums in namespaces can quickly have stringification enabled using REGISTER_ENUM']
##teamcity[testFinished name='Enums in namespaces can quickly have stringification enabled using REGISTER_ENUM' duration="{duration}"]
##teamcity[testStarted name='Epsilon only applies to Approx|'s value']
##teamcity[testFinished name='Epsilon only applies to Approx|'s value' duration="{duration}"]
##teamcity[testStarted name='Equality checks that should fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven == 6 )|nwith expansion:|n 7 == 6|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven == 8 )|nwith expansion:|n 7 == 8|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven == 0 )|nwith expansion:|n 7 == 0|n- failure ignore as test marked as |'ok to fail|'|n']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 9.11f ) )|nwith expansion:|n 9.100000381f|n==|nApprox( 9.10999965667724609 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 9.0f ) )|nwith expansion:|n 9.100000381f == Approx( 9.0 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 1 ) )|nwith expansion:|n 9.100000381f == Approx( 1.0 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one == Approx( 0 ) )|nwith expansion:|n 9.100000381f == Approx( 0.0 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.double_pi == Approx( 3.1415 ) )|nwith expansion:|n 3.14159265350000005|n==|nApprox( 3.14150000000000018 )|n- failure ignore as test marked as |'ok to fail|'|n']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello == "goodbye" )|nwith expansion:|n "hello" == "goodbye"|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello == "hell" )|nwith expansion:|n "hello" == "hell"|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello == "hello1" )|nwith expansion:|n "hello" == "hello1"|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello.size() == 6 )|nwith expansion:|n 5 == 6|n- failure ignore as test marked as |'ok to fail|'|n']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testIgnored name='Equality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( x == Approx( 1.301 ) )|nwith expansion:|n 1.30000000000000027|n==|nApprox( 1.30099999999999993 )|n- failure ignore as test marked as |'ok to fail|'|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Equality checks that should fail' duration="{duration}"]
##teamcity[testStarted name='Equality checks that should succeed']
##teamcity[testFinished name='Equality checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='Equals']
##teamcity[testFinished name='Equals' duration="{duration}"]
##teamcity[testStarted name='Equals string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Equals string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Equals( "this string contains |'ABC|' as a substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" equals: "this string contains |'ABC|' as a substring"|n']
##teamcity[testFailed name='Equals string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Equals( "something else", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" equals: "something else" (case insensitive)|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Equals string matcher' duration="{duration}"]
##teamcity[testStarted name='Exception as a value (e.g. in REQUIRE_THROWS_MATCHES) can be stringified']
##teamcity[testFinished name='Exception as a value (e.g. in REQUIRE_THROWS_MATCHES) can be stringified' duration="{duration}"]
##teamcity[testStarted name='Exception matchers that fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Exception matchers that fail' message='-------------------------------------------------------------------------------|nNo exception|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nno exception was thrown where one was expected|n CHECK_THROWS_MATCHES( doesNotThrow(), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n doesNotThrow(), SpecialException, ExceptionMatcher{ 1 }|n']
##teamcity[testFailed name='Exception matchers that fail' message='Matchers.tests.cpp:<line number>|nno exception was thrown where one was expected|n REQUIRE_THROWS_MATCHES( doesNotThrow(), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n doesNotThrow(), SpecialException, ExceptionMatcher{ 1 }|n']
##teamcity[testFailed name='Exception matchers that fail' message='-------------------------------------------------------------------------------|nType mismatch|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nunexpected exception with message:|n "Unknown exception"|n CHECK_THROWS_MATCHES( throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 }|n']
##teamcity[testFailed name='Exception matchers that fail' message='Matchers.tests.cpp:<line number>|nunexpected exception with message:|n "Unknown exception"|n REQUIRE_THROWS_MATCHES( throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n throwsAsInt( 1 ), SpecialException, ExceptionMatcher{ 1 }|n']
##teamcity[testFailed name='Exception matchers that fail' message='-------------------------------------------------------------------------------|nContents are wrong|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THROWS_MATCHES( throwsSpecialException( 3 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n SpecialException::what special exception has value of 1|n']
##teamcity[testFailed name='Exception matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n REQUIRE_THROWS_MATCHES( throwsSpecialException( 4 ), SpecialException, ExceptionMatcher{ 1 } )|nwith expansion:|n SpecialException::what special exception has value of 1|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Exception matchers that fail' duration="{duration}"]
##teamcity[testStarted name='Exception matchers that succeed']
##teamcity[testFinished name='Exception matchers that succeed' duration="{duration}"]
Add MessageMatches matcher for exception (#2570)
2022-12-12 00:40:47 +01:00
##teamcity[testStarted name='Exception message can be matched']
##teamcity[testFinished name='Exception message can be matched' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Exception messages can be tested for']
##teamcity[testFinished name='Exception messages can be tested for' duration="{duration}"]
##teamcity[testStarted name='Exceptions matchers']
##teamcity[testFinished name='Exceptions matchers' duration="{duration}"]
##teamcity[testStarted name='Expected exceptions that don|'t throw or unexpected exceptions fail the test']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Expected exceptions that don|'t throw or unexpected exceptions fail the test' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK_THROWS_AS( thisThrows(), std::string )|nwith expansion:|n thisThrows(), std::string|n']
##teamcity[testFailed name='Expected exceptions that don|'t throw or unexpected exceptions fail the test' message='Exception.tests.cpp:<line number>|nno exception was thrown where one was expected|n CHECK_THROWS_AS( thisDoesntThrow(), std::domain_error )|nwith expansion:|n thisDoesntThrow(), std::domain_error|n']
##teamcity[testFailed name='Expected exceptions that don|'t throw or unexpected exceptions fail the test' message='Exception.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK_NOTHROW( thisThrows() )|nwith expansion:|n thisThrows()|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Expected exceptions that don|'t throw or unexpected exceptions fail the test' duration="{duration}"]
##teamcity[testStarted name='FAIL aborts the test']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='FAIL aborts the test' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "This is a failure"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='FAIL aborts the test' duration="{duration}"]
##teamcity[testStarted name='FAIL does not require an argument']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='FAIL does not require an argument' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='FAIL does not require an argument' duration="{duration}"]
##teamcity[testStarted name='FAIL_CHECK does not abort the test']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='FAIL_CHECK does not abort the test' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "This is a failure"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='FAIL_CHECK does not abort the test' duration="{duration}"]
##teamcity[testStarted name='Factorials are computed']
##teamcity[testFinished name='Factorials are computed' duration="{duration}"]
Add test for empty result of filter generator
2023-02-01 18:27:41 +01:00
##teamcity[testStarted name='Filter generator throws exception for empty generator']
##teamcity[testFinished name='Filter generator throws exception for empty generator' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Floating point matchers: double']
##teamcity[testFinished name='Floating point matchers: double' duration="{duration}"]
##teamcity[testStarted name='Floating point matchers: float']
##teamcity[testFinished name='Floating point matchers: float' duration="{duration}"]
Add test showing literals and complex generators in one GENERATE
2023-12-10 16:38:48 +01:00
##teamcity[testStarted name='GENERATE can combine literals and generators']
##teamcity[testFinished name='GENERATE can combine literals and generators' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Generators -- adapters']
##teamcity[testFinished name='Generators -- adapters' duration="{duration}"]
##teamcity[testStarted name='Generators -- simple']
##teamcity[testFinished name='Generators -- simple' duration="{duration}"]
##teamcity[testStarted name='Generators internals']
##teamcity[testFinished name='Generators internals' duration="{duration}"]
##teamcity[testStarted name='Greater-than inequalities with different epsilons']
##teamcity[testFinished name='Greater-than inequalities with different epsilons' duration="{duration}"]
Refactor tests
2022-03-31 22:30:38 +02:00
##teamcity[testStarted name='Hashers with different seed produce different hash with same test case']
##teamcity[testFinished name='Hashers with different seed produce different hash with same test case' duration="{duration}"]
##teamcity[testStarted name='Hashers with same seed produce same hash']
##teamcity[testFinished name='Hashers with same seed produce same hash' duration="{duration}"]
##teamcity[testStarted name='Hashing different test cases produces different result']
##teamcity[testFinished name='Hashing different test cases produces different result' duration="{duration}"]
##teamcity[testStarted name='Hashing test case produces same hash across multiple calls']
##teamcity[testFinished name='Hashing test case produces same hash across multiple calls' duration="{duration}"]
Add test for multiple streaming parts in UNSCOPED_INFO
2023-09-17 10:44:31 +02:00
##teamcity[testStarted name='INFO and UNSCOPED_INFO can stream multiple arguments']
##teamcity[testFailed name='INFO and UNSCOPED_INFO can stream multiple arguments' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with messages:|n "This info has multiple parts."|n "This unscoped info has multiple parts."|n "Show infos!"']
##teamcity[testFinished name='INFO and UNSCOPED_INFO can stream multiple arguments' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='INFO and WARN do not abort tests']
##teamcity[testFinished name='INFO and WARN do not abort tests' duration="{duration}"]
##teamcity[testStarted name='INFO gets logged on failure']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='INFO gets logged on failure' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "this message should be logged"|n "so should this"|n REQUIRE( a == 1 )|nwith expansion:|n 2 == 1|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='INFO gets logged on failure' duration="{duration}"]
##teamcity[testStarted name='INFO gets logged on failure, even if captured before successful assertions']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='INFO gets logged on failure, even if captured before successful assertions' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "this message may be logged later"|n "this message should be logged"|n CHECK( a == 1 )|nwith expansion:|n 2 == 1|n']
##teamcity[testFailed name='INFO gets logged on failure, even if captured before successful assertions' message='Message.tests.cpp:<line number>|nexpression failed with messages:|n "this message may be logged later"|n "this message should be logged"|n "and this, but later"|n CHECK( a == 0 )|nwith expansion:|n 2 == 0|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='INFO gets logged on failure, even if captured before successful assertions' duration="{duration}"]
##teamcity[testStarted name='INFO is reset for each loop']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='INFO is reset for each loop' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "current counter 10"|n "i := 10"|n REQUIRE( i < 10 )|nwith expansion:|n 10 < 10|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='INFO is reset for each loop' duration="{duration}"]
Assert Info reset need to also reset result disposition to normal to handle uncaught exception correctly (#2723) * AssertionEnd does not reset the assertion info yet. That is done after populateReaction. And reset assertion info would also reset the result disposition to normal, so that any uncaught exception would be reported as failure * Approving test output changes due to added unit tests * Unit tests to throw std::runtime_error instead of std::exception * Add a unit test to test incomplete assertion handler --------- Co-authored-by: Ross <ross.tang@gfo-x.com>
2023-08-07 22:07:31 +02:00
##teamcity[testStarted name='Incomplete AssertionHandler']
##teamcity[testIgnored name='Incomplete AssertionHandler' message='AssertionHandler.tests.cpp:<line number>|n...............................................................................|n|nAssertionHandler.tests.cpp:<line number>|nunexpected exception with message:|n "Exception translation was disabled by CATCH_CONFIG_FAST_COMPILE"|n REQUIRE( Dummy )|nwith expansion:|n Dummy|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Incomplete AssertionHandler' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Inequality checks that should fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven != 7 )|nwith expansion:|n 7 != 7|n- failure ignore as test marked as |'ok to fail|'|n']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one != Approx( 9.1f ) )|nwith expansion:|n 9.100000381f|n!=|nApprox( 9.10000038146972656 )|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.double_pi != Approx( 3.1415926535 ) )|nwith expansion:|n 3.14159265350000005|n!=|nApprox( 3.14159265350000005 )|n- failure ignore as test marked as |'ok to fail|'|n']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello != "hello" )|nwith expansion:|n "hello" != "hello"|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Inequality checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello.size() != 5 )|nwith expansion:|n 5 != 5|n- failure ignore as test marked as |'ok to fail|'|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Inequality checks that should fail' duration="{duration}"]
##teamcity[testStarted name='Inequality checks that should succeed']
##teamcity[testFinished name='Inequality checks that should succeed' duration="{duration}"]
Add JSON reporter (#2706) Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-11-14 22:52:15 +01:00
##teamcity[testStarted name='JsonWriter']
##teamcity[testFinished name='JsonWriter' duration="{duration}"]
Fix special character escaping in JsonWriter
2023-11-14 23:35:22 +01:00
##teamcity[testStarted name='JsonWriter escapes charaters in strings properly']
##teamcity[testFinished name='JsonWriter escapes charaters in strings properly' duration="{duration}"]
Fix compilation error when lambdas are used in assertions This is a partial revert of b7b346c3e5603.
2020-03-19 13:32:45 +01:00
##teamcity[testStarted name='Lambdas in assertions']
##teamcity[testFinished name='Lambdas in assertions' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Less-than inequalities with different epsilons']
##teamcity[testFinished name='Less-than inequalities with different epsilons' duration="{duration}"]
##teamcity[testStarted name='ManuallyRegistered']
##teamcity[testFinished name='ManuallyRegistered' duration="{duration}"]
##teamcity[testStarted name='Matchers can be (AllOf) composed with the && operator']
##teamcity[testFinished name='Matchers can be (AllOf) composed with the && operator' duration="{duration}"]
##teamcity[testStarted name='Matchers can be (AnyOf) composed with the |||| operator']
##teamcity[testFinished name='Matchers can be (AnyOf) composed with the |||| operator' duration="{duration}"]
##teamcity[testStarted name='Matchers can be composed with both && and ||||']
##teamcity[testFinished name='Matchers can be composed with both && and ||||' duration="{duration}"]
##teamcity[testStarted name='Matchers can be composed with both && and |||| - failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Matchers can be composed with both && and |||| - failing' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), ( ContainsSubstring( "string" ) |||| ContainsSubstring( "different" ) ) && ContainsSubstring( "random" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" ( ( contains: "string" or contains: "different" ) and contains: "random" )|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Matchers can be composed with both && and |||| - failing' duration="{duration}"]
##teamcity[testStarted name='Matchers can be negated (Not) with the ! operator']
##teamcity[testFinished name='Matchers can be negated (Not) with the ! operator' duration="{duration}"]
##teamcity[testStarted name='Matchers can be negated (Not) with the ! operator - failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Matchers can be negated (Not) with the ! operator - failing' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), !ContainsSubstring( "substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" not contains: "substring"|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Matchers can be negated (Not) with the ! operator - failing' duration="{duration}"]
Mark !mayfail tests as skipped in the JUnit reporter Should fix #2116
2021-06-07 19:53:34 +02:00
##teamcity[testStarted name='Mayfail test case with nested sections']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Mayfail test case with nested sections' message='-------------------------------------------------------------------------------|nA|n1|n-------------------------------------------------------------------------------|nCondition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Mayfail test case with nested sections' message='-------------------------------------------------------------------------------|nA|n2|n-------------------------------------------------------------------------------|nCondition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Mayfail test case with nested sections' message='-------------------------------------------------------------------------------|nB|n1|n-------------------------------------------------------------------------------|nCondition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='Mayfail test case with nested sections' message='-------------------------------------------------------------------------------|nB|n2|n-------------------------------------------------------------------------------|nCondition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
Mark !mayfail tests as skipped in the JUnit reporter Should fix #2116
2021-06-07 19:53:34 +02:00
##teamcity[testFinished name='Mayfail test case with nested sections' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Mismatching exception messages failing the test']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Mismatching exception messages failing the test' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nexpression failed|n REQUIRE_THROWS_WITH( thisThrows(), "should fail" )|nwith expansion:|n "expected exception" equals: "should fail"|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Mismatching exception messages failing the test' duration="{duration}"]
Add more unit tests * Call order of listeners/reporters in multireporter * Adding listeners/reporters properly updates reporter preferences
2021-12-19 15:16:56 +01:00
##teamcity[testStarted name='Multireporter calls reporters and listeners in correct order']
##teamcity[testFinished name='Multireporter calls reporters and listeners in correct order' duration="{duration}"]
##teamcity[testStarted name='Multireporter updates ReporterPreferences properly']
##teamcity[testFinished name='Multireporter updates ReporterPreferences properly' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Nested generators and captured variables']
##teamcity[testFinished name='Nested generators and captured variables' duration="{duration}"]
##teamcity[testStarted name='Nice descriptive name']
##teamcity[testFinished name='Nice descriptive name' duration="{duration}"]
##teamcity[testStarted name='Non-std exceptions can be translated']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Non-std exceptions can be translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "custom exception"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Non-std exceptions can be translated' duration="{duration}"]
##teamcity[testStarted name='Objects that evaluated in boolean contexts can be checked']
##teamcity[testFinished name='Objects that evaluated in boolean contexts can be checked' duration="{duration}"]
##teamcity[testStarted name='Optionally static assertions']
##teamcity[testFinished name='Optionally static assertions' duration="{duration}"]
##teamcity[testStarted name='Ordering comparison checks that should fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|n...............................................................................|n|nCondition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven > 7 )|nwith expansion:|n 7 > 7|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < 7 )|nwith expansion:|n 7 < 7|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven > 8 )|nwith expansion:|n 7 > 8|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < 6 )|nwith expansion:|n 7 < 6|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < 0 )|nwith expansion:|n 7 < 0|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven < -1 )|nwith expansion:|n 7 < -1|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven >= 8 )|nwith expansion:|n 7 >= 8|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.int_seven <= 6 )|nwith expansion:|n 7 <= 6|n']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one < 9 )|nwith expansion:|n 9.100000381f < 9|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one > 10 )|nwith expansion:|n 9.100000381f > 10|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.float_nine_point_one > 9.2 )|nwith expansion:|n 9.100000381f > 9.19999999999999929|n']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello > "hello" )|nwith expansion:|n "hello" > "hello"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello < "hello" )|nwith expansion:|n "hello" < "hello"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello > "hellp" )|nwith expansion:|n "hello" > "hellp"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello > "z" )|nwith expansion:|n "hello" > "z"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello < "hellm" )|nwith expansion:|n "hello" < "hellm"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello < "a" )|nwith expansion:|n "hello" < "a"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello >= "z" )|nwith expansion:|n "hello" >= "z"|n']
##teamcity[testFailed name='Ordering comparison checks that should fail' message='Condition.tests.cpp:<line number>|nexpression failed|n CHECK( data.str_hello <= "a" )|nwith expansion:|n "hello" <= "a"|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Ordering comparison checks that should fail' duration="{duration}"]
##teamcity[testStarted name='Ordering comparison checks that should succeed']
##teamcity[testFinished name='Ordering comparison checks that should succeed' duration="{duration}"]
##teamcity[testStarted name='Our PCG implementation provides expected results for known seeds']
##teamcity[testFinished name='Our PCG implementation provides expected results for known seeds' duration="{duration}"]
##teamcity[testStarted name='Output from all sections is reported']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Output from all sections is reported' message='-------------------------------------------------------------------------------|none|n-------------------------------------------------------------------------------|nMessage.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "Message from section one"']
##teamcity[testFailed name='Output from all sections is reported' message='-------------------------------------------------------------------------------|ntwo|n-------------------------------------------------------------------------------|nMessage.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "Message from section two"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Output from all sections is reported' duration="{duration}"]
Feature: generic matchers (#1843) This commit extends the Matchers feature with the ability to have type-independent (e.g. templated) matchers. This is done by adding a new base type that Matchers can extend, `MatcherGenericBase`, and overloads of operators `!`, `&&` and `||` that handle matchers extending `MatcherGenericBase` in a special manner. These new matchers can also take their arguments as values and non-const references. Closes #1307 Closes #1553 Closes #1554 Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-02-16 11:19:10 +01:00
##teamcity[testStarted name='Overloaded comma or address-of operators are not used']
##teamcity[testFinished name='Overloaded comma or address-of operators are not used' duration="{duration}"]
Add parseUInt utility function There is an increasing number of places where Catch2 wants to parse strings into numbers, but being stuck in C++14 world, we do not have good stdlib facilities to do this (`strtoul` and `stoul` are both bad).
2022-10-20 15:06:26 +02:00
##teamcity[testStarted name='Parse uints']
##teamcity[testFinished name='Parse uints' duration="{duration}"]
Add more tests for various less common tag-related cases
2021-12-27 10:04:48 +01:00
##teamcity[testStarted name='Parsed tags are matched case insensitive']
##teamcity[testFinished name='Parsed tags are matched case insensitive' duration="{duration}"]
Improve shardIndex/Count cli argument parsing
2021-10-26 23:26:07 +02:00
##teamcity[testStarted name='Parsing sharding-related cli flags']
##teamcity[testFinished name='Parsing sharding-related cli flags' duration="{duration}"]
Add more tests for various less common tag-related cases
2021-12-27 10:04:48 +01:00
##teamcity[testStarted name='Parsing tags with non-alphabetical characters is pass-through']
##teamcity[testFinished name='Parsing tags with non-alphabetical characters is pass-through' duration="{duration}"]
Allow enabling multiple warnings in one invocation
2021-12-18 20:48:43 +01:00
##teamcity[testStarted name='Parsing warnings']
##teamcity[testFinished name='Parsing warnings' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Pointers can be compared to null']
##teamcity[testFinished name='Pointers can be compared to null' duration="{duration}"]
##teamcity[testStarted name='Precision of floating point stringification can be set']
##teamcity[testFinished name='Precision of floating point stringification can be set' duration="{duration}"]
##teamcity[testStarted name='Predicate matcher can accept const char*']
##teamcity[testFinished name='Predicate matcher can accept const char*' duration="{duration}"]
##teamcity[testStarted name='Process can be configured on command line']
##teamcity[testFinished name='Process can be configured on command line' duration="{duration}"]
##teamcity[testStarted name='Product with differing arities - std::tuple<int, double, float>']
##teamcity[testFinished name='Product with differing arities - std::tuple<int, double, float>' duration="{duration}"]
##teamcity[testStarted name='Product with differing arities - std::tuple<int, double>']
##teamcity[testFinished name='Product with differing arities - std::tuple<int, double>' duration="{duration}"]
##teamcity[testStarted name='Product with differing arities - std::tuple<int>']
##teamcity[testFinished name='Product with differing arities - std::tuple<int>' duration="{duration}"]
Add generateRandomSeed utility to generate randomness seed
2021-10-08 20:02:24 +02:00
##teamcity[testStarted name='Random seed generation accepts known methods']
##teamcity[testFinished name='Random seed generation accepts known methods' duration="{duration}"]
##teamcity[testStarted name='Random seed generation reports unknown methods']
##teamcity[testFinished name='Random seed generation reports unknown methods' duration="{duration}"]
Support sentinel-based ranges in default stringify (#2004)
2020-08-18 10:34:47 +02:00
##teamcity[testStarted name='Range type with sentinel']
##teamcity[testFinished name='Range type with sentinel' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Reconstruction should be based on stringification: #914']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Reconstruction should be based on stringification: #914' message='Decomposition.tests.cpp:<line number>|n...............................................................................|n|nDecomposition.tests.cpp:<line number>|nexpression failed|n CHECK( truthy(false) )|nwith expansion:|n Hey, its truthy!|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Reconstruction should be based on stringification: #914' duration="{duration}"]
##teamcity[testStarted name='Regex string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Regex string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Matches( "this STRING contains |'abc|' as a substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" matches "this STRING contains |'abc|' as a substring" case sensitively|n']
##teamcity[testFailed name='Regex string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Matches( "contains |'abc|' as a substring" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" matches "contains |'abc|' as a substring" case sensitively|n']
##teamcity[testFailed name='Regex string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), Matches( "this string contains |'abc|' as a" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" matches "this string contains |'abc|' as a" case sensitively|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Regex string matcher' duration="{duration}"]
Error out reporter registration for reporter names with '::'
2022-01-09 19:01:14 +01:00
##teamcity[testStarted name='Registering reporter with |'::|' in name fails']
##teamcity[testFinished name='Registering reporter with |'::|' in name fails' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Regression test #1']
##teamcity[testFinished name='Regression test #1' duration="{duration}"]
--list-* flags write to target specified by the -o flag Also added tests for the default implementations of list* reporter helpers. Closes #2061
2021-01-26 17:53:59 +01:00
##teamcity[testStarted name='Reporter|'s write listings to provided stream']
##teamcity[testFinished name='Reporter|'s write listings to provided stream' duration="{duration}"]
Fix bad indentation calculation in the console reporter The problem came from the console reporter trying to provide a fancy linebreaking (primarily for things like `SCENARIO` or the BDD macros), so that new lines start with extra indentation if the text being line broken starts as "{text}: ". The console reporter did not properly take into account cases where the ": " part would already be in a later line, in which case it would ask for non-sensical level of indentation (larger than single line length). We fixed this by also enforcing that the special indentation case only triggers if the ": " is found early enough in the line, so that we also avoid degenerate cases like this: ``` blablabla: F a n c y . . . ``` Fixes #2309
2021-10-25 15:10:35 +02:00
##teamcity[testStarted name='Reproducer for #2309 - a very long description past 80 chars (default console width) with a late colon : blablabla']
##teamcity[testFinished name='Reproducer for #2309 - a very long description past 80 chars (default console width) with a late colon : blablabla' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='SUCCEED counts as a test pass']
##teamcity[testFinished name='SUCCEED counts as a test pass' duration="{duration}"]
##teamcity[testStarted name='SUCCEED does not require an argument']
##teamcity[testFinished name='SUCCEED does not require an argument' duration="{duration}"]
##teamcity[testStarted name='Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or methods']
##teamcity[testFinished name='Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or methods' duration="{duration}"]
##teamcity[testStarted name='Scenario: Do that thing with the thing']
##teamcity[testFinished name='Scenario: Do that thing with the thing' duration="{duration}"]
##teamcity[testStarted name='Scenario: This is a really long scenario name to see how the list command deals with wrapping']
##teamcity[testFinished name='Scenario: This is a really long scenario name to see how the list command deals with wrapping' duration="{duration}"]
##teamcity[testStarted name='Scenario: Vector resizing affects size and capacity']
##teamcity[testFinished name='Scenario: Vector resizing affects size and capacity' duration="{duration}"]
##teamcity[testStarted name='Sends stuff to stdout and stderr']
##teamcity[testStdOut name='Sends stuff to stdout and stderr' out='A string sent directly to stdout|n']
##teamcity[testStdErr name='Sends stuff to stdout and stderr' out='A string sent directly to stderr|nA string sent to stderr via clog|n']
##teamcity[testFinished name='Sends stuff to stdout and stderr' duration="{duration}"]
##teamcity[testStarted name='Some simple comparisons between doubles']
##teamcity[testFinished name='Some simple comparisons between doubles' duration="{duration}"]
##teamcity[testStarted name='Standard output from all sections is reported']
##teamcity[testStdOut name='Standard output from all sections is reported' out='Message from section one|nMessage from section two|n']
##teamcity[testFinished name='Standard output from all sections is reported' duration="{duration}"]
##teamcity[testStarted name='StartsWith string matcher']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='StartsWith string matcher' message='Matchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), StartsWith( "This String" ) )|nwith expansion:|n "this string contains |'abc|' as a substring" starts with: "This String"|n']
##teamcity[testFailed name='StartsWith string matcher' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( testStringForMatching(), StartsWith( "string", Catch::CaseSensitive::No ) )|nwith expansion:|n "this string contains |'abc|' as a substring" starts with: "string" (case insensitive)|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='StartsWith string matcher' duration="{duration}"]
##teamcity[testStarted name='Static arrays are convertible to string']
##teamcity[testFinished name='Static arrays are convertible to string' duration="{duration}"]
##teamcity[testStarted name='String matchers']
##teamcity[testFinished name='String matchers' duration="{duration}"]
##teamcity[testStarted name='StringRef']
##teamcity[testFinished name='StringRef' duration="{duration}"]
##teamcity[testStarted name='StringRef at compilation time']
##teamcity[testFinished name='StringRef at compilation time' duration="{duration}"]
Cleanup and optimize stringifying of string-like types More specifically, made the actual implementation of string-like type handling take argument as `Catch::StringRef`, instead of taking `std::string const&`. This means that string-like types that are not `std::string` no longer need to pay for an extra construction of `std::string` (including the potential allocation), before they can be stringified. The actual string stringification routine is now also better about reserving sufficient space.
2021-05-18 00:02:46 +02:00
##teamcity[testStarted name='Stringifying char arrays with statically known sizes - char']
##teamcity[testFinished name='Stringifying char arrays with statically known sizes - char' duration="{duration}"]
##teamcity[testStarted name='Stringifying char arrays with statically known sizes - signed char']
##teamcity[testFinished name='Stringifying char arrays with statically known sizes - signed char' duration="{duration}"]
##teamcity[testStarted name='Stringifying char arrays with statically known sizes - unsigned char']
##teamcity[testFinished name='Stringifying char arrays with statically known sizes - unsigned char' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Stringifying std::chrono::duration helpers']
##teamcity[testFinished name='Stringifying std::chrono::duration helpers' duration="{duration}"]
##teamcity[testStarted name='Stringifying std::chrono::duration with weird ratios']
##teamcity[testFinished name='Stringifying std::chrono::duration with weird ratios' duration="{duration}"]
##teamcity[testStarted name='Stringifying std::chrono::time_point<system_clock>']
##teamcity[testFinished name='Stringifying std::chrono::time_point<system_clock>' duration="{duration}"]
##teamcity[testStarted name='Tabs and newlines show in output']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Tabs and newlines show in output' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n CHECK( s1 == s2 )|nwith expansion:|n "if ($b == 10) {|n $a = 20;|n}"|n==|n"if ($b == 10) {|n $a = 20;|n}|n"|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Tabs and newlines show in output' duration="{duration}"]
##teamcity[testStarted name='Tag alias can be registered against tag patterns']
##teamcity[testFinished name='Tag alias can be registered against tag patterns' duration="{duration}"]
Add more tests for various less common tag-related cases
2021-12-27 10:04:48 +01:00
##teamcity[testStarted name='Tags with spaces and non-alphanumerical characters are accepted']
##teamcity[testFinished name='Tags with spaces and non-alphanumerical characters are accepted' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 0']
##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 1']
##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case method with test types specified inside std::tuple - MyTypes - 2']
##teamcity[testFinished name='Template test case method with test types specified inside std::tuple - MyTypes - 2' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 0']
##teamcity[testFinished name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 1']
##teamcity[testFinished name='Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 0']
##teamcity[testFinished name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 1']
##teamcity[testFinished name='Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside std::tuple - MyTypes - 0']
##teamcity[testFinished name='Template test case with test types specified inside std::tuple - MyTypes - 0' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside std::tuple - MyTypes - 1']
##teamcity[testFinished name='Template test case with test types specified inside std::tuple - MyTypes - 1' duration="{duration}"]
##teamcity[testStarted name='Template test case with test types specified inside std::tuple - MyTypes - 2']
##teamcity[testFinished name='Template test case with test types specified inside std::tuple - MyTypes - 2' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - float']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - float' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - int']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - int' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - std::string']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - std::string' duration="{duration}"]
##teamcity[testStarted name='TemplateTest: vectors can be sized and resized - std::tuple<int,float>']
##teamcity[testFinished name='TemplateTest: vectors can be sized and resized - std::tuple<int,float>' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - (std::tuple<int, float>), 6']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - (std::tuple<int, float>), 6' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - float,4']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - float,4' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - int,5']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - int,5' duration="{duration}"]
##teamcity[testStarted name='TemplateTestSig: vectors can be sized and resized - std::string,15']
##teamcity[testFinished name='TemplateTestSig: vectors can be sized and resized - std::string,15' duration="{duration}"]
Add more tests for various less common tag-related cases
2021-12-27 10:04:48 +01:00
##teamcity[testStarted name='Test case with identical tags keeps just one']
##teamcity[testFinished name='Test case with identical tags keeps just one' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Test case with one argument']
##teamcity[testFinished name='Test case with one argument' duration="{duration}"]
##teamcity[testStarted name='Test enum bit values']
##teamcity[testFinished name='Test enum bit values' duration="{duration}"]
##teamcity[testStarted name='Test with special, characters "in name']
##teamcity[testFinished name='Test with special, characters "in name' duration="{duration}"]
Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187) Resolves #1390 Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
##teamcity[testStarted name='Testing checked-if']
##teamcity[testFinished name='Testing checked-if' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if 2']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Testing checked-if 2' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187) Resolves #1390 Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
##teamcity[testFinished name='Testing checked-if 2' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if 3']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testIgnored name='Testing checked-if 3' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
Suppress failure of CHECKED_IF and CHECKED_ELSE (#2187) Resolves #1390 Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2021-05-10 21:42:47 +02:00
##teamcity[testFinished name='Testing checked-if 3' duration="{duration}"]
Assert Info reset need to also reset result disposition to normal to handle uncaught exception correctly (#2723) * AssertionEnd does not reset the assertion info yet. That is done after populateReaction. And reset assertion info would also reset the result disposition to normal, so that any uncaught exception would be reported as failure * Approving test output changes due to added unit tests * Unit tests to throw std::runtime_error instead of std::exception * Add a unit test to test incomplete assertion handler --------- Co-authored-by: Ross <ross.tang@gfo-x.com>
2023-08-07 22:07:31 +02:00
##teamcity[testStarted name='Testing checked-if 4']
##teamcity[testIgnored name='Testing checked-if 4' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nunexpected exception with message:|n "Uncaught exception should fail!"|n {Unknown expression after the reported line}|nwith expansion:|n {Unknown expression after the reported line}|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Testing checked-if 4' duration="{duration}"]
##teamcity[testStarted name='Testing checked-if 5']
##teamcity[testIgnored name='Testing checked-if 5' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nunexpected exception with message:|n "Uncaught exception should fail!"|n {Unknown expression after the reported line}|nwith expansion:|n {Unknown expression after the reported line}|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='Testing checked-if 5' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='The NO_FAIL macro reports a failure but does not fail the test']
##teamcity[testFinished name='The NO_FAIL macro reports a failure but does not fail the test' duration="{duration}"]
--list-* flags write to target specified by the -o flag Also added tests for the default implementations of list* reporter helpers. Closes #2061
2021-01-26 17:53:59 +01:00
##teamcity[testStarted name='The default listing implementation write to provided stream']
##teamcity[testFinished name='The default listing implementation write to provided stream' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='This test |'should|' fail but doesn|'t']
##teamcity[testFinished name='This test |'should|' fail but doesn|'t' duration="{duration}"]
##teamcity[testStarted name='Thrown string literals are translated']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Thrown string literals are translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "For some reason someone is throwing a string literal!"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Thrown string literals are translated' duration="{duration}"]
##teamcity[testStarted name='Tracker']
##teamcity[testFinished name='Tracker' duration="{duration}"]
##teamcity[testStarted name='Trim strings']
##teamcity[testFinished name='Trim strings' duration="{duration}"]
Added ElementsAre and UnorderedElementsAre (#2377) Co-authored-by: Garz4 <fancygarz4@gmail.com> Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-22 00:33:04 +01:00
##teamcity[testStarted name='Type conversions of RangeEquals and similar']
##teamcity[testFinished name='Type conversions of RangeEquals and similar' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Unexpected exceptions can be translated']
Default StringMaker<FloatingPointType>::precision to max_digits10 `max_digits10` is the number of decimal digits that the type _can_ represent, in other words, the number of decimal digits needed to disambiguate any two floating point numbers when serialized to a string.
2024-04-30 16:43:05 +02:00
##teamcity[testFailed name='Unexpected exceptions can be translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "3.14000000000000012"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Unexpected exceptions can be translated' duration="{duration}"]
Implement a simplified variant of std::unique_ptr<T> This simplified variant supports only a subset of the functionality in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports single element pointer (no array support) with default deleter. By removing the support for custom deleters, we also avoid requiring significant machinery to support EBO, speeding up instantiations of `unique_ptr<T>` significantly. Catch2 also currently does not need to support `unique_ptr<T[]>`, so that is not supported either.
2020-05-24 23:41:02 +02:00
##teamcity[testStarted name='Upcasting special member functions']
##teamcity[testFinished name='Upcasting special member functions' duration="{duration}"]
Add more comprehensive tests for the quantifier matchers This includes * Testing both positive and negative path through the matchers * Testing them with types whose `begin` and `end` member functions require ADL * Testing them with types that return different types from `begin` and `end`
2020-12-26 23:09:51 +01:00
##teamcity[testStarted name='Usage of AllMatch range matcher']
##teamcity[testFinished name='Usage of AllMatch range matcher' duration="{duration}"]
Add AllTrue, AnyTrue, NoneTrue matchers
2021-11-14 17:05:31 +01:00
##teamcity[testStarted name='Usage of AllTrue range matcher']
##teamcity[testFinished name='Usage of AllTrue range matcher' duration="{duration}"]
Add more comprehensive tests for the quantifier matchers This includes * Testing both positive and negative path through the matchers * Testing them with types whose `begin` and `end` member functions require ADL * Testing them with types that return different types from `begin` and `end`
2020-12-26 23:09:51 +01:00
##teamcity[testStarted name='Usage of AnyMatch range matcher']
##teamcity[testFinished name='Usage of AnyMatch range matcher' duration="{duration}"]
Add AllTrue, AnyTrue, NoneTrue matchers
2021-11-14 17:05:31 +01:00
##teamcity[testStarted name='Usage of AnyTrue range matcher']
##teamcity[testFinished name='Usage of AnyTrue range matcher' duration="{duration}"]
Add more comprehensive tests for the quantifier matchers This includes * Testing both positive and negative path through the matchers * Testing them with types whose `begin` and `end` member functions require ADL * Testing them with types that return different types from `begin` and `end`
2020-12-26 23:09:51 +01:00
##teamcity[testStarted name='Usage of NoneMatch range matcher']
##teamcity[testFinished name='Usage of NoneMatch range matcher' duration="{duration}"]
Add AllTrue, AnyTrue, NoneTrue matchers
2021-11-14 17:05:31 +01:00
##teamcity[testStarted name='Usage of NoneTrue range matcher']
##teamcity[testFinished name='Usage of NoneTrue range matcher' duration="{duration}"]
Added ElementsAre and UnorderedElementsAre (#2377) Co-authored-by: Garz4 <fancygarz4@gmail.com> Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-22 00:33:04 +01:00
##teamcity[testStarted name='Usage of RangeEquals range matcher']
##teamcity[testFinished name='Usage of RangeEquals range matcher' duration="{duration}"]
##teamcity[testStarted name='Usage of UnorderedRangeEquals range matcher']
##teamcity[testFinished name='Usage of UnorderedRangeEquals range matcher' duration="{duration}"]
Add IsEmpty and SizeIs matchers for ranges/containers `SizeIs` can accept both `size_t` and a matcher. In the first case, it checks whether the size of the range is equal to specified size. In the second case, it checks whether the provided matcher accepts the size of the range.
2020-03-09 11:13:07 +01:00
##teamcity[testStarted name='Usage of the SizeIs range matcher']
##teamcity[testFinished name='Usage of the SizeIs range matcher' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='Use a custom approx']
##teamcity[testFinished name='Use a custom approx' duration="{duration}"]
##teamcity[testStarted name='Variadic macros']
##teamcity[testFinished name='Variadic macros' duration="{duration}"]
##teamcity[testStarted name='Vector Approx matcher']
##teamcity[testFinished name='Vector Approx matcher' duration="{duration}"]
##teamcity[testStarted name='Vector Approx matcher -- failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Vector Approx matcher -- failing' message='-------------------------------------------------------------------------------|nEmpty and non empty vectors are not approx equal|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, Approx( t1 ) )|nwith expansion:|n { } is approx: { 1.0, 2.0 }|n']
##teamcity[testFailed name='Vector Approx matcher -- failing' message='-------------------------------------------------------------------------------|nJust different vectors|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v1, Approx( v2 ) )|nwith expansion:|n { 2.0, 4.0, 6.0 } is approx: { 1.0, 3.0, 5.0 }|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Vector Approx matcher -- failing' duration="{duration}"]
##teamcity[testStarted name='Vector matchers']
##teamcity[testFinished name='Vector matchers' duration="{duration}"]
##teamcity[testStarted name='Vector matchers that fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='Vector matchers that fail' message='-------------------------------------------------------------------------------|nContains (element)|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, VectorContains( -1 ) )|nwith expansion:|n { 1, 2, 3 } Contains: -1|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, VectorContains( 1 ) )|nwith expansion:|n { } Contains: 1|n']
##teamcity[testFailed name='Vector matchers that fail' message='-------------------------------------------------------------------------------|nContains (vector)|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, Contains( v ) )|nwith expansion:|n { } Contains: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, Contains( v2 ) )|nwith expansion:|n { 1, 2, 3 } Contains: { 1, 2, 4 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='-------------------------------------------------------------------------------|nEquals|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, Equals( v2 ) )|nwith expansion:|n { 1, 2, 3 } Equals: { 1, 2 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v2, Equals( v ) )|nwith expansion:|n { 1, 2 } Equals: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, Equals( v ) )|nwith expansion:|n { } Equals: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, Equals( empty ) )|nwith expansion:|n { 1, 2, 3 } Equals: { }|n']
##teamcity[testFailed name='Vector matchers that fail' message='-------------------------------------------------------------------------------|nUnorderedEquals|n-------------------------------------------------------------------------------|nMatchers.tests.cpp:<line number>|n...............................................................................|n|nMatchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( v, UnorderedEquals( empty ) )|nwith expansion:|n { 1, 2, 3 } UnorderedEquals: { }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( empty, UnorderedEquals( v ) )|nwith expansion:|n { } UnorderedEquals: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( permuted, UnorderedEquals( v ) )|nwith expansion:|n { 1, 3 } UnorderedEquals: { 1, 2, 3 }|n']
##teamcity[testFailed name='Vector matchers that fail' message='Matchers.tests.cpp:<line number>|nexpression failed|n CHECK_THAT( permuted, UnorderedEquals( v ) )|nwith expansion:|n { 3, 1 } UnorderedEquals: { 1, 2, 3 }|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='Vector matchers that fail' duration="{duration}"]
##teamcity[testStarted name='When checked exceptions are thrown they can be expected or unexpected']
##teamcity[testFinished name='When checked exceptions are thrown they can be expected or unexpected' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown directly they are always failures']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown directly they are always failures' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "unexpected exception"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown directly they are always failures' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown during a CHECK the test should continue']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown during a CHECK the test should continue' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK( thisThrows() == 0 )|nwith expansion:|n thisThrows() == 0|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown during a CHECK the test should continue' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown during a REQUIRE the test should abort fail']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown during a REQUIRE the test should abort fail' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n REQUIRE( thisThrows() == 0 )|nwith expansion:|n thisThrows() == 0|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown during a REQUIRE the test should abort fail' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown from functions they are always failures']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown from functions they are always failures' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "expected exception"|n CHECK( thisThrows() == 0 )|nwith expansion:|n thisThrows() == 0|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown from functions they are always failures' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown from sections they are always failures']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='When unchecked exceptions are thrown from sections they are always failures' message='-------------------------------------------------------------------------------|nsection name|n-------------------------------------------------------------------------------|nException.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "unexpected exception"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='When unchecked exceptions are thrown from sections they are always failures' duration="{duration}"]
##teamcity[testStarted name='When unchecked exceptions are thrown, but caught, they do not affect the test']
##teamcity[testFinished name='When unchecked exceptions are thrown, but caught, they do not affect the test' duration="{duration}"]
##teamcity[testStarted name='X/level/0/a']
##teamcity[testFinished name='X/level/0/a' duration="{duration}"]
##teamcity[testStarted name='X/level/0/b']
##teamcity[testFinished name='X/level/0/b' duration="{duration}"]
##teamcity[testStarted name='X/level/1/a']
##teamcity[testFinished name='X/level/1/a' duration="{duration}"]
##teamcity[testStarted name='X/level/1/b']
##teamcity[testFinished name='X/level/1/b' duration="{duration}"]
##teamcity[testStarted name='XmlEncode']
##teamcity[testFinished name='XmlEncode' duration="{duration}"]
Add explicit test for serialization of boolean attributes in XML
2021-05-30 20:02:02 +02:00
##teamcity[testStarted name='XmlWriter writes boolean attributes as true/false']
##teamcity[testFinished name='XmlWriter writes boolean attributes as true/false' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='a succeeding test can still be skipped']
##teamcity[testIgnored name='a succeeding test can still be skipped' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='a succeeding test can still be skipped' duration="{duration}"]
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle Now that Catch2 is a proper library, we can always build the full library (comparatively minor slowdown) and the user can avoid including benchmarking headers to avoid the compilation slowdown.
2020-02-03 18:08:15 +01:00
##teamcity[testStarted name='analyse no analysis']
##teamcity[testFinished name='analyse no analysis' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='array<int, N> -> toString']
##teamcity[testFinished name='array<int, N> -> toString' duration="{duration}"]
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle Now that Catch2 is a proper library, we can always build the full library (comparatively minor slowdown) and the user can avoid including benchmarking headers to avoid the compilation slowdown.
2020-02-03 18:08:15 +01:00
##teamcity[testStarted name='benchmark function call']
##teamcity[testFinished name='benchmark function call' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='boolean member']
##teamcity[testFinished name='boolean member' duration="{duration}"]
##teamcity[testStarted name='checkedElse']
##teamcity[testFinished name='checkedElse' duration="{duration}"]
##teamcity[testStarted name='checkedElse, failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='checkedElse, failing' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n REQUIRE( testCheckedElse( false ) )|nwith expansion:|n false|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='checkedElse, failing' duration="{duration}"]
##teamcity[testStarted name='checkedIf']
##teamcity[testFinished name='checkedIf' duration="{duration}"]
##teamcity[testStarted name='checkedIf, failing']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='checkedIf, failing' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n REQUIRE( testCheckedIf( false ) )|nwith expansion:|n false|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='checkedIf, failing' duration="{duration}"]
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle Now that Catch2 is a proper library, we can always build the full library (comparatively minor slowdown) and the user can avoid including benchmarking headers to avoid the compilation slowdown.
2020-02-03 18:08:15 +01:00
##teamcity[testStarted name='classify_outliers']
##teamcity[testFinished name='classify_outliers' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='comparisons between const int variables']
##teamcity[testFinished name='comparisons between const int variables' duration="{duration}"]
##teamcity[testStarted name='comparisons between int variables']
##teamcity[testFinished name='comparisons between int variables' duration="{duration}"]
Fix ulp distance calculation for numbers with different signs This is a simplification of the fix proposed in #2152, with the critical function split out so that it can be tested directly, without having to go through the ULP matcher. Closes #2152
2021-07-26 22:01:04 +02:00
##teamcity[testStarted name='convertToBits']
##teamcity[testFinished name='convertToBits' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='dynamic skipping works with generators']
##teamcity[testIgnored name='dynamic skipping works with generators' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 41"']
##teamcity[testIgnored name='dynamic skipping works with generators' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 43"']
##teamcity[testFinished name='dynamic skipping works with generators' duration="{duration}"]
Disallow empty tags in test case specification
2021-05-28 20:49:05 +02:00
##teamcity[testStarted name='empty tags are not allowed']
##teamcity[testFinished name='empty tags are not allowed' duration="{duration}"]
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle Now that Catch2 is a proper library, we can always build the full library (comparatively minor slowdown) and the user can avoid including benchmarking headers to avoid the compilation slowdown.
2020-02-03 18:08:15 +01:00
##teamcity[testStarted name='erfc_inv']
##teamcity[testFinished name='erfc_inv' duration="{duration}"]
##teamcity[testStarted name='estimate_clock_resolution']
##teamcity[testFinished name='estimate_clock_resolution' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='even more nested SECTION tests']
##teamcity[testFinished name='even more nested SECTION tests' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='failed assertions before SKIP cause test case to fail']
##teamcity[testIgnored name='failed assertions before SKIP cause test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexpression failed|n CHECK( 3 == 4 )|nwith expansion:|n 3 == 4|n- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='failed assertions before SKIP cause test case to fail' message='Skip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='failed assertions before SKIP cause test case to fail' duration="{duration}"]
##teamcity[testStarted name='failing for some generator values causes entire test case to fail']
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='failing for some generator values causes entire test case to fail' duration="{duration}"]
##teamcity[testStarted name='failing in some unskipped sections causes entire test case to fail']
##teamcity[testIgnored name='failing in some unskipped sections causes entire test case to fail' message='-------------------------------------------------------------------------------|nskipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testIgnored name='failing in some unskipped sections causes entire test case to fail' message='-------------------------------------------------------------------------------|nnot skipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
##teamcity[testFinished name='failing in some unskipped sections causes entire test case to fail' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='first tag']
##teamcity[testFinished name='first tag' duration="{duration}"]
##teamcity[testStarted name='has printf']
loose text artifact
##teamcity[testFinished name='has printf' duration="{duration}"]
Add support for multiply calling lambda parsers in Clara Previously a lambda parser in Clara could only be invoked once, even if it internally was ok with being invoked multiple times. With this change, a lambda parser can mark itself as `accept_many`, in which case it will be invoked multiple times if the appropriate flag was supplied multiple times by the user.
2021-10-27 18:57:56 +02:00
##teamcity[testStarted name='is_unary_function']
##teamcity[testFinished name='is_unary_function' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='just failure']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='just failure' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "Previous info should not be seen"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='just failure' duration="{duration}"]
##teamcity[testStarted name='just failure after unscoped info']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='just failure after unscoped info' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexplicit failure with message:|n "previous unscoped info SHOULD not be seen"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='just failure after unscoped info' duration="{duration}"]
##teamcity[testStarted name='just info']
##teamcity[testFinished name='just info' duration="{duration}"]
##teamcity[testStarted name='just unscoped info']
##teamcity[testFinished name='just unscoped info' duration="{duration}"]
##teamcity[testStarted name='long long']
##teamcity[testFinished name='long long' duration="{duration}"]
##teamcity[testStarted name='looped SECTION tests']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='looped SECTION tests' message='-------------------------------------------------------------------------------|nb is currently: 0|n-------------------------------------------------------------------------------|nMisc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n CHECK( b > a )|nwith expansion:|n 0 > 1|n']
##teamcity[testFailed name='looped SECTION tests' message='-------------------------------------------------------------------------------|nb is currently: 1|n-------------------------------------------------------------------------------|nMisc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n CHECK( b > a )|nwith expansion:|n 1 > 1|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='looped SECTION tests' duration="{duration}"]
##teamcity[testStarted name='looped tests']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[0|] (1) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[1|] (1) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[3|] (3) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[4|] (5) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[6|] (13) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
##teamcity[testFailed name='looped tests' message='Misc.tests.cpp:<line number>|nexpression failed with message:|n "Testing if fib|[7|] (21) is even"|n CHECK( ( fib|[i|] % 2 ) == 0 )|nwith expansion:|n 1 == 0|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='looped tests' duration="{duration}"]
Add test for %debug stream name
2022-04-23 23:34:27 +02:00
##teamcity[testStarted name='makeStream recognizes %debug stream name']
##teamcity[testFinished name='makeStream recognizes %debug stream name' duration="{duration}"]
Implement a simplified variant of std::unique_ptr<T> This simplified variant supports only a subset of the functionality in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports single element pointer (no array support) with default deleter. By removing the support for custom deleters, we also avoid requiring significant machinery to support EBO, speeding up instantiations of `unique_ptr<T>` significantly. Catch2 also currently does not need to support `unique_ptr<T[]>`, so that is not supported either.
2020-05-24 23:41:02 +02:00
##teamcity[testStarted name='make_unique reimplementation']
##teamcity[testFinished name='make_unique reimplementation' duration="{duration}"]
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle Now that Catch2 is a proper library, we can always build the full library (comparatively minor slowdown) and the user can avoid including benchmarking headers to avoid the compilation slowdown.
2020-02-03 18:08:15 +01:00
##teamcity[testStarted name='mean']
##teamcity[testFinished name='mean' duration="{duration}"]
##teamcity[testStarted name='measure']
##teamcity[testFinished name='measure' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='mix info, unscoped info and warning']
##teamcity[testFinished name='mix info, unscoped info and warning' duration="{duration}"]
##teamcity[testStarted name='more nested SECTION tests']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='more nested SECTION tests' message='-------------------------------------------------------------------------------|ndoesn|'t equal|nequal|n-------------------------------------------------------------------------------|nMisc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed|n REQUIRE( a == b )|nwith expansion:|n 1 == 2|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='more nested SECTION tests' duration="{duration}"]
##teamcity[testStarted name='nested SECTION tests']
##teamcity[testFinished name='nested SECTION tests' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='nested sections can be skipped dynamically at runtime']
##teamcity[testIgnored name='nested sections can be skipped dynamically at runtime' message='-------------------------------------------------------------------------------|nB|nB2|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testStdOut name='nested sections can be skipped dynamically at runtime' out='a!|nb1!|n!|n']
##teamcity[testFinished name='nested sections can be skipped dynamically at runtime' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='non streamable - with conv. op']
##teamcity[testFinished name='non streamable - with conv. op' duration="{duration}"]
##teamcity[testStarted name='non-copyable objects']
##teamcity[testFinished name='non-copyable objects' duration="{duration}"]
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle Now that Catch2 is a proper library, we can always build the full library (comparatively minor slowdown) and the user can avoid including benchmarking headers to avoid the compilation slowdown.
2020-02-03 18:08:15 +01:00
##teamcity[testStarted name='normal_quantile']
##teamcity[testFinished name='normal_quantile' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='not allowed']
##teamcity[testFinished name='not allowed' duration="{duration}"]
##teamcity[testStarted name='not prints unscoped info from previous failures']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='not prints unscoped info from previous failures' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with message:|n "this SHOULD be seen"|n REQUIRE( false )|nwith expansion:|n false|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='not prints unscoped info from previous failures' duration="{duration}"]
##teamcity[testStarted name='null strings']
##teamcity[testFinished name='null strings' duration="{duration}"]
##teamcity[testStarted name='null_ptr']
##teamcity[testFinished name='null_ptr' duration="{duration}"]
##teamcity[testStarted name='pair<pair<int,const char *,pair<std::string,int> > -> toString']
##teamcity[testFinished name='pair<pair<int,const char *,pair<std::string,int> > -> toString' duration="{duration}"]
##teamcity[testStarted name='parseEnums']
##teamcity[testFinished name='parseEnums' duration="{duration}"]
##teamcity[testStarted name='pointer to class']
##teamcity[testFinished name='pointer to class' duration="{duration}"]
##teamcity[testStarted name='print unscoped info if passing unscoped info is printed']
##teamcity[testFinished name='print unscoped info if passing unscoped info is printed' duration="{duration}"]
##teamcity[testStarted name='prints unscoped info on failure']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='prints unscoped info on failure' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "this SHOULD be seen"|n "this SHOULD also be seen"|n REQUIRE( false )|nwith expansion:|n false|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='prints unscoped info on failure' duration="{duration}"]
##teamcity[testStarted name='prints unscoped info only for the first assertion']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='prints unscoped info only for the first assertion' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with message:|n "this SHOULD be seen only ONCE"|n CHECK( false )|nwith expansion:|n false|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='prints unscoped info only for the first assertion' duration="{duration}"]
##teamcity[testStarted name='random SECTION tests']
##teamcity[testFinished name='random SECTION tests' duration="{duration}"]
##teamcity[testStarted name='replaceInPlace']
##teamcity[testFinished name='replaceInPlace' duration="{duration}"]
Add %stderr and %stdout as recognized magic stream names
2022-03-07 10:42:26 +01:00
##teamcity[testStarted name='request an unknown %-starting stream fails']
##teamcity[testFinished name='request an unknown %-starting stream fails' duration="{duration}"]
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle Now that Catch2 is a proper library, we can always build the full library (comparatively minor slowdown) and the user can avoid including benchmarking headers to avoid the compilation slowdown.
2020-02-03 18:08:15 +01:00
##teamcity[testStarted name='resolution']
##teamcity[testFinished name='resolution' duration="{duration}"]
##teamcity[testStarted name='run_for_at_least, chronometer']
##teamcity[testFinished name='run_for_at_least, chronometer' duration="{duration}"]
##teamcity[testStarted name='run_for_at_least, int']
##teamcity[testFinished name='run_for_at_least, int' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='second tag']
##teamcity[testFinished name='second tag' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='sections can be skipped dynamically at runtime']
##teamcity[testIgnored name='sections can be skipped dynamically at runtime' message='-------------------------------------------------------------------------------|nskipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='sections can be skipped dynamically at runtime' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='send a single char to INFO']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='send a single char to INFO' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed with message:|n "3"|n REQUIRE( false )|nwith expansion:|n false|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='send a single char to INFO' duration="{duration}"]
##teamcity[testStarted name='sends information to INFO']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='sends information to INFO' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "hi"|n "i := 7"|n REQUIRE( false )|nwith expansion:|n false|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='sends information to INFO' duration="{duration}"]
##teamcity[testStarted name='shortened hide tags are split apart']
##teamcity[testFinished name='shortened hide tags are split apart' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='skipped tests can optionally provide a reason']
##teamcity[testIgnored name='skipped tests can optionally provide a reason' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 43"']
##teamcity[testFinished name='skipped tests can optionally provide a reason' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='splitString']
##teamcity[testFinished name='splitString' duration="{duration}"]
##teamcity[testStarted name='stacks unscoped info in loops']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='stacks unscoped info in loops' message='Message.tests.cpp:<line number>|n...............................................................................|n|nMessage.tests.cpp:<line number>|nexpression failed with messages:|n "Count 1 to 3..."|n "1"|n "2"|n "3"|n CHECK( false )|nwith expansion:|n false|n']
##teamcity[testFailed name='stacks unscoped info in loops' message='Message.tests.cpp:<line number>|nexpression failed with messages:|n "Count 4 to 6..."|n "4"|n "5"|n "6"|n CHECK( false )|nwith expansion:|n false|n']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='stacks unscoped info in loops' duration="{duration}"]
Change startsWith(char) to take StringRef as argument
2021-09-27 14:52:44 +02:00
##teamcity[testStarted name='startsWith']
##teamcity[testFinished name='startsWith' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='std::map is convertible string']
##teamcity[testFinished name='std::map is convertible string' duration="{duration}"]
##teamcity[testStarted name='std::pair<int,const std::string> -> toString']
##teamcity[testFinished name='std::pair<int,const std::string> -> toString' duration="{duration}"]
##teamcity[testStarted name='std::pair<int,std::string> -> toString']
##teamcity[testFinished name='std::pair<int,std::string> -> toString' duration="{duration}"]
##teamcity[testStarted name='std::set is convertible string']
##teamcity[testFinished name='std::set is convertible string' duration="{duration}"]
##teamcity[testStarted name='std::vector<std::pair<std::string,int> > -> toString']
##teamcity[testFinished name='std::vector<std::pair<std::string,int> > -> toString' duration="{duration}"]
Add %stderr and %stdout as recognized magic stream names
2022-03-07 10:42:26 +01:00
##teamcity[testStarted name='stdout and stderr streams have %-starting name']
##teamcity[testFinished name='stdout and stderr streams have %-starting name' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='stringify ranges']
##teamcity[testFinished name='stringify ranges' duration="{duration}"]
##teamcity[testStarted name='stringify( has_maker )']
##teamcity[testFinished name='stringify( has_maker )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_maker_and_operator )']
##teamcity[testFinished name='stringify( has_maker_and_operator )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_neither )']
##teamcity[testFinished name='stringify( has_neither )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_operator )']
##teamcity[testFinished name='stringify( has_operator )' duration="{duration}"]
##teamcity[testStarted name='stringify( has_template_operator )']
##teamcity[testFinished name='stringify( has_template_operator )' duration="{duration}"]
##teamcity[testStarted name='stringify( vectors<has_maker> )']
##teamcity[testFinished name='stringify( vectors<has_maker> )' duration="{duration}"]
##teamcity[testStarted name='stringify( vectors<has_maker_and_operator> )']
##teamcity[testFinished name='stringify( vectors<has_maker_and_operator> )' duration="{duration}"]
##teamcity[testStarted name='stringify( vectors<has_operator> )']
##teamcity[testFinished name='stringify( vectors<has_operator> )' duration="{duration}"]
##teamcity[testStarted name='strlen3']
##teamcity[testFinished name='strlen3' duration="{duration}"]
##teamcity[testStarted name='tables']
##teamcity[testFinished name='tables' duration="{duration}"]
Add test for dots in tag names
2021-05-12 21:43:14 +02:00
##teamcity[testStarted name='tags with dots in later positions are not parsed as hidden']
##teamcity[testFinished name='tags with dots in later positions are not parsed as hidden' duration="{duration}"]
Add new SKIP macro for skipping tests at runtime (#2360) * Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2023-01-12 15:01:47 +01:00
##teamcity[testStarted name='tests can be skipped dynamically at runtime']
##teamcity[testIgnored name='tests can be skipped dynamically at runtime' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
##teamcity[testFinished name='tests can be skipped dynamically at runtime' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='thrown std::strings are translated']
Improved path normalization in approvalTests.py Instead of redoing the whole line where path was found, only the directory part of the path is removed, instead of removing all of the line before the path starts. This results in slight change in how junit and sonarqube approvals come out, and significant change in how TeamCity reporter approvals come out. This latter difference is the reason for the change, as now the lines with `testFailed` and `testIgnored` messages are not completely butchered.
2022-12-31 21:17:54 +01:00
##teamcity[testFailed name='thrown std::strings are translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "Why would you throw a std::string?"']
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testFinished name='thrown std::strings are translated' duration="{duration}"]
##teamcity[testStarted name='toString on const wchar_t const pointer returns the string contents']
##teamcity[testFinished name='toString on const wchar_t const pointer returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString on const wchar_t pointer returns the string contents']
##teamcity[testFinished name='toString on const wchar_t pointer returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString on wchar_t const pointer returns the string contents']
##teamcity[testFinished name='toString on wchar_t const pointer returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString on wchar_t returns the string contents']
##teamcity[testFinished name='toString on wchar_t returns the string contents' duration="{duration}"]
##teamcity[testStarted name='toString(enum class w/operator<<)']
##teamcity[testFinished name='toString(enum class w/operator<<)' duration="{duration}"]
##teamcity[testStarted name='toString(enum class)']
##teamcity[testFinished name='toString(enum class)' duration="{duration}"]
##teamcity[testStarted name='toString(enum w/operator<<)']
##teamcity[testFinished name='toString(enum w/operator<<)' duration="{duration}"]
##teamcity[testStarted name='toString(enum)']
##teamcity[testFinished name='toString(enum)' duration="{duration}"]
##teamcity[testStarted name='tuple<>']
##teamcity[testFinished name='tuple<>' duration="{duration}"]
##teamcity[testStarted name='tuple<float,int>']
##teamcity[testFinished name='tuple<float,int>' duration="{duration}"]
##teamcity[testStarted name='tuple<int>']
##teamcity[testFinished name='tuple<int>' duration="{duration}"]
##teamcity[testStarted name='tuple<string,string>']
##teamcity[testFinished name='tuple<string,string>' duration="{duration}"]
##teamcity[testStarted name='tuple<tuple<int>,tuple<>,float>']
##teamcity[testFinished name='tuple<tuple<int>,tuple<>,float>' duration="{duration}"]
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle Now that Catch2 is a proper library, we can always build the full library (comparatively minor slowdown) and the user can avoid including benchmarking headers to avoid the compilation slowdown.
2020-02-03 18:08:15 +01:00
##teamcity[testStarted name='uniform samples']
##teamcity[testFinished name='uniform samples' duration="{duration}"]
Add uniform_integer_distribution
2023-12-08 22:01:36 +01:00
##teamcity[testStarted name='uniform_integer_distribution can return the bounds']
##teamcity[testFinished name='uniform_integer_distribution can return the bounds' duration="{duration}"]
Implement a simplified variant of std::unique_ptr<T> This simplified variant supports only a subset of the functionality in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports single element pointer (no array support) with default deleter. By removing the support for custom deleters, we also avoid requiring significant machinery to support EBO, speeding up instantiations of `unique_ptr<T>` significantly. Catch2 also currently does not need to support `unique_ptr<T[]>`, so that is not supported either.
2020-05-24 23:41:02 +02:00
##teamcity[testStarted name='unique_ptr reimplementation: basic functionality']
##teamcity[testFinished name='unique_ptr reimplementation: basic functionality' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='vec<vec<string,alloc>> -> toString']
##teamcity[testFinished name='vec<vec<string,alloc>> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<bool> -> toString']
##teamcity[testFinished name='vector<bool> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<int,allocator> -> toString']
##teamcity[testFinished name='vector<int,allocator> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<int> -> toString']
##teamcity[testFinished name='vector<int> -> toString' duration="{duration}"]
##teamcity[testStarted name='vector<string> -> toString']
##teamcity[testFinished name='vector<string> -> toString' duration="{duration}"]
##teamcity[testStarted name='vectors can be sized and resized']
##teamcity[testFinished name='vectors can be sized and resized' duration="{duration}"]
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle Now that Catch2 is a proper library, we can always build the full library (comparatively minor slowdown) and the user can avoid including benchmarking headers to avoid the compilation slowdown.
2020-02-03 18:08:15 +01:00
##teamcity[testStarted name='warmup']
##teamcity[testFinished name='warmup' duration="{duration}"]
##teamcity[testStarted name='weighted_average_quantile']
##teamcity[testFinished name='weighted_average_quantile' duration="{duration}"]
Add all in-repo reporters to approval tests
2020-02-02 15:04:19 +01:00
##teamcity[testStarted name='xmlentitycheck']
##teamcity[testFinished name='xmlentitycheck' duration="{duration}"]
##teamcity[testSuiteFinished name='<exe-name>']
Reference in New Issue Copy Permalink