mirror of
https://github.com/catchorg/Catch2.git
synced 2024-11-21 21:06:11 +01:00
Add new SKIP macro for skipping tests at runtime (#2360)
* Add new SKIP macro for skipping tests at runtime This adds a new `SKIP` macro for dynamically skipping tests at runtime. The "skipped" status of a test case is treated as a first-class citizen, like "succeeded" or "failed", and is reported with a new color on the console. * Don't show "skipped assertions" in console/compact reporters Also extend skip tests to cover a few more use cases. * Return exit code 4 if all test cases are skipped * Use LightGrey for the skip colour This isn't great, but is better than the deep blue that was borderline invisible on dark backgrounds. The fix is to redo the colouring a bit, including introducing light-blue that is actually visible. * Add support for explicit skips in all reporters * --allow-running-no-tests also allows all tests to be skipped * Add docs for SKIP macro, deprecate IEventListener::skipTest Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
This commit is contained in:
Philip Salzmann
GitHub
parent
52066dbc2a
commit
d548be26e3
@ -11,6 +11,7 @@ Once you're up and running consider the following reference material.
|
||||
* [Logging macros](logging.md#top)
|
||||
* [Test cases and sections](test-cases-and-sections.md#top)
|
||||
* [Test fixtures](test-fixtures.md#top)
|
||||
* [Skipping tests at runtime](skipping.md#top)
|
||||
* [Reporters (output customization)](reporters.md#top)
|
||||
* [Event Listeners](event-listeners.md#top)
|
||||
* [Data Generators (value parameterized tests)](generators.md#top)
|
||||
|
||||
@ -561,10 +561,10 @@ processes, as is done with the [Bazel test sharding](https://docs.bazel.build/ve
|
||||
|
||||
> Introduced in Catch2 3.0.1.
|
||||
|
||||
By default, Catch2 test binaries return non-0 exit code if no tests were
|
||||
run, e.g. if the binary was compiled with no tests, or the provided test
|
||||
spec matched no tests. This flag overrides that, so a test run with no
|
||||
tests still returns 0.
|
||||
By default, Catch2 test binaries return non-0 exit code if no tests were run,
|
||||
e.g. if the binary was compiled with no tests, the provided test spec matched no
|
||||
tests, or all tests [were skipped at runtime](skipping.md#top). This flag
|
||||
overrides that, so a test run with no tests still returns 0.
|
||||
|
||||
## Output verbosity
|
||||
```
|
||||
|
||||
@ -26,6 +26,15 @@ to accurately probe the environment for this information so the flag
|
||||
where it will export `BAZEL_TEST=1` for purposes like the above. Catch2
|
||||
will now instead inspect the environment instead of relying on build configuration.
|
||||
|
||||
### `IEventLister::skipTest( TestCaseInfo const& testInfo )`
|
||||
|
||||
This event (including implementations in derived classes such as `ReporterBase`)
|
||||
is deprecated and will be removed in the next major release. It is currently
|
||||
invoked for all test cases that are not going to be executed due to the test run
|
||||
being aborted (when using `--abort` or `--abortx`). It is however
|
||||
**NOT** invoked for test cases that are [explicitly skipped using the `SKIP`
|
||||
macro](skipping.md#top).
|
||||
|
||||
---
|
||||
|
||||
[Home](Readme.md#top)
|
||||
|
||||
72
docs/skipping.md
Normal file
72
docs/skipping.md
Normal file
@ -0,0 +1,72 @@
|
||||
<a id="top"></a>
|
||||
# Skipping Test Cases at Runtime
|
||||
|
||||
> [Introduced](https://github.com/catchorg/Catch2/pull/2360) in Catch2 X.Y.Z.
|
||||
|
||||
In some situations it may not be possible to meaningfully execute a test case, for example when the system under test is missing certain hardware capabilities.
|
||||
If the required conditions can only be determined at runtime, it often doesn't make sense to consider such a test case as either passed or failed, because it simply can not run at all.
|
||||
To properly express such scenarios, Catch2 allows to explicitly _skip_ test cases, using the `SKIP` macro:
|
||||
|
||||
**SKIP(** _message expression_ **)**
|
||||
|
||||
Example usage:
|
||||
|
||||
```c++
|
||||
TEST_CASE("copy files between drives") {
|
||||
if(getNumberOfHardDrives() < 2) {
|
||||
SKIP("at least two hard drives required");
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
This test case is then reported as _skipped_ instead of _passed_ or _failed_.
|
||||
|
||||
The `SKIP` macro behaves similarly to an explicit [`FAIL`](logging.md#top), in that it is the last expression that will be executed:
|
||||
|
||||
```c++
|
||||
TEST_CASE("my test") {
|
||||
printf("foo");
|
||||
SKIP();
|
||||
printf("bar"); // not printed
|
||||
}
|
||||
```
|
||||
|
||||
However a failed assertion _before_ a `SKIP` still causes the entire test case to fail:
|
||||
|
||||
```c++
|
||||
TEST_CASE("failing test") {
|
||||
CHECK(1 == 2);
|
||||
SKIP();
|
||||
}
|
||||
```
|
||||
|
||||
## Interaction with Sections and Generators
|
||||
|
||||
Sections, nested sections as well as test cases with [generators](generators.md#top) can all be individually skipped, with the rest executing as usual:
|
||||
|
||||
```c++
|
||||
TEST_CASE("complex test case") {
|
||||
int value = GENERATE(2, 4, 6);
|
||||
SECTION("a") {
|
||||
SECTION("a1") { CHECK(value < 8); }
|
||||
SECTION("a2") {
|
||||
if (value == 4) {
|
||||
SKIP();
|
||||
}
|
||||
CHECK(value % 2 == 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This test case will report 5 passing assertions; one for each of the three values in section `a1`, as well as one for each in `a2`, except for when `value == 4`.
|
||||
|
||||
Note that as soon as one section is skipped, the entire test case will be reported as _skipped_ (unless there is a failing assertion, in which case it will be reported as _failed_ instead).
|
||||
|
||||
If all test cases in a run are skipped, Catch2 returns a non-zero exit code by default.
|
||||
This can be overridden using the [--allow-running-no-tests](command-line.md#no-tests-override) flag.
|
||||
|
||||
---
|
||||
|
||||
[Home](Readme.md#top)
|
||||
@ -341,6 +341,12 @@ namespace Catch {
|
||||
return 2;
|
||||
}
|
||||
|
||||
if ( totals.testCases.total() > 0 &&
|
||||
totals.testCases.total() == totals.testCases.skipped
|
||||
&& !m_config->zeroTestsCountAsSuccess() ) {
|
||||
return 4;
|
||||
}
|
||||
|
||||
// Note that on unices only the lower 8 bits are usually used, clamping
|
||||
// the return value to 255 prevents false negative when some multiple
|
||||
// of 256 tests has failed
|
||||
|
||||
@ -49,6 +49,7 @@
|
||||
#define CATCH_FAIL( ... ) INTERNAL_CATCH_MSG( "CATCH_FAIL", Catch::ResultWas::ExplicitFailure, Catch::ResultDisposition::Normal, __VA_ARGS__ )
|
||||
#define CATCH_FAIL_CHECK( ... ) INTERNAL_CATCH_MSG( "CATCH_FAIL_CHECK", Catch::ResultWas::ExplicitFailure, Catch::ResultDisposition::ContinueOnFailure, __VA_ARGS__ )
|
||||
#define CATCH_SUCCEED( ... ) INTERNAL_CATCH_MSG( "CATCH_SUCCEED", Catch::ResultWas::Ok, Catch::ResultDisposition::ContinueOnFailure, __VA_ARGS__ )
|
||||
#define CATCH_SKIP( ... ) INTERNAL_CATCH_MSG( "SKIP", Catch::ResultWas::ExplicitSkip, Catch::ResultDisposition::Normal, __VA_ARGS__ )
|
||||
|
||||
|
||||
#if !defined(CATCH_CONFIG_RUNTIME_STATIC_REQUIRE)
|
||||
@ -102,6 +103,7 @@
|
||||
#define CATCH_FAIL( ... ) (void)(0)
|
||||
#define CATCH_FAIL_CHECK( ... ) (void)(0)
|
||||
#define CATCH_SUCCEED( ... ) (void)(0)
|
||||
#define CATCH_SKIP( ... ) (void)(0)
|
||||
|
||||
#define CATCH_STATIC_REQUIRE( ... ) (void)(0)
|
||||
#define CATCH_STATIC_REQUIRE_FALSE( ... ) (void)(0)
|
||||
@ -146,6 +148,7 @@
|
||||
#define FAIL( ... ) INTERNAL_CATCH_MSG( "FAIL", Catch::ResultWas::ExplicitFailure, Catch::ResultDisposition::Normal, __VA_ARGS__ )
|
||||
#define FAIL_CHECK( ... ) INTERNAL_CATCH_MSG( "FAIL_CHECK", Catch::ResultWas::ExplicitFailure, Catch::ResultDisposition::ContinueOnFailure, __VA_ARGS__ )
|
||||
#define SUCCEED( ... ) INTERNAL_CATCH_MSG( "SUCCEED", Catch::ResultWas::Ok, Catch::ResultDisposition::ContinueOnFailure, __VA_ARGS__ )
|
||||
#define SKIP( ... ) INTERNAL_CATCH_MSG( "SKIP", Catch::ResultWas::ExplicitSkip, Catch::ResultDisposition::Normal, __VA_ARGS__ )
|
||||
|
||||
|
||||
#if !defined(CATCH_CONFIG_RUNTIME_STATIC_REQUIRE)
|
||||
@ -198,6 +201,7 @@
|
||||
#define FAIL( ... ) (void)(0)
|
||||
#define FAIL_CHECK( ... ) (void)(0)
|
||||
#define SUCCEED( ... ) (void)(0)
|
||||
#define SKIP( ... ) (void)(0)
|
||||
|
||||
#define STATIC_REQUIRE( ... ) (void)(0)
|
||||
#define STATIC_REQUIRE_FALSE( ... ) (void)(0)
|
||||
|
||||
@ -14,6 +14,7 @@ namespace Catch {
|
||||
diff.passed = passed - other.passed;
|
||||
diff.failed = failed - other.failed;
|
||||
diff.failedButOk = failedButOk - other.failedButOk;
|
||||
diff.skipped = skipped - other.skipped;
|
||||
return diff;
|
||||
}
|
||||
|
||||
@ -21,14 +22,15 @@ namespace Catch {
|
||||
passed += other.passed;
|
||||
failed += other.failed;
|
||||
failedButOk += other.failedButOk;
|
||||
skipped += other.skipped;
|
||||
return *this;
|
||||
}
|
||||
|
||||
std::uint64_t Counts::total() const {
|
||||
return passed + failed + failedButOk;
|
||||
return passed + failed + failedButOk + skipped;
|
||||
}
|
||||
bool Counts::allPassed() const {
|
||||
return failed == 0 && failedButOk == 0;
|
||||
return failed == 0 && failedButOk == 0 && skipped == 0;
|
||||
}
|
||||
bool Counts::allOk() const {
|
||||
return failed == 0;
|
||||
@ -53,6 +55,8 @@ namespace Catch {
|
||||
++diff.testCases.failed;
|
||||
else if( diff.assertions.failedButOk > 0 )
|
||||
++diff.testCases.failedButOk;
|
||||
else if ( diff.assertions.skipped > 0 )
|
||||
++ diff.testCases.skipped;
|
||||
else
|
||||
++diff.testCases.passed;
|
||||
return diff;
|
||||
|
||||
@ -23,6 +23,7 @@ namespace Catch {
|
||||
std::uint64_t passed = 0;
|
||||
std::uint64_t failed = 0;
|
||||
std::uint64_t failedButOk = 0;
|
||||
std::uint64_t skipped = 0;
|
||||
};
|
||||
|
||||
struct Totals {
|
||||
|
||||
@ -242,7 +242,12 @@ namespace Catch {
|
||||
*/
|
||||
virtual void testRunEnded( TestRunStats const& testRunStats ) = 0;
|
||||
|
||||
//! Called with test cases that are skipped due to the test run aborting
|
||||
/**
|
||||
* Called with test cases that are skipped due to the test run aborting.
|
||||
* NOT called for test cases that are explicitly skipped using the `SKIP` macro.
|
||||
*
|
||||
* Deprecated - will be removed in the next major release.
|
||||
*/
|
||||
virtual void skipTest( TestCaseInfo const& testInfo ) = 0;
|
||||
|
||||
//! Called if a fatal error (signal/structured exception) occured
|
||||
|
||||
@ -50,6 +50,13 @@ namespace Catch {
|
||||
if (m_reaction.shouldThrow) {
|
||||
throw_test_failure_exception();
|
||||
}
|
||||
if ( m_reaction.shouldSkip ) {
|
||||
#if !defined( CATCH_CONFIG_DISABLE_EXCEPTIONS )
|
||||
throw Catch::TestSkipException();
|
||||
#else
|
||||
CATCH_ERROR( "Explicitly skipping tests during runtime requires exceptions" );
|
||||
#endif
|
||||
}
|
||||
}
|
||||
void AssertionHandler::setCompleted() {
|
||||
m_completed = true;
|
||||
|
||||
@ -22,6 +22,7 @@ namespace Catch {
|
||||
struct AssertionReaction {
|
||||
bool shouldDebugBreak = false;
|
||||
bool shouldThrow = false;
|
||||
bool shouldSkip = false;
|
||||
};
|
||||
|
||||
class AssertionHandler {
|
||||
|
||||
@ -47,6 +47,7 @@ namespace Catch {
|
||||
|
||||
Error = BrightRed,
|
||||
Success = Green,
|
||||
Skip = LightGrey,
|
||||
|
||||
OriginalExpression = Cyan,
|
||||
ReconstructedExpression = BrightYellow,
|
||||
|
||||
@ -44,6 +44,9 @@ namespace Catch {
|
||||
catch( TestFailureException& ) {
|
||||
std::rethrow_exception(std::current_exception());
|
||||
}
|
||||
catch( TestSkipException& ) {
|
||||
std::rethrow_exception(std::current_exception());
|
||||
}
|
||||
catch( std::exception const& ex ) {
|
||||
return ex.what();
|
||||
}
|
||||
|
||||
@ -16,6 +16,8 @@ namespace Catch {
|
||||
Ok = 0,
|
||||
Info = 1,
|
||||
Warning = 2,
|
||||
// TODO: Should explicit skip be considered "not OK" (cf. isOk)? I.e., should it have the failure bit?
|
||||
ExplicitSkip = 4,
|
||||
|
||||
FailureBit = 0x10,
|
||||
|
||||
|
||||
@ -270,6 +270,9 @@ namespace Catch {
|
||||
if (result.getResultType() == ResultWas::Ok) {
|
||||
m_totals.assertions.passed++;
|
||||
m_lastAssertionPassed = true;
|
||||
} else if (result.getResultType() == ResultWas::ExplicitSkip) {
|
||||
m_totals.assertions.skipped++;
|
||||
m_lastAssertionPassed = true;
|
||||
} else if (!result.succeeded()) {
|
||||
m_lastAssertionPassed = false;
|
||||
if (result.isOk()) {
|
||||
@ -475,6 +478,8 @@ namespace Catch {
|
||||
duration = timer.getElapsedSeconds();
|
||||
} CATCH_CATCH_ANON (TestFailureException&) {
|
||||
// This just means the test was aborted due to failure
|
||||
} CATCH_CATCH_ANON (TestSkipException&) {
|
||||
// This just means the test was explicitly skipped
|
||||
} CATCH_CATCH_ALL {
|
||||
// Under CATCH_CONFIG_FAST_COMPILE, unexpected exceptions under REQUIRE assertions
|
||||
// are reported without translation at the point of origin.
|
||||
@ -571,8 +576,13 @@ namespace Catch {
|
||||
data.message = static_cast<std::string>(message);
|
||||
AssertionResult assertionResult{ m_lastAssertionInfo, data };
|
||||
assertionEnded( assertionResult );
|
||||
if( !assertionResult.isOk() )
|
||||
if ( !assertionResult.isOk() ) {
|
||||
populateReaction( reaction );
|
||||
} else if ( resultType == ResultWas::ExplicitSkip ) {
|
||||
// TODO: Need to handle this explicitly, as ExplicitSkip is
|
||||
// considered "OK"
|
||||
reaction.shouldSkip = true;
|
||||
}
|
||||
}
|
||||
void RunContext::handleUnexpectedExceptionNotThrown(
|
||||
AssertionInfo const& info,
|
||||
|
||||
@ -20,6 +20,9 @@ namespace Catch {
|
||||
*/
|
||||
[[noreturn]] void throw_test_failure_exception();
|
||||
|
||||
//! Used to signal that the remainder of a test should be skipped
|
||||
struct TestSkipException{};
|
||||
|
||||
} // namespace Catch
|
||||
|
||||
#endif // CATCH_TEST_FAILURE_EXCEPTION_HPP_INCLUDED
|
||||
|
||||
@ -17,7 +17,9 @@ namespace Catch {
|
||||
void AutomakeReporter::testCaseEnded(TestCaseStats const& _testCaseStats) {
|
||||
// Possible values to emit are PASS, XFAIL, SKIP, FAIL, XPASS and ERROR.
|
||||
m_stream << ":test-result: ";
|
||||
if (_testCaseStats.totals.assertions.allPassed()) {
|
||||
if ( _testCaseStats.totals.testCases.skipped > 0 ) {
|
||||
m_stream << "SKIP";
|
||||
} else if (_testCaseStats.totals.assertions.allPassed()) {
|
||||
m_stream << "PASS";
|
||||
} else if (_testCaseStats.totals.assertions.allOk()) {
|
||||
m_stream << "XFAIL";
|
||||
|
||||
@ -105,6 +105,11 @@ public:
|
||||
printIssue("explicitly");
|
||||
printRemainingMessages(Colour::None);
|
||||
break;
|
||||
case ResultWas::ExplicitSkip:
|
||||
printResultType(Colour::Skip, "skipped"_sr);
|
||||
printMessage();
|
||||
printRemainingMessages();
|
||||
break;
|
||||
// These cases are here to prevent compiler warnings
|
||||
case ResultWas::Unknown:
|
||||
case ResultWas::FailureBit:
|
||||
@ -220,7 +225,7 @@ private:
|
||||
|
||||
// Drop out if result was successful and we're not printing those
|
||||
if( !m_config->includeSuccessfulResults() && result.isOk() ) {
|
||||
if( result.getResultType() != ResultWas::Warning )
|
||||
if( result.getResultType() != ResultWas::Warning && result.getResultType() != ResultWas::ExplicitSkip )
|
||||
return;
|
||||
printInfoMessages = false;
|
||||
}
|
||||
|
||||
@ -111,6 +111,14 @@ public:
|
||||
if (_stats.infoMessages.size() > 1)
|
||||
messageLabel = "explicitly with messages";
|
||||
break;
|
||||
case ResultWas::ExplicitSkip:
|
||||
colour = Colour::Skip;
|
||||
passOrFail = "SKIPPED"_sr;
|
||||
if (_stats.infoMessages.size() == 1)
|
||||
messageLabel = "explicitly with message";
|
||||
if (_stats.infoMessages.size() > 1)
|
||||
messageLabel = "explicitly with messages";
|
||||
break;
|
||||
// These cases are here to prevent compiler warnings
|
||||
case ResultWas::Unknown:
|
||||
case ResultWas::FailureBit:
|
||||
@ -185,13 +193,16 @@ std::size_t makeRatio( std::uint64_t number, std::uint64_t total ) {
|
||||
return (ratio == 0 && number > 0) ? 1 : static_cast<std::size_t>(ratio);
|
||||
}
|
||||
|
||||
std::size_t& findMax( std::size_t& i, std::size_t& j, std::size_t& k ) {
|
||||
if (i > j && i > k)
|
||||
std::size_t&
|
||||
findMax( std::size_t& i, std::size_t& j, std::size_t& k, std::size_t& l ) {
|
||||
if (i > j && i > k && i > l)
|
||||
return i;
|
||||
else if (j > k)
|
||||
else if (j > k && j > l)
|
||||
return j;
|
||||
else
|
||||
else if (k > l)
|
||||
return k;
|
||||
else
|
||||
return l;
|
||||
}
|
||||
|
||||
enum class Justification { Left, Right };
|
||||
@ -400,7 +411,8 @@ void ConsoleReporter::assertionEnded(AssertionStats const& _assertionStats) {
|
||||
bool includeResults = m_config->includeSuccessfulResults() || !result.isOk();
|
||||
|
||||
// Drop out if result was successful but we're not printing them.
|
||||
if (!includeResults && result.getResultType() != ResultWas::Warning)
|
||||
// TODO: Make configurable whether skips should be printed
|
||||
if (!includeResults && result.getResultType() != ResultWas::Warning && result.getResultType() != ResultWas::ExplicitSkip)
|
||||
return;
|
||||
|
||||
lazyPrint();
|
||||
@ -603,10 +615,11 @@ void ConsoleReporter::printTotalsDivider(Totals const& totals) {
|
||||
std::size_t failedRatio = makeRatio(totals.testCases.failed, totals.testCases.total());
|
||||
std::size_t failedButOkRatio = makeRatio(totals.testCases.failedButOk, totals.testCases.total());
|
||||
std::size_t passedRatio = makeRatio(totals.testCases.passed, totals.testCases.total());
|
||||
while (failedRatio + failedButOkRatio + passedRatio < CATCH_CONFIG_CONSOLE_WIDTH - 1)
|
||||
findMax(failedRatio, failedButOkRatio, passedRatio)++;
|
||||
std::size_t skippedRatio = makeRatio(totals.testCases.skipped, totals.testCases.total());
|
||||
while (failedRatio + failedButOkRatio + passedRatio + skippedRatio < CATCH_CONFIG_CONSOLE_WIDTH - 1)
|
||||
findMax(failedRatio, failedButOkRatio, passedRatio, skippedRatio)++;
|
||||
while (failedRatio + failedButOkRatio + passedRatio > CATCH_CONFIG_CONSOLE_WIDTH - 1)
|
||||
findMax(failedRatio, failedButOkRatio, passedRatio)--;
|
||||
findMax(failedRatio, failedButOkRatio, passedRatio, skippedRatio)--;
|
||||
|
||||
m_stream << m_colour->guardColour( Colour::Error )
|
||||
<< std::string( failedRatio, '=' )
|
||||
@ -619,6 +632,8 @@ void ConsoleReporter::printTotalsDivider(Totals const& totals) {
|
||||
m_stream << m_colour->guardColour( Colour::Success )
|
||||
<< std::string( passedRatio, '=' );
|
||||
}
|
||||
m_stream << m_colour->guardColour( Colour::Skip )
|
||||
<< std::string( skippedRatio, '=' );
|
||||
} else {
|
||||
m_stream << m_colour->guardColour( Colour::Warning )
|
||||
<< std::string( CATCH_CONFIG_CONSOLE_WIDTH - 1, '=' );
|
||||
|
||||
@ -316,15 +316,22 @@ namespace Catch {
|
||||
}
|
||||
|
||||
std::vector<SummaryColumn> columns;
|
||||
// Don't include "skipped assertions" in total count
|
||||
const auto totalAssertionCount =
|
||||
totals.assertions.total() - totals.assertions.skipped;
|
||||
columns.push_back( SummaryColumn( "", Colour::None )
|
||||
.addRow( totals.testCases.total() )
|
||||
.addRow( totals.assertions.total() ) );
|
||||
.addRow( totalAssertionCount ) );
|
||||
columns.push_back( SummaryColumn( "passed", Colour::Success )
|
||||
.addRow( totals.testCases.passed )
|
||||
.addRow( totals.assertions.passed ) );
|
||||
columns.push_back( SummaryColumn( "failed", Colour::ResultError )
|
||||
.addRow( totals.testCases.failed )
|
||||
.addRow( totals.assertions.failed ) );
|
||||
columns.push_back( SummaryColumn( "skipped", Colour::Skip )
|
||||
.addRow( totals.testCases.skipped )
|
||||
// Don't print "skipped assertions"
|
||||
.addRow( 0 ) );
|
||||
columns.push_back(
|
||||
SummaryColumn( "failed as expected", Colour::ResultExpectedFailure )
|
||||
.addRow( totals.testCases.failedButOk )
|
||||
|
||||
@ -132,6 +132,7 @@ namespace Catch {
|
||||
xml.writeAttribute( "name"_sr, stats.runInfo.name );
|
||||
xml.writeAttribute( "errors"_sr, unexpectedExceptions );
|
||||
xml.writeAttribute( "failures"_sr, stats.totals.assertions.failed-unexpectedExceptions );
|
||||
xml.writeAttribute( "skipped"_sr, stats.totals.assertions.skipped );
|
||||
xml.writeAttribute( "tests"_sr, stats.totals.assertions.total() );
|
||||
xml.writeAttribute( "hostname"_sr, "tbd"_sr ); // !TBD
|
||||
if( m_config->showDurations() == ShowDurations::Never )
|
||||
@ -244,7 +245,8 @@ namespace Catch {
|
||||
|
||||
void JunitReporter::writeAssertion( AssertionStats const& stats ) {
|
||||
AssertionResult const& result = stats.assertionResult;
|
||||
if( !result.isOk() ) {
|
||||
if ( !result.isOk() ||
|
||||
result.getResultType() == ResultWas::ExplicitSkip ) {
|
||||
std::string elementName;
|
||||
switch( result.getResultType() ) {
|
||||
case ResultWas::ThrewException:
|
||||
@ -256,7 +258,9 @@ namespace Catch {
|
||||
case ResultWas::DidntThrowException:
|
||||
elementName = "failure";
|
||||
break;
|
||||
|
||||
case ResultWas::ExplicitSkip:
|
||||
elementName = "skipped";
|
||||
break;
|
||||
// We should never see these here:
|
||||
case ResultWas::Info:
|
||||
case ResultWas::Warning:
|
||||
@ -274,7 +278,9 @@ namespace Catch {
|
||||
xml.writeAttribute( "type"_sr, result.getTestMacroName() );
|
||||
|
||||
ReusableStringStream rss;
|
||||
if (stats.totals.assertions.total() > 0) {
|
||||
if ( result.getResultType() == ResultWas::ExplicitSkip ) {
|
||||
rss << "SKIPPED\n";
|
||||
} else {
|
||||
rss << "FAILED" << ":\n";
|
||||
if (result.hasExpression()) {
|
||||
rss << " ";
|
||||
@ -285,8 +291,6 @@ namespace Catch {
|
||||
rss << "with expansion:\n";
|
||||
rss << TextFlow::Column(result.getExpandedExpression()).indent(2) << '\n';
|
||||
}
|
||||
} else {
|
||||
rss << '\n';
|
||||
}
|
||||
|
||||
if( !result.getMessage().empty() )
|
||||
|
||||
@ -97,7 +97,8 @@ namespace Catch {
|
||||
|
||||
void SonarQubeReporter::writeAssertion(AssertionStats const& stats, bool okToFail) {
|
||||
AssertionResult const& result = stats.assertionResult;
|
||||
if (!result.isOk()) {
|
||||
if ( !result.isOk() ||
|
||||
result.getResultType() == ResultWas::ExplicitSkip ) {
|
||||
std::string elementName;
|
||||
if (okToFail) {
|
||||
elementName = "skipped";
|
||||
@ -108,15 +109,13 @@ namespace Catch {
|
||||
elementName = "error";
|
||||
break;
|
||||
case ResultWas::ExplicitFailure:
|
||||
elementName = "failure";
|
||||
break;
|
||||
case ResultWas::ExpressionFailed:
|
||||
elementName = "failure";
|
||||
break;
|
||||
case ResultWas::DidntThrowException:
|
||||
elementName = "failure";
|
||||
break;
|
||||
|
||||
case ResultWas::ExplicitSkip:
|
||||
elementName = "skipped";
|
||||
break;
|
||||
// We should never see these here:
|
||||
case ResultWas::Info:
|
||||
case ResultWas::Warning:
|
||||
@ -136,7 +135,9 @@ namespace Catch {
|
||||
xml.writeAttribute("message"_sr, messageRss.str());
|
||||
|
||||
ReusableStringStream textRss;
|
||||
if (stats.totals.assertions.total() > 0) {
|
||||
if ( result.getResultType() == ResultWas::ExplicitSkip ) {
|
||||
textRss << "SKIPPED\n";
|
||||
} else {
|
||||
textRss << "FAILED:\n";
|
||||
if (result.hasExpression()) {
|
||||
textRss << '\t' << result.getExpressionInMacro() << '\n';
|
||||
|
||||
@ -100,6 +100,12 @@ namespace Catch {
|
||||
printIssue("explicitly"_sr);
|
||||
printRemainingMessages(Colour::None);
|
||||
break;
|
||||
case ResultWas::ExplicitSkip:
|
||||
printResultType(tapPassedString);
|
||||
printIssue(" # SKIP"_sr);
|
||||
printMessage();
|
||||
printRemainingMessages();
|
||||
break;
|
||||
// These cases are here to prevent compiler warnings
|
||||
case ResultWas::Unknown:
|
||||
case ResultWas::FailureBit:
|
||||
|
||||
@ -59,7 +59,8 @@ namespace Catch {
|
||||
|
||||
void TeamCityReporter::assertionEnded(AssertionStats const& assertionStats) {
|
||||
AssertionResult const& result = assertionStats.assertionResult;
|
||||
if (!result.isOk()) {
|
||||
if ( !result.isOk() ||
|
||||
result.getResultType() == ResultWas::ExplicitSkip ) {
|
||||
|
||||
ReusableStringStream msg;
|
||||
if (!m_headerPrintedForThisSection)
|
||||
@ -84,6 +85,9 @@ namespace Catch {
|
||||
case ResultWas::ExplicitFailure:
|
||||
msg << "explicit failure";
|
||||
break;
|
||||
case ResultWas::ExplicitSkip:
|
||||
msg << "explicit skip";
|
||||
break;
|
||||
|
||||
// We shouldn't get here because of the isOk() test
|
||||
case ResultWas::Ok:
|
||||
@ -111,18 +115,16 @@ namespace Catch {
|
||||
" " << result.getExpandedExpression() << '\n';
|
||||
}
|
||||
|
||||
if (currentTestCaseInfo->okToFail()) {
|
||||
if ( result.getResultType() == ResultWas::ExplicitSkip ) {
|
||||
m_stream << "##teamcity[testIgnored";
|
||||
} else if ( currentTestCaseInfo->okToFail() ) {
|
||||
msg << "- failure ignore as test marked as 'ok to fail'\n";
|
||||
m_stream << "##teamcity[testIgnored"
|
||||
<< " name='" << escape(currentTestCaseInfo->name) << '\''
|
||||
<< " message='" << escape(msg.str()) << '\''
|
||||
<< "]\n";
|
||||
m_stream << "##teamcity[testIgnored";
|
||||
} else {
|
||||
m_stream << "##teamcity[testFailed"
|
||||
<< " name='" << escape(currentTestCaseInfo->name) << '\''
|
||||
<< " message='" << escape(msg.str()) << '\''
|
||||
<< "]\n";
|
||||
m_stream << "##teamcity[testFailed";
|
||||
}
|
||||
m_stream << " name='" << escape( currentTestCaseInfo->name ) << '\''
|
||||
<< " message='" << escape( msg.str() ) << '\'' << "]\n";
|
||||
}
|
||||
m_stream.flush();
|
||||
}
|
||||
|
||||
@ -108,9 +108,10 @@ namespace Catch {
|
||||
}
|
||||
|
||||
// Drop out if result was successful but we're not printing them.
|
||||
if( !includeResults && result.getResultType() != ResultWas::Warning )
|
||||
if ( !includeResults && result.getResultType() != ResultWas::Warning &&
|
||||
result.getResultType() != ResultWas::ExplicitSkip ) {
|
||||
return;
|
||||
|
||||
}
|
||||
|
||||
// Print the expression if there is one.
|
||||
if( result.hasExpression() ) {
|
||||
@ -153,6 +154,12 @@ namespace Catch {
|
||||
m_xml.writeText( result.getMessage() );
|
||||
m_xml.endElement();
|
||||
break;
|
||||
case ResultWas::ExplicitSkip:
|
||||
m_xml.startElement( "Skip" );
|
||||
writeSourceInfo( result.getSourceInfo() );
|
||||
m_xml.writeText( result.getMessage() );
|
||||
m_xml.endElement();
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
@ -163,15 +170,18 @@ namespace Catch {
|
||||
|
||||
void XmlReporter::sectionEnded( SectionStats const& sectionStats ) {
|
||||
StreamingReporterBase::sectionEnded( sectionStats );
|
||||
if( --m_sectionDepth > 0 ) {
|
||||
XmlWriter::ScopedElement e = m_xml.scopedElement( "OverallResults" );
|
||||
e.writeAttribute( "successes"_sr, sectionStats.assertions.passed );
|
||||
e.writeAttribute( "failures"_sr, sectionStats.assertions.failed );
|
||||
e.writeAttribute( "expectedFailures"_sr, sectionStats.assertions.failedButOk );
|
||||
|
||||
if ( m_config->showDurations() == ShowDurations::Always )
|
||||
e.writeAttribute( "durationInSeconds"_sr, sectionStats.durationInSeconds );
|
||||
if ( --m_sectionDepth > 0 ) {
|
||||
{
|
||||
XmlWriter::ScopedElement e = m_xml.scopedElement( "OverallResults" );
|
||||
e.writeAttribute( "successes"_sr, sectionStats.assertions.passed );
|
||||
e.writeAttribute( "failures"_sr, sectionStats.assertions.failed );
|
||||
e.writeAttribute( "expectedFailures"_sr, sectionStats.assertions.failedButOk );
|
||||
e.writeAttribute( "skipped"_sr, sectionStats.assertions.skipped > 0 );
|
||||
|
||||
if ( m_config->showDurations() == ShowDurations::Always )
|
||||
e.writeAttribute( "durationInSeconds"_sr, sectionStats.durationInSeconds );
|
||||
}
|
||||
// Ends assertion tag
|
||||
m_xml.endElement();
|
||||
}
|
||||
}
|
||||
@ -180,6 +190,7 @@ namespace Catch {
|
||||
StreamingReporterBase::testCaseEnded( testCaseStats );
|
||||
XmlWriter::ScopedElement e = m_xml.scopedElement( "OverallResult" );
|
||||
e.writeAttribute( "success"_sr, testCaseStats.totals.assertions.allOk() );
|
||||
e.writeAttribute( "skips"_sr, testCaseStats.totals.assertions.skipped );
|
||||
|
||||
if ( m_config->showDurations() == ShowDurations::Always )
|
||||
e.writeAttribute( "durationInSeconds"_sr, m_testCaseTimer.getElapsedSeconds() );
|
||||
@ -197,11 +208,13 @@ namespace Catch {
|
||||
m_xml.scopedElement( "OverallResults" )
|
||||
.writeAttribute( "successes"_sr, testRunStats.totals.assertions.passed )
|
||||
.writeAttribute( "failures"_sr, testRunStats.totals.assertions.failed )
|
||||
.writeAttribute( "expectedFailures"_sr, testRunStats.totals.assertions.failedButOk );
|
||||
.writeAttribute( "expectedFailures"_sr, testRunStats.totals.assertions.failedButOk )
|
||||
.writeAttribute( "skips"_sr, testRunStats.totals.assertions.skipped );
|
||||
m_xml.scopedElement( "OverallResultsCases")
|
||||
.writeAttribute( "successes"_sr, testRunStats.totals.testCases.passed )
|
||||
.writeAttribute( "failures"_sr, testRunStats.totals.testCases.failed )
|
||||
.writeAttribute( "expectedFailures"_sr, testRunStats.totals.testCases.failedButOk );
|
||||
.writeAttribute( "expectedFailures"_sr, testRunStats.totals.testCases.failedButOk )
|
||||
.writeAttribute( "skips"_sr, testRunStats.totals.testCases.skipped );
|
||||
m_xml.endElement();
|
||||
}
|
||||
|
||||
|
||||
@ -116,6 +116,7 @@ set(TEST_SOURCES
|
||||
${SELF_TEST_DIR}/UsageTests/Generators.tests.cpp
|
||||
${SELF_TEST_DIR}/UsageTests/Message.tests.cpp
|
||||
${SELF_TEST_DIR}/UsageTests/Misc.tests.cpp
|
||||
${SELF_TEST_DIR}/UsageTests/Skip.tests.cpp
|
||||
${SELF_TEST_DIR}/UsageTests/ToStringByte.tests.cpp
|
||||
${SELF_TEST_DIR}/UsageTests/ToStringChrono.tests.cpp
|
||||
${SELF_TEST_DIR}/UsageTests/ToStringGeneral.tests.cpp
|
||||
@ -272,6 +273,10 @@ add_test(NAME TestSpecs::OverrideFailureWithNoMatchedTests
|
||||
COMMAND $<TARGET_FILE:SelfTest> "___nonexistent_test___" --allow-running-no-tests
|
||||
)
|
||||
|
||||
add_test(NAME TestSpecs::OverrideAllSkipFailure
|
||||
COMMAND $<TARGET_FILE:SelfTest> "tests can be skipped dynamically at runtime" --allow-running-no-tests
|
||||
)
|
||||
|
||||
add_test(NAME TestSpecs::NonMatchingTestSpecIsRoundTrippable
|
||||
COMMAND $<TARGET_FILE:SelfTest> Tracker, "this test does not exist" "[nor does this tag]"
|
||||
)
|
||||
|
||||
@ -488,15 +488,32 @@ set_tests_properties(TestSpecs::EmptySpecWithNoTestsFails
|
||||
PROPERTIES
|
||||
WILL_FAIL ON
|
||||
)
|
||||
|
||||
add_test(
|
||||
NAME TestSpecs::OverrideFailureWithEmptySpec
|
||||
COMMAND $<TARGET_FILE:NoTests> --allow-running-no-tests
|
||||
)
|
||||
|
||||
add_test(
|
||||
NAME List::Listeners::WorksWithoutRegisteredListeners
|
||||
COMMAND $<TARGET_FILE:NoTests> --list-listeners
|
||||
)
|
||||
|
||||
|
||||
add_executable(AllSkipped ${TESTS_DIR}/X93-AllSkipped.cpp)
|
||||
target_link_libraries(AllSkipped PRIVATE Catch2::Catch2WithMain)
|
||||
|
||||
add_test(
|
||||
NAME TestSpecs::SkippingAllTestsFails
|
||||
COMMAND $<TARGET_FILE:AllSkipped>
|
||||
)
|
||||
set_tests_properties(TestSpecs::SkippingAllTestsFails
|
||||
PROPERTIES
|
||||
WILL_FAIL ON
|
||||
)
|
||||
|
||||
set( EXTRA_TEST_BINARIES
|
||||
AllSkipped
|
||||
PrefixedMacros
|
||||
DisabledMacros
|
||||
DisabledExceptions-DefaultHandler
|
||||
|
||||
16
tests/ExtraTests/X93-AllSkipped.cpp
Normal file
16
tests/ExtraTests/X93-AllSkipped.cpp
Normal file
@ -0,0 +1,16 @@
|
||||
|
||||
// Copyright Catch2 Authors
|
||||
// Distributed under the Boost Software License, Version 1.0.
|
||||
// (See accompanying file LICENSE.txt or copy at
|
||||
// https://www.boost.org/LICENSE_1_0.txt)
|
||||
|
||||
// SPDX-License-Identifier: BSL-1.0
|
||||
|
||||
#include <catch2/catch_test_macros.hpp>
|
||||
|
||||
TEST_CASE( "this test case is being skipped" ) { SKIP(); }
|
||||
|
||||
TEST_CASE( "all sections in this test case are being skipped" ) {
|
||||
SECTION( "A" ) { SKIP(); }
|
||||
SECTION( "B" ) { SKIP(); }
|
||||
}
|
||||
@ -297,6 +297,7 @@ Message from section two
|
||||
:test-result: PASS X/level/1/b
|
||||
:test-result: PASS XmlEncode
|
||||
:test-result: PASS XmlWriter writes boolean attributes as true/false
|
||||
:test-result: SKIP a succeeding test can still be skipped
|
||||
:test-result: PASS analyse no analysis
|
||||
:test-result: PASS array<int, N> -> toString
|
||||
:test-result: PASS benchmark function call
|
||||
@ -309,10 +310,14 @@ Message from section two
|
||||
:test-result: PASS comparisons between const int variables
|
||||
:test-result: PASS comparisons between int variables
|
||||
:test-result: PASS convertToBits
|
||||
:test-result: SKIP dynamic skipping works with generators
|
||||
:test-result: PASS empty tags are not allowed
|
||||
:test-result: PASS erfc_inv
|
||||
:test-result: PASS estimate_clock_resolution
|
||||
:test-result: PASS even more nested SECTION tests
|
||||
:test-result: XFAIL failed assertions before SKIP cause test case to fail
|
||||
:test-result: XFAIL failing for some generator values causes entire test case to fail
|
||||
:test-result: XFAIL failing in some unskipped sections causes entire test case to fail
|
||||
:test-result: FAIL first tag
|
||||
loose text artifact
|
||||
:test-result: FAIL has printf
|
||||
@ -331,6 +336,10 @@ loose text artifact
|
||||
:test-result: FAIL mix info, unscoped info and warning
|
||||
:test-result: FAIL more nested SECTION tests
|
||||
:test-result: PASS nested SECTION tests
|
||||
a!
|
||||
b1!
|
||||
!
|
||||
:test-result: FAIL nested sections can be skipped dynamically at runtime
|
||||
:test-result: PASS non streamable - with conv. op
|
||||
:test-result: PASS non-copyable objects
|
||||
:test-result: PASS normal_cdf
|
||||
@ -352,9 +361,11 @@ loose text artifact
|
||||
:test-result: PASS run_for_at_least, chronometer
|
||||
:test-result: PASS run_for_at_least, int
|
||||
:test-result: FAIL second tag
|
||||
:test-result: SKIP sections can be skipped dynamically at runtime
|
||||
:test-result: FAIL send a single char to INFO
|
||||
:test-result: FAIL sends information to INFO
|
||||
:test-result: PASS shortened hide tags are split apart
|
||||
:test-result: SKIP skipped tests can optionally provide a reason
|
||||
:test-result: PASS splitString
|
||||
:test-result: FAIL stacks unscoped info in loops
|
||||
:test-result: PASS startsWith
|
||||
@ -376,6 +387,7 @@ loose text artifact
|
||||
:test-result: PASS strlen3
|
||||
:test-result: PASS tables
|
||||
:test-result: PASS tags with dots in later positions are not parsed as hidden
|
||||
:test-result: SKIP tests can be skipped dynamically at runtime
|
||||
:test-result: FAIL thrown std::strings are translated
|
||||
:test-result: PASS toString on const wchar_t const pointer returns the string contents
|
||||
:test-result: PASS toString on const wchar_t pointer returns the string contents
|
||||
|
||||
@ -290,6 +290,7 @@
|
||||
:test-result: PASS X/level/1/b
|
||||
:test-result: PASS XmlEncode
|
||||
:test-result: PASS XmlWriter writes boolean attributes as true/false
|
||||
:test-result: SKIP a succeeding test can still be skipped
|
||||
:test-result: PASS analyse no analysis
|
||||
:test-result: PASS array<int, N> -> toString
|
||||
:test-result: PASS benchmark function call
|
||||
@ -302,10 +303,14 @@
|
||||
:test-result: PASS comparisons between const int variables
|
||||
:test-result: PASS comparisons between int variables
|
||||
:test-result: PASS convertToBits
|
||||
:test-result: SKIP dynamic skipping works with generators
|
||||
:test-result: PASS empty tags are not allowed
|
||||
:test-result: PASS erfc_inv
|
||||
:test-result: PASS estimate_clock_resolution
|
||||
:test-result: PASS even more nested SECTION tests
|
||||
:test-result: XFAIL failed assertions before SKIP cause test case to fail
|
||||
:test-result: XFAIL failing for some generator values causes entire test case to fail
|
||||
:test-result: XFAIL failing in some unskipped sections causes entire test case to fail
|
||||
:test-result: FAIL first tag
|
||||
:test-result: FAIL has printf
|
||||
:test-result: PASS is_unary_function
|
||||
@ -323,6 +328,7 @@
|
||||
:test-result: FAIL mix info, unscoped info and warning
|
||||
:test-result: FAIL more nested SECTION tests
|
||||
:test-result: PASS nested SECTION tests
|
||||
:test-result: FAIL nested sections can be skipped dynamically at runtime
|
||||
:test-result: PASS non streamable - with conv. op
|
||||
:test-result: PASS non-copyable objects
|
||||
:test-result: PASS normal_cdf
|
||||
@ -344,9 +350,11 @@
|
||||
:test-result: PASS run_for_at_least, chronometer
|
||||
:test-result: PASS run_for_at_least, int
|
||||
:test-result: FAIL second tag
|
||||
:test-result: SKIP sections can be skipped dynamically at runtime
|
||||
:test-result: FAIL send a single char to INFO
|
||||
:test-result: FAIL sends information to INFO
|
||||
:test-result: PASS shortened hide tags are split apart
|
||||
:test-result: SKIP skipped tests can optionally provide a reason
|
||||
:test-result: PASS splitString
|
||||
:test-result: FAIL stacks unscoped info in loops
|
||||
:test-result: PASS startsWith
|
||||
@ -368,6 +376,7 @@
|
||||
:test-result: PASS strlen3
|
||||
:test-result: PASS tables
|
||||
:test-result: PASS tags with dots in later positions are not parsed as hidden
|
||||
:test-result: SKIP tests can be skipped dynamically at runtime
|
||||
:test-result: FAIL thrown std::strings are translated
|
||||
:test-result: PASS toString on const wchar_t const pointer returns the string contents
|
||||
:test-result: PASS toString on const wchar_t pointer returns the string contents
|
||||
|
||||
@ -2060,6 +2060,8 @@ Xml.tests.cpp:<line number>: passed: encode( "[\x7F]" ) == "[\\x7F]" for: "[\x7F
|
||||
Xml.tests.cpp:<line number>: passed: stream.str(), ContainsSubstring(R"(attr1="true")") && ContainsSubstring(R"(attr2="false")") for: "<?xml version="1.0" encoding="UTF-8"?>
|
||||
<Element1 attr1="true" attr2="false"/>
|
||||
" ( contains: "attr1="true"" and contains: "attr2="false"" )
|
||||
Skip.tests.cpp:<line number>: passed:
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.point.count() == 23 for: 23.0 == 23
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.lower_bound.count() == 23 for: 23.0 == 23
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.upper_bound.count() == 23 for: 23.0 == 23
|
||||
@ -2149,6 +2151,9 @@ FloatingPoint.tests.cpp:<line number>: passed: convertToBits( -0. ) == ( 1ULL <<
|
||||
9223372036854775808 (0x<hex digits>)
|
||||
FloatingPoint.tests.cpp:<line number>: passed: convertToBits( std::numeric_limits<float>::denorm_min() ) == 1 for: 1 == 1
|
||||
FloatingPoint.tests.cpp:<line number>: passed: convertToBits( std::numeric_limits<double>::denorm_min() ) == 1 for: 1 == 1
|
||||
Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 41'
|
||||
Skip.tests.cpp:<line number>: passed:
|
||||
Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 43'
|
||||
Tag.tests.cpp:<line number>: passed: Catch::TestCaseInfo("", { "test with an empty tag", "[]" }, dummySourceLineInfo)
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: erfc_inv(1.103560) == Approx(-0.09203687623843015) for: -0.0920368762 == Approx( -0.0920368762 )
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: erfc_inv(1.067400) == Approx(-0.05980291115763361) for: -0.0598029112 == Approx( -0.0598029112 )
|
||||
@ -2158,6 +2163,14 @@ InternalBenchmark.tests.cpp:<line number>: passed: res.outliers.total() == 0 for
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
Skip.tests.cpp:<line number>: failed: 3 == 4
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: failed: explicitly
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: failed: explicitly
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: failed: explicitly
|
||||
loose text artifact
|
||||
Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary1)>::value'
|
||||
Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary2)>::value'
|
||||
@ -2215,6 +2228,10 @@ Misc.tests.cpp:<line number>: passed: a < b for: 1 < 2
|
||||
Misc.tests.cpp:<line number>: passed: a != b for: 1 != 2
|
||||
Misc.tests.cpp:<line number>: passed: b != a for: 2 != 1
|
||||
Misc.tests.cpp:<line number>: passed: a != b for: 1 != 2
|
||||
a!
|
||||
b1!
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
!
|
||||
Tricky.tests.cpp:<line number>: passed: s == "7" for: "7" == "7"
|
||||
Tricky.tests.cpp:<line number>: passed: ti == typeid(int) for: {?} == {?}
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: normal_cdf(0.000000) == Approx(0.50000000000000000) for: 0.5 == Approx( 0.5 )
|
||||
@ -2299,9 +2316,13 @@ InternalBenchmark.tests.cpp:<line number>: passed: x >= old_x for: 128 >= 64
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: Timing.elapsed >= time for: 128 ns >= 100 ns
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: Timing.result == Timing.iterations + 17 for: 145 == 145
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: Timing.iterations >= time.count() for: 128 >= 100
|
||||
Skip.tests.cpp:<line number>: passed:
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: passed:
|
||||
Misc.tests.cpp:<line number>: failed: false with 1 message: '3'
|
||||
Message.tests.cpp:<line number>: failed: false with 2 messages: 'hi' and 'i := 7'
|
||||
Tag.tests.cpp:<line number>: passed: testcase.tags, VectorContains( Tag( "magic-tag" ) ) && VectorContains( Tag( "."_catch_sr ) ) for: { {?}, {?} } ( Contains: {?} and Contains: {?} )
|
||||
Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 43'
|
||||
StringManip.tests.cpp:<line number>: passed: splitStringRef("", ','), Equals(std::vector<StringRef>()) for: { } Equals: { }
|
||||
StringManip.tests.cpp:<line number>: passed: splitStringRef("abc", ','), Equals(std::vector<StringRef>{"abc"}) for: { abc } Equals: { abc }
|
||||
StringManip.tests.cpp:<line number>: passed: splitStringRef("abc,def", ','), Equals(std::vector<StringRef>{"abc", "def"}) for: { abc, def } Equals: { abc, def }
|
||||
@ -2367,6 +2388,7 @@ Generators.tests.cpp:<line number>: passed: strlen(std::get<0>(data)) == static_
|
||||
Generators.tests.cpp:<line number>: passed: strlen(std::get<0>(data)) == static_cast<size_t>(std::get<1>(data)) for: 6 == 6
|
||||
Tag.tests.cpp:<line number>: passed: testcase.tags.size() == 1 for: 1 == 1
|
||||
Tag.tests.cpp:<line number>: passed: testcase.tags[0].original == "magic.tag"_catch_sr for: magic.tag == magic.tag
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'Why would you throw a std::string?'
|
||||
Misc.tests.cpp:<line number>: passed: result == "\"wide load\"" for: ""wide load"" == ""wide load""
|
||||
Misc.tests.cpp:<line number>: passed: result == "\"wide load\"" for: ""wide load"" == ""wide load""
|
||||
@ -2463,7 +2485,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
test cases: 395 | 305 passed | 83 failed | 7 failed as expected
|
||||
assertions: 2163 | 1993 passed | 143 failed | 27 failed as expected
|
||||
test cases: 404 | 305 passed | 84 failed | 5 skipped | 10 failed as expected
|
||||
assertions: 2173 | 1997 passed | 145 failed | 31 failed as expected
|
||||
|
||||
|
||||
|
||||
@ -2053,6 +2053,8 @@ Xml.tests.cpp:<line number>: passed: encode( "[\x7F]" ) == "[\\x7F]" for: "[\x7F
|
||||
Xml.tests.cpp:<line number>: passed: stream.str(), ContainsSubstring(R"(attr1="true")") && ContainsSubstring(R"(attr2="false")") for: "<?xml version="1.0" encoding="UTF-8"?>
|
||||
<Element1 attr1="true" attr2="false"/>
|
||||
" ( contains: "attr1="true"" and contains: "attr2="false"" )
|
||||
Skip.tests.cpp:<line number>: passed:
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.point.count() == 23 for: 23.0 == 23
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.lower_bound.count() == 23 for: 23.0 == 23
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: analysis.mean.upper_bound.count() == 23 for: 23.0 == 23
|
||||
@ -2142,6 +2144,9 @@ FloatingPoint.tests.cpp:<line number>: passed: convertToBits( -0. ) == ( 1ULL <<
|
||||
9223372036854775808 (0x<hex digits>)
|
||||
FloatingPoint.tests.cpp:<line number>: passed: convertToBits( std::numeric_limits<float>::denorm_min() ) == 1 for: 1 == 1
|
||||
FloatingPoint.tests.cpp:<line number>: passed: convertToBits( std::numeric_limits<double>::denorm_min() ) == 1 for: 1 == 1
|
||||
Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 41'
|
||||
Skip.tests.cpp:<line number>: passed:
|
||||
Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 43'
|
||||
Tag.tests.cpp:<line number>: passed: Catch::TestCaseInfo("", { "test with an empty tag", "[]" }, dummySourceLineInfo)
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: erfc_inv(1.103560) == Approx(-0.09203687623843015) for: -0.0920368762 == Approx( -0.0920368762 )
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: erfc_inv(1.067400) == Approx(-0.05980291115763361) for: -0.0598029112 == Approx( -0.0598029112 )
|
||||
@ -2151,6 +2156,14 @@ InternalBenchmark.tests.cpp:<line number>: passed: res.outliers.total() == 0 for
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
Skip.tests.cpp:<line number>: failed: 3 == 4
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: failed: explicitly
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: failed: explicitly
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: failed: explicitly
|
||||
Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary1)>::value'
|
||||
Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary2)>::value'
|
||||
Clara.tests.cpp:<line number>: passed: with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary3)>::value'
|
||||
@ -2207,6 +2220,7 @@ Misc.tests.cpp:<line number>: passed: a < b for: 1 < 2
|
||||
Misc.tests.cpp:<line number>: passed: a != b for: 1 != 2
|
||||
Misc.tests.cpp:<line number>: passed: b != a for: 2 != 1
|
||||
Misc.tests.cpp:<line number>: passed: a != b for: 1 != 2
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Tricky.tests.cpp:<line number>: passed: s == "7" for: "7" == "7"
|
||||
Tricky.tests.cpp:<line number>: passed: ti == typeid(int) for: {?} == {?}
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: normal_cdf(0.000000) == Approx(0.50000000000000000) for: 0.5 == Approx( 0.5 )
|
||||
@ -2291,9 +2305,13 @@ InternalBenchmark.tests.cpp:<line number>: passed: x >= old_x for: 128 >= 64
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: Timing.elapsed >= time for: 128 ns >= 100 ns
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: Timing.result == Timing.iterations + 17 for: 145 == 145
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: Timing.iterations >= time.count() for: 128 >= 100
|
||||
Skip.tests.cpp:<line number>: passed:
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Skip.tests.cpp:<line number>: passed:
|
||||
Misc.tests.cpp:<line number>: failed: false with 1 message: '3'
|
||||
Message.tests.cpp:<line number>: failed: false with 2 messages: 'hi' and 'i := 7'
|
||||
Tag.tests.cpp:<line number>: passed: testcase.tags, VectorContains( Tag( "magic-tag" ) ) && VectorContains( Tag( "."_catch_sr ) ) for: { {?}, {?} } ( Contains: {?} and Contains: {?} )
|
||||
Skip.tests.cpp:<line number>: skipped: 'skipping because answer = 43'
|
||||
StringManip.tests.cpp:<line number>: passed: splitStringRef("", ','), Equals(std::vector<StringRef>()) for: { } Equals: { }
|
||||
StringManip.tests.cpp:<line number>: passed: splitStringRef("abc", ','), Equals(std::vector<StringRef>{"abc"}) for: { abc } Equals: { abc }
|
||||
StringManip.tests.cpp:<line number>: passed: splitStringRef("abc,def", ','), Equals(std::vector<StringRef>{"abc", "def"}) for: { abc, def } Equals: { abc, def }
|
||||
@ -2359,6 +2377,7 @@ Generators.tests.cpp:<line number>: passed: strlen(std::get<0>(data)) == static_
|
||||
Generators.tests.cpp:<line number>: passed: strlen(std::get<0>(data)) == static_cast<size_t>(std::get<1>(data)) for: 6 == 6
|
||||
Tag.tests.cpp:<line number>: passed: testcase.tags.size() == 1 for: 1 == 1
|
||||
Tag.tests.cpp:<line number>: passed: testcase.tags[0].original == "magic.tag"_catch_sr for: magic.tag == magic.tag
|
||||
Skip.tests.cpp:<line number>: skipped:
|
||||
Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'Why would you throw a std::string?'
|
||||
Misc.tests.cpp:<line number>: passed: result == "\"wide load\"" for: ""wide load"" == ""wide load""
|
||||
Misc.tests.cpp:<line number>: passed: result == "\"wide load\"" for: ""wide load"" == ""wide load""
|
||||
@ -2455,7 +2474,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
|
||||
InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
Misc.tests.cpp:<line number>: passed:
|
||||
test cases: 395 | 305 passed | 83 failed | 7 failed as expected
|
||||
assertions: 2163 | 1993 passed | 143 failed | 27 failed as expected
|
||||
test cases: 404 | 305 passed | 84 failed | 5 skipped | 10 failed as expected
|
||||
assertions: 2173 | 1997 passed | 145 failed | 31 failed as expected
|
||||
|
||||
|
||||
|
||||
@ -1164,6 +1164,14 @@ Exception.tests.cpp:<line number>: FAILED:
|
||||
due to unexpected exception with message:
|
||||
unexpected exception
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
a succeeding test can still be skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
checkedElse, failing
|
||||
-------------------------------------------------------------------------------
|
||||
@ -1186,6 +1194,87 @@ Misc.tests.cpp:<line number>: FAILED:
|
||||
with expansion:
|
||||
false
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
dynamic skipping works with generators
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
explicitly with message:
|
||||
skipping because answer = 41
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
dynamic skipping works with generators
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
explicitly with message:
|
||||
skipping because answer = 43
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failed assertions before SKIP cause test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
CHECK( 3 == 4 )
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing in some unskipped sections causes entire test case to fail
|
||||
skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing in some unskipped sections causes entire test case to fail
|
||||
not skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
|
||||
loose text artifact
|
||||
-------------------------------------------------------------------------------
|
||||
just failure
|
||||
@ -1304,6 +1393,19 @@ Misc.tests.cpp:<line number>: FAILED:
|
||||
with expansion:
|
||||
1 == 2
|
||||
|
||||
a!
|
||||
b1!
|
||||
-------------------------------------------------------------------------------
|
||||
nested sections can be skipped dynamically at runtime
|
||||
B
|
||||
B2
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
!
|
||||
-------------------------------------------------------------------------------
|
||||
not prints unscoped info from previous failures
|
||||
-------------------------------------------------------------------------------
|
||||
@ -1338,6 +1440,15 @@ Message.tests.cpp:<line number>: FAILED:
|
||||
with message:
|
||||
this SHOULD be seen only ONCE
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
sections can be skipped dynamically at runtime
|
||||
skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
send a single char to INFO
|
||||
-------------------------------------------------------------------------------
|
||||
@ -1361,6 +1472,16 @@ with messages:
|
||||
hi
|
||||
i := 7
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
skipped tests can optionally provide a reason
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
explicitly with message:
|
||||
skipping because answer = 43
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
stacks unscoped info in loops
|
||||
-------------------------------------------------------------------------------
|
||||
@ -1383,6 +1504,14 @@ with messages:
|
||||
5
|
||||
6
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
tests can be skipped dynamically at runtime
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
thrown std::strings are translated
|
||||
-------------------------------------------------------------------------------
|
||||
@ -1394,6 +1523,6 @@ due to unexpected exception with message:
|
||||
Why would you throw a std::string?
|
||||
|
||||
===============================================================================
|
||||
test cases: 395 | 319 passed | 69 failed | 7 failed as expected
|
||||
assertions: 2148 | 1993 passed | 128 failed | 27 failed as expected
|
||||
test cases: 404 | 319 passed | 69 failed | 6 skipped | 10 failed as expected
|
||||
assertions: 2156 | 1997 passed | 128 failed | 31 failed as expected
|
||||
|
||||
|
||||
@ -14659,6 +14659,16 @@ with expansion:
|
||||
<Element1 attr1="true" attr2="false"/>
|
||||
" ( contains: "attr1="true"" and contains: "attr2="false"" )
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
a succeeding test can still be skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: PASSED:
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
analyse no analysis
|
||||
-------------------------------------------------------------------------------
|
||||
@ -15204,6 +15214,34 @@ FloatingPoint.tests.cpp:<line number>: PASSED:
|
||||
with expansion:
|
||||
1 == 1
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
dynamic skipping works with generators
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
explicitly with message:
|
||||
skipping because answer = 41
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
dynamic skipping works with generators
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: PASSED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
dynamic skipping works with generators
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
explicitly with message:
|
||||
skipping because answer = 43
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
empty tags are not allowed
|
||||
-------------------------------------------------------------------------------
|
||||
@ -15279,6 +15317,67 @@ Misc.tests.cpp:<line number>
|
||||
|
||||
Misc.tests.cpp:<line number>: PASSED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failed assertions before SKIP cause test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
CHECK( 3 == 4 )
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing in some unskipped sections causes entire test case to fail
|
||||
skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing in some unskipped sections causes entire test case to fail
|
||||
not skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
first tag
|
||||
-------------------------------------------------------------------------------
|
||||
@ -15775,6 +15874,40 @@ Misc.tests.cpp:<line number>: PASSED:
|
||||
with expansion:
|
||||
1 != 2
|
||||
|
||||
a-------------------------------------------------------------------------------
|
||||
nested sections can be skipped dynamically at runtime
|
||||
A
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
|
||||
No assertions in section 'A'
|
||||
|
||||
!
|
||||
b1-------------------------------------------------------------------------------
|
||||
nested sections can be skipped dynamically at runtime
|
||||
B
|
||||
B1
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
|
||||
No assertions in section 'B1'
|
||||
|
||||
!
|
||||
-------------------------------------------------------------------------------
|
||||
nested sections can be skipped dynamically at runtime
|
||||
B
|
||||
B2
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
!
|
||||
-------------------------------------------------------------------------------
|
||||
non streamable - with conv. op
|
||||
-------------------------------------------------------------------------------
|
||||
@ -16376,6 +16509,33 @@ Misc.tests.cpp:<line number>
|
||||
|
||||
No assertions in test case 'second tag'
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
sections can be skipped dynamically at runtime
|
||||
not skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: PASSED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
sections can be skipped dynamically at runtime
|
||||
skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
sections can be skipped dynamically at runtime
|
||||
also not skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: PASSED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
send a single char to INFO
|
||||
-------------------------------------------------------------------------------
|
||||
@ -16410,6 +16570,16 @@ Tag.tests.cpp:<line number>: PASSED:
|
||||
with expansion:
|
||||
{ {?}, {?} } ( Contains: {?} and Contains: {?} )
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
skipped tests can optionally provide a reason
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
explicitly with message:
|
||||
skipping because answer = 43
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
splitString
|
||||
-------------------------------------------------------------------------------
|
||||
@ -16837,6 +17007,14 @@ Tag.tests.cpp:<line number>: PASSED:
|
||||
with expansion:
|
||||
magic.tag == magic.tag
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
tests can be skipped dynamically at runtime
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
thrown std::strings are translated
|
||||
-------------------------------------------------------------------------------
|
||||
@ -17544,6 +17722,6 @@ Misc.tests.cpp:<line number>
|
||||
Misc.tests.cpp:<line number>: PASSED:
|
||||
|
||||
===============================================================================
|
||||
test cases: 395 | 305 passed | 83 failed | 7 failed as expected
|
||||
assertions: 2163 | 1993 passed | 143 failed | 27 failed as expected
|
||||
test cases: 404 | 305 passed | 84 failed | 5 skipped | 10 failed as expected
|
||||
assertions: 2173 | 1997 passed | 145 failed | 31 failed as expected
|
||||
|
||||
|
||||
@ -14652,6 +14652,16 @@ with expansion:
|
||||
<Element1 attr1="true" attr2="false"/>
|
||||
" ( contains: "attr1="true"" and contains: "attr2="false"" )
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
a succeeding test can still be skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: PASSED:
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
analyse no analysis
|
||||
-------------------------------------------------------------------------------
|
||||
@ -15197,6 +15207,34 @@ FloatingPoint.tests.cpp:<line number>: PASSED:
|
||||
with expansion:
|
||||
1 == 1
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
dynamic skipping works with generators
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
explicitly with message:
|
||||
skipping because answer = 41
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
dynamic skipping works with generators
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: PASSED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
dynamic skipping works with generators
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
explicitly with message:
|
||||
skipping because answer = 43
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
empty tags are not allowed
|
||||
-------------------------------------------------------------------------------
|
||||
@ -15272,6 +15310,67 @@ Misc.tests.cpp:<line number>
|
||||
|
||||
Misc.tests.cpp:<line number>: PASSED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failed assertions before SKIP cause test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
CHECK( 3 == 4 )
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing for some generator values causes entire test case to fail
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing in some unskipped sections causes entire test case to fail
|
||||
skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
failing in some unskipped sections causes entire test case to fail
|
||||
not skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: FAILED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
first tag
|
||||
-------------------------------------------------------------------------------
|
||||
@ -15767,6 +15866,37 @@ Misc.tests.cpp:<line number>: PASSED:
|
||||
with expansion:
|
||||
1 != 2
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
nested sections can be skipped dynamically at runtime
|
||||
A
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
|
||||
No assertions in section 'A'
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
nested sections can be skipped dynamically at runtime
|
||||
B
|
||||
B1
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
|
||||
No assertions in section 'B1'
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
nested sections can be skipped dynamically at runtime
|
||||
B
|
||||
B2
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
non streamable - with conv. op
|
||||
-------------------------------------------------------------------------------
|
||||
@ -16368,6 +16498,33 @@ Misc.tests.cpp:<line number>
|
||||
|
||||
No assertions in test case 'second tag'
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
sections can be skipped dynamically at runtime
|
||||
not skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: PASSED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
sections can be skipped dynamically at runtime
|
||||
skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
sections can be skipped dynamically at runtime
|
||||
also not skipped
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: PASSED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
send a single char to INFO
|
||||
-------------------------------------------------------------------------------
|
||||
@ -16402,6 +16559,16 @@ Tag.tests.cpp:<line number>: PASSED:
|
||||
with expansion:
|
||||
{ {?}, {?} } ( Contains: {?} and Contains: {?} )
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
skipped tests can optionally provide a reason
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
explicitly with message:
|
||||
skipping because answer = 43
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
splitString
|
||||
-------------------------------------------------------------------------------
|
||||
@ -16829,6 +16996,14 @@ Tag.tests.cpp:<line number>: PASSED:
|
||||
with expansion:
|
||||
magic.tag == magic.tag
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
tests can be skipped dynamically at runtime
|
||||
-------------------------------------------------------------------------------
|
||||
Skip.tests.cpp:<line number>
|
||||
...............................................................................
|
||||
|
||||
Skip.tests.cpp:<line number>: SKIPPED:
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
thrown std::strings are translated
|
||||
-------------------------------------------------------------------------------
|
||||
@ -17536,6 +17711,6 @@ Misc.tests.cpp:<line number>
|
||||
Misc.tests.cpp:<line number>: PASSED:
|
||||
|
||||
===============================================================================
|
||||
test cases: 395 | 305 passed | 83 failed | 7 failed as expected
|
||||
assertions: 2163 | 1993 passed | 143 failed | 27 failed as expected
|
||||
test cases: 404 | 305 passed | 84 failed | 5 skipped | 10 failed as expected
|
||||
assertions: 2173 | 1997 passed | 145 failed | 31 failed as expected
|
||||
|
||||
|
||||
@ -6,3 +6,6 @@ A string sent to stderr via clog
|
||||
Message from section one
|
||||
Message from section two
|
||||
loose text artifact
|
||||
a!
|
||||
b1!
|
||||
!
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<testsuitesloose text artifact
|
||||
>
|
||||
<testsuite name="<exe-name>" errors="17" failures="126" tests="2163" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
|
||||
<testsuite name="<exe-name>" errors="17" failures="128" skipped="11" tests="2184" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
|
||||
<properties>
|
||||
<property name="random-seed" value="1"/>
|
||||
<property name="filters" value=""*" ~[!nonportable] ~[!benchmark] ~[approvals]"/>
|
||||
@ -1591,6 +1591,12 @@ at Exception.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="XmlEncode/string with control char (1)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="XmlEncode/string with control char (x7F)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="XmlWriter writes boolean attributes as true/false" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="a succeeding test can still be skipped" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="analyse no analysis" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="array<int, N> -> toString" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="benchmark function call/without chronometer" time="{duration}" status="run"/>
|
||||
@ -1625,12 +1631,67 @@ at Misc.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="comparisons between const int variables" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="comparisons between int variables" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="convertToBits" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="dynamic skipping works with generators" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
skipping because answer = 41
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
skipping because answer = 43
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="empty tags are not allowed" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="erfc_inv" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="estimate_clock_resolution" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="even more nested SECTION tests/c/d (leaf)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="even more nested SECTION tests/c/e (leaf)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="even more nested SECTION tests/f (leaf)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="failed assertions before SKIP cause test case to fail" time="{duration}" status="run">
|
||||
<skipped message="TEST_CASE tagged with !mayfail"/>
|
||||
<failure message="3 == 4" type="CHECK">
|
||||
FAILED:
|
||||
CHECK( 3 == 4 )
|
||||
at Skip.tests.cpp:<line number>
|
||||
</failure>
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="failing for some generator values causes entire test case to fail" time="{duration}" status="run">
|
||||
<failure type="FAIL">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</failure>
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<failure type="FAIL">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</failure>
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="failing in some unskipped sections causes entire test case to fail/skipped" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="failing in some unskipped sections causes entire test case to fail/not skipped" time="{duration}" status="run">
|
||||
<skipped message="TEST_CASE tagged with !mayfail"/>
|
||||
<failure type="FAIL">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</failure>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="is_unary_function" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="just failure" time="{duration}" status="run">
|
||||
<failure type="FAIL">
|
||||
@ -1743,6 +1804,19 @@ at Misc.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="more nested SECTION tests/doesn't equal/less than" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="nested SECTION tests/doesn't equal" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="nested SECTION tests/doesn't equal/not equal" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="nested sections can be skipped dynamically at runtime/B2/B" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="nested sections can be skipped dynamically at runtime/B" time="{duration}" status="run">
|
||||
<system-out>
|
||||
a!
|
||||
b1!
|
||||
!
|
||||
</system-out>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="non streamable - with conv. op" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="non-copyable objects" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="normal_cdf" time="{duration}" status="run"/>
|
||||
@ -1794,6 +1868,14 @@ at Message.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="resolution" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="run_for_at_least, chronometer" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="run_for_at_least, int" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="sections can be skipped dynamically at runtime/not skipped" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="sections can be skipped dynamically at runtime/skipped" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="sections can be skipped dynamically at runtime/also not skipped" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="send a single char to INFO" time="{duration}" status="run">
|
||||
<failure message="false" type="REQUIRE">
|
||||
FAILED:
|
||||
@ -1812,6 +1894,13 @@ at Message.tests.cpp:<line number>
|
||||
</failure>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="shortened hide tags are split apart" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="skipped tests can optionally provide a reason" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
skipping because answer = 43
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="splitString" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="stacks unscoped info in loops" time="{duration}" status="run">
|
||||
<failure message="false" type="CHECK">
|
||||
@ -1856,6 +1945,12 @@ at Message.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="strlen3" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="tables" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="tags with dots in later positions are not parsed as hidden" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="tests can be skipped dynamically at runtime" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="thrown std::strings are translated" time="{duration}" status="run">
|
||||
<error type="TEST_CASE">
|
||||
FAILED:
|
||||
@ -1905,6 +2000,9 @@ This would not be caught previously
|
||||
A string sent directly to stdout
|
||||
Message from section one
|
||||
Message from section two
|
||||
a!
|
||||
b1!
|
||||
!
|
||||
</system-out>
|
||||
<system-err>
|
||||
Nor would this
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<testsuites>
|
||||
<testsuite name="<exe-name>" errors="17" failures="126" tests="2163" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
|
||||
<testsuite name="<exe-name>" errors="17" failures="128" skipped="11" tests="2184" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
|
||||
<properties>
|
||||
<property name="random-seed" value="1"/>
|
||||
<property name="filters" value=""*" ~[!nonportable] ~[!benchmark] ~[approvals]"/>
|
||||
@ -1590,6 +1590,12 @@ at Exception.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="XmlEncode/string with control char (1)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="XmlEncode/string with control char (x7F)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="XmlWriter writes boolean attributes as true/false" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="a succeeding test can still be skipped" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="analyse no analysis" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="array<int, N> -> toString" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="benchmark function call/without chronometer" time="{duration}" status="run"/>
|
||||
@ -1624,12 +1630,67 @@ at Misc.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="comparisons between const int variables" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="comparisons between int variables" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="convertToBits" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="dynamic skipping works with generators" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
skipping because answer = 41
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
skipping because answer = 43
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="empty tags are not allowed" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="erfc_inv" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="estimate_clock_resolution" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="even more nested SECTION tests/c/d (leaf)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="even more nested SECTION tests/c/e (leaf)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="even more nested SECTION tests/f (leaf)" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="failed assertions before SKIP cause test case to fail" time="{duration}" status="run">
|
||||
<skipped message="TEST_CASE tagged with !mayfail"/>
|
||||
<failure message="3 == 4" type="CHECK">
|
||||
FAILED:
|
||||
CHECK( 3 == 4 )
|
||||
at Skip.tests.cpp:<line number>
|
||||
</failure>
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="failing for some generator values causes entire test case to fail" time="{duration}" status="run">
|
||||
<failure type="FAIL">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</failure>
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<failure type="FAIL">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</failure>
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="failing in some unskipped sections causes entire test case to fail/skipped" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="failing in some unskipped sections causes entire test case to fail/not skipped" time="{duration}" status="run">
|
||||
<skipped message="TEST_CASE tagged with !mayfail"/>
|
||||
<failure type="FAIL">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</failure>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="is_unary_function" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="just failure" time="{duration}" status="run">
|
||||
<failure type="FAIL">
|
||||
@ -1742,6 +1803,19 @@ at Misc.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="more nested SECTION tests/doesn't equal/less than" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="nested SECTION tests/doesn't equal" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="nested SECTION tests/doesn't equal/not equal" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="nested sections can be skipped dynamically at runtime/B2/B" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="nested sections can be skipped dynamically at runtime/B" time="{duration}" status="run">
|
||||
<system-out>
|
||||
a!
|
||||
b1!
|
||||
!
|
||||
</system-out>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="non streamable - with conv. op" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="non-copyable objects" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="normal_cdf" time="{duration}" status="run"/>
|
||||
@ -1793,6 +1867,14 @@ at Message.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="resolution" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="run_for_at_least, chronometer" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="run_for_at_least, int" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="sections can be skipped dynamically at runtime/not skipped" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="sections can be skipped dynamically at runtime/skipped" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="sections can be skipped dynamically at runtime/also not skipped" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="send a single char to INFO" time="{duration}" status="run">
|
||||
<failure message="false" type="REQUIRE">
|
||||
FAILED:
|
||||
@ -1811,6 +1893,13 @@ at Message.tests.cpp:<line number>
|
||||
</failure>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="shortened hide tags are split apart" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="skipped tests can optionally provide a reason" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
skipping because answer = 43
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="splitString" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="stacks unscoped info in loops" time="{duration}" status="run">
|
||||
<failure message="false" type="CHECK">
|
||||
@ -1855,6 +1944,12 @@ at Message.tests.cpp:<line number>
|
||||
<testcase classname="<exe-name>.global" name="strlen3" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="tables" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="tags with dots in later positions are not parsed as hidden" time="{duration}" status="run"/>
|
||||
<testcase classname="<exe-name>.global" name="tests can be skipped dynamically at runtime" time="{duration}" status="run">
|
||||
<skipped type="SKIP">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testcase>
|
||||
<testcase classname="<exe-name>.global" name="thrown std::strings are translated" time="{duration}" status="run">
|
||||
<error type="TEST_CASE">
|
||||
FAILED:
|
||||
@ -1904,6 +1999,9 @@ This would not be caught previously
|
||||
A string sent directly to stdout
|
||||
Message from section one
|
||||
Message from section two
|
||||
a!
|
||||
b1!
|
||||
!
|
||||
</system-out>
|
||||
<system-err>
|
||||
Nor would this
|
||||
|
||||
@ -1832,6 +1832,95 @@ at Misc.tests.cpp:<line number>
|
||||
<testCase name="xmlentitycheck/embedded xml: <test>it should be possible to embed xml characters, such as <, " or &, or even whole <xml>documents</xml> within an attribute</test>" duration="{duration}"/>
|
||||
<testCase name="xmlentitycheck/encoded chars: these should all be encoded: &&&"""<<<&"<<&"" duration="{duration}"/>
|
||||
</file>
|
||||
<file path="tests/<exe-name>/UsageTests/Skip.tests.cpp">
|
||||
<testCase name="a succeeding test can still be skipped" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="dynamic skipping works with generators" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
skipping because answer = 41
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
skipping because answer = 43
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="failed assertions before SKIP cause test case to fail" duration="{duration}">
|
||||
<skipped message="CHECK(3 == 4)">
|
||||
FAILED:
|
||||
CHECK( 3 == 4 )
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="failing for some generator values causes entire test case to fail" duration="{duration}">
|
||||
<skipped message="FAIL()">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="FAIL()">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="failing in some unskipped sections causes entire test case to fail/skipped" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="failing in some unskipped sections causes entire test case to fail/not skipped" duration="{duration}">
|
||||
<skipped message="FAIL()">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="nested sections can be skipped dynamically at runtime/B2/B" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="nested sections can be skipped dynamically at runtime/B" duration="{duration}"/>
|
||||
<testCase name="sections can be skipped dynamically at runtime/not skipped" duration="{duration}"/>
|
||||
<testCase name="sections can be skipped dynamically at runtime/skipped" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="sections can be skipped dynamically at runtime/also not skipped" duration="{duration}"/>
|
||||
<testCase name="skipped tests can optionally provide a reason" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
skipping because answer = 43
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="tests can be skipped dynamically at runtime" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
</file>
|
||||
<file path="tests/<exe-name>/UsageTests/ToStringChrono.tests.cpp">
|
||||
<testCase name="Stringifying std::chrono::duration helpers" duration="{duration}"/>
|
||||
<testCase name="Stringifying std::chrono::duration with weird ratios" duration="{duration}"/>
|
||||
|
||||
@ -1831,6 +1831,95 @@ at Misc.tests.cpp:<line number>
|
||||
<testCase name="xmlentitycheck/embedded xml: <test>it should be possible to embed xml characters, such as <, " or &, or even whole <xml>documents</xml> within an attribute</test>" duration="{duration}"/>
|
||||
<testCase name="xmlentitycheck/encoded chars: these should all be encoded: &&&"""<<<&"<<&"" duration="{duration}"/>
|
||||
</file>
|
||||
<file path="tests/<exe-name>/UsageTests/Skip.tests.cpp">
|
||||
<testCase name="a succeeding test can still be skipped" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="dynamic skipping works with generators" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
skipping because answer = 41
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
skipping because answer = 43
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="failed assertions before SKIP cause test case to fail" duration="{duration}">
|
||||
<skipped message="CHECK(3 == 4)">
|
||||
FAILED:
|
||||
CHECK( 3 == 4 )
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="failing for some generator values causes entire test case to fail" duration="{duration}">
|
||||
<skipped message="FAIL()">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="FAIL()">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="failing in some unskipped sections causes entire test case to fail/skipped" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="failing in some unskipped sections causes entire test case to fail/not skipped" duration="{duration}">
|
||||
<skipped message="FAIL()">
|
||||
FAILED:
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="nested sections can be skipped dynamically at runtime/B2/B" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="nested sections can be skipped dynamically at runtime/B" duration="{duration}"/>
|
||||
<testCase name="sections can be skipped dynamically at runtime/not skipped" duration="{duration}"/>
|
||||
<testCase name="sections can be skipped dynamically at runtime/skipped" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="sections can be skipped dynamically at runtime/also not skipped" duration="{duration}"/>
|
||||
<testCase name="skipped tests can optionally provide a reason" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
skipping because answer = 43
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
<testCase name="tests can be skipped dynamically at runtime" duration="{duration}">
|
||||
<skipped message="SKIP()">
|
||||
SKIPPED
|
||||
at Skip.tests.cpp:<line number>
|
||||
</skipped>
|
||||
</testCase>
|
||||
</file>
|
||||
<file path="tests/<exe-name>/UsageTests/ToStringChrono.tests.cpp">
|
||||
<testCase name="Stringifying std::chrono::duration helpers" duration="{duration}"/>
|
||||
<testCase name="Stringifying std::chrono::duration with weird ratios" duration="{duration}"/>
|
||||
|
||||
@ -3609,6 +3609,10 @@ ok {test-number} - encode( "[\x01]" ) == "[\\x01]" for: "[\x01]" == "[\x01]"
|
||||
ok {test-number} - encode( "[\x7F]" ) == "[\\x7F]" for: "[\x7F]" == "[\x7F]"
|
||||
# XmlWriter writes boolean attributes as true/false
|
||||
ok {test-number} - stream.str(), ContainsSubstring(R"(attr1="true")") && ContainsSubstring(R"(attr2="false")") for: "<?xml version="1.0" encoding="UTF-8"?> <Element1 attr1="true" attr2="false"/> " ( contains: "attr1="true"" and contains: "attr2="false"" )
|
||||
# a succeeding test can still be skipped
|
||||
ok {test-number} -
|
||||
# a succeeding test can still be skipped
|
||||
ok {test-number} - # SKIP
|
||||
# analyse no analysis
|
||||
ok {test-number} - analysis.mean.point.count() == 23 for: 23.0 == 23
|
||||
# analyse no analysis
|
||||
@ -3779,6 +3783,12 @@ ok {test-number} - convertToBits( -0. ) == ( 1ULL << 63 ) for: 92233720368547758
|
||||
ok {test-number} - convertToBits( std::numeric_limits<float>::denorm_min() ) == 1 for: 1 == 1
|
||||
# convertToBits
|
||||
ok {test-number} - convertToBits( std::numeric_limits<double>::denorm_min() ) == 1 for: 1 == 1
|
||||
# dynamic skipping works with generators
|
||||
ok {test-number} - # SKIP 'skipping because answer = 41'
|
||||
# dynamic skipping works with generators
|
||||
ok {test-number} -
|
||||
# dynamic skipping works with generators
|
||||
ok {test-number} - # SKIP 'skipping because answer = 43'
|
||||
# empty tags are not allowed
|
||||
ok {test-number} - Catch::TestCaseInfo("", { "test with an empty tag", "[]" }, dummySourceLineInfo)
|
||||
# erfc_inv
|
||||
@ -3797,6 +3807,22 @@ ok {test-number} -
|
||||
ok {test-number} -
|
||||
# even more nested SECTION tests
|
||||
ok {test-number} -
|
||||
# failed assertions before SKIP cause test case to fail
|
||||
not ok {test-number} - 3 == 4
|
||||
# failed assertions before SKIP cause test case to fail
|
||||
ok {test-number} - # SKIP
|
||||
# failing for some generator values causes entire test case to fail
|
||||
not ok {test-number} - explicitly
|
||||
# failing for some generator values causes entire test case to fail
|
||||
ok {test-number} - # SKIP
|
||||
# failing for some generator values causes entire test case to fail
|
||||
not ok {test-number} - explicitly
|
||||
# failing for some generator values causes entire test case to fail
|
||||
ok {test-number} - # SKIP
|
||||
# failing in some unskipped sections causes entire test case to fail
|
||||
ok {test-number} - # SKIP
|
||||
# failing in some unskipped sections causes entire test case to fail
|
||||
not ok {test-number} - explicitly
|
||||
loose text artifact
|
||||
# is_unary_function
|
||||
ok {test-number} - with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary1)>::value'
|
||||
@ -3906,6 +3932,11 @@ ok {test-number} - a != b for: 1 != 2
|
||||
ok {test-number} - b != a for: 2 != 1
|
||||
# nested SECTION tests
|
||||
ok {test-number} - a != b for: 1 != 2
|
||||
a!
|
||||
b1!
|
||||
# nested sections can be skipped dynamically at runtime
|
||||
ok {test-number} - # SKIP
|
||||
!
|
||||
# non streamable - with conv. op
|
||||
ok {test-number} - s == "7" for: "7" == "7"
|
||||
# non-copyable objects
|
||||
@ -4070,12 +4101,20 @@ ok {test-number} - Timing.elapsed >= time for: 128 ns >= 100 ns
|
||||
ok {test-number} - Timing.result == Timing.iterations + 17 for: 145 == 145
|
||||
# run_for_at_least, int
|
||||
ok {test-number} - Timing.iterations >= time.count() for: 128 >= 100
|
||||
# sections can be skipped dynamically at runtime
|
||||
ok {test-number} -
|
||||
# sections can be skipped dynamically at runtime
|
||||
ok {test-number} - # SKIP
|
||||
# sections can be skipped dynamically at runtime
|
||||
ok {test-number} -
|
||||
# send a single char to INFO
|
||||
not ok {test-number} - false with 1 message: '3'
|
||||
# sends information to INFO
|
||||
not ok {test-number} - false with 2 messages: 'hi' and 'i := 7'
|
||||
# shortened hide tags are split apart
|
||||
ok {test-number} - testcase.tags, VectorContains( Tag( "magic-tag" ) ) && VectorContains( Tag( "."_catch_sr ) ) for: { {?}, {?} } ( Contains: {?} and Contains: {?} )
|
||||
# skipped tests can optionally provide a reason
|
||||
ok {test-number} - # SKIP 'skipping because answer = 43'
|
||||
# splitString
|
||||
ok {test-number} - splitStringRef("", ','), Equals(std::vector<StringRef>()) for: { } Equals: { }
|
||||
# splitString
|
||||
@ -4158,6 +4197,8 @@ ok {test-number} - strlen(std::get<0>(data)) == static_cast<size_t>(std::get<1>(
|
||||
ok {test-number} - testcase.tags.size() == 1 for: 1 == 1
|
||||
# tags with dots in later positions are not parsed as hidden
|
||||
ok {test-number} - testcase.tags[0].original == "magic.tag"_catch_sr for: magic.tag == magic.tag
|
||||
# tests can be skipped dynamically at runtime
|
||||
ok {test-number} - # SKIP
|
||||
# thrown std::strings are translated
|
||||
not ok {test-number} - unexpected exception with message: 'Why would you throw a std::string?'
|
||||
# toString on const wchar_t const pointer returns the string contents
|
||||
@ -4330,5 +4371,5 @@ ok {test-number} - q3 == 23. for: 23.0 == 23.0
|
||||
ok {test-number} -
|
||||
# xmlentitycheck
|
||||
ok {test-number} -
|
||||
1..2163
|
||||
1..2184
|
||||
|
||||
|
||||
@ -3602,6 +3602,10 @@ ok {test-number} - encode( "[\x01]" ) == "[\\x01]" for: "[\x01]" == "[\x01]"
|
||||
ok {test-number} - encode( "[\x7F]" ) == "[\\x7F]" for: "[\x7F]" == "[\x7F]"
|
||||
# XmlWriter writes boolean attributes as true/false
|
||||
ok {test-number} - stream.str(), ContainsSubstring(R"(attr1="true")") && ContainsSubstring(R"(attr2="false")") for: "<?xml version="1.0" encoding="UTF-8"?> <Element1 attr1="true" attr2="false"/> " ( contains: "attr1="true"" and contains: "attr2="false"" )
|
||||
# a succeeding test can still be skipped
|
||||
ok {test-number} -
|
||||
# a succeeding test can still be skipped
|
||||
ok {test-number} - # SKIP
|
||||
# analyse no analysis
|
||||
ok {test-number} - analysis.mean.point.count() == 23 for: 23.0 == 23
|
||||
# analyse no analysis
|
||||
@ -3772,6 +3776,12 @@ ok {test-number} - convertToBits( -0. ) == ( 1ULL << 63 ) for: 92233720368547758
|
||||
ok {test-number} - convertToBits( std::numeric_limits<float>::denorm_min() ) == 1 for: 1 == 1
|
||||
# convertToBits
|
||||
ok {test-number} - convertToBits( std::numeric_limits<double>::denorm_min() ) == 1 for: 1 == 1
|
||||
# dynamic skipping works with generators
|
||||
ok {test-number} - # SKIP 'skipping because answer = 41'
|
||||
# dynamic skipping works with generators
|
||||
ok {test-number} -
|
||||
# dynamic skipping works with generators
|
||||
ok {test-number} - # SKIP 'skipping because answer = 43'
|
||||
# empty tags are not allowed
|
||||
ok {test-number} - Catch::TestCaseInfo("", { "test with an empty tag", "[]" }, dummySourceLineInfo)
|
||||
# erfc_inv
|
||||
@ -3790,6 +3800,22 @@ ok {test-number} -
|
||||
ok {test-number} -
|
||||
# even more nested SECTION tests
|
||||
ok {test-number} -
|
||||
# failed assertions before SKIP cause test case to fail
|
||||
not ok {test-number} - 3 == 4
|
||||
# failed assertions before SKIP cause test case to fail
|
||||
ok {test-number} - # SKIP
|
||||
# failing for some generator values causes entire test case to fail
|
||||
not ok {test-number} - explicitly
|
||||
# failing for some generator values causes entire test case to fail
|
||||
ok {test-number} - # SKIP
|
||||
# failing for some generator values causes entire test case to fail
|
||||
not ok {test-number} - explicitly
|
||||
# failing for some generator values causes entire test case to fail
|
||||
ok {test-number} - # SKIP
|
||||
# failing in some unskipped sections causes entire test case to fail
|
||||
ok {test-number} - # SKIP
|
||||
# failing in some unskipped sections causes entire test case to fail
|
||||
not ok {test-number} - explicitly
|
||||
# is_unary_function
|
||||
ok {test-number} - with 1 message: 'Catch::Clara::Detail::is_unary_function<decltype(unary1)>::value'
|
||||
# is_unary_function
|
||||
@ -3898,6 +3924,8 @@ ok {test-number} - a != b for: 1 != 2
|
||||
ok {test-number} - b != a for: 2 != 1
|
||||
# nested SECTION tests
|
||||
ok {test-number} - a != b for: 1 != 2
|
||||
# nested sections can be skipped dynamically at runtime
|
||||
ok {test-number} - # SKIP
|
||||
# non streamable - with conv. op
|
||||
ok {test-number} - s == "7" for: "7" == "7"
|
||||
# non-copyable objects
|
||||
@ -4062,12 +4090,20 @@ ok {test-number} - Timing.elapsed >= time for: 128 ns >= 100 ns
|
||||
ok {test-number} - Timing.result == Timing.iterations + 17 for: 145 == 145
|
||||
# run_for_at_least, int
|
||||
ok {test-number} - Timing.iterations >= time.count() for: 128 >= 100
|
||||
# sections can be skipped dynamically at runtime
|
||||
ok {test-number} -
|
||||
# sections can be skipped dynamically at runtime
|
||||
ok {test-number} - # SKIP
|
||||
# sections can be skipped dynamically at runtime
|
||||
ok {test-number} -
|
||||
# send a single char to INFO
|
||||
not ok {test-number} - false with 1 message: '3'
|
||||
# sends information to INFO
|
||||
not ok {test-number} - false with 2 messages: 'hi' and 'i := 7'
|
||||
# shortened hide tags are split apart
|
||||
ok {test-number} - testcase.tags, VectorContains( Tag( "magic-tag" ) ) && VectorContains( Tag( "."_catch_sr ) ) for: { {?}, {?} } ( Contains: {?} and Contains: {?} )
|
||||
# skipped tests can optionally provide a reason
|
||||
ok {test-number} - # SKIP 'skipping because answer = 43'
|
||||
# splitString
|
||||
ok {test-number} - splitStringRef("", ','), Equals(std::vector<StringRef>()) for: { } Equals: { }
|
||||
# splitString
|
||||
@ -4150,6 +4186,8 @@ ok {test-number} - strlen(std::get<0>(data)) == static_cast<size_t>(std::get<1>(
|
||||
ok {test-number} - testcase.tags.size() == 1 for: 1 == 1
|
||||
# tags with dots in later positions are not parsed as hidden
|
||||
ok {test-number} - testcase.tags[0].original == "magic.tag"_catch_sr for: magic.tag == magic.tag
|
||||
# tests can be skipped dynamically at runtime
|
||||
ok {test-number} - # SKIP
|
||||
# thrown std::strings are translated
|
||||
not ok {test-number} - unexpected exception with message: 'Why would you throw a std::string?'
|
||||
# toString on const wchar_t const pointer returns the string contents
|
||||
@ -4322,5 +4360,5 @@ ok {test-number} - q3 == 23. for: 23.0 == 23.0
|
||||
ok {test-number} -
|
||||
# xmlentitycheck
|
||||
ok {test-number} -
|
||||
1..2163
|
||||
1..2184
|
||||
|
||||
|
||||
@ -722,6 +722,9 @@
|
||||
##teamcity[testFinished name='XmlEncode' duration="{duration}"]
|
||||
##teamcity[testStarted name='XmlWriter writes boolean attributes as true/false']
|
||||
##teamcity[testFinished name='XmlWriter writes boolean attributes as true/false' duration="{duration}"]
|
||||
##teamcity[testStarted name='a succeeding test can still be skipped']
|
||||
##teamcity[testIgnored name='a succeeding test can still be skipped' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='a succeeding test can still be skipped' duration="{duration}"]
|
||||
##teamcity[testStarted name='analyse no analysis']
|
||||
##teamcity[testFinished name='analyse no analysis' duration="{duration}"]
|
||||
##teamcity[testStarted name='array<int, N> -> toString']
|
||||
@ -748,6 +751,10 @@
|
||||
##teamcity[testFinished name='comparisons between int variables' duration="{duration}"]
|
||||
##teamcity[testStarted name='convertToBits']
|
||||
##teamcity[testFinished name='convertToBits' duration="{duration}"]
|
||||
##teamcity[testStarted name='dynamic skipping works with generators']
|
||||
##teamcity[testIgnored name='dynamic skipping works with generators' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 41"']
|
||||
##teamcity[testIgnored name='dynamic skipping works with generators' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 43"']
|
||||
##teamcity[testFinished name='dynamic skipping works with generators' duration="{duration}"]
|
||||
##teamcity[testStarted name='empty tags are not allowed']
|
||||
##teamcity[testFinished name='empty tags are not allowed' duration="{duration}"]
|
||||
##teamcity[testStarted name='erfc_inv']
|
||||
@ -756,6 +763,20 @@
|
||||
##teamcity[testFinished name='estimate_clock_resolution' duration="{duration}"]
|
||||
##teamcity[testStarted name='even more nested SECTION tests']
|
||||
##teamcity[testFinished name='even more nested SECTION tests' duration="{duration}"]
|
||||
##teamcity[testStarted name='failed assertions before SKIP cause test case to fail']
|
||||
##teamcity[testIgnored name='failed assertions before SKIP cause test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexpression failed|n CHECK( 3 == 4 )|nwith expansion:|n 3 == 4|n- failure ignore as test marked as |'ok to fail|'|n']
|
||||
##teamcity[testIgnored name='failed assertions before SKIP cause test case to fail' message='Skip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='failed assertions before SKIP cause test case to fail' duration="{duration}"]
|
||||
##teamcity[testStarted name='failing for some generator values causes entire test case to fail']
|
||||
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
|
||||
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
|
||||
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='failing for some generator values causes entire test case to fail' duration="{duration}"]
|
||||
##teamcity[testStarted name='failing in some unskipped sections causes entire test case to fail']
|
||||
##teamcity[testIgnored name='failing in some unskipped sections causes entire test case to fail' message='-------------------------------------------------------------------------------|nskipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testIgnored name='failing in some unskipped sections causes entire test case to fail' message='-------------------------------------------------------------------------------|nnot skipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
|
||||
##teamcity[testFinished name='failing in some unskipped sections causes entire test case to fail' duration="{duration}"]
|
||||
##teamcity[testStarted name='first tag']
|
||||
##teamcity[testFinished name='first tag' duration="{duration}"]
|
||||
##teamcity[testStarted name='has printf']
|
||||
@ -802,6 +823,10 @@ loose text artifact
|
||||
##teamcity[testFinished name='more nested SECTION tests' duration="{duration}"]
|
||||
##teamcity[testStarted name='nested SECTION tests']
|
||||
##teamcity[testFinished name='nested SECTION tests' duration="{duration}"]
|
||||
##teamcity[testStarted name='nested sections can be skipped dynamically at runtime']
|
||||
##teamcity[testIgnored name='nested sections can be skipped dynamically at runtime' message='-------------------------------------------------------------------------------|nB|nB2|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testStdOut name='nested sections can be skipped dynamically at runtime' out='a!|nb1!|n!|n']
|
||||
##teamcity[testFinished name='nested sections can be skipped dynamically at runtime' duration="{duration}"]
|
||||
##teamcity[testStarted name='non streamable - with conv. op']
|
||||
##teamcity[testFinished name='non streamable - with conv. op' duration="{duration}"]
|
||||
##teamcity[testStarted name='non-copyable objects']
|
||||
@ -847,6 +872,9 @@ loose text artifact
|
||||
##teamcity[testFinished name='run_for_at_least, int' duration="{duration}"]
|
||||
##teamcity[testStarted name='second tag']
|
||||
##teamcity[testFinished name='second tag' duration="{duration}"]
|
||||
##teamcity[testStarted name='sections can be skipped dynamically at runtime']
|
||||
##teamcity[testIgnored name='sections can be skipped dynamically at runtime' message='-------------------------------------------------------------------------------|nskipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='sections can be skipped dynamically at runtime' duration="{duration}"]
|
||||
##teamcity[testStarted name='send a single char to INFO']
|
||||
##teamcity[testFailed name='send a single char to INFO' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed with message:|n "3"|n REQUIRE( false )|nwith expansion:|n false|n']
|
||||
##teamcity[testFinished name='send a single char to INFO' duration="{duration}"]
|
||||
@ -855,6 +883,9 @@ loose text artifact
|
||||
##teamcity[testFinished name='sends information to INFO' duration="{duration}"]
|
||||
##teamcity[testStarted name='shortened hide tags are split apart']
|
||||
##teamcity[testFinished name='shortened hide tags are split apart' duration="{duration}"]
|
||||
##teamcity[testStarted name='skipped tests can optionally provide a reason']
|
||||
##teamcity[testIgnored name='skipped tests can optionally provide a reason' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 43"']
|
||||
##teamcity[testFinished name='skipped tests can optionally provide a reason' duration="{duration}"]
|
||||
##teamcity[testStarted name='splitString']
|
||||
##teamcity[testFinished name='splitString' duration="{duration}"]
|
||||
##teamcity[testStarted name='stacks unscoped info in loops']
|
||||
@ -899,6 +930,9 @@ loose text artifact
|
||||
##teamcity[testFinished name='tables' duration="{duration}"]
|
||||
##teamcity[testStarted name='tags with dots in later positions are not parsed as hidden']
|
||||
##teamcity[testFinished name='tags with dots in later positions are not parsed as hidden' duration="{duration}"]
|
||||
##teamcity[testStarted name='tests can be skipped dynamically at runtime']
|
||||
##teamcity[testIgnored name='tests can be skipped dynamically at runtime' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='tests can be skipped dynamically at runtime' duration="{duration}"]
|
||||
##teamcity[testStarted name='thrown std::strings are translated']
|
||||
##teamcity[testFailed name='thrown std::strings are translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "Why would you throw a std::string?"']
|
||||
##teamcity[testFinished name='thrown std::strings are translated' duration="{duration}"]
|
||||
|
||||
@ -722,6 +722,9 @@
|
||||
##teamcity[testFinished name='XmlEncode' duration="{duration}"]
|
||||
##teamcity[testStarted name='XmlWriter writes boolean attributes as true/false']
|
||||
##teamcity[testFinished name='XmlWriter writes boolean attributes as true/false' duration="{duration}"]
|
||||
##teamcity[testStarted name='a succeeding test can still be skipped']
|
||||
##teamcity[testIgnored name='a succeeding test can still be skipped' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='a succeeding test can still be skipped' duration="{duration}"]
|
||||
##teamcity[testStarted name='analyse no analysis']
|
||||
##teamcity[testFinished name='analyse no analysis' duration="{duration}"]
|
||||
##teamcity[testStarted name='array<int, N> -> toString']
|
||||
@ -748,6 +751,10 @@
|
||||
##teamcity[testFinished name='comparisons between int variables' duration="{duration}"]
|
||||
##teamcity[testStarted name='convertToBits']
|
||||
##teamcity[testFinished name='convertToBits' duration="{duration}"]
|
||||
##teamcity[testStarted name='dynamic skipping works with generators']
|
||||
##teamcity[testIgnored name='dynamic skipping works with generators' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 41"']
|
||||
##teamcity[testIgnored name='dynamic skipping works with generators' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 43"']
|
||||
##teamcity[testFinished name='dynamic skipping works with generators' duration="{duration}"]
|
||||
##teamcity[testStarted name='empty tags are not allowed']
|
||||
##teamcity[testFinished name='empty tags are not allowed' duration="{duration}"]
|
||||
##teamcity[testStarted name='erfc_inv']
|
||||
@ -756,6 +763,20 @@
|
||||
##teamcity[testFinished name='estimate_clock_resolution' duration="{duration}"]
|
||||
##teamcity[testStarted name='even more nested SECTION tests']
|
||||
##teamcity[testFinished name='even more nested SECTION tests' duration="{duration}"]
|
||||
##teamcity[testStarted name='failed assertions before SKIP cause test case to fail']
|
||||
##teamcity[testIgnored name='failed assertions before SKIP cause test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexpression failed|n CHECK( 3 == 4 )|nwith expansion:|n 3 == 4|n- failure ignore as test marked as |'ok to fail|'|n']
|
||||
##teamcity[testIgnored name='failed assertions before SKIP cause test case to fail' message='Skip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='failed assertions before SKIP cause test case to fail' duration="{duration}"]
|
||||
##teamcity[testStarted name='failing for some generator values causes entire test case to fail']
|
||||
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
|
||||
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
|
||||
##teamcity[testIgnored name='failing for some generator values causes entire test case to fail' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='failing for some generator values causes entire test case to fail' duration="{duration}"]
|
||||
##teamcity[testStarted name='failing in some unskipped sections causes entire test case to fail']
|
||||
##teamcity[testIgnored name='failing in some unskipped sections causes entire test case to fail' message='-------------------------------------------------------------------------------|nskipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testIgnored name='failing in some unskipped sections causes entire test case to fail' message='-------------------------------------------------------------------------------|nnot skipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit failure- failure ignore as test marked as |'ok to fail|'|n']
|
||||
##teamcity[testFinished name='failing in some unskipped sections causes entire test case to fail' duration="{duration}"]
|
||||
##teamcity[testStarted name='first tag']
|
||||
##teamcity[testFinished name='first tag' duration="{duration}"]
|
||||
##teamcity[testStarted name='has printf']
|
||||
@ -801,6 +822,10 @@
|
||||
##teamcity[testFinished name='more nested SECTION tests' duration="{duration}"]
|
||||
##teamcity[testStarted name='nested SECTION tests']
|
||||
##teamcity[testFinished name='nested SECTION tests' duration="{duration}"]
|
||||
##teamcity[testStarted name='nested sections can be skipped dynamically at runtime']
|
||||
##teamcity[testIgnored name='nested sections can be skipped dynamically at runtime' message='-------------------------------------------------------------------------------|nB|nB2|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testStdOut name='nested sections can be skipped dynamically at runtime' out='a!|nb1!|n!|n']
|
||||
##teamcity[testFinished name='nested sections can be skipped dynamically at runtime' duration="{duration}"]
|
||||
##teamcity[testStarted name='non streamable - with conv. op']
|
||||
##teamcity[testFinished name='non streamable - with conv. op' duration="{duration}"]
|
||||
##teamcity[testStarted name='non-copyable objects']
|
||||
@ -846,6 +871,9 @@
|
||||
##teamcity[testFinished name='run_for_at_least, int' duration="{duration}"]
|
||||
##teamcity[testStarted name='second tag']
|
||||
##teamcity[testFinished name='second tag' duration="{duration}"]
|
||||
##teamcity[testStarted name='sections can be skipped dynamically at runtime']
|
||||
##teamcity[testIgnored name='sections can be skipped dynamically at runtime' message='-------------------------------------------------------------------------------|nskipped|n-------------------------------------------------------------------------------|nSkip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='sections can be skipped dynamically at runtime' duration="{duration}"]
|
||||
##teamcity[testStarted name='send a single char to INFO']
|
||||
##teamcity[testFailed name='send a single char to INFO' message='Misc.tests.cpp:<line number>|n...............................................................................|n|nMisc.tests.cpp:<line number>|nexpression failed with message:|n "3"|n REQUIRE( false )|nwith expansion:|n false|n']
|
||||
##teamcity[testFinished name='send a single char to INFO' duration="{duration}"]
|
||||
@ -854,6 +882,9 @@
|
||||
##teamcity[testFinished name='sends information to INFO' duration="{duration}"]
|
||||
##teamcity[testStarted name='shortened hide tags are split apart']
|
||||
##teamcity[testFinished name='shortened hide tags are split apart' duration="{duration}"]
|
||||
##teamcity[testStarted name='skipped tests can optionally provide a reason']
|
||||
##teamcity[testIgnored name='skipped tests can optionally provide a reason' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip with message:|n "skipping because answer = 43"']
|
||||
##teamcity[testFinished name='skipped tests can optionally provide a reason' duration="{duration}"]
|
||||
##teamcity[testStarted name='splitString']
|
||||
##teamcity[testFinished name='splitString' duration="{duration}"]
|
||||
##teamcity[testStarted name='stacks unscoped info in loops']
|
||||
@ -898,6 +929,9 @@
|
||||
##teamcity[testFinished name='tables' duration="{duration}"]
|
||||
##teamcity[testStarted name='tags with dots in later positions are not parsed as hidden']
|
||||
##teamcity[testFinished name='tags with dots in later positions are not parsed as hidden' duration="{duration}"]
|
||||
##teamcity[testStarted name='tests can be skipped dynamically at runtime']
|
||||
##teamcity[testIgnored name='tests can be skipped dynamically at runtime' message='Skip.tests.cpp:<line number>|n...............................................................................|n|nSkip.tests.cpp:<line number>|nexplicit skip']
|
||||
##teamcity[testFinished name='tests can be skipped dynamically at runtime' duration="{duration}"]
|
||||
##teamcity[testStarted name='thrown std::strings are translated']
|
||||
##teamcity[testFailed name='thrown std::strings are translated' message='Exception.tests.cpp:<line number>|n...............................................................................|n|nException.tests.cpp:<line number>|nunexpected exception with message:|n "Why would you throw a std::string?"']
|
||||
##teamcity[testFinished name='thrown std::strings are translated' duration="{duration}"]
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
73
tests/SelfTest/UsageTests/Skip.tests.cpp
Normal file
73
tests/SelfTest/UsageTests/Skip.tests.cpp
Normal file
@ -0,0 +1,73 @@
|
||||
|
||||
// Copyright Catch2 Authors
|
||||
// Distributed under the Boost Software License, Version 1.0.
|
||||
// (See accompanying file LICENSE.txt or copy at
|
||||
// https://www.boost.org/LICENSE_1_0.txt)
|
||||
|
||||
// SPDX-License-Identifier: BSL-1.0
|
||||
|
||||
#include <catch2/catch_test_macros.hpp>
|
||||
#include <catch2/generators/catch_generators_range.hpp>
|
||||
|
||||
#include <iostream>
|
||||
|
||||
TEST_CASE( "tests can be skipped dynamically at runtime", "[skipping]" ) {
|
||||
SKIP();
|
||||
FAIL( "this is not reached" );
|
||||
}
|
||||
|
||||
TEST_CASE( "skipped tests can optionally provide a reason", "[skipping]" ) {
|
||||
const int answer = 43;
|
||||
SKIP( "skipping because answer = " << answer );
|
||||
FAIL( "this is not reached" );
|
||||
}
|
||||
|
||||
TEST_CASE( "sections can be skipped dynamically at runtime", "[skipping]" ) {
|
||||
SECTION( "not skipped" ) { SUCCEED(); }
|
||||
SECTION( "skipped" ) { SKIP(); }
|
||||
SECTION( "also not skipped" ) { SUCCEED(); }
|
||||
}
|
||||
|
||||
TEST_CASE( "nested sections can be skipped dynamically at runtime",
|
||||
"[skipping]" ) {
|
||||
SECTION( "A" ) { std::cout << "a"; }
|
||||
SECTION( "B" ) {
|
||||
SECTION( "B1" ) { std::cout << "b1"; }
|
||||
SECTION( "B2" ) { SKIP(); }
|
||||
}
|
||||
std::cout << "!\n";
|
||||
}
|
||||
|
||||
TEST_CASE( "dynamic skipping works with generators", "[skipping]" ) {
|
||||
const int answer = GENERATE( 41, 42, 43 );
|
||||
if ( answer != 42 ) { SKIP( "skipping because answer = " << answer ); }
|
||||
SUCCEED();
|
||||
}
|
||||
|
||||
TEST_CASE( "failed assertions before SKIP cause test case to fail",
|
||||
"[skipping][!shouldfail]" ) {
|
||||
CHECK( 3 == 4 );
|
||||
SKIP();
|
||||
}
|
||||
|
||||
TEST_CASE( "a succeeding test can still be skipped",
|
||||
"[skipping][!shouldfail]" ) {
|
||||
SUCCEED();
|
||||
SKIP();
|
||||
}
|
||||
|
||||
TEST_CASE( "failing in some unskipped sections causes entire test case to fail",
|
||||
"[skipping][!shouldfail]" ) {
|
||||
SECTION( "skipped" ) { SKIP(); }
|
||||
SECTION( "not skipped" ) { FAIL(); }
|
||||
}
|
||||
|
||||
TEST_CASE( "failing for some generator values causes entire test case to fail",
|
||||
"[skipping][!shouldfail]" ) {
|
||||
int i = GENERATE( 1, 2, 3, 4 );
|
||||
if ( i % 2 == 0 ) {
|
||||
SKIP();
|
||||
} else {
|
||||
FAIL();
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user