This fixes result disposition being ContinueOnFailure |
ContinueOnFailure for CHECK_THAT (obviously an error) and Normal |
ContinueOnFailure for REQUIRE_THAT (less obviously an error, but worse,
as that signals to the pipeline that assertion failure should both abort
and continue the test with ???? happening).
Jenkins groups junit test results by loosely interpreting the
classname attribute of the <testcase> element as a package-qualified
java class name such as java.util.String. It ignores the <testsuite>
elements in the xml. To organize test results we therefore need to
embed the suite name in the classname attribute as if it was a java
package name.
Fixes#922.
If [#filename] is present in tags, use it for the classname attribute,
rather than "global". If the test fixture is present that still takes
precedence.
Previously, some errors in Catch configuration would cause exceptions to
be thrown before main was even entered. This leads to call to
`std::terminate`, which is not a particularly nice way of ending the
binary.
Now these exceptions are registered with a global collector and used
once Catch enters main. They can also be optionally ignored, if user
supplies his own main and opts not to check them (or ignored them
intentionally).
Closes#921
This prevents Catch from disabling `Wpadded` for Clang inside test files
(files that do not define either `CATCH_CONFIG_MAIN` or
`CATCH_CONFIG_RUNNER`).
catch_suppress_warnings.h and catch_reenable_warnings.h should be
included only once*, so that the stitching script includes them as the
first and last header respectively, since it only includes each header
once. This caused a bug, where the first one was included properly, but
the second one was included prematurely, from catch_xmlwriter.hpp, and
thus was guarded by `CATCH_IMPL`.
* At least until the stitching script is changed to accomodate common
warning disabling header.
Fixes#871
Change it so the classname attribute on the <testcase> element is the
test fixture name or "global" regardless of whether the TEST_CASE
contains SECTIONs. This way the output is not changed substantially,
just because a SECTION is added to a TEST_CASE.
Types which are truthy, but have more information than the truthiness in their string conversion were showing up as 'true' or 'false' instead of showing the underlying type's string value.
Previously, this would not print out any messages for the last CHECK
```cpp
TEST_CASE("Foo") {
INFO("Test case start");
for (int i = 0; i < 2; ++i) {
INFO("The number is " << i);
CHECK(i == 0);
}
CHECK(false);
}
```
now it does.
This means that `REQUIRE(std::vector<int>{1, 2} == std::vector<int>{1,
2});` works as expected.
Note that assertion macros taking more than 1 argument are currently not
variadic, because variadic args have to come last, which would make the
interface of these ugly: `REQUIRE_THROWS_AS(std::exception const&, ....
)`
All C++11 toggles are now removed. What is left is either platform
specific (POSIX_SIGNALS, WINDOWS_SEH), or possibly still needed
(USE_COUNTER).
If current CLion is compatible with `__COUNTER__`, then we should also
force `__COUNTER__` usage.
Changed
* CATCH_AUTO_PTR -> std::unique_ptr
* CATCH_OVERRIDE -> override
* CATCH_NULL -> nullptr
* CATCH_NOEXCEPT -> noexcept
* CATCH_NOEXCEPT_IS -> noexcept
Removed
* CATCH_CONFIG_CPP11_UNIQUE_PTR
* CATCH_CONFIG_CPP11_SHUFFLE
* CATCH_CONFIG_CPP11_TYPE_TRAITS
* CATCH_CONFIG_CPP11_OVERRIDE
* CATCH_CONFIG_CPP11_LONG_LONG
* CATCH_CONFIG_CPP11_TUPLE
* CATCH_CONFIG_CPP11_IS_ENUM
* CATCH_CONFIG_CPP11_GENERATED_METHODS
* CATCH_CONFIG_CPP11_NOEXCEPT
* CATCH_CONFIG_CPP11_NULLPTR
* CATCH_CONFIG_VARIADIC_MACROS
- typedefs long long for MSVC
- typedefs uint64_t otherwise
Should probably do finer grained compiler checking - but this should at least be better than what was there before
When using C++11, comparison operators are already templated to take
anything that can be explicitly converted to double, but constructor
took only doubles. This lead to warnings when an `Approx` was
constructed from floats, which was problematic for some users.
Since just adding float constructor would be a large breaking change, as
suddenly `Approx( 1 )` would become ambiguous, I added a templated
constructor that will take anything that is explicitly convertible to
double. This has the added benefit of allowing constructing `Approx`
instances from instances of strong typedefs, ie allowing
`calculated_temp == Approx( known_temp)`.
Closes#873
Unexpected exceptions no longer cause abort and there should be no more
potential for false negatives.
The trade-off now is that exceptions are no longer translated.
This is another warning that follows test macros, making it painful to
suppress without leaking outside. Luckily clang's `_Pragma`
implementation works.
Should fix#308
Effectively a revert of previous commit, fixing #542, where this was
added to stop linters complaining about `REQUIRE_THROWS_AS` used like
`REQUIRE_THROWS_AS(expr, std::exception);`, which would be slicing the
caught exception. Now it is user's responsibility to pass us proper
exception type.
Closes#833 which wanted to add `typename`, so that the construct works
in a template, but that would not work with MSVC and older GCC's, as
having `typename` outside of a template is allowed only from C++11
onward.
This seems to give about 15% speedup when compiling tests using GCC.
The tradeoff is that under certain circumstances, there is a chance for
false negative result, when the expression under test throws exception
and the test code catches it before it gets to the test runner.
Example:
``` cpp
TEST_CASE("False negative") {
try {
REQUIRE(throws() == "");
} catch (...) {}
}
```
This test case will succeed, reporting no assertions checked, instead of
failing as it would with `CATCH_CONFIG_FAST_COMPILE` disabled. However,
just removing the try-catch block inside client's code will fix this, so
it is worthwhile.
This change does not apply to CHECK* macros, because these are currently
specified as continuing on exception and thus need the local try-catch
to work as intended.
std::ifstream in libstdc++ contains a bug, where it sets errno to zero.
To work around it, we manually save the errno before using std::ifstream
in debugger check, and reset it after we are done.
We also preventively save errno before using sprintf.
Fixes#835
In some cases, like when given
```cpp
std::vector<char>* str =
reinterpret_cast<std::vector<char>*>(0x1234458);
CHECK(*str == std::vector<char>());
```
reconstructing the expression to report it would cause another fatal
error. Instead we just put together an AssertionResult without
reconstructing the expression fully.
This should fully fix#810
This is a hackish attempt to add a TAP reporter (see
philsquared/Catch#309 ) by following the TAP 12 specification
<http://testanything.org/tap-specification.html>. I'm unsure how well I
did in following the spec or with following good C++ guidelines.
Comments are appreciated.
Signed-off-by: Colton Wolkins (Ogre) <frostyfrog2@gmail.com>
If the gcc version supports `_Pragma` properly, we use that to disable
it locally inside assertions.
Otherwise we disable it for the entire TU.
Fixes#674
Some versions of MinGW do not support enough of Win32 API to let us work
with SEH, so SEH is now MSVC only (+ configurable toggle).
Also made use of gmtime_s MSVC only (as oposed to Windows only).
Fixes#805
Because the signal changes were in a different branch from the windows.h
related changes, the SEH handling code included the header directly.
Fixes#803
This prevents Clang from complaining about unused value in expressions
containing explicit casts used in the THROW assertion macro family.
Example:
`REQUIRE_THROWS_AS(static_cast<bool>(object), std::bad_cast);` would
trigger `-Wunused-value` warning. Now it does not.
Credits to Arto Bendiken, who submitted a PR almost 3 years ago, but his
branch has since died and I was unable to merge it.
The integrated assembler segment was missing an underscore:
"_asm__". Also we remove the "DEBUG" macro check, so we are consistent
with the linux and windows variant.
Now breaking into gdb on failure should work via:
gdb --args test_executable --break
Using sizeof(expr) can trigger a compile-time error,
"lambda expressions are not allowed in an unevaluated context", when passing
expression containing lambda, like a std algorithm. This error is considered
a standard defect, as it is meant to prevent lambdas in decltype
or templates, but not in sizeof.
This reverts commit 227598af47.
In reality, this is a relatively small performance improvement,
especially with the previous improvements removing lots of superfluous
string handling, but still was measurable.
This is sane, because those `const char*`s are given to us by compiler,
from the text area and thus we do not have to manage their lifetimes. We
also never want to change them.
Also moved copy constructor to compiler-generated methods, not sure why
it wasn't -- even before it was the same as a compiler would generate.
This means that all tabs used in indentation are now 4 spaces and that
there should be no more trailing whitespace.
Ill also look into creating a pre-commit hook that will prevent this
from happening in the future.
Fixes#105
Rewrote main wrapping loop. Now uses iterators instead of indices and intermediate strings.
Differentiates between chars to wrap before, after or instead of.
Doesn’t preserve trailing newlines.
Wraps or more characters.
Dropped support for using tab character as an indent setting control char.
Hopefully avoids all the undefined behaviour and other bugs of the previous implementation.
Don't duplicate Catch::isDebuggerActive() check many times, do it just once
in CATCH_BREAK_INTO_DEBUGGER() definition and use a separate CATCH_TRAP()
macro for the really platform-dependent part.
* Empty strings are now direct constructed as `std::string()`, not as empty string literals.
* `startsWith` and `endsWith` no longer construct new a string. This should be an improvement
for libstdc++ when using older standards, as it doesn't use SSO but COW and thus even short
strings are expensive to first create.
* Various places now use char literal instead of string literals containing single char.
** `startsWith` and `endsWith` now also have overload that takes single char.
Generally the performance improvements under VS2015 are small, as going from short string
to char is mostly meaningless because of SSO (Catch doesn't push string handling that hard)
and previous commit removed most string handling if tests pass, which is the expect case.
This fixes the case when we pass signal to previously registered
handler, and it needs to transform the signal into different one.
Still problematic: What if the signal handler we replaced does not
terminate the application? We can end up in a weird state and loop
forever.
Possible solution: Deregister our signal handlers, CALL the previous
signal handler explicitly and if control returns, abort. This would
however complicate our code quite a bit, as we would have to parse the
sigaction we delegate to, decide whether to use signal handler or signal
action, etc...
Only some "signals" are handled under Windows, because Windows does not
use signals per-se and the mechanics are different. For now, we handle
sigsegv, stack overflow, div-by-zero and sigill. We can also
meaningfully
add various floating point errors, but not sigterm and family, because
sigterm is not a structured exception under Windows.
There is also no catch-all, because that would also catch various
debugger-related exceptions, like EXCEPTION_BREAKPOINT.
Also stops Catch from assuming its the only signal user in the binary,
and makes it restore the signal handlers it has replaced. Same goes for
the signal stack.
The signal stack itself probably shouldn't be always reallocated for
fragmentation reasons, but that can be fixed later on.
Now if we detect C++11 compiler, or MSVC in version corresponding to VS2015,
we switch from using `std::random_shuffle` to `std::shuffle`.
`std::random_shuffle` was officially deprecated in C++14, and removed in C++17.
Also removed guarded inclusion of `<random>` header, as there was nothing
in the header that used it.
Catch passes ::tolower into std::transform with string iterators.
::tolower has the signature int(int), which triggers a stealth narrowing
warning inside std::transform, because transform calls
*_Dest = _Fn(*_First), which implicitly narrows an int to a char.
For this particular application the narrowing is fine, so explicitly
narrow in a lambda.
Catch passes an RNG which accepts int to random_shuffle. Inside
random_shuffle, the STL tries to call that RNG with the difference_type
of the user provided iterators. For std::vector, this is ptrdiff_t,
which on amd64 builds is wider than int. This triggers a narrowing
warning because the 64 bit difference is being truncated to 32 bits.
Note that this RNG implementation still does not produce a correctly
uniformly shuffled result -- it's currently asserting that std::rand
can produce 1000000 which is false -- but I don't know enough about
how much repeatable shuffles are necessary here, so I'm leaving that
alone for now.
SCENARIO does not add leading spaces to the test name (only BDD-style section
names are modified), so the trimming is not necessary. But if the name is
trimmed, it makes it harder to correlate the output of XML reporter with tests
that have leading spaces in their name: e.g. these tests will have the same name
attribute:
TEST_CASE("Test") {}
TEST_CASE(" Test") {}
This commit fixes the following scenario:
* You have a test that compares strings with embedded control
characters.
* The test fails.
* You are using JUnit tests within TeamCity.
Before this commit, the JUnit report watcher fails on parsing the XML
for two reasons: the control characters are missing a semicolon at the
end, and the XML document doesn't specify that it is XML 1.1.
XML 1.0 --- what we get if we don't specify an XML version --- doesn't support embedding control characters --- see
http://stackoverflow.com/questions/404107/why-are-control-characters-illegal-in-xml
for all of the gory details.
This is based on PR #588 by @mrpi
Instead of `exit(1)`, it now throws `std::runtime_error` with the details
of the failure. This exception is handled in `run()` at a higher level where
the log is printed to cerr and the test gracefully exits.
