Commit Graph

382 Commits

Author SHA1 Message Date
Alexander Stein 2fbd66c51c Improve JUnit failure report
The JUnit report is improved in that:
* The message shows the testing condition, not the result
* The actual message has similar output than the console one
2019-10-27 18:02:14 +01:00
sp-dani-garcia 51b29ced1a Add SonarQube Generic Test Data reporter
It outputs reports in the `Generic Execution Test Data` format, see
https://docs.sonarqube.org/latest/analysis/generic-test/, specifically
https://docs.sonarqube.org/latest/analysis/generic-test/#header-2

Close #1738 (this is a cherry-pick and fixup of that PR)
2019-10-27 10:02:41 +01:00
Manuel Drews 4327baba40 Enable `range` generator for floating point types 2019-10-26 20:52:09 +02:00
Martin Hořeňovský 50cc14c94c
Rework StringRef interface and internals
Now it no longer tries to be this weird hybrid between an owning
and non-owning reference, and is only ever non-owning. This is also
reflected in its interface, for example `StringRef::isNullTerminated`
is now public, and `StringRef::c_str()` has the precondition that it
is true.

Overview of the changes:
* The `StringRef::m_data` member has been completely removed, as it
had no more uses.
* `StringRef::isSubstring()` has been made public and renamed to
`StringRef::isNullTerminated()`, so that the name reflects what the
method actually does.
* `StringRef::currentData()` has been renamed to `StringRef::data()`,
to be in line with common C++ containers and container-alikes.
* `StringRef::c_str()` will no longer silently make copies. It instead
has a precondition that `isNullTerminated()` is true.
* If the user needs a null-terminated string, they should use the
`std::string` conversion operator and call `c_str()` on the resulting
`std::string`.
* Some small optimizations in various places.
* Basic functionality is now `constexpr`.
2019-10-25 13:57:52 +02:00
Martin Hořeňovský 9e8ae7d470
Use scientific notation for the WithinULP matcher
This should now properly handle small numbers which would previously
output something like `[0.00000000000000019, 0.00000000000000019]`,
which does not allow user to read the numbers properly.

Closes #1760
2019-10-20 12:30:21 +02:00
Martin Hořeňovský e4d61e4cd8
Fix baselines 2019-10-13 21:26:51 +02:00
Martin Hořeňovský 2cc0c71856
Add a matcher that checks exception's message
Only works for exceptions that publicly derive from `std::exception`
and the matching is done exactly, including case and whitespace.

Closes #1649
Closes #1728

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch master
# Your branch is up-to-date with 'origin/master'.
#
# Changes to be committed:
#	modified:   ../docs/matchers.md
#	modified:   ../include/internal/catch_capture_matchers.h
#	modified:   ../projects/CMakeLists.txt
#	modified:   ../projects/SelfTest/Baselines/compact.sw.approved.txt
#	modified:   ../projects/SelfTest/Baselines/console.std.approved.txt
#	modified:   ../projects/SelfTest/Baselines/console.sw.approved.txt
#	modified:   ../projects/SelfTest/Baselines/junit.sw.approved.txt
#	modified:   ../projects/SelfTest/Baselines/xml.sw.approved.txt
#	modified:   ../projects/SelfTest/UsageTests/Matchers.tests.cpp
#
# Untracked files:
#	./
#	../clang-full/
#	../clang-test/
#	../clang10-build/
#	../coverage-build/
#	../gcc-build/
#	../gcc-full/
#	../include/internal/catch_matchers_exception.cpp
#	../include/internal/catch_matchers_exception.hpp
#	../misc-build/
#	../msvc-sln/
#	../notes.txt
#	../test-install/
#
2019-10-13 20:37:07 +02:00
Martin Hořeňovský d2d418a9cb
Add a Relative Comparison matcher for floating point numbers
It checks Knuth's _close enough with tolerance_ relationship, that
is `|lhs - rhs| <= epsilon * max(|lhs|, |rhs|)`, rather then the
_very close with tolerance_ relationship that can be written down as
`|lhs - rhs| <= epsilon * min(|lhs|, |rhs|)`.

This is because it is the more common model around the internet, and
as such is likely to be less surprising to the users. In the future
we might want to provide the other model as well.

Closes #1746
2019-10-13 11:56:50 +02:00
Martin Hořeňovský c8db4e77c4
Add a from_range(Container) overload to the generator helpers 2019-10-09 14:51:36 +02:00
Martin Hořeňovský 535da5c513
Introduce Catch's own RNG based on the PCG family of RNGs
In the future, we will also want to introduce our own
`uniform_int_distribution` and `uniform_real_distribution` to get
repeatable test runs across different platforms.
2019-10-07 19:56:23 +02:00
Martin Hořeňovský 319cb9e1da
Add a generator that takes an iterator pair 2019-10-06 13:55:10 +02:00
Martin Hořeňovský c38a5caa2e
Allow full range of target ULP values for the ULPMatcher
Previously it was limited to roughly 2 billion ULPs, rather than
the roughly 2^64 possible ones.
2019-10-05 13:38:22 +02:00
Martin Hořeňovský 02ee130bd0
Special case Approx(inf) to better follow user expectations 2019-10-04 13:28:43 +02:00
Martin Hořeňovský fc2066bf18
Add tests for escaping special characters in CLI 2019-09-21 00:50:17 +02:00
Martin Hořeňovský af8b2538a6
Ignore leading/trailing whitespace in test/section specs
The leading/trailing whitespace is problematic because of e.g.
`WHEN` macro having preceeding whitespace for alignment, and it is
generally messy.

Credits to Phil who did lot of the original work.

Closes #1708
2019-09-09 14:28:11 +02:00
Martin Hořeňovský dab0296b64
Add test for past-the-end substr on StringRef 2019-09-08 21:07:21 +02:00
Martin Hořeňovský 7b865daccc
Make StringRef's operator std::string explicit
This way it is explicit when there is a `StringRef` -> `std::string`
conversion and makes it easier to look for allocations that could
be avoided.

Doing this has already removed one allocation per registered test
case, as there was a completely pointless `StringRef` -> `std::string`
conversion when parsing tags of a test case.
2019-09-08 15:58:10 +02:00
Martin Hořeňovský 923db16322
Use StringRefs through the enum registration 2019-09-08 14:14:46 +02:00
Martin Hořeňovský 6da00c1b64
Split out the rest of string manipulation tests 2019-09-07 20:22:36 +02:00
Martin Hořeňovský fe967b1f41
Remove the ill-considered StringRef::numberOfCharacters
It never counted characters, only codepoints. If you think these
are interchangeable, you should not touch non-ascii text.
2019-09-07 20:13:22 +02:00
Martin Hořeňovský f2c2711bdc
Add trim for StringRef 2019-09-07 11:31:00 +02:00
Martin Hořeňovský b77ab74b72
Start adding tests for string manip functions 2019-09-06 18:50:57 +02:00
Benjamin Worpitz 18d597cf10 Allow to use non-copyable and non-movable types in TEMPLATE_LIST_TEST_CASE
The parameter given to `convert` may not be copyable therefore it has to be
captured by const reference. For example an `std::tuple` that contains a
non-copyable type is itself non-copyable.

The NonDefaultConstructible test-case was reduced by one example type
because it did not add any value.
2019-09-06 12:15:09 +02:00
Martin Hořeňovský ebe6a07c23
Remove the new tests from approvals
They lead to stringification of file (which is ok) and file line
(not ok) to the approvals, which makes them exceedingly brittle
and not worth approval testing. Instead we just run them as part
of the base test run.
2019-08-09 11:13:28 +02:00
Martin Hořeňovský 738818ae1d
Add a test for the internals 2019-08-09 10:50:53 +02:00
Martin Hořeňovský 3701c2e2e6
Small cleanup of self tests 2019-08-04 13:50:25 +02:00
Benjamin Worpitz e7c26f09d1 Fix non-default-constructible type lists used in TEMPLATE_LIST_TEST_CASE (#1697)
* Fix non-default-constructible type lists used in TEMPLATE_LIST_TEST_CASE

std::tuple is not default constructible when the first type is not
default-constuctible. Therefore it can not be instantiated.
to circumvent this, we have to use std::declval in the unevaluate decltype
context.
2019-07-22 12:56:33 +02:00
David Sommerich 9cf5897a11 Fix ChunkGenerator with chunk-size 0
Fixes #1671
2019-06-29 23:48:30 +10:00
Jozef Grajciar f41051f22a test approvals for TEMPLATE_LIST_TEST_CASE 2019-06-15 15:40:39 +02:00
Martin Hořeňovský 7142d5a8c9
Show bound when stringifying the WithinULP matcher
Closes #1581
2019-06-15 07:12:27 +02:00
Martin Hořeňovský 10067a47da
Add an extra test for benchmarking macros
Also updated baselines
2019-06-06 21:29:25 +02:00
Jozef Grajciar 84f8e806b8 Nttp support tests approvals 2019-05-23 21:03:47 +02:00
Phil Nash 31537c43d9 Rebased approvals 2019-05-21 00:05:39 +01:00
Phil Nash d30f1dda02 Rebased approvals 2019-05-18 20:53:28 +01:00
Martin Hořeňovský 1161011dd0
Refactor custom precision in floating point stringification
Also fixup tests.
2019-05-03 15:38:06 +02:00
Petr Ledvina 9c741fe960 Allow quotes in CAPTURE arguments (#1608)
* Allow quotes in CAPTURE arguments

Fix CAPTURE to handle string and character literals properly
2019-05-01 19:12:44 +02:00
Phil nash 9837c35df1 Rebased approvals for stringified enum tests 2019-04-26 11:30:19 +01:00
Will Handley 91b617c462 Added a vector approximate matcher 2019-04-20 10:02:17 +02:00
Martin Hořeňovský 36fb856163
Fix JUnit reporter output so that it conforms to JUnit schema
The `properties` can only go under the `testsuite` tag, not under
the `testsuites` tag.

Fixes #1598
2019-04-11 13:04:54 +02:00
Martin Hořeňovský 4e32e0a563
Fix vendored dep because upstream is non-responsive
Eventually this needs to be fixed in the textflow project by Phil,
but he has not done so in the half a year this bug has been known
to be there, so...

Closes #1470
Closes #1455
2019-04-10 20:17:25 +02:00
Ryan Pavlik edde6f4736 Fix typos identified by codespell.
Self test baselines also modified accordingly, due to
one typo found in a string in test code.
2019-04-10 09:42:11 +02:00
Ziv Shahaf 8af8704089 support for printing test filters (PR #1585) 2019-04-03 20:24:52 +02:00
Martin Hořeňovský 3816e99d0c
Add GENERATE_COPY and GENERATE_VAR capturing generator macros 2019-03-31 14:11:10 +02:00
Martin Hořeňovský b77cec05c0
Fix test tag parsing to split [.foo] into [.][foo] 2019-03-29 10:48:56 +01:00
Omer Ozarslan 54089c4c8c Deducing return type of map generator helper (#1576)
* Deduce map return type implicitly

Giving the first template argument to map generator function to deduce
return type is now optional even if the return type is different from
the type generated by mapped generator.
2019-03-24 15:44:22 +01:00
Omer Ozarslan 7aee973a4a Add tests for unscoped info
Update approval tests as new tests are added for messaging.
2019-03-06 22:04:35 +01:00
Martin Hořeňovský 1a03918455
Force flush streams in tests for #1514
This avoids the problem where writes to stderr/stdout stop being
line-buffered when stderr/stdout is redirected to a file, which led
to different order of outputs between Linux and Windows in our tests.
2019-03-06 21:53:26 +01:00
Martin Hořeňovský 28db5ed4c9
Add tests for #1514
If the regression comes back, it will only be caught by approvals,
but that's better than nothing.
2019-03-02 21:22:47 +01:00
Martin Hořeňovský 29b3b7ae6b
Namespace our type traits in catch_meta
Previously they could conflict with user-defined type traits that
were in the global namespace.

Fixes #1548
2019-02-23 21:06:20 +01:00
Martin Hořeňovský ef5fd8d42f
Add another test for FilterGenerator 2019-02-23 20:37:35 +01:00