Commit Graph

736 Commits

Author SHA1 Message Date
Martin Hořeňovský 90e2549cec
Rewrite includes in tests to use library's include paths 2019-12-02 12:23:10 +01:00
Martin Hořeňovský 443fa0fc88
Fixing paths in SelfTest 2019-11-30 17:41:17 +01:00
Martin Hořeňovský c9067b2253
Refactor SectionInfo constructors 2019-11-16 21:53:32 +01:00
Martin Hořeňovský d36c15c3ca
Store tags in one big pre-allocated string and only work with refs
This should decrease the number of allocations before main is entered
significantly, but complicates the code somewhat in return.

Assuming I used `massif` right, doing just `SelfTest --list-tests`
went from 929 allocations at "Remove gcc-4.9 from the travis builds"
(2 commits up), to 614 allocations with this commit.
2019-11-14 10:53:12 +01:00
Martin Hořeňovský 302e2c0b06
Do not copy around TestCaseInfo
Now a `TEST_CASE` macro should create a single TestCaseInfo and then
it should never be copied around. This, together with latter changes,
should significantly decrease the number of allocations made before
`main` is even entered.
2019-11-14 10:52:34 +01:00
Martin Hořeňovský eabb8a6af7
Cleanup tests for C++20 2019-11-04 13:08:10 +01:00
Martin Hořeňovský 2c6ace04a7
Redo build matrix on .travis for C++14 and up
* Use Xenial as the base distribution
* Remove C++11 builds
* Add a lot more C++14 builds
* Add some C++17 builds
* Include newer versions of Clang and GCC
2019-10-31 16:19:56 +01:00
Martin Hořeňovský 4f47d1c6c1
Hidden tests now require positive filter match to be selected
This also required some refactoring of how the pattern matching
works. This means that the concepts of include and exclude patterns
are no longer unified, with exclusion patterns working as just
negation of an inclusion patterns (which led to including hidden
tags by default, as they did not match the exclusion), but rather
both include and exclude patterns are handled separately.

The new logic is that given a filter and a test case, the test
case must match _all_ include patterns and _no_ exclude patterns
to be included by the filter. Furthermore, if the test case is
hidden, then the filter must have at least one include pattern
for the test case to be used.

Closes #1184
2019-10-29 14:07:18 +01:00
Martin Hořeňovský 557b336125
Remove superfluous semicolon in the INFO macro
Closes #1456
2019-10-29 14:07:18 +01:00
Martin Hořeňovský 74e0e737a6
Remove useless test 2019-10-29 11:44:24 +01:00
Martin Hořeňovský 849f2848bd
Rework how warning suppression in macros is done
Previously, each warning suppression was self-contained, with its
own pair of `SUPPRESS_X_WARNING` and `UNSUPPRESS_X_WARNING` macros.
This had the obvious advantage of being self-containing, but it
also meant that if we needed to suppress more than one warning
in a single place, then we would manipulate the compiler's warning
state multiple times, even though logically we would only need one
layer.

The new way of suppressing warnings in macros is to push compiler's
warning state with `CATCH_INTERNAL_START_WARNINGS_SUPPRESSION` macro,
then disable whatever macros we need with the
`CATCH_INTERNAL_SUPPRESS_X_WARNINGS` macro, and then return to the
previous state using `CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION`.
2019-10-27 21:07:21 +01:00
Alexander Stein 2fbd66c51c Improve JUnit failure report
The JUnit report is improved in that:
* The message shows the testing condition, not the result
* The actual message has similar output than the console one
2019-10-27 18:02:14 +01:00
sp-dani-garcia 51b29ced1a Add SonarQube Generic Test Data reporter
It outputs reports in the `Generic Execution Test Data` format, see
https://docs.sonarqube.org/latest/analysis/generic-test/, specifically
https://docs.sonarqube.org/latest/analysis/generic-test/#header-2

Close #1738 (this is a cherry-pick and fixup of that PR)
2019-10-27 10:02:41 +01:00
Manuel Drews 4327baba40 Enable `range` generator for floating point types 2019-10-26 20:52:09 +02:00
Martin Hořeňovský 50cc14c94c
Rework StringRef interface and internals
Now it no longer tries to be this weird hybrid between an owning
and non-owning reference, and is only ever non-owning. This is also
reflected in its interface, for example `StringRef::isNullTerminated`
is now public, and `StringRef::c_str()` has the precondition that it
is true.

Overview of the changes:
* The `StringRef::m_data` member has been completely removed, as it
had no more uses.
* `StringRef::isSubstring()` has been made public and renamed to
`StringRef::isNullTerminated()`, so that the name reflects what the
method actually does.
* `StringRef::currentData()` has been renamed to `StringRef::data()`,
to be in line with common C++ containers and container-alikes.
* `StringRef::c_str()` will no longer silently make copies. It instead
has a precondition that `isNullTerminated()` is true.
* If the user needs a null-terminated string, they should use the
`std::string` conversion operator and call `c_str()` on the resulting
`std::string`.
* Some small optimizations in various places.
* Basic functionality is now `constexpr`.
2019-10-25 13:57:52 +02:00
Martin Hořeňovský 90825a4f7a
Add more tests for reading test specs from file
Related to #1770
2019-10-20 15:14:50 +02:00
Martin Hořeňovský 9e8ae7d470
Use scientific notation for the WithinULP matcher
This should now properly handle small numbers which would previously
output something like `[0.00000000000000019, 0.00000000000000019]`,
which does not allow user to read the numbers properly.

Closes #1760
2019-10-20 12:30:21 +02:00
Martin Hořeňovský e4d61e4cd8
Fix baselines 2019-10-13 21:26:51 +02:00
Martin Hořeňovský 2cc0c71856
Add a matcher that checks exception's message
Only works for exceptions that publicly derive from `std::exception`
and the matching is done exactly, including case and whitespace.

Closes #1649
Closes #1728

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch master
# Your branch is up-to-date with 'origin/master'.
#
# Changes to be committed:
#	modified:   ../docs/matchers.md
#	modified:   ../include/internal/catch_capture_matchers.h
#	modified:   ../projects/CMakeLists.txt
#	modified:   ../projects/SelfTest/Baselines/compact.sw.approved.txt
#	modified:   ../projects/SelfTest/Baselines/console.std.approved.txt
#	modified:   ../projects/SelfTest/Baselines/console.sw.approved.txt
#	modified:   ../projects/SelfTest/Baselines/junit.sw.approved.txt
#	modified:   ../projects/SelfTest/Baselines/xml.sw.approved.txt
#	modified:   ../projects/SelfTest/UsageTests/Matchers.tests.cpp
#
# Untracked files:
#	./
#	../clang-full/
#	../clang-test/
#	../clang10-build/
#	../coverage-build/
#	../gcc-build/
#	../gcc-full/
#	../include/internal/catch_matchers_exception.cpp
#	../include/internal/catch_matchers_exception.hpp
#	../misc-build/
#	../msvc-sln/
#	../notes.txt
#	../test-install/
#
2019-10-13 20:37:07 +02:00
Martin Hořeňovský 28663fb959
Use the right overload of std::nextafter in tests 2019-10-13 13:36:22 +02:00
Martin Hořeňovský d2d418a9cb
Add a Relative Comparison matcher for floating point numbers
It checks Knuth's _close enough with tolerance_ relationship, that
is `|lhs - rhs| <= epsilon * max(|lhs|, |rhs|)`, rather then the
_very close with tolerance_ relationship that can be written down as
`|lhs - rhs| <= epsilon * min(|lhs|, |rhs|)`.

This is because it is the more common model around the internet, and
as such is likely to be less surprising to the users. In the future
we might want to provide the other model as well.

Closes #1746
2019-10-13 11:56:50 +02:00
Martin Hořeňovský c8db4e77c4
Add a from_range(Container) overload to the generator helpers 2019-10-09 14:51:36 +02:00
Martin Hořeňovský 3109add95c
Add tests for the -f/--input-file command line argument 2019-10-09 12:54:13 +02:00
Martin Hořeňovský a5a22cdadb
Have the random generators use the global rng instance
This means that if you nest multiple random generators inside one
test case, they will not return the same sequence of numbers.

Idea taken from #1736 by Amit Herman.

Closes #1736
Closes #1734
2019-10-07 21:53:07 +02:00
Martin Hořeňovský 535da5c513
Introduce Catch's own RNG based on the PCG family of RNGs
In the future, we will also want to introduce our own
`uniform_int_distribution` and `uniform_real_distribution` to get
repeatable test runs across different platforms.
2019-10-07 19:56:23 +02:00
Martin Hořeňovský 319cb9e1da
Add a generator that takes an iterator pair 2019-10-06 13:55:10 +02:00
Martin Hořeňovský c38a5caa2e
Allow full range of target ULP values for the ULPMatcher
Previously it was limited to roughly 2 billion ULPs, rather than
the roughly 2^64 possible ones.
2019-10-05 13:38:22 +02:00
Martin Hořeňovský ebc5609484
Add test for INF == Approx(1) 2019-10-04 14:19:39 +02:00
Martin Hořeňovský 02ee130bd0
Special case Approx(inf) to better follow user expectations 2019-10-04 13:28:43 +02:00
Martin Hořeňovský fc2066bf18
Add tests for escaping special characters in CLI 2019-09-21 00:50:17 +02:00
Martin Hořeňovský af8b2538a6
Ignore leading/trailing whitespace in test/section specs
The leading/trailing whitespace is problematic because of e.g.
`WHEN` macro having preceeding whitespace for alignment, and it is
generally messy.

Credits to Phil who did lot of the original work.

Closes #1708
2019-09-09 14:28:11 +02:00
Martin Hořeňovský dab0296b64
Add test for past-the-end substr on StringRef 2019-09-08 21:07:21 +02:00
Martin Hořeňovský 7b865daccc
Make StringRef's operator std::string explicit
This way it is explicit when there is a `StringRef` -> `std::string`
conversion and makes it easier to look for allocations that could
be avoided.

Doing this has already removed one allocation per registered test
case, as there was a completely pointless `StringRef` -> `std::string`
conversion when parsing tags of a test case.
2019-09-08 15:58:10 +02:00
Martin Hořeňovský 923db16322
Use StringRefs through the enum registration 2019-09-08 14:14:46 +02:00
Martin Hořeňovský 6da00c1b64
Split out the rest of string manipulation tests 2019-09-07 20:22:36 +02:00
Martin Hořeňovský fe967b1f41
Remove the ill-considered StringRef::numberOfCharacters
It never counted characters, only codepoints. If you think these
are interchangeable, you should not touch non-ascii text.
2019-09-07 20:13:22 +02:00
Martin Hořeňovský f2c2711bdc
Add trim for StringRef 2019-09-07 11:31:00 +02:00
Martin Hořeňovský b77ab74b72
Start adding tests for string manip functions 2019-09-06 18:50:57 +02:00
Benjamin Worpitz 18d597cf10 Allow to use non-copyable and non-movable types in TEMPLATE_LIST_TEST_CASE
The parameter given to `convert` may not be copyable therefore it has to be
captured by const reference. For example an `std::tuple` that contains a
non-copyable type is itself non-copyable.

The NonDefaultConstructible test-case was reduced by one example type
because it did not add any value.
2019-09-06 12:15:09 +02:00
Martin Hořeňovský e2d863b090
Actually run the random generator tests 2019-08-09 11:21:05 +02:00
Martin Hořeňovský ebe6a07c23
Remove the new tests from approvals
They lead to stringification of file (which is ok) and file line
(not ok) to the approvals, which makes them exceedingly brittle
and not worth approval testing. Instead we just run them as part
of the base test run.
2019-08-09 11:13:28 +02:00
Martin Hořeňovský 738818ae1d
Add a test for the internals 2019-08-09 10:50:53 +02:00
Steven Franzen 6070745cab Improve reporting of unmatched filters (#1684)
This PR ultimately does 3 things
* Separately tracks matched tests per each filter part (that is, a set of filters separated by an OR (`,`)), which allows Catch2 to report each of the alternative filters that don't match any tests.
* Fixes `-w NoTests` to return non-zero in the process
* Adds tests for `-w NoTests`.
2019-08-06 20:51:19 +02:00
Martin Hořeňovský 3701c2e2e6
Small cleanup of self tests 2019-08-04 13:50:25 +02:00
Benjamin Worpitz e7c26f09d1 Fix non-default-constructible type lists used in TEMPLATE_LIST_TEST_CASE (#1697)
* Fix non-default-constructible type lists used in TEMPLATE_LIST_TEST_CASE

std::tuple is not default constructible when the first type is not
default-constuctible. Therefore it can not be instantiated.
to circumvent this, we have to use std::declval in the unevaluate decltype
context.
2019-07-22 12:56:33 +02:00
Stuart Dootson 87739ad3fe Add `std::byte` stringification support (#1686)
* Add `std::byte` support

1. Add byte detection in Catch config
2. Add a `std::byte`specialisation for Catch2::Detail::StringMaker
2019-07-13 14:47:56 +02:00
Martin Hořeňovský c0d3a2e08f
Merge pull request #1672 from sommd/master
Fix ChunkGenerator with chunk-size 0
2019-07-01 18:07:37 +02:00
Martin Hořeňovský 9a6551b22b
Add integration test for the children completion check fix 2019-06-30 00:00:41 +02:00
David Sommerich 9cf5897a11 Fix ChunkGenerator with chunk-size 0
Fixes #1671
2019-06-29 23:48:30 +10:00
Jozef Grajciar f41051f22a test approvals for TEMPLATE_LIST_TEST_CASE 2019-06-15 15:40:39 +02:00