Commit Graph

127 Commits

Author SHA1 Message Date
Martin Hořeňovský
4d8acafecb
Rename isStdout => isConsole and add doccomment 2022-03-24 16:02:51 +01:00
Martin Hořeňovský
4acc520f76
Event listeners no longer take reporter config in constructor
This also required splitting out Listener factory from
the reporter factory hierarchy. In return, the listener
factories only need to take in `IConfig const*`, which
opens up further refactorings down the road in the colour
selection and implementation.
2022-03-18 00:36:18 +01:00
Martin Hořeňovský
913f79a661
Each reporter keeps its own colour implementation
This opens path to per-reporter colour output customization,
and fixes multiple issues with the old colour implementation.

Under the old implementation, using Win32-backed colouring
would always change the colour used by the console, even if the
actual output was written elsewhere, such as a file passed by
the `--out` flag. This will no longer happen, as the reporter's
colour impl will check that the reporter's stream is pointed
to console before trying to change the colours.

POSIX/ANSI colour implementation suffered a similar-ish issue,
in that it only wrote the colour escape codes into the default
output stream, even if the reporter asking for colouring was
actually writing to a completely different output stream.
2022-03-08 12:51:13 +01:00
Martin Hořeňovský
06f74a0f8e
Add %stderr and %stdout as recognized magic stream names 2022-03-07 10:59:27 +01:00
Martin Hořeňovský
61d0f7a9af
Pass the whole IStream wrapper into reporter
This will become useful when reworking colour support, because
Win32 colour support requires checking whether the output is
stdout, which is done through the `IStream` wrapper.
2022-03-06 16:34:13 +01:00
Martin Hořeňovský
ff5b311898
Remove superfluous warning suppression in tests 2022-02-22 20:13:17 +01:00
Martin Hořeňovský
715cd25081
Add IStream::isStdOut 2022-02-22 15:45:51 +01:00
Martin Hořeňovský
fc5552d27b
Push down handling of default reporter to Config's constructor
This simplifies the handling of default reporter in console
parsing, at the cost of making `Config`'s constructor responsible
for more things.
2022-02-17 23:02:30 +01:00
Martin Hořeňovský
7a2f9f4633
Extend checkLicense.py to also check tests 2022-01-29 13:10:54 +01:00
Martin Hořeňovský
f83332d89b
Use the new licence header in SelfTest's cpp files 2022-01-29 00:03:43 +01:00
Martin Hořeňovský
b6ff2c3dda
Error out reporter registration for reporter names with '::' 2022-01-09 19:01:14 +01:00
Martin Hořeňovský
103cb16696
Rename listening reporter to multi reporter 2022-01-05 00:00:28 +01:00
Martin Hořeňovský
fae0be25b3 Improve error handling in parsing reporter specs 2022-01-01 14:16:11 +01:00
Martin Hořeňovský
8baf9c05a3 Add more unit tests
* Call order of listeners/reporters in multireporter
* Adding listeners/reporters properly updates reporter preferences
2022-01-01 14:10:52 +01:00
Martin Jeřábek
ccd67b293d Add support for multiple parallel reporters
This requires a bunch of different changes across the reporter
subsystem.

* We need to handle multiple reporters and their differing
  preferences in `ListeningReporter`, e.g. what to do when
  we mix reporters that capture and don't capture stdout.
* We need to change how the reporter is given output and
  how we parse reporter's output destination from CLI.
* Approval tests need to handle multireporter option
2022-01-01 14:02:23 +01:00
Martin Jeřábek
8711b63a0a
Add tests for StringRef's copy operations being shallow 2021-12-29 20:20:29 +01:00
Martin Hořeňovský
72a09de236
Add tests for Optional's op== and != 2021-12-29 16:58:22 +01:00
Martin Hořeňovský
f00b6e2019
Add more tests for various less common tag-related cases 2021-12-28 00:22:53 +01:00
Martin Hořeňovský
45577a1f4c
Refactor implementation of case-insensitivity in tags
By not materializing the lower cased tags ahead of time, we
save allocations at the cost of worsened performance when comparing
two tags.

Since there are rarely many tags, and commonly they are not
compared even if present, this is almost always a win. The new
implementation also improves the robustness of the code
responsible for handling tags in a case-insensitive manner.
2021-12-28 00:20:55 +01:00
Martin Hořeňovský
cbb6764fb1
Add CaseInsensitiveEqualTo comparison type 2021-12-26 18:54:47 +01:00
Martin Hořeňovský
3cc0c033e4
Allow enabling multiple warnings in one invocation 2021-12-18 20:50:05 +01:00
Martin Hořeňovský
931f41b4d6
Add some tests for TextFlow::Column 2021-11-01 22:51:17 +01:00
Martin Hořeňovský
3c5c86a4e4
Add test for filtering out multiple initial values in filter gen 2021-10-28 11:26:53 +02:00
Martin Hořeňovský
12d14a3c63
Add support for multiply calling lambda parsers in Clara
Previously a lambda parser in Clara could only be invoked once,
even if it internally was ok with being invoked multiple times.

With this change, a lambda parser can mark itself as `accept_many`,
in which case it will be invoked multiple times if the appropriate
flag was supplied multiple times by the user.
2021-10-27 20:15:28 +02:00
Martin Hořeňovský
5ac1ffe9ee Improve shardIndex/Count cli argument parsing 2021-10-27 17:24:30 +02:00
Ben Dunkin
3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský
905bf438ae
Fix bad indentation calculation in the console reporter
The problem came from the console reporter trying to provide a
fancy linebreaking (primarily for things like `SCENARIO` or the
BDD macros), so that new lines start with extra indentation if
the text being line broken starts as "{text}: ".

The console reporter did not properly take into account cases
where the ": " part would already be in a later line, in which
case it would ask for non-sensical level of indentation (larger
than single line length).

We fixed this by also enforcing that the special indentation case
only triggers if the ": " is found early enough in the line, so
that we also avoid degenerate cases like this:
```
blablabla: F
           a
           n
           c
           y
           .
           .
           .
```

Fixes #2309
2021-10-25 15:21:28 +02:00
Martin Hořeňovský
200a487cf2
Add generateRandomSeed utility to generate randomness seed 2021-10-08 21:35:41 +02:00
Martin Hořeňovský
13670f535f
Add more tests for XmlWrite::write* members 2021-10-02 14:39:52 +02:00
Martin Hořeňovský
426954032f
Rename Contains string matcher builder to ContainsSubstring
The problem with the old name was that it collided with the
range matcher `Contains`, and it was not really possible to
disambiguate them just with argument types.

Closes #2131
2021-09-27 21:28:33 +02:00
Martin Hořeňovský
f02c2678a1
Use StringRef for className in TestCaseInfo 2021-09-27 19:07:45 +02:00
Martin Hořeňovský
21b99d6f58
Add StringRef::compare for three way comparison 2021-09-27 18:41:17 +02:00
Martin Hořeňovský
d42e7a23a0
Change startsWith(char) to take StringRef as argument 2021-09-27 14:55:28 +02:00
Martin Hořeňovský
1cbbc5d2cb
Add test for !shouldfail and failed benchmarks 2021-08-21 00:06:31 +02:00
Martin Hořeňovský
0a3f511cfe
Fix inconsistencies in reporting benchmarking failures
With these changes, all these benchmarks
```cpp
BENCHMARK("Empty benchmark") {};
BENCHMARK("Throwing benchmark") {
    throw "just a plain literal, bleh";
};
BENCHMARK("Asserting benchmark") {
    REQUIRE(1 == 2);
};
BENCHMARK("FAIL'd benchmark") {
    FAIL("This benchmark only fails, nothing else");
};
```

report the respective failure and mark the outer `TEST_CASE` as
failed. Previously, the first two would not fail the `TEST_CASE`,
and the latter two would break xml reporter's formatting, because
`benchmarkFailed`, `benchmarkEnded` etc would not be be called
properly in failure cases.
2021-08-07 15:16:11 +02:00
Martin Hořeňovský
3d1cf95b32
Fix ulp distance calculation for numbers with different signs
This is a simplification of the fix proposed in #2152, with the
critical function split out so that it can be tested directly,
without having to go through the ULP matcher.

Closes #2152
2021-07-27 21:04:41 +02:00
Martin Hořeňovský
816f69416b
Use TestCaseInfo in tests directly 2021-06-01 00:17:19 +02:00
Martin Hořeňovský
aee31d0620
Disallow empty tags in test case specification 2021-06-01 00:17:13 +02:00
Martin Hořeňovský
5741de9ccd
Add explicit test for serialization of boolean attributes in XML 2021-05-31 08:41:28 +02:00
Martin Hořeňovský
9137e591fa
Cleanup and optimize stringifying of string-like types
More specifically, made the actual implementation of string-like
type handling take argument as `Catch::StringRef`, instead of
taking `std::string const&`.

This means that string-like types that are not `std::string` no
longer need to pay for an extra construction of `std::string`
(including the potential allocation), before they can be stringified.

The actual string stringification routine is now also better about
reserving sufficient space.
2021-05-18 23:15:22 +02:00
Martin Hořeňovský
5eb7aa4f90
Add test for dots in tag names 2021-05-12 21:43:14 +02:00
Martin Hořeňovský
f1d7a10e06
Remove StringRef::isNullTerminated and StringRef::c_str
Both of them were fundamentally unsafe to use and shouldn't be used
at all.
2021-03-26 21:11:44 +01:00
Martin Hořeňovský
971b1fc32a
--list-* flags write to target specified by the -o flag
Also added tests for the default implementations of list* reporter
helpers.

Closes #2061
2021-01-26 18:36:54 +01:00
Roman Proskuryakov
0acb371b92
Fix Wold-style-cast error (#2125)
* Add Wold-style-cast to cmake flags
* Fix old style cast in catch_stats.hpp
* Fix old style cast in catch_stats.cpp
2020-12-28 14:00:19 +01:00
Martin Hořeňovský
135103bacf
Arg::parse accepts plain Args as argument 2020-11-07 18:00:27 +01:00
Martin Hořeňovský
6ffac61719
Fix types in generators: decay types, properly constrain forwarding
Fixes #2040
Closes #2012
2020-10-20 10:57:37 +02:00
Sean Middleditch
31d4831245
Support sentinel-based ranges in default stringify (#2004) 2020-09-07 14:23:47 +02:00
Martin Hořeňovský
cc18bd719d
Sweep out Wshadow
Most of the changes are completely pointless renaming of constructor
arguments so that they do not use the same name as the type members,
but 🤷

Closes #2015
2020-09-06 13:11:42 +02:00
Martin Hořeňovský
1a97af45f1
Cleanup some stuff found by MSVC's /analyze
* Added some missing `noexcept`s on custom destructors.
* Fixed `std::move` being called on a const-reference.
* Initialized `ScopedMessage::m_moved` in class definition, instead
of doing so in constructors explicitly.
* Turned some `enum`s into `enum class`es.
* Initialized `StreamingReporterBase::currentTestCaseInfo` in class
definition.
* Some cleanups in SelfTest code.
2020-08-20 20:42:21 +02:00
Martin Hořeňovský
f16be402f7
Make XmlEncoding tests slightly more efficient 2020-08-18 21:02:31 +02:00
Martin Hořeňovský
4565b826cf
Modify generator tracking to allow GENERATEs between SECTIONs
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-26 11:35:06 +02:00
Martin Hořeňovský
317145514f
Add op+(StringRef, StringRef) -> std::string 2020-05-31 22:30:41 +02:00
Martin Hořeňovský
1d1ccf8f3c
Replace all uses of std::unique_ptr with Catch::Detail::unique_ptr
Doing some benchmarking with ClangBuildAnalyzer suggests that
compiling Catch2's `SelfTest` spends 10% of the time instantiating
`std::unique_ptr` for some interface types required for registering
and running tests.

The lesser compilation overhead of `Catch::Detail::unique_ptr` should
significantly reduce that time.

The compiled implementation was also changed to use the custom impl,
to avoid having to convert between using `std::unique_ptr` and
`Catch::Detail::unique_ptr`. This will likely also improve the compile
times of the implementation, but that is less important than improving
compilation times of the user's TUs with tests.
2020-05-31 15:20:24 +02:00
Martin Hořeňovský
41bbaa6d57
Implement a simplified variant of std::unique_ptr<T>
This simplified variant supports only a subset of the functionality
in `std::unique_ptr<T>`. `Catch::Detail::unique_ptr<T>` only supports
single element pointer (no array support) with default deleter.

By removing the support for custom deleters, we also avoid requiring
significant machinery to support EBO, speeding up instantiations of
`unique_ptr<T>` significantly. Catch2 also currently does not need
to support `unique_ptr<T[]>`, so that is not supported either.
2020-05-31 15:08:47 +02:00
Martin Hořeňovský
9e498278be
Move StringRef header to internals 2020-05-10 10:09:01 +02:00
Martin Hořeňovský
895d0a0696
Small improvements for StringRef
* `operator[]` is constexpr
* `operator<<` and `operator+=` are hidden friends
2020-05-10 07:22:11 +02:00
Martin Hořeňovský
0c223bb751
Decrease chance of false positive in random generator testing 2020-05-03 19:01:21 +02:00
Martin Hořeňovský
dd35430a2b
Add more tests for test spec parser
Originally the tests were from #1912, but as it turned out, the issue
was somewhere else. Still, the inputs provided were interesting, so
they are now part of our test suite.
2020-05-02 18:24:07 +02:00
Martin Hořeňovský
87a8b61d5a
Fix bug in test spec parser handling of escaping in ORed patterns
It did not clear out all of its internal state when switching from
one pattern to another, so when it should've escaped `,`, it took
its position from its position in the original user-provided string,
rather than its position in the current pattern.

Fixes #1905
2020-05-01 09:56:34 +02:00
Martin Hořeňovský
60cfaa38fb
Make warnings in assertions fire for GCC/Clang again
The old code caused warnings to fire under MSVC, and Clang <3.8.
I could not find a GCC version where it worked, but I assume that it
did at some point.

This new code causes all of MSVC, GCC, Clang, in current versions,
to emit signed/unsigned comparison warning in test like this:

```cpp
TEST_CASE() {
    int32_t i = -1;
    uint32_t j = 1;
    REQUIRE(i != j);
}
```

Where previously only MSVC would emit the warning.

Fixes #1880
2020-04-29 21:27:57 +02:00
Martin Hořeňovský
e1e6872c4c
Standardize header names and file locations
This is both a really big and a really small commit. It is small in
that it only contains renaming, moving and modification of include
directives caused by this.

It is really big in the obvious way of touching something like 200
files.

The new rules for naming files is simple: headers use the `.hpp`
extension. The rules for physical file layout is still kinda in
progress, but the basics are also simple:
 * Significant parts of functionality get their own subfolder
   * Benchmarking is in `catch2/benchmark`
   * Matchers are in `catch2/matchers`
   * Generators are in `catch2/generators`
   * Reporters are in `catch2/reporters`
   * Baseline testing facilities are in `catch2/`
 * Various top level folders also contain `internal` subfolder,
   with files that users probably do not want to include directly,
   at least not until they have to write something like their own
   reporter.
    * The exact files in these subfolders is likely to change later
      on

Note that while some includes were cleaned up in this commit, it
is only the low hanging fruit and further cleanup using automatic
tooling will happen later.

Also note that various include guards, copyright notices and file
headers will also be standardized later, rather than in this commit.
2020-04-24 18:58:44 +02:00
Martin Hořeňovský
7892954c99
Remove global warning manipulation headers
They were unused except for one test file anyway.
2020-04-05 19:14:16 +02:00
Martin Hořeňovský
c2daf468bb
Standardize matcher headers to use .hpp suffix 2020-03-27 10:22:25 +01:00
Martin Hořeňovský
c3013a6251
Move matcher implementation to their own subfolder
In the future we can expect many more matchers, so let's give them
a place to live.

Also moved matcher-related internal files to `internal` subfolder.
Ideally we should sort out all of our source code, but that will
have to come later.
2020-02-20 17:42:11 +01:00
Martin Hořeňovský
cf6575576f
Start fixing up Matchers: namespaces, composition ops
This commit also forbids composing lvalues of composed matchers, as
per previous deprecation notice. I do not expect this to be contentious
in practice, because there was a bug in that usage for years, and
nobody complained.
2020-02-20 13:03:30 +01:00
Tristan Stenner
6a2c025bfc
Add command line option 'never' to --wait-for-keypress (#1866)
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2020-02-16 16:11:11 +01:00
khyperia
3c7e737a7b
Allow configuring of benchmark warmup time 2020-02-16 15:44:23 +01:00
Martin Hořeňovský
db1a0465dc
Outline GeneratorException from generators header 2020-02-14 16:15:47 +01:00
Martin Hořeňovský
b2a6523d85
Fix Clang-3.8 compilation of tests
As far as I know, the compilation error is a compiler bug, but I
do not want to just drop the support for something that is trivial
to work around.
2020-02-13 16:57:37 +01:00
Joe Burzinski
82baef62e2
Fix forwarding in SingleValueGenerator and generator creation
Fixes #1809
2020-02-13 16:19:50 +01:00
Martin Hořeňovský
2945b80f61
Add more constexpr to StringRef 2020-02-13 15:01:03 +01:00
Martin Hořeňovský
c50ba09cde
Split [.foo] into [.][foo] when parsing test specs
b77cec05c0 fixed this problem for tagging tests, so that a test
case tagged with `[.foo]` would be parsed as tagged with `[.][foo]`.
This does the same for the test spec parsing.

Fixes #1798
2020-02-13 13:35:10 +01:00
Martin Hořeňovský
cd7d7a1c67
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle
Now that Catch2 is a proper library, we can always build the full
library (comparatively minor slowdown) and the user can avoid
including benchmarking headers to avoid the compilation slowdown.
2020-02-06 11:36:46 +01:00
Martin Hořeňovský
c3a5e21648
Move Approx out of the Detail namespace 2020-02-03 15:14:59 +01:00
Martin Hořeňovský
6c3a5ef625
Remove CATCH_CONFIG_DISABLE_MATCHERS
Now that the recommended distribution and usage method is proper
library, users can just avoid including the matcher headers to get
basically the same effect.
2020-01-25 09:07:36 +01:00
Martin Hořeňovský
26f78f96aa
Start using piecemeal includes in test files 2020-01-21 10:03:54 +01:00
Martin Hořeňovský
0fea081ad1
Move tests from projects/ to tests/ 2019-12-05 16:00:20 +01:00