Commit Graph

54 Commits

Author SHA1 Message Date
Martin Hořeňovský c65e5b6514
Add 'uses-python' label to tests using Python 2022-04-24 13:31:28 +02:00
Martin Hořeňovský 05e85c5652
Split out Catch::cout/cerr/clog into their own file 2022-04-16 16:33:50 +02:00
Martin Hořeňovský c742ea9ad9
Some IWYU include cleanups 2022-04-14 17:16:01 +02:00
Martin Hořeňovský 79d1e82381
Store IStream instance owning ptrs in reporter instances
This ended up being a surprisingly large refactoring, motivated
by removing a `const_cast` from `Config`'s handling of reporter
streams, forced by previous commit.
2022-04-13 15:00:50 +02:00
Lukasz Okraszewski cb551b4f6d
Add default reporter for Bazel integration (#2399)
When the added Bazel configuration flag is enabled,
a default JUnit reporter will be added if the XML
envrioment variable is defined.
Fix include paths for generated config header.
Enable Bazel config by default when building with
Bazel.


Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
2022-04-08 18:01:59 +02:00
Martin Hořeňovský 423e1d2ebb
Integrate the new reporter specs into Catch2
This means that the CLI interface now uses the new key-value oriented
reporter spec, the common reporter base creates the colour implementation
based on the reporter-specific configuration, and it also stores the
custom configuration options for each reporter instance.

Closes #339 as it allows per-reporter forcing of ansi colour codes.
2022-04-07 12:50:08 +02:00
Martin Hořeňovský 4acc520f76
Event listeners no longer take reporter config in constructor
This also required splitting out Listener factory from
the reporter factory hierarchy. In return, the listener
factories only need to take in `IConfig const*`, which
opens up further refactorings down the road in the colour
selection and implementation.
2022-03-18 00:36:18 +01:00
John Beard f9facc1881 Add variadic CAPTURE tests
This makes sure that CAPTURE works when called with variadic arguments,
and also works that way when disabled.

The underlying fix to #2316 is not applicable (CAPTURE is already
variadic when disabled).

This is a port of 5e94498ed0 to the
devel branch.
2022-03-05 16:06:54 +01:00
Martin Hořeňovský 79e4cd1af4
Cleanup summary comments in ExtraTests 2022-01-29 00:40:03 +01:00
Martin Hořeňovský 635db2785f
Use the new licence header in ExtraTests's cpp files 2022-01-29 00:07:44 +01:00
Martin Hořeňovský e53a75b425
Add integration test for listener-reporter event ordering 2022-01-28 00:00:52 +01:00
Martin Hořeňovský 88b28ab592
Extra compilation checks for compiling without exceptions 2022-01-01 23:21:37 +01:00
Martin Hořeňovský ef3374ed81
Add more macros to the DisabledMacros test 2022-01-01 16:45:28 +01:00
Martin Hořeňovský 899554bff2 Add test for handling of reporter's assertion preferences 2022-01-01 14:15:44 +01:00
Martin Hořeňovský b4efa4751a Add test for handling of Listeners requesting stdout capture 2022-01-01 14:13:48 +01:00
Martin Hořeňovský 6b55f5d780
Don't send full stdout so far to testCasePartialEnded event 2022-01-01 12:21:41 +01:00
Martin Hořeňovský 9f2dca5384
Exit with non-0 return code if no tests were run
A new flag, `--allow-running-no-tests` was added to override this
behaviour if exit code of 0 was desired.

This change also made `-w NoTests` obsolete, and so it has been
removed.
2021-12-18 20:50:00 +01:00
Martin Hořeňovský d012735c6e
Add test for runtime behaviour of STATIC_CHECK 2021-11-15 20:06:29 +01:00
Morwenn f41d761674
Add STATIC_CHECK and STATIC_CHECK_FALSE (#2318) 2021-11-15 00:28:27 +01:00
Martin Hořeňovský 9200b4078b
Move reporter_registrars.hpp to reporters/ 2021-11-10 23:52:59 +01:00
Martin Hořeňovský 62d8913d67
Cumulative reporter base records benchmark results 2021-11-09 11:52:50 +01:00
Martin Hořeňovský 8780425385
Make reporter lookup case insensitive, registration case preserving
Previously registration was case preserving, but lookup used
lowercased reporter name, so a reporter whose name contained
upper case character could not be requested by the user.
2021-11-09 11:50:03 +01:00
Ben Dunkin 3087e19cc7 Allow test sharding for e.g. Bazel test sharding feature
This greatly simplifies running Catch2 tests in single binary
in parallel from external test runners. Instead of having to
shard the tests by tags/test names, an external test runner
can now just ask for test shard 2 (out of X), and execute that
in single process, without having to know what tests are actually
in the shard.

Note that sharding also applies to test listing, and happens after
tests were ordered according to the `--order` feature.
2021-10-27 17:24:30 +02:00
Martin Hořeňovský fb4153e05e
Multiple tests can have same name as long as their tags differ
This change also changes it so that test case macros using a
class name can have same name **and** tags as long as the
used class name differs.

Closes #1915
Closes #1999
2021-09-25 21:37:03 +02:00
Martin Hořeňovský 10fb93cce8
Add new reporter event called for each test case enter/exit
This means that e.g. for `TEST_CASE` with two sibling `SECTION`s
the event will fire twice, because the `TEST_CASE` will be entered
twice.

Closes #2107 (the event mentioned there already exists, but this
is its counterpart that we also want to provide to users)
2021-09-14 13:19:45 +02:00
Martin Hořeňovský 3afea8128a
Increase timing window for min duration tests to 1s
This test tends to be brittle on Mac CI machines, which are
heavily loaded and bursty. Since the tests are only run as part
of the "extra tests" test set, this increase should not have
a significant impact on the total duration of CI runs.
2021-02-20 23:09:02 +01:00
Martin Hořeňovský 19cbdebb0e
Lenghten timing threshold in --min-duration integration test 2020-11-28 21:00:09 +01:00
Martin Hořeňovský 60dfec559f
Provide 1 .hpp + 1 .cpp distribution of Catch2
This commits also adds a script that does the amalgamation of headers
and .cpp files into the distributable version, removes the old
`generateSingleHeader` script, and also adds a very simple compilation
test for the amalgamated distribution.
2020-09-09 13:02:50 +02:00
Martin Hořeňovský 2d4f8ac8e6
Disable problematic test on Windows 2020-09-06 20:41:58 +02:00
Martin Hořeňovský 90d6fd849e
Increase tolerances in --min-duration tests
The underpowered and oversubscribed CI servers are hell.
2020-07-26 10:48:03 +02:00
Martin Hořeňovský 13917c44b4
--min-duration is overriden by -d no 2020-07-26 10:48:01 +02:00
Martin Hořeňovský e6d947f6d4
Refactor tests for duration reporting threshold 2020-07-26 10:47:58 +02:00
Martin Hořeňovský db32550898
Remove catch_default_main.hpp
There are two reasons for this:

1) It is highly unlikely that someone has use for this header,
which has no customization points and only provides simplest
possible main, and cannot link the static library which also
provides a default main implementation.
2) It being a header was causing extra complications with
the convenience headers, and our checking script. This would either
require special handling in the checking script, or would break user's
of the main convenience header.

All in all, it is simpler and better in the long term to remove it,
than to fix its problems.
2020-05-09 18:00:49 +02:00
Martin Hořeňovský c086746cc9
Use main-as-static-lib in ExtraTests 2020-05-03 19:21:58 +02:00
Martin Hořeňovský e1e6872c4c
Standardize header names and file locations
This is both a really big and a really small commit. It is small in
that it only contains renaming, moving and modification of include
directives caused by this.

It is really big in the obvious way of touching something like 200
files.

The new rules for naming files is simple: headers use the `.hpp`
extension. The rules for physical file layout is still kinda in
progress, but the basics are also simple:
 * Significant parts of functionality get their own subfolder
   * Benchmarking is in `catch2/benchmark`
   * Matchers are in `catch2/matchers`
   * Generators are in `catch2/generators`
   * Reporters are in `catch2/reporters`
   * Baseline testing facilities are in `catch2/`
 * Various top level folders also contain `internal` subfolder,
   with files that users probably do not want to include directly,
   at least not until they have to write something like their own
   reporter.
    * The exact files in these subfolders is likely to change later
      on

Note that while some includes were cleaned up in this commit, it
is only the low hanging fruit and further cleanup using automatic
tooling will happen later.

Also note that various include guards, copyright notices and file
headers will also be standardized later, rather than in this commit.
2020-04-24 18:58:44 +02:00
Martin Hořeňovský ab0ca2f566
Rename some matcher-related files
The two changes are
`catch_matchers_templates` -> `catch_matchers_templated` and
`catch_matchers_generic` -> `catch_matchers_predicate`. The former
is mostly cosmetic, but the second was previously significantly
misleading, and as the library is now to be consumed by including
specific headers, this needed to be fixed.
2020-03-27 10:24:08 +01:00
Martin Hořeňovský c2daf468bb
Standardize matcher headers to use .hpp suffix 2020-03-27 10:22:25 +01:00
Martin Hořeňovský d1ffaf55a1
Fix warnings in ExtraTests and Examples 2020-02-26 16:07:54 +01:00
Martin Hořeňovský 6e270958a2
Add development build option to CMake and enable it on CI
Development build enables warnings and and `Werror` or equivalent.
2020-02-26 16:07:26 +01:00
Martin Hořeňovský 34e7a5e0cf
Bunch of warning fixes 2020-02-21 23:10:22 +01:00
Martin Hořeňovský c3013a6251
Move matcher implementation to their own subfolder
In the future we can expect many more matchers, so let's give them
a place to live.

Also moved matcher-related internal files to `internal` subfolder.
Ideally we should sort out all of our source code, but that will
have to come later.
2020-02-20 17:42:11 +01:00
Martin Hořeňovský a1be19aa1b
Remove the separately compiled test for Benchmarking Macros
Thanks to the changes to compilation model, the tests for benchmarking
macros have been made part of the normal test run. This means that
the only purpose these separately compiled tests served was to waste
CI time.
2020-02-18 13:48:34 +01:00
Martin Hořeňovský c745adb81c
Temporarily disable the test for DebugBreakMacro
The change to static library means that the new implementation needs
to be somewhat different, and I do not want to fix it right now.
2020-02-18 13:43:37 +01:00
Martin Hořeňovský 06c135706e
Move X12-CustomDebugBreakMacro.cpp to the right place
This is a small fixup because cherry-picking the relevant commit
threw it into the wrong place...
2020-02-18 10:20:37 +01:00
Martin Hořeňovský 89f18f15ca
Add a test for custom debug break macros
See #1846
2020-02-16 15:47:00 +01:00
Martin Hořeňovský 2788897051
Minor cleanup in the benchmarking tests 2020-02-13 15:12:57 +01:00
Martin Hořeňovský cd7d7a1c67
Remove CATCH_CONFIG_ENABLE_BENCHMARKING compilation toggle
Now that Catch2 is a proper library, we can always build the full
library (comparatively minor slowdown) and the user can avoid
including benchmarking headers to avoid the compilation slowdown.
2020-02-06 11:36:46 +01:00
Martin Hořeňovský a3ffc20f57
Provide CTest a hint on test costs (and thus ordering)
When running tests in parallel, CTest runs the tests in decreasing
order of cost (time required), to get the largest speed up from
parallelism. However, the initial cost estimates for all tests are
0, and they are only updated after a test run. This works on a dev
machine, where the tests are ran over and over again, because
eventually the estimates become quite precise, but CI always does
a clean build with 0 estimates.

Because we have 2 slow tests, we want them to run first to avoid
losing parallelism. To do this, we provide them with a cost estimate
manually.
2020-02-03 09:07:23 +01:00
Martin Hořeňovský 6c3a5ef625
Remove CATCH_CONFIG_DISABLE_MATCHERS
Now that the recommended distribution and usage method is proper
library, users can just avoid including the matcher headers to get
basically the same effect.
2020-01-25 09:07:36 +01:00
Martin Hořeňovský 2b696c4388
Piecemeal includes in extra-tests 2020-01-21 15:03:07 +01:00