mirror of
https://github.com/catchorg/Catch2.git
synced 2024-12-22 19:33:29 +01:00
Rewrite event listener docs and add event documentation
This commit is contained in:
Martin Hořeňovský
parent
dd12ce8141
commit
0ccb1c30c6
@ -1,74 +1,43 @@
|
||||
<a id="top"></a>
|
||||
# Event Listeners
|
||||
|
||||
A `Listener` is a class you can register with Catch that will then be passed events,
|
||||
such as a test case starting or ending, as they happen during a test run.
|
||||
`Listeners` are actually types of `Reporters`, with a few small differences:
|
||||
|
||||
1. Once registered in code they are automatically used - you don't need to specify them on the command line
|
||||
2. They are called in addition to (just before) any reporters, and you can register multiple listeners.
|
||||
3. They derive from `Catch::TestEventListenerBase`, which has default stubs for all the events,
|
||||
so you are not forced to implement events you're not interested in.
|
||||
4. You register a listener with `CATCH_REGISTER_LISTENER`
|
||||
An event listener is a bit like a reporter, in that it responds to various
|
||||
reporter events in Catch2, but it is not expected to write any output.
|
||||
Instead, an event listener performs actions within the test process, such
|
||||
as performing global initialization (e.g. of a C library), or cleaning out
|
||||
in-memory logs if they are not needed (the test case passed).
|
||||
|
||||
Unlike reporters, each registered event listener is always active. Event
|
||||
listeners are always notified before reporter(s).
|
||||
|
||||
## Implementing a Listener
|
||||
Simply derive a class from `Catch::TestEventListenerBase` and implement the methods you are interested in, either in
|
||||
the main source file (i.e. the one that defines `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`), or in a
|
||||
file that defines `CATCH_CONFIG_EXTERNAL_INTERFACES`.
|
||||
To write your own event listener, you should derive from `Catch::TestEventListenerBase`,
|
||||
as it provides empty stubs for all reporter events, allowing you to
|
||||
only override events you care for. Afterwards you have to register it
|
||||
with Catch2 using `CATCH_REGISTER_LISTENER` macro, so that Catch2 knows
|
||||
about it and instantiates it before running tests.
|
||||
|
||||
Then register it using `CATCH_REGISTER_LISTENER`.
|
||||
Example event listener:
|
||||
```cpp
|
||||
#include <catch2/reporters/catch_reporter_event_listener.hpp>
|
||||
#include <catch2/catch_reporter_registrars.hpp>
|
||||
|
||||
For example ([complete source code](../examples/210-Evt-EventListeners.cpp)):
|
||||
class testRunListener : public Catch::EventListenerBase {
|
||||
public:
|
||||
using Catch::EventListenerBase::EventListenerBase;
|
||||
|
||||
```c++
|
||||
#define CATCH_CONFIG_MAIN
|
||||
#include "catch.hpp"
|
||||
|
||||
struct MyListener : Catch::TestEventListenerBase {
|
||||
|
||||
using TestEventListenerBase::TestEventListenerBase; // inherit constructor
|
||||
|
||||
void testCaseStarting( Catch::TestCaseInfo const& testInfo ) override {
|
||||
// Perform some setup before a test case is run
|
||||
}
|
||||
|
||||
void testCaseEnded( Catch::TestCaseStats const& testCaseStats ) override {
|
||||
// Tear-down after a test case is run
|
||||
void testRunStarting(Catch::TestRunInfo const&) override {
|
||||
lib_foo_init();
|
||||
}
|
||||
};
|
||||
CATCH_REGISTER_LISTENER( MyListener )
|
||||
|
||||
CATCH_REGISTER_LISTENER(testRunListener)
|
||||
```
|
||||
|
||||
_Note that you should not use any assertion macros within a Listener!_
|
||||
|
||||
## Events that can be hooked
|
||||
[You can find the list of events that the listeners can react to on its
|
||||
own page](reporter-events.md#top).
|
||||
|
||||
The following are the methods that can be overridden in the Listener:
|
||||
|
||||
```c++
|
||||
// The whole test run, starting and ending
|
||||
virtual void testRunStarting( TestRunInfo const& testRunInfo );
|
||||
virtual void testRunEnded( TestRunStats const& testRunStats );
|
||||
|
||||
// Test cases starting and ending
|
||||
virtual void testCaseStarting( TestCaseInfo const& testInfo );
|
||||
virtual void testCaseEnded( TestCaseStats const& testCaseStats );
|
||||
|
||||
// Sections starting and ending
|
||||
virtual void sectionStarting( SectionInfo const& sectionInfo );
|
||||
virtual void sectionEnded( SectionStats const& sectionStats );
|
||||
|
||||
// Assertions before/ after
|
||||
virtual void assertionStarting( AssertionInfo const& assertionInfo );
|
||||
virtual bool assertionEnded( AssertionStats const& assertionStats );
|
||||
|
||||
// A test is being skipped (because it is "hidden")
|
||||
virtual void skipTest( TestCaseInfo const& testInfo );
|
||||
```
|
||||
|
||||
More information about the events (e.g. name of the test case) is contained in the structs passed as arguments -
|
||||
just look in the source code to see what fields are available.
|
||||
|
||||
---
|
||||
|
||||
|
||||
175
docs/reporter-events.md
Normal file
175
docs/reporter-events.md
Normal file
@ -0,0 +1,175 @@
|
||||
<a id="top"></a>
|
||||
# Reporter events
|
||||
|
||||
**Contents**<br>
|
||||
[Test running events](#test-running-events)<br>
|
||||
[Benchmarking events](#benchmarking-events)<br>
|
||||
[Listings events](#listings-events)<br>
|
||||
[Miscellaneous events](#miscellaneous-events)<br>
|
||||
|
||||
Reporter events are one of the customization points for user code. They
|
||||
are used by [reporters](reporters.md#top) to customize Catch2's output,
|
||||
and by [event listeners](event-listeners.md#top) to perform in-process
|
||||
actions under some conditions.
|
||||
|
||||
There are currently 21 reporter events in Catch2, split between 4 distinct
|
||||
event groups:
|
||||
* test running events (10 events)
|
||||
* benchmarking (4 events)
|
||||
* listings (3 events)
|
||||
* miscellaneous (4 events)
|
||||
|
||||
## Test running events
|
||||
|
||||
Test running events are always paired so that for each `fooStarting` event,
|
||||
there is a `fooEnded` event. This means that the 10 test running events
|
||||
consist of 5 pairs of events:
|
||||
|
||||
* `testRunStarting` and `testRunEnded`,
|
||||
* `testCaseStarting` and `testCaseEnded`,
|
||||
* `testCasePartialStarting` and `testCasePartialEnded`,
|
||||
* `sectionStarting` and `sectionEnded`,
|
||||
* `assertionStarting` and `assertionEnded`
|
||||
|
||||
### `testRun` events
|
||||
|
||||
```cpp
|
||||
void testRunStarting( TestRunInfo const& testRunInfo );
|
||||
void testRunEnded( TestRunStats const& testRunStats );
|
||||
```
|
||||
|
||||
The `testRun` events bookend the entire test run. `testRunStarting` is
|
||||
emitted before the first test case is executed, and `testRunEnded` is
|
||||
emitted after all the test cases have been executed.
|
||||
|
||||
### `testCase` events
|
||||
|
||||
```cpp
|
||||
void testCaseStarting( TestCaseInfo const& testInfo );
|
||||
void testCaseEnded( TestCaseStats const& testCaseStats );
|
||||
```
|
||||
|
||||
The `testCase` events bookend one _full_ run of a specific test case.
|
||||
Individual runs through a test case, e.g. due to `SECTION`s or `GENERATE`s,
|
||||
are handled by a different event.
|
||||
|
||||
|
||||
### `testCasePartial` events
|
||||
|
||||
> Introduced in Catch2 X.Y.Z
|
||||
|
||||
```cpp
|
||||
void testCasePartialStarting( TestCaseInfo const& testInfo, uint64_t partNumber );
|
||||
void testCasePartialEnded(TestCaseStats const& testCaseStats, uint64_t partNumber );
|
||||
```
|
||||
|
||||
`testCasePartial` events bookend one _partial_ run of a specific test case.
|
||||
This means that for any given test case, these events can be emitted
|
||||
multiple times, e.g. due to multiple leaf sections.
|
||||
|
||||
In regards to nesting with `testCase` events, `testCasePartialStarting`
|
||||
will never be emitted before the corresponding `testCaseStarting`, and
|
||||
`testCasePartialEnded` will always be emitted before the corresponding
|
||||
`testCaseEnded`.
|
||||
|
||||
|
||||
### `section` events
|
||||
|
||||
```cpp
|
||||
void sectionStarting( SectionInfo const& sectionInfo );
|
||||
void sectionEnded( SectionStats const& sectionStats );
|
||||
```
|
||||
|
||||
`section` events are emitted only for active `SECTION`s, that is, sections
|
||||
that are entered. Sections that are skipped in this test case run-through
|
||||
do not cause events to be emitted.
|
||||
|
||||
_Note that test cases always contain one implicit section. The event for
|
||||
this section is emitted after the corresponding `testCasePartialStarting`
|
||||
event._
|
||||
|
||||
|
||||
### `assertion` events
|
||||
|
||||
```cpp
|
||||
void assertionStarting( AssertionInfo const& assertionInfo );
|
||||
void assertionEnded( AssertionStats const& assertionStats );
|
||||
```
|
||||
|
||||
`assertionStarting` is called after the expression is captured, but before
|
||||
the assertion expression is evaluated. This might seem like a minor
|
||||
distinction, but what it means is that if you have assertion like
|
||||
`REQUIRE( a + b == c + d )`, then what happens is that `a + b` and `c + d`
|
||||
are evaluated before `assertionStarting` is emitted, while the `==` is
|
||||
evaluated after the event.
|
||||
|
||||
|
||||
## Benchmarking events
|
||||
|
||||
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
|
||||
|
||||
```cpp
|
||||
void benchmarkPreparing( StringRef name ) override;
|
||||
void benchmarkStarting( BenchmarkInfo const& benchmarkInfo ) override;
|
||||
void benchmarkEnded( BenchmarkStats<> const& benchmarkStats ) override;
|
||||
void benchmarkFailed( StringRef error ) override;
|
||||
```
|
||||
|
||||
Due to the benchmark lifecycle being bit more complicated, the benchmarking
|
||||
events have their own category, even though they could be seen as parallel
|
||||
to the `assertion*` events. You should expect running a benchmark to
|
||||
generate at least 2 of the events above.
|
||||
|
||||
To understand the explanation below, you should read the [benchmarking
|
||||
documentation](benchmarks.md#top) first.
|
||||
|
||||
* `benchmarkPreparing` event is sent after the environmental probe
|
||||
finishes, but before the user code is first estimated.
|
||||
* `benchmarkStarting` event is sent after the user code is estimated,
|
||||
but has not been benchmarked yet.
|
||||
* `benchmarkEnded` event is sent after the user code has been benchmarked,
|
||||
and contains the benchmarking results.
|
||||
* `benchmarkFailed` event is sent if either the estimation or the
|
||||
benchmarking itself fails.
|
||||
|
||||
|
||||
## Listings events
|
||||
|
||||
> Introduced in Catch2 X.Y.Z.
|
||||
|
||||
Listings events are events that correspond to the test binary being
|
||||
invoked with `--list-foo` flag.
|
||||
|
||||
There are currently 3 listing events, one for reporters, one for tests,
|
||||
and one for tags. Note that they are not exclusive to each other.
|
||||
|
||||
```cpp
|
||||
void listReporters( std::vector<ReporterDescription> const& descriptions );
|
||||
void listTests( std::vector<TestCaseHandle> const& tests );
|
||||
void listTags( std::vector<TagInfo> const& tagInfos );
|
||||
```
|
||||
|
||||
|
||||
## Miscellaneous events
|
||||
|
||||
```cpp
|
||||
void reportInvalidArguments( StringRef unmatchedSpec );
|
||||
void fatalErrorEncountered( StringRef error );
|
||||
void noMatchingTestCases( StringRef unmatchedSpec );
|
||||
```
|
||||
|
||||
These are one-off events that do not neatly fit into other categories.
|
||||
|
||||
`reportInvalidArguments` is sent for each [test specification command line
|
||||
argument](command-line.md#specifying-which-tests-to-run) that wasn't
|
||||
parsed into a valid spec.
|
||||
|
||||
`fatalErrorEncountered` is sent when Catch2's POSIX signal handling
|
||||
or Windows SE handler is called into with a fatal signal/exception.
|
||||
|
||||
`noMatchingTestCases` is sent for each user provided test specification
|
||||
that did not match any registered tests.
|
||||
|
||||
---
|
||||
|
||||
[Home](Readme.md#top)
|
||||
Loading…
Reference in New Issue
Block a user