mirror of
https://github.com/catchorg/Catch2.git
synced 2024-12-23 03:43:28 +01:00
Rewrite event listener docs and add event documentation
This commit is contained in:
parent
dd12ce8141
commit
0ccb1c30c6
@ -1,74 +1,43 @@
|
|||||||
<a id="top"></a>
|
<a id="top"></a>
|
||||||
# Event Listeners
|
# Event Listeners
|
||||||
|
|
||||||
A `Listener` is a class you can register with Catch that will then be passed events,
|
An event listener is a bit like a reporter, in that it responds to various
|
||||||
such as a test case starting or ending, as they happen during a test run.
|
reporter events in Catch2, but it is not expected to write any output.
|
||||||
`Listeners` are actually types of `Reporters`, with a few small differences:
|
Instead, an event listener performs actions within the test process, such
|
||||||
|
as performing global initialization (e.g. of a C library), or cleaning out
|
||||||
1. Once registered in code they are automatically used - you don't need to specify them on the command line
|
in-memory logs if they are not needed (the test case passed).
|
||||||
2. They are called in addition to (just before) any reporters, and you can register multiple listeners.
|
|
||||||
3. They derive from `Catch::TestEventListenerBase`, which has default stubs for all the events,
|
|
||||||
so you are not forced to implement events you're not interested in.
|
|
||||||
4. You register a listener with `CATCH_REGISTER_LISTENER`
|
|
||||||
|
|
||||||
|
Unlike reporters, each registered event listener is always active. Event
|
||||||
|
listeners are always notified before reporter(s).
|
||||||
|
|
||||||
## Implementing a Listener
|
To write your own event listener, you should derive from `Catch::TestEventListenerBase`,
|
||||||
Simply derive a class from `Catch::TestEventListenerBase` and implement the methods you are interested in, either in
|
as it provides empty stubs for all reporter events, allowing you to
|
||||||
the main source file (i.e. the one that defines `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`), or in a
|
only override events you care for. Afterwards you have to register it
|
||||||
file that defines `CATCH_CONFIG_EXTERNAL_INTERFACES`.
|
with Catch2 using `CATCH_REGISTER_LISTENER` macro, so that Catch2 knows
|
||||||
|
about it and instantiates it before running tests.
|
||||||
|
|
||||||
Then register it using `CATCH_REGISTER_LISTENER`.
|
Example event listener:
|
||||||
|
```cpp
|
||||||
|
#include <catch2/reporters/catch_reporter_event_listener.hpp>
|
||||||
|
#include <catch2/catch_reporter_registrars.hpp>
|
||||||
|
|
||||||
For example ([complete source code](../examples/210-Evt-EventListeners.cpp)):
|
class testRunListener : public Catch::EventListenerBase {
|
||||||
|
public:
|
||||||
|
using Catch::EventListenerBase::EventListenerBase;
|
||||||
|
|
||||||
```c++
|
void testRunStarting(Catch::TestRunInfo const&) override {
|
||||||
#define CATCH_CONFIG_MAIN
|
lib_foo_init();
|
||||||
#include "catch.hpp"
|
|
||||||
|
|
||||||
struct MyListener : Catch::TestEventListenerBase {
|
|
||||||
|
|
||||||
using TestEventListenerBase::TestEventListenerBase; // inherit constructor
|
|
||||||
|
|
||||||
void testCaseStarting( Catch::TestCaseInfo const& testInfo ) override {
|
|
||||||
// Perform some setup before a test case is run
|
|
||||||
}
|
|
||||||
|
|
||||||
void testCaseEnded( Catch::TestCaseStats const& testCaseStats ) override {
|
|
||||||
// Tear-down after a test case is run
|
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
CATCH_REGISTER_LISTENER( MyListener )
|
|
||||||
|
CATCH_REGISTER_LISTENER(testRunListener)
|
||||||
```
|
```
|
||||||
|
|
||||||
_Note that you should not use any assertion macros within a Listener!_
|
_Note that you should not use any assertion macros within a Listener!_
|
||||||
|
|
||||||
## Events that can be hooked
|
[You can find the list of events that the listeners can react to on its
|
||||||
|
own page](reporter-events.md#top).
|
||||||
|
|
||||||
The following are the methods that can be overridden in the Listener:
|
|
||||||
|
|
||||||
```c++
|
|
||||||
// The whole test run, starting and ending
|
|
||||||
virtual void testRunStarting( TestRunInfo const& testRunInfo );
|
|
||||||
virtual void testRunEnded( TestRunStats const& testRunStats );
|
|
||||||
|
|
||||||
// Test cases starting and ending
|
|
||||||
virtual void testCaseStarting( TestCaseInfo const& testInfo );
|
|
||||||
virtual void testCaseEnded( TestCaseStats const& testCaseStats );
|
|
||||||
|
|
||||||
// Sections starting and ending
|
|
||||||
virtual void sectionStarting( SectionInfo const& sectionInfo );
|
|
||||||
virtual void sectionEnded( SectionStats const& sectionStats );
|
|
||||||
|
|
||||||
// Assertions before/ after
|
|
||||||
virtual void assertionStarting( AssertionInfo const& assertionInfo );
|
|
||||||
virtual bool assertionEnded( AssertionStats const& assertionStats );
|
|
||||||
|
|
||||||
// A test is being skipped (because it is "hidden")
|
|
||||||
virtual void skipTest( TestCaseInfo const& testInfo );
|
|
||||||
```
|
|
||||||
|
|
||||||
More information about the events (e.g. name of the test case) is contained in the structs passed as arguments -
|
|
||||||
just look in the source code to see what fields are available.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
175
docs/reporter-events.md
Normal file
175
docs/reporter-events.md
Normal file
@ -0,0 +1,175 @@
|
|||||||
|
<a id="top"></a>
|
||||||
|
# Reporter events
|
||||||
|
|
||||||
|
**Contents**<br>
|
||||||
|
[Test running events](#test-running-events)<br>
|
||||||
|
[Benchmarking events](#benchmarking-events)<br>
|
||||||
|
[Listings events](#listings-events)<br>
|
||||||
|
[Miscellaneous events](#miscellaneous-events)<br>
|
||||||
|
|
||||||
|
Reporter events are one of the customization points for user code. They
|
||||||
|
are used by [reporters](reporters.md#top) to customize Catch2's output,
|
||||||
|
and by [event listeners](event-listeners.md#top) to perform in-process
|
||||||
|
actions under some conditions.
|
||||||
|
|
||||||
|
There are currently 21 reporter events in Catch2, split between 4 distinct
|
||||||
|
event groups:
|
||||||
|
* test running events (10 events)
|
||||||
|
* benchmarking (4 events)
|
||||||
|
* listings (3 events)
|
||||||
|
* miscellaneous (4 events)
|
||||||
|
|
||||||
|
## Test running events
|
||||||
|
|
||||||
|
Test running events are always paired so that for each `fooStarting` event,
|
||||||
|
there is a `fooEnded` event. This means that the 10 test running events
|
||||||
|
consist of 5 pairs of events:
|
||||||
|
|
||||||
|
* `testRunStarting` and `testRunEnded`,
|
||||||
|
* `testCaseStarting` and `testCaseEnded`,
|
||||||
|
* `testCasePartialStarting` and `testCasePartialEnded`,
|
||||||
|
* `sectionStarting` and `sectionEnded`,
|
||||||
|
* `assertionStarting` and `assertionEnded`
|
||||||
|
|
||||||
|
### `testRun` events
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
void testRunStarting( TestRunInfo const& testRunInfo );
|
||||||
|
void testRunEnded( TestRunStats const& testRunStats );
|
||||||
|
```
|
||||||
|
|
||||||
|
The `testRun` events bookend the entire test run. `testRunStarting` is
|
||||||
|
emitted before the first test case is executed, and `testRunEnded` is
|
||||||
|
emitted after all the test cases have been executed.
|
||||||
|
|
||||||
|
### `testCase` events
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
void testCaseStarting( TestCaseInfo const& testInfo );
|
||||||
|
void testCaseEnded( TestCaseStats const& testCaseStats );
|
||||||
|
```
|
||||||
|
|
||||||
|
The `testCase` events bookend one _full_ run of a specific test case.
|
||||||
|
Individual runs through a test case, e.g. due to `SECTION`s or `GENERATE`s,
|
||||||
|
are handled by a different event.
|
||||||
|
|
||||||
|
|
||||||
|
### `testCasePartial` events
|
||||||
|
|
||||||
|
> Introduced in Catch2 X.Y.Z
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
void testCasePartialStarting( TestCaseInfo const& testInfo, uint64_t partNumber );
|
||||||
|
void testCasePartialEnded(TestCaseStats const& testCaseStats, uint64_t partNumber );
|
||||||
|
```
|
||||||
|
|
||||||
|
`testCasePartial` events bookend one _partial_ run of a specific test case.
|
||||||
|
This means that for any given test case, these events can be emitted
|
||||||
|
multiple times, e.g. due to multiple leaf sections.
|
||||||
|
|
||||||
|
In regards to nesting with `testCase` events, `testCasePartialStarting`
|
||||||
|
will never be emitted before the corresponding `testCaseStarting`, and
|
||||||
|
`testCasePartialEnded` will always be emitted before the corresponding
|
||||||
|
`testCaseEnded`.
|
||||||
|
|
||||||
|
|
||||||
|
### `section` events
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
void sectionStarting( SectionInfo const& sectionInfo );
|
||||||
|
void sectionEnded( SectionStats const& sectionStats );
|
||||||
|
```
|
||||||
|
|
||||||
|
`section` events are emitted only for active `SECTION`s, that is, sections
|
||||||
|
that are entered. Sections that are skipped in this test case run-through
|
||||||
|
do not cause events to be emitted.
|
||||||
|
|
||||||
|
_Note that test cases always contain one implicit section. The event for
|
||||||
|
this section is emitted after the corresponding `testCasePartialStarting`
|
||||||
|
event._
|
||||||
|
|
||||||
|
|
||||||
|
### `assertion` events
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
void assertionStarting( AssertionInfo const& assertionInfo );
|
||||||
|
void assertionEnded( AssertionStats const& assertionStats );
|
||||||
|
```
|
||||||
|
|
||||||
|
`assertionStarting` is called after the expression is captured, but before
|
||||||
|
the assertion expression is evaluated. This might seem like a minor
|
||||||
|
distinction, but what it means is that if you have assertion like
|
||||||
|
`REQUIRE( a + b == c + d )`, then what happens is that `a + b` and `c + d`
|
||||||
|
are evaluated before `assertionStarting` is emitted, while the `==` is
|
||||||
|
evaluated after the event.
|
||||||
|
|
||||||
|
|
||||||
|
## Benchmarking events
|
||||||
|
|
||||||
|
> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
void benchmarkPreparing( StringRef name ) override;
|
||||||
|
void benchmarkStarting( BenchmarkInfo const& benchmarkInfo ) override;
|
||||||
|
void benchmarkEnded( BenchmarkStats<> const& benchmarkStats ) override;
|
||||||
|
void benchmarkFailed( StringRef error ) override;
|
||||||
|
```
|
||||||
|
|
||||||
|
Due to the benchmark lifecycle being bit more complicated, the benchmarking
|
||||||
|
events have their own category, even though they could be seen as parallel
|
||||||
|
to the `assertion*` events. You should expect running a benchmark to
|
||||||
|
generate at least 2 of the events above.
|
||||||
|
|
||||||
|
To understand the explanation below, you should read the [benchmarking
|
||||||
|
documentation](benchmarks.md#top) first.
|
||||||
|
|
||||||
|
* `benchmarkPreparing` event is sent after the environmental probe
|
||||||
|
finishes, but before the user code is first estimated.
|
||||||
|
* `benchmarkStarting` event is sent after the user code is estimated,
|
||||||
|
but has not been benchmarked yet.
|
||||||
|
* `benchmarkEnded` event is sent after the user code has been benchmarked,
|
||||||
|
and contains the benchmarking results.
|
||||||
|
* `benchmarkFailed` event is sent if either the estimation or the
|
||||||
|
benchmarking itself fails.
|
||||||
|
|
||||||
|
|
||||||
|
## Listings events
|
||||||
|
|
||||||
|
> Introduced in Catch2 X.Y.Z.
|
||||||
|
|
||||||
|
Listings events are events that correspond to the test binary being
|
||||||
|
invoked with `--list-foo` flag.
|
||||||
|
|
||||||
|
There are currently 3 listing events, one for reporters, one for tests,
|
||||||
|
and one for tags. Note that they are not exclusive to each other.
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
void listReporters( std::vector<ReporterDescription> const& descriptions );
|
||||||
|
void listTests( std::vector<TestCaseHandle> const& tests );
|
||||||
|
void listTags( std::vector<TagInfo> const& tagInfos );
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## Miscellaneous events
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
void reportInvalidArguments( StringRef unmatchedSpec );
|
||||||
|
void fatalErrorEncountered( StringRef error );
|
||||||
|
void noMatchingTestCases( StringRef unmatchedSpec );
|
||||||
|
```
|
||||||
|
|
||||||
|
These are one-off events that do not neatly fit into other categories.
|
||||||
|
|
||||||
|
`reportInvalidArguments` is sent for each [test specification command line
|
||||||
|
argument](command-line.md#specifying-which-tests-to-run) that wasn't
|
||||||
|
parsed into a valid spec.
|
||||||
|
|
||||||
|
`fatalErrorEncountered` is sent when Catch2's POSIX signal handling
|
||||||
|
or Windows SE handler is called into with a fatal signal/exception.
|
||||||
|
|
||||||
|
`noMatchingTestCases` is sent for each user provided test specification
|
||||||
|
that did not match any registered tests.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
[Home](Readme.md#top)
|
Loading…
Reference in New Issue
Block a user