Rewrite event listener docs and add event documentation

docs/event-listeners.md

@@ -1,74 +1,43 @@
 <a id="top"></a>
 # Event Listeners
 
-A `Listener` is a class you can register with Catch that will then be passed events,
+An event listener is a bit like a reporter, in that it responds to various
-such as a test case starting or ending, as they happen during a test run.
+reporter events in Catch2, but it is not expected to write any output.
-`Listeners` are actually types of `Reporters`, with a few small differences:
+Instead, an event listener performs actions within the test process, such
+as performing global initialization (e.g. of a C library), or cleaning out
+in-memory logs if they are not needed (the test case passed).
 
-1. Once registered in code they are automatically used - you don't need to specify them on the command line
+Unlike reporters, each registered event listener is always active. Event
-2. They are called in addition to (just before) any reporters, and you can register multiple listeners.
+listeners are always notified before reporter(s).
-3. They derive from `Catch::TestEventListenerBase`, which has default stubs for all the events,
-so you are not forced to implement events you're not interested in.
-4. You register a listener with `CATCH_REGISTER_LISTENER`
 
+To write your own event listener, you should derive from `Catch::TestEventListenerBase`,
+as it provides empty stubs for all reporter events, allowing you to
+only override events you care for. Afterwards you have to register it
+with Catch2 using `CATCH_REGISTER_LISTENER` macro, so that Catch2 knows
+about it and instantiates it before running tests.
 
-## Implementing a Listener
+Example event listener:
-Simply derive a class from `Catch::TestEventListenerBase` and implement the methods you are interested in, either in
+```cpp
-the main source file (i.e. the one that defines `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`), or in a
+#include <catch2/reporters/catch_reporter_event_listener.hpp>
-file that defines `CATCH_CONFIG_EXTERNAL_INTERFACES`.
+#include <catch2/catch_reporter_registrars.hpp>
 
-Then register it using `CATCH_REGISTER_LISTENER`.
+class testRunListener : public Catch::EventListenerBase {
+public:
+    using Catch::EventListenerBase::EventListenerBase;
 
-For example ([complete source code](../examples/210-Evt-EventListeners.cpp)):
+    void testRunStarting(Catch::TestRunInfo const&) override {
-
+        lib_foo_init();
-```c++
-#define CATCH_CONFIG_MAIN
-#include "catch.hpp"
-
-struct MyListener : Catch::TestEventListenerBase {
-
-    using TestEventListenerBase::TestEventListenerBase; // inherit constructor
-
-    void testCaseStarting( Catch::TestCaseInfo const& testInfo ) override {
-        // Perform some setup before a test case is run
-    }
-    
-    void testCaseEnded( Catch::TestCaseStats const& testCaseStats ) override {
-        // Tear-down after a test case is run
     }
 };
-CATCH_REGISTER_LISTENER( MyListener )
+
+CATCH_REGISTER_LISTENER(testRunListener)
 ```
 
 _Note that you should not use any assertion macros within a Listener!_ 
 
-## Events that can be hooked
+[You can find the list of events that the listeners can react to on its
+own page](reporter-events.md#top).
 
-The following are the methods that can be overridden in the Listener:
-
-```c++
-// The whole test run, starting and ending
-virtual void testRunStarting( TestRunInfo const& testRunInfo );
-virtual void testRunEnded( TestRunStats const& testRunStats );
-
-// Test cases starting and ending
-virtual void testCaseStarting( TestCaseInfo const& testInfo );
-virtual void testCaseEnded( TestCaseStats const& testCaseStats );
-
-// Sections starting and ending
-virtual void sectionStarting( SectionInfo const& sectionInfo );
-virtual void sectionEnded( SectionStats const& sectionStats );
-
-// Assertions before/ after
-virtual void assertionStarting( AssertionInfo const& assertionInfo );
-virtual bool assertionEnded( AssertionStats const& assertionStats );
-
-// A test is being skipped (because it is "hidden")
-virtual void skipTest( TestCaseInfo const& testInfo );
-```
-
-More information about the events (e.g. name of the test case) is contained in the structs passed as arguments -
-just look in the source code to see what fields are available. 
 
 ---
 

docs/reporter-events.md

@@ -0,0 +1,175 @@
+<a id="top"></a>
+# Reporter events
+
+**Contents**<br>
+[Test running events](#test-running-events)<br>
+[Benchmarking events](#benchmarking-events)<br>
+[Listings events](#listings-events)<br>
+[Miscellaneous events](#miscellaneous-events)<br>
+
+Reporter events are one of the customization points for user code. They
+are used by [reporters](reporters.md#top) to customize Catch2's output,
+and by [event listeners](event-listeners.md#top) to perform in-process
+actions under some conditions.
+
+There are currently 21 reporter events in Catch2, split between 4 distinct
+event groups:
+* test running events (10 events)
+* benchmarking (4 events)
+* listings (3 events)
+* miscellaneous (4 events)
+
+## Test running events
+
+Test running events are always paired so that for each `fooStarting` event,
+there is a `fooEnded` event. This means that the 10 test running events
+consist of 5 pairs of events:
+
+* `testRunStarting` and `testRunEnded`,
+* `testCaseStarting` and `testCaseEnded`,
+* `testCasePartialStarting` and `testCasePartialEnded`,
+* `sectionStarting` and `sectionEnded`,
+* `assertionStarting` and `assertionEnded`
+
+### `testRun` events
+
+```cpp
+void testRunStarting( TestRunInfo const& testRunInfo );
+void testRunEnded( TestRunStats const& testRunStats );
+```
+
+The `testRun` events bookend the entire test run. `testRunStarting` is
+emitted before the first test case is executed, and `testRunEnded` is
+emitted after all the test cases have been executed.
+
+### `testCase` events
+
+```cpp
+void testCaseStarting( TestCaseInfo const& testInfo );
+void testCaseEnded( TestCaseStats const& testCaseStats );
+```
+
+The `testCase` events bookend one _full_ run of a specific test case.
+Individual runs through a test case, e.g. due to `SECTION`s or `GENERATE`s,
+are handled by a different event.
+
+
+### `testCasePartial` events
+
+> Introduced in Catch2 X.Y.Z
+
+```cpp
+void testCasePartialStarting( TestCaseInfo const& testInfo, uint64_t partNumber );
+void testCasePartialEnded(TestCaseStats const& testCaseStats, uint64_t partNumber );
+```
+
+`testCasePartial` events bookend one _partial_ run of a specific test case.
+This means that for any given test case, these events can be emitted
+multiple times, e.g. due to multiple leaf sections.
+
+In regards to nesting with `testCase` events, `testCasePartialStarting`
+will never be emitted before the corresponding `testCaseStarting`, and
+`testCasePartialEnded` will always be emitted before the corresponding
+`testCaseEnded`.
+
+
+### `section` events
+
+```cpp
+void sectionStarting( SectionInfo const& sectionInfo );
+void sectionEnded( SectionStats const& sectionStats );
+```
+
+`section` events are emitted only for active `SECTION`s, that is, sections
+that are entered. Sections that are skipped in this test case run-through
+do not cause events to be emitted.
+
+_Note that test cases always contain one implicit section. The event for
+this section is emitted after the corresponding `testCasePartialStarting`
+event._
+
+
+### `assertion` events
+
+```cpp
+void assertionStarting( AssertionInfo const& assertionInfo );
+void assertionEnded( AssertionStats const& assertionStats );
+```
+
+`assertionStarting` is called after the expression is captured, but before
+the assertion expression is evaluated. This might seem like a minor
+distinction, but what it means is that if you have assertion like
+`REQUIRE( a + b == c + d )`, then what happens is that `a + b` and `c + d`
+are evaluated before `assertionStarting` is emitted, while the `==` is
+evaluated after the event.
+
+
+## Benchmarking events
+
+> [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0.
+
+```cpp
+void benchmarkPreparing( StringRef name ) override;
+void benchmarkStarting( BenchmarkInfo const& benchmarkInfo ) override;
+void benchmarkEnded( BenchmarkStats<> const& benchmarkStats ) override;
+void benchmarkFailed( StringRef error ) override;
+```
+
+Due to the benchmark lifecycle being bit more complicated, the benchmarking
+events have their own category, even though they could be seen as parallel
+to the `assertion*` events. You should expect running a benchmark to
+generate at least 2 of the events above.
+
+To understand the explanation below, you should read the [benchmarking
+documentation](benchmarks.md#top) first.
+
+* `benchmarkPreparing` event is sent after the environmental probe
+finishes, but before the user code is first estimated.
+* `benchmarkStarting` event is sent after the user code is estimated,
+but has not been benchmarked yet.
+* `benchmarkEnded` event is sent after the user code has been benchmarked,
+and contains the benchmarking results.
+* `benchmarkFailed` event is sent if either the estimation or the
+benchmarking itself fails.
+
+
+## Listings events
+
+> Introduced in Catch2 X.Y.Z.
+
+Listings events are events that correspond to the test binary being
+invoked with `--list-foo` flag. 
+
+There are currently 3 listing events, one for reporters, one for tests,
+and one for tags. Note that they are not exclusive to each other.
+
+```cpp
+void listReporters( std::vector<ReporterDescription> const& descriptions );
+void listTests( std::vector<TestCaseHandle> const& tests );
+void listTags( std::vector<TagInfo> const& tagInfos );
+```
+
+
+## Miscellaneous events
+
+```cpp
+void reportInvalidArguments( StringRef unmatchedSpec );
+void fatalErrorEncountered( StringRef error );
+void noMatchingTestCases( StringRef unmatchedSpec );
+```
+
+These are one-off events that do not neatly fit into other categories.
+
+`reportInvalidArguments` is sent for each [test specification command line
+argument](command-line.md#specifying-which-tests-to-run) that wasn't
+parsed into a valid spec.
+
+`fatalErrorEncountered` is sent when Catch2's POSIX signal handling
+or Windows SE handler is called into with a fatal signal/exception.
+
+`noMatchingTestCases` is sent for each user provided test specification
+that did not match any registered tests.
+
+---
+
+[Home](Readme.md#top)