Add support for multiple parallel reporters

This requires a bunch of different changes across the reporter
subsystem.

* We need to handle multiple reporters and their differing
  preferences in `ListeningReporter`, e.g. what to do when
  we mix reporters that capture and don't capture stdout.
* We need to change how the reporter is given output and
  how we parse reporter's output destination from CLI.
* Approval tests need to handle multireporter option
This commit is contained in:
Martin Jeřábek
2021-02-06 20:12:07 +01:00
committed by Martin Hořeňovský
parent 6b55f5d780
commit ccd67b293d
33 changed files with 51234 additions and 224 deletions

View File

@@ -123,7 +123,7 @@ Test names containing special characters, such as `,` or `[` can specify them on
<a id="choosing-a-reporter-to-use"></a>
## Choosing a reporter to use
<pre>-r, --reporter &lt;reporter></pre>
<pre>-r, --reporter &lt;reporter[::output-file]&gt;</pre>
A reporter is an object that formats and structures the output of running tests, and potentially summarises the results. By default a console reporter is used that writes, IDE friendly, textual output. Catch comes bundled with some alternative reporters, but more can be added in client code.<br />
The bundled reporters are:
@@ -136,6 +136,13 @@ The bundled reporters are:
The JUnit reporter is an xml format that follows the structure of the JUnit XML Report ANT task, as consumed by a number of third-party tools, including Continuous Integration servers such as Jenkins. If not otherwise needed, the standard XML reporter is preferred as this is a streaming reporter, whereas the Junit reporter needs to hold all its results until the end so it can write the overall results into attributes of the root node.
This option may be passed multiple times to use multiple (different) reporters at the same time. See [Reporters](reporters.md#multiple-reporters) for details.
_Note: There is currently no way to escape `::` in the reporter spec,
and thus reporter/file names with `::` in them will not work properly.
As `::` in paths is relatively obscure (unlike `:`), we do not consider
this an issue._
<a id="breaking-into-the-debugger"></a>
## Breaking into the debugger
<pre>-b, --break</pre>
@@ -178,7 +185,7 @@ If one or more test-specs have been supplied too then only the matching tests wi
<a id="sending-output-to-a-file"></a>
## Sending output to a file
<pre>-o, --out &lt;filename>
<pre>-o, --out &lt;filename&gt;
</pre>
Use this option to send all output to a file. By default output is sent to stdout (note that uses of stdout and stderr *from within test cases* are redirected and included in the report - so even stderr will effectively end up on stdout).
@@ -414,7 +421,7 @@ There are some limitations of this feature to be aware of:
- Code outside of sections being skipped will still be executed - e.g. any set-up code in the TEST_CASE before the
start of the first section.</br>
- At time of writing, wildcards are not supported in section names.
- If you specify a section without narrowing to a test case first then all test cases will be executed
- If you specify a section without narrowing to a test case first then all test cases will be executed
(but only matching sections within them).
@@ -422,7 +429,7 @@ start of the first section.</br>
## Filenames as tags
<pre>-#, --filenames-as-tags</pre>
When this option is used then every test is given an additional tag which is formed of the unqualified
When this option is used then every test is given an additional tag which is formed of the unqualified
filename it is found in, with any extension stripped, prefixed with the `#` character.
So, for example, tests within the file `~\Dev\MyProject\Ferrets.cpp` would be tagged `[#Ferrets]`.