Merged from develop

This commit is contained in:
Phil Nash
2015-03-27 17:54:45 +00:00
83 changed files with 4428 additions and 1430 deletions

View File

@@ -9,6 +9,8 @@ Before looking at this material be sure to read the [tutorial](tutorial.md)
* [Command line](command-line.md)
* [Build systems](build-systems.md)
* [Supplying your own main()](own-main.md)
* [Configuration](configuration.md)
* [String Conversions](tostring.md)
* [Why are my tests slow to compile?](slow-compiles.md)
Other
@@ -16,8 +18,3 @@ Other
* [Why Catch?](why-catch.md)
* [What's changed](whats-changed.md)
* [Contributing](contributing.md)
---
[Home](../README.md)

View File

@@ -17,12 +17,12 @@ The ```CHECK``` family are equivalent but execution continues in the same test c
Evaluates the expression and records the result. If an exception is thrown it is caught, reported, and counted as a failure. These are the macros you will use most of the time
Examples:
```c++
```
CHECK( str == "string value" );
CHECK( thisReturnsTrue() );
REQUIRE( i == 42 );
```
* **REQUIRE_FALSE(** _expression_ **)** and
* **CHECK_FALSE(** _expression_ **)**
@@ -30,10 +30,28 @@ Evaluates the expression and records the _logical NOT_ of the result. If an exce
(these forms exist as a workaround for the fact that ! prefixed expressions cannot be decomposed).
Example:
```c++
```
REQUIRE_FALSE( thisReturnsFalse() );
```
### Floating point comparisons
When comparing floating point numbers - especially if at least one of them has been computed - great care must be taken to allow for rounding errors and inexact representations.
Catch provides a way to perform tolerant comparisons of floating point values through use of a wrapper class called ```Approx```. ```Approx``` can be used on either side of a comparison expression. It overloads the comparisons operators to take a tolerance into account. Here's a simple example:
```
REQUIRE( performComputation() == Approx( 2.1 ) );
```
By default a small epsilon value is used that covers many simple cases of rounding errors. When this is insufficent the epsilon value (the amount within which a difference either way is ignored) can be specified by calling the ```epsilon()``` method on the ```Approx``` instance. e.g.:
```
REQUIRE( 22/7 == Approx( 3.141 ).epsilon( 0.01 ) );
```
When dealing with very large or very small numbers it can be useful to specify a scale, which can be achieved by calling the ```scale()``` method on the ```Approx``` instance.
## Exceptions
* **REQUIRE_THROWS(** _expression_ **)** and
@@ -61,4 +79,4 @@ To support Matchers a slightly different form is used. Matchers will be more ful
---
[Home](../README.md)
[Home](Readme.md)

View File

@@ -1,5 +1,50 @@
# Integration with build systems
Build Systems may refer to low-level tools, like CMake, or larger systems that run on servers, like Jenkins or TeamCity. This page will talk about both.
# Continuous Integration systems
Probably the most important aspect to using Catch with a build server is the use of different reporters. Catch comes bundled with three reporters that should cover the majority of build servers out there - although adding more for better integration with some is always a possibility (as has been done with TeamCity).
Two of these reporters are built in (XML and JUnit) and the third (TeamCity) is included as a separate header. It's possible that the other two may be split out in the future too - as that would make the core of Catch smaller for those that don't need them.
## XML Reporter
```-r xml```
The XML Reporter writes in an XML format that is specific to Catch.
The advantage of this format is that it corresponds well to the way Catch works (especially the more unusual features, such as nested sections) and is a fully streaming format - that is it writes output as it goes, without having to store up all its results before it can start writing.
The disadvantage is that, being specific to Catch, no existing build servers understand the format natively. It can be used as input to an XSLT transformation that could covert it to, say, HTML - although this loses the streaming advantage, of course.
## JUnit Reporter
```-r junit```
The JUnit Reporter writes in an XML format that mimics the JUnit ANT schema.
The advantage of this format is that the JUnit Ant schema is widely understood by most build servers and so can usually be consumed with no additional work.
The disadvantage is that this schema was designed to correspond to how JUnit works - and there is a significant mismatch with how Catch works. Additionally the format is not streamable (because opening elements hold counts of failed and passing tests as attributes) - so the whole test run must complete before it can be written.
## TeamCity Reporter
```-r teamcity```
The TeamCity Reporter writes TeamCity service messages to stdout. In order to be able to use this reporter an additional header must also be included.
```catch_reporter_teamcity.hpp``` can be found in the ```include\reporters``` directory. It should be included in the same file that ```#define```s ```CATCH_CONFIG_MAIN``` or ```CATCH_CONFIG_RUNNER```. The ```#include``` should be placed after ```#include```ing Catch itself.
e.g.:
```
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
#include "catch_reporter_teamcity.hpp"
```
Being specific to TeamCity this is the best reporter to use with it - but it is completely unsuitable for any other purpose. It is a streaming format (it writes as it goes) - although test results don't appear in the TeamCity interface until the completion of a suite (usually the whole test run).
# Low-level tools
## CMake
You can use the following CMake script to automatically fetch Catch from github and configure it as an external project:
@@ -35,3 +80,7 @@ add_subdirectory(${EXT_PROJECTS_DIR}/catch)
include_directories(${CATCH_INCLUDE_DIR} ${COMMON_INCLUDES})
enable_testing(true) # Enables unit-testing.
```
---
[Home](Readme.md)

View File

@@ -1,18 +1,33 @@
Catch works quite nicely without any command line options at all - but for those times when you want greater control the following options are available.
Note that options are described according to the following pattern:
Click one of the followings links to take you straight to that option - or scroll on to browse the available options.
<a href="#specifying-which-tests-to-run"> ` <test-spec> ...`</a><br />
<a href="#choosing-a-reporter-to-use"> ` -r, --reporter`</a><br />
<a href="#breaking-into-the-debugger"> ` -b, --break`</a><br />
<a href="#usage"> ` -h, -?, --help`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -l, --list-tests`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -t, --list-tags`</a><br />
<a href="#showing-results-for-successful-tests"> ` -s, --success`</a><br />
<a href="#aborting-after-a-certain-number-of-failures"> ` -a, --abort`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` -l, --list`</a><br />
<a href="#sending-output-to-a-file"> ` -o, --out`</a><br />
<a href="#naming-a-test-run"> ` -n, --name`</a><br />
<a href="#breaking-into-the-debugger"> ` -b, --break`</a><br />
<a href="#eliding-assertions-expected-to-throw"> ` -e, --nothrow`</a><br />
<a href="#invisibles"> ` -i, --invisibles`</a><br />
<a href="#sending-output-to-a-file"> ` -o, --out`</a><br />
<a href="#choosing-a-reporter-to-use"> ` -r, --reporter`</a><br />
<a href="#naming-a-test-run"> ` -n, --name`</a><br />
<a href="#aborting-after-a-certain-number-of-failures"> ` -a, --abort`</a><br />
<a href="#aborting-after-a-certain-number-of-failures"> ` -x, --abortx`</a><br />
<a href="#warnings"> ` -w, --warn`</a><br />
<a href="#reporting-timings"> ` -d, --durations`</a><br />
<a href="#usage"> ` -h, -?, --help`</a><br />
<a href="#input-file"> ` -f, --input-file`</a><br />
</br>
<a href="#list-test-names-only"> ` --list-test-names-only`</a><br />
<a href="#listing-available-tests-tags-or-reporters"> ` --list-reporters`</a><br />
<a href="#order"> ` --order`</a><br />
<a href="#rng-seed"> ` --rng-seed`</a><br />
</br>
<a id="specifying-which-tests-to-run"></a>
## Specifying which tests to run
@@ -21,7 +36,8 @@ Note that options are described according to the following pattern:
Test cases, wildcarded test cases, tags and tag expressions are all passed directly as arguments. Tags are distinguished by being enclosed in square brackets.
If no test specs are supplied then all test cases, except "hidden" tests (tagged ```[hide]```, ```[.]``` or, in the legacy case, prefixed by `'./'`) are run.
If no test specs are supplied then all test cases, except "hidden" tests, are run.
A test is hidden by giving it any tag starting with (or just) a period (```.```) - or, in the deprecated case, tagged ```[hide]``` or given name starting with `'./'`. To specify hidden tests from the command line ```[.]``` or ```[hide]``` can be used *regardless of how they were declared*.
Specs must be enclosed in quotes if they contain spaces. If they do not contain spaces the quotes are optional.
@@ -79,7 +95,7 @@ In addition to the command line option, ensure you have built your code with the
<pre>-s, --success</pre>
Usually you only want to see reporting for failed tests. Sometimes it's useful to see *all* the output (especially when you don't trust that that test you just added worked first time!).
To see successul, as well as failing, test results just pass this option. Note that each reporter may treat this option differently. The Junit reporter, for example, logs all results regardless.
To see successful, as well as failing, test results just pass this option. Note that each reporter may treat this option differently. The Junit reporter, for example, logs all results regardless.
<a id="aborting-after-a-certain-number-of-failures"></a>
## Aborting after a certain number of failures
@@ -131,6 +147,13 @@ Sometimes exceptions are expected outside of one of the assertions that tests fo
When running with this option any throw checking assertions are skipped so as not to contribute additional noise. Be careful if this affects the behaviour of subsequent tests.
<a id="invisibles"></a>
## Make whitespace visible
<pre>-i, --invisibles</pre>
If a string comparison fails due to differences in whitespace - especially leading or trailing whitespace - it can be hard to see what's going on.
This option transforms tabs and newline characters into ```\t``` and ```\n``` respectively when printing.
<a id="warnings"></a>
## Warnings
<pre>-w, --warn &lt;warning name></pre>
@@ -145,6 +168,47 @@ The ony available warning, presently, is ```NoAssertions```. This warning fails
When set to ```yes``` Catch will report the duration of each test case, in milliseconds. Note that it does this regardless of whether a test case passes or fails. Note, also, the certain reporters (e.g. Junit) always report test case durations regardless of this option being set or not.
<a id="input-file"></a>
## Load test names to run from a file
<pre>-f, --input-file &lt;filename></pre>
Provide the name of a file that contains a list of test case names - one per line. Blank lines are skipped and anything after the comment character, ```#```, is ignored.
A useful way to generate an initial instance of this file is to use the <a href="#list-test-names-only">list-test-names-only</a> option. This can then be manually curated to specify a specific subset of tests - or in a specific order.
<a id="list-test-names-only"></a>
## Just test names
<pre>--list-test-names-only</pre>
This option lists all available tests in a non-indented form, one on each line. This makes it ideal for saving to a file and feeding back into the <a href="#input-file">```-f``` or ```--input-file```</a> option.
<a id="order"></a>
## Specify the order test cases are run
<pre>--order &lt;decl|lex|rand&gt;</pre>
Test cases are ordered one of three ways:
### decl
Declaration order. The order the tests were originally declared in. Note that ordering between files is not guaranteed and is implementation dependent.
### lex
Lexicographically sorted. Tests are sorted, alpha-numerically, by name.
### rand
Randomly sorted. Test names are sorted using ```std::random_shuffle()```. By default the random number generator is seeded with 0 - and so the order is repeatable. To control the random seed see <a href="#rng-seed">rng-seed</a>.
<a id="rng-seed"></a>
## Specify a seed for the Random Number Generator
<pre>--rng-seed &lt;'time'|number&gt;</pre>
Sets a seed for the random number generator using ```std::srand()```.
If a number is provided this is used directly as the seed so the random pattern is repeatable.
Alternatively if the keyword ```time``` is provided then the result of calling ```std::time(0)``` is used and so the pattern becomes unpredictable.
In either case the actual value for the seed is printed as part of Catch's output so if an issue is discovered that is sensitive to test ordering the ordering can be reproduced - even if it was originally seeded from ```std::time(0)```.
<a id="usage"></a>
## Usage
<pre>-h, -?, --help</pre>
@@ -153,4 +217,4 @@ Prints the command line arguments to stdout
---
[Home](../README.md)
[Home](Readme.md)

66
docs/configuration.md Normal file
View File

@@ -0,0 +1,66 @@
Catch is designed to "just work" as much as possible. For most people the only configuration needed is telling Catch which source file should host all the implementation code (```CATCH_CONFIG_MAIN```).
Nonetheless there are still some occasions where finer control is needed. For these occasions Catch exposes a small set of macros for configuring how it is built.
# main()/ implementation
CATCH_CONFIG_MAIN // Designates this as implementation file and defines main()
CATCH_CONFIG_RUNNER // Designates this as implementation file
Although Catch is header only it still, internally, maintains a distinction between interface headers and headers that contain implementation. Only one source file in your test project should compile the implementation headers and this is controlled through the use of one of these macros - one of these identifiers should be defined before including Catch in *exactly one implementation file in your project*.
# Prefixing Catch macros
CATCH_CONFIG_PREFIX_ALL
To keep test code clean and uncluttered Catch uses short macro names (e.g. ```TEST_CASE``` and ```REQUIRE```). Occasionally these may conflict with identifiers from platform headers or the system under test. In this case the above identifier can be defined. This will cause all the Catch user macros to be prefixed with ```CATCH_``` (e.g. ```CATCH_TEST_CASE``` and ```CATCH_REQUIRE```).
# Terminal colour
CATCH_CONFIG_COLOUR_NONE // completely disables all text colouring
CATCH_CONFIG_COLOUR_WINDOWS // forces the Win32 console API to be used
CATCH_CONFIG_COLOUR_ANSI // forces ANSI colour codes to be used
Yes, I am English, so I will continue to spell "colour" with a 'u'.
When sending output to the terminal, if it detects that it can, Catch will use colourised text. On Windows the Win32 API, ```SetConsoleTextAttribute```, is used. On POSIX systems ANSI colour escape codes are inserted into the stream.
For finer control you can define one of the above identifiers (these are mutually exclusive - but that is not checked so may behave unexpectedly if you mix them):
Note that when ANSI colour codes are used "unistd.h" must be includable - along with a definition of ```isatty()```
Typically you should place the ```#define``` before #including "catch.hpp" in your main source file - but if you prefer you can define it for your whole project by whatever your IDE or build system provides for you to do so.
# Console width
CATCH_CONFIG_CONSOLE_WIDTH = x // where x is a number
Catch formats output intended for the console to fit within a fixed number of characters. This is especially important as indentation is used extensively and uncontrolled line wraps break this.
By default a console width of 80 is assumed but this can be controlled by defining the above identifier to be a different value.
# stdout
CATCH_CONFIG_NOSTDOUT
Catch does not use ```std::cout``` and ```std::cerr``` directly but gets them from ```Catch::cout()``` and ```Catch::cerr()``` respectively. If the above identifier is defined these functions are left unimplemented and you must implement them yourself. Their signatures are:
std::ostream& cout();
std::ostream& cerr();
This can be useful on certain platforms that do not provide ```std::cout``` and ```std::cerr```, such as certain embedded systems.
# C++ conformance toggles
CATCH_CONFIG_CPP11_NULLPTR
CATCH_CONFIG_CPP11_NOEXCEPT
CATCH_CONFIG_SFINAE // Basic, C++03, support for SFINAE
CATCH_CONFIG_VARIADIC_MACROS // Usually pre-C++11 compiler extensions are sufficient
CATCH_CONFIG_NO_VARIADIC_MACROS // Suppress if Catch is too eager to enable it
Catch has some basic compiler detection that will attempt to select the appropriate mix of these macros. However being incomplete - and often without access to the respective compilers - this detection tends to be conservative.
So overriding control is given to the user. If a compiler supports a feature (and Catch does not already detect it) then one or more of these may be defined to enable it (or suppress it, in some cases). If you do do this please raise an issue, specifying your compiler version (ideally with an idea of how to detect it) and stating that it has such support.
---
[Home](Readme.md)

View File

@@ -16,8 +16,8 @@ In addition to the include files and IDE projects there are a number of tests in
The other directories are ```scripts``` which contains a set of python scripts to help in testing Catch as well as generating the single include, and docs, which contains the documentation as a set of markdown files.
*This document is a work in progress and will be updated soon with new information.*
*this document is in-progress...*
---
[Home](../README.md)
[Home](Readme.md)

View File

@@ -49,4 +49,4 @@ These macros are now deprecated and are just aliases for INFO and CAPTURE (which
---
[Home](../README.md)
[Home](Readme.md)

View File

@@ -47,9 +47,9 @@ int main( int argc, char* const argv[] )
if( returnCode != 0 ) // Indicates a command line error
return returnCode;
// Writing to session.configData() or session.Config() here
// overrides command line args.
// Only do this if you know you need to.
// writing to session.configData() or session.Config() here
// overrides command line args
// only do this if you know you need to
return session.run();
}
@@ -65,4 +65,4 @@ Catch embeds a powerful command line parser which you can also use to parse your
---
[Home](../README.md)
[Home](Readme.md)

View File

@@ -19,4 +19,4 @@ As a result the main source file *does* compile the whole of Catch every time! S
---
[Home](../README.md)
[Home](Readme.md)

View File

@@ -9,7 +9,7 @@ Test cases and sections are very easy to use in practice:
* **TEST_CASE(** _test name_ \[, _tags_ \] **)**
* **SECTION(** _section name_ **)**
_test name_ and _section name_ are free-form quoted strings. The optional _tags_ argument is a quoted string containing one or more tags enclosed in square brackets. Tags are discussed below. Test names must be unique within the Catch executable.
_test name_ and _section name_ are free form, quoted, strings. The optional _tags_ argument is a quoted string containing one or more tags enclosed in square brackets. Tags are discussed below. Test names must be unique within the Catch executable.
For examples see the [Tutorial](tutorial.md)
@@ -57,4 +57,4 @@ Other than the additional prefixes and the formatting in the console reporter th
---
[Home](../README.md)
[Home](Readme.md)

View File

@@ -1,4 +1,4 @@
Although Catch allows you to group tests together as sections within a test case, it can still be convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too. You define the test fixture as a simple structure:
Although Catch allows you to group tests together as sections within a test case, it can still convenient, sometimes, to group them using a more traditional test fixture. Catch fully supports this too. You define the test fixture as a simple structure:
```c++
class UniqueTestsFixture {
@@ -29,4 +29,4 @@ The two test cases here will create uniquely-named derived classes of UniqueTest
---
[Home](../README.md)
[Home](Readme.md)

46
docs/tostring.md Normal file
View File

@@ -0,0 +1,46 @@
# String conversions
Catch needs to be able to convert types you use in assertions and logging expressions into strings (for logging and reporting purposes).
Most built-in or std types are supported out of the box but there are two ways that you can tell Catch how to convert your own types (or other, third-party types) into strings.
## operator << overload for std::ostream
This is the standard way of providing string conversions in C++ - and the chances are you may already provide this for your own purposes. If you're not familiar with this idiom it involves writing a free function of the form:
```
std::ostream& operator << ( std::ostream& os, T const& value ) {
os << convertMyTypeToString( value );
return os;
}
```
(where ```T``` is your type and ```convertMyTypeToString``` is where you'll write whatever code is necessary to make your type printable - it doesn't have to be in another function).
You should put this function in the same namespace as your type.
Alternatively you may prefer to write it as a member function:
```
std::ostream& T::operator << ( std::ostream& os ) const {
os << convertMyTypeToString( *this );
return os;
}
```
## Catch::toString overload
If you don't want to provide an ```operator <<``` overload, or you want to convert your type differently for testing purposes, you can provide an overload for ```Catch::toString()``` for your type.
```
namespace Catch {
std::string toString( T const& value ) {
return convertMyTypeToString( value );
}
}
```
Again ```T``` is your type and ```convertMyTypeToString``` is where you'll write whatever code is necessary to make your type printable. Note that the function must be in the Catch namespace, which itself must be in the global namespace.
---
[Home](Readme.md)

View File

@@ -86,10 +86,10 @@ Of course there are still more issues to do deal with. For example we'll hit pro
Although this was a simple test it's been enough to demonstrate a few things about how Catch is used. Let's take moment to consider those before we move on.
1. All we did was ```#define``` one identifier and ```#include``` one header and we got everything - even an implementation of ```main()``` that will [respond to command line arguments](command-line.md). You can only use that ```#define``` in one implementation file, for (hopefully) obvious reasons. Once you have more than one file with unit tests in you'll just ```#include "catch.hpp"``` and go. Usually it's a good idea to have a dedicated implementation file that just has ```#define CATCH_CONFIG_MAIN``` and ```#include "catch.hpp"```. You can also provide your own implementation of main and drive Catch yourself (see [Supplying-your-own-main()](own-main.md).
2. We introduce test cases with the TEST_CASE macro. This macro takes one or two arguments - a free form test name and, optionally, one or more tags (for more see <a href="#test-cases-and-sections">Test cases and Sections</a>, below. The test name must be unique. You can run sets of tests by specifying a wildcarded test name or a tag expression. See the [command line docs](command-line.md) for more information on running tests.
1. All we did was ```#define``` one identifier and ```#include``` one header and we got everything - even an implementation of ```main()``` that will [respond to command line arguments](command-line.md). You can only use that ```#define``` in one implementation file, for (hopefully) obvious reasons. Once you have more than one file with unit tests in you'll just ```#include "catch.hpp"``` and go. Usually it's a good idea to have a dedicated implementation file that just has ```#define CATCH_CONFIG_MAIN``` and ```#include "catch.hpp"```. You can also provide your own implementation of main and drive Catch yourself (see [Supplying-your-own-main()](own-main.md)).
2. We introduce test cases with the ```TEST_CASE``` macro. This macro takes one or two arguments - a free form test name and, optionally, one or more tags (for more see <a href="#test-cases-and-sections">Test cases and Sections</a>, below. The test name must be unique. You can run sets of tests by specifying a wildcarded test name or a tag expression. See the [command line docs](command-line.md) for more information on running tests.
3. The name and tags arguments are just strings. We haven't had to declare a function or method - or explicitly register the test case anywhere. Behind the scenes a function with a generated name is defined for you, and automatically registered using static registry classes. By abstracting the function name away we can name our tests without the constraints of identifier names.
4. We write our individual test assertions using the REQUIRE macro. Rather than a separate macro for each type of condition we express the condition naturally using C/C++ syntax. Behind the scenes a simple set of expression templates captures the left-hand-side and right-hand-side of the expression so we can display the values in our test report. As we'll see later there _are_ other assertion macros - but because of this technique the number of them is drastically reduced.
4. We write our individual test assertions using the ```REQUIRE``` macro. Rather than a separate macro for each type of condition we express the condition naturally using C/C++ syntax. Behind the scenes a simple set of expression templates captures the left-hand-side and right-hand-side of the expression so we can display the values in our test report. As we'll see later there _are_ other assertion macros - but because of this technique the number of them is drastically reduced.
<a id="test-cases-and-sections"></a>
## Test cases and sections
@@ -157,7 +157,7 @@ The power of sections really shows, however, when we need to execute a sequence
}
```
Sections can be nested to an arbitrary depth (limited only by your stack size). Each leaf section (i.e. a section that contains no nested sections) will be executed exactly once, on a separate path of execution from any other leaf section (so no leaf section can interfere with another). Obviously a failure in a parent section will prevent nested sections from running - but that's the idea.
Sections can be nested to an arbitrary depth (limited only by your stack size). Each leaf section (i.e. a section that contains no nested sections) will be executed exactly once, on a separate path of execution from any other leaf section (so no leaf section can interfere with another). A failure in a parent section will prevent nested sections from running - but then that's the idea.
## BDD-Style
@@ -220,9 +220,11 @@ Scenario: vectors can be sized and resized
```
## Next steps
For more specific information see the [Reference pages](Readme.md)
This has been a brief introduction to get you up and running with Catch, and to point out some of the key differences between Catch and other frameworks you may already be familiar with. This will get you going quite far already and you are now in a position to dive in and write some tests.
Of course there is more to learn - most of which you should be able to page-fault in as you go. Please see the every-growing [Reference section](Readme.md) for what's available.
---
[Home](../README.md)
[Home](Readme.md)

View File

@@ -21,4 +21,4 @@ If you find any issues please raise issue tickets on the [issue tracker on GitHu
---
[Home](../README.md)
[Home](Readme.md)

View File

@@ -39,4 +39,4 @@ See the [tutorial](tutorial.md) to get more of a taste of using CATCH in practic
---
[Home](../README.md)
[Home](Readme.md)