mirror of
				https://github.com/catchorg/Catch2.git
				synced 2025-10-31 20:27:11 +01:00 
			
		
		
		
	 Jozef Grajciar
					Jozef Grajciar
				
			
				
					committed by
					
						 Martin Hořeňovský
						Martin Hořeňovský
					
				
			
			
				
	
			
			
			 Martin Hořeňovský
						Martin Hořeňovský
					
				
			
						parent
						
							6d803cba5d
						
					
				
				
					commit
					037ddbc75c
				
			| @@ -1,7 +1,7 @@ | ||||
| <a id="top"></a> | ||||
| # Authoring benchmarks | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0. | ||||
|  | ||||
| Writing benchmarks is not easy. Catch simplifies certain aspects but you'll | ||||
| always need to take care about various aspects. Understanding a few things about | ||||
|   | ||||
| @@ -226,7 +226,7 @@ When set to ```yes``` Catch will report the duration of each test case, in milli | ||||
|  | ||||
| <pre>-D, --min-duration <value></pre> | ||||
|  | ||||
| > `--min-duration` was [introduced](https://github.com/catchorg/Catch2/pull/1910) in Catch 2.13.0 | ||||
| > `--min-duration` was [introduced](https://github.com/catchorg/Catch2/pull/1910) in Catch2 2.13.0 | ||||
|  | ||||
| When set, Catch will report the duration of each test case that took more | ||||
| than <value> seconds, in milliseconds. This option is overriden by both | ||||
| @@ -302,7 +302,7 @@ either before running any tests, after running all tests - or both, depending on | ||||
| ## Specify the number of benchmark samples to collect | ||||
| <pre>--benchmark-samples <# of samples></pre> | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0. | ||||
|  | ||||
| When running benchmarks a number of "samples" is collected. This is the base data for later statistical analysis. | ||||
| Per sample a clock resolution dependent number of iterations of the user code is run, which is independent of the number of samples. Defaults to 100. | ||||
| @@ -311,7 +311,7 @@ Per sample a clock resolution dependent number of iterations of the user code is | ||||
| ## Specify the number of resamples for bootstrapping | ||||
| <pre>--benchmark-resamples <# of resamples></pre> | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0. | ||||
|  | ||||
| After the measurements are performed, statistical [bootstrapping] is performed | ||||
| on the samples. The number of resamples for that bootstrapping is configurable | ||||
| @@ -326,7 +326,7 @@ defaults to 95%). | ||||
| ## Specify the confidence-interval for bootstrapping | ||||
| <pre>--benchmark-confidence-interval <confidence-interval></pre> | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0. | ||||
|  | ||||
| The confidence-interval is used for statistical bootstrapping on the samples to | ||||
| calculate the upper and lower bounds of mean and standard deviation. | ||||
| @@ -336,7 +336,7 @@ Must be between 0 and 1 and defaults to 0.95. | ||||
| ## Disable statistical analysis of collected benchmark samples | ||||
| <pre>--benchmark-no-analysis</pre> | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch 2.9.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1616) in Catch2 2.9.0. | ||||
|  | ||||
| When this flag is specified no bootstrapping or any other statistical analysis is performed. | ||||
| Instead the user code is only measured and the plain mean from the samples is reported. | ||||
| @@ -345,7 +345,7 @@ Instead the user code is only measured and the plain mean from the samples is re | ||||
| ## Specify the amount of time in milliseconds spent on warming up each test | ||||
| <pre>--benchmark-warmup-time</pre> | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/pull/1844) in Catch 2.11.2. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/pull/1844) in Catch2 2.11.2. | ||||
|  | ||||
| Configure the amount of time spent warming up each test. | ||||
|  | ||||
|   | ||||
| @@ -117,7 +117,7 @@ Catch's selection, by defining either `CATCH_CONFIG_CPP11_TO_STRING` or | ||||
|     CATCH_CONFIG_CPP17_OPTIONAL             // Override std::optional support detection (checked by CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER) | ||||
|     CATCH_CONFIG_CPP17_BYTE                 // Override std::byte support detection (Catch provides a StringMaker specialization by default) | ||||
|  | ||||
| > `CATCH_CONFIG_CPP17_STRING_VIEW` was [introduced](https://github.com/catchorg/Catch2/issues/1376) in Catch 2.4.1. | ||||
| > `CATCH_CONFIG_CPP17_STRING_VIEW` was [introduced](https://github.com/catchorg/Catch2/issues/1376) in Catch2 2.4.1. | ||||
|  | ||||
| Catch contains basic compiler/standard detection and attempts to use | ||||
| some C++17 features whenever appropriate. This automatic detection | ||||
| @@ -141,7 +141,7 @@ by using `_NO_` in the macro, e.g. `CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS`. | ||||
|     CATCH_CONFIG_ANDROID_LOGWRITE           // Use android's logging system for debug output | ||||
|     CATCH_CONFIG_GLOBAL_NEXTAFTER           // Use nextafter{,f,l} instead of std::nextafter | ||||
|  | ||||
| > [`CATCH_CONFIG_ANDROID_LOGWRITE`](https://github.com/catchorg/Catch2/issues/1743) and [`CATCH_CONFIG_GLOBAL_NEXTAFTER`](https://github.com/catchorg/Catch2/pull/1739) were introduced in Catch 2.10.0 | ||||
| > [`CATCH_CONFIG_ANDROID_LOGWRITE`](https://github.com/catchorg/Catch2/issues/1743) and [`CATCH_CONFIG_GLOBAL_NEXTAFTER`](https://github.com/catchorg/Catch2/pull/1739) were introduced in Catch2 2.10.0 | ||||
|  | ||||
| Currently Catch enables `CATCH_CONFIG_WINDOWS_SEH` only when compiled with MSVC, because some versions of MinGW do not have the necessary Win32 API support. | ||||
|  | ||||
| @@ -197,13 +197,13 @@ By default, Catch does not stringify some types from the standard library. This | ||||
|     CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER // Provide StringMaker specialization for std::optional (on C++17) | ||||
|     CATCH_CONFIG_ENABLE_ALL_STRINGMAKERS     // Defines all of the above | ||||
|  | ||||
| > `CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1380) in Catch 2.4.1. | ||||
| > `CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1380) in Catch2 2.4.1. | ||||
|  | ||||
| > `CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1510) in Catch 2.6.0. | ||||
| > `CATCH_CONFIG_ENABLE_OPTIONAL_STRINGMAKER` was [introduced](https://github.com/catchorg/Catch2/issues/1510) in Catch2 2.6.0. | ||||
|  | ||||
| ## Disabling exceptions | ||||
|  | ||||
| > Introduced in Catch 2.4.0. | ||||
| > Introduced in Catch2 2.4.0. | ||||
|  | ||||
| By default, Catch2 uses exceptions to signal errors and to abort tests | ||||
| when an assertion from the `REQUIRE` family of assertions fails. We also | ||||
| @@ -239,7 +239,7 @@ namespace Catch { | ||||
|  | ||||
| ## Overriding Catch's debug break (`-b`) | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/pull/1846) in Catch 2.11.2. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/pull/1846) in Catch2 2.11.2. | ||||
|  | ||||
| You can override Catch2's break-into-debugger code by defining the | ||||
| `CATCH_BREAK_INTO_DEBUGGER()` macro. This can be used if e.g. Catch2 does | ||||
|   | ||||
| @@ -147,9 +147,9 @@ be replaced with the actual version upon release. There are 2 styles | ||||
| of placeholders used through the documentation, you should pick one that | ||||
| fits your text better (if in doubt, take a look at the existing version | ||||
| tags for other features). | ||||
|   * `> [Introduced](link-to-issue-or-PR) in Catch X.Y.Z` - this | ||||
|   * `> [Introduced](link-to-issue-or-PR) in Catch2 X.Y.Z` - this | ||||
|   placeholder is usually used after a section heading | ||||
|   * `> X (Y and Z) was [introduced](link-to-issue-or-PR) in Catch X.Y.Z` | ||||
|   * `> X (Y and Z) was [introduced](link-to-issue-or-PR) in Catch2 X.Y.Z` | ||||
|   - this placeholder is used when you need to tag a subpart of something, | ||||
|   e.g. a list | ||||
|  | ||||
|   | ||||
| @@ -1,7 +1,7 @@ | ||||
| <a id="top"></a> | ||||
| # Data Generators | ||||
|  | ||||
| > Introduced in Catch 2.6.0. | ||||
| > Introduced in Catch2 2.6.0. | ||||
|  | ||||
| Data generators (also known as _data driven/parametrized test cases_) | ||||
| let you reuse the same set of assertions across different input values. | ||||
| @@ -92,7 +92,7 @@ TEST_CASE("Complex mix of sections and generates") { | ||||
| } | ||||
| ``` | ||||
|  | ||||
| > The ability to place `GENERATE` between two `SECTION`s was [introduced](https://github.com/catchorg/Catch2/issues/1938) in Catch 2.13.0. | ||||
| > The ability to place `GENERATE` between two `SECTION`s was [introduced](https://github.com/catchorg/Catch2/issues/1938) in Catch2 2.13.0. | ||||
|  | ||||
| ## Provided generators | ||||
|  | ||||
| @@ -117,9 +117,9 @@ a test case, | ||||
|   * `RangeGenerator<T>` -- generates all values inside an arithmetic range | ||||
|   * `IteratorGenerator<T>` -- copies and returns values from an iterator range | ||||
|  | ||||
| > `ChunkGenerator<T>`, `RandomIntegerGenerator<Integral>`, `RandomFloatGenerator<Float>` and `RangeGenerator<T>` were introduced in Catch 2.7.0. | ||||
| > `ChunkGenerator<T>`, `RandomIntegerGenerator<Integral>`, `RandomFloatGenerator<Float>` and `RangeGenerator<T>` were introduced in Catch2 2.7.0. | ||||
|  | ||||
| > `IteratorGenerator<T>` was introduced in Catch 2.10.0. | ||||
| > `IteratorGenerator<T>` was introduced in Catch2 2.10.0. | ||||
|  | ||||
| The generators also have associated helper functions that infer their | ||||
| type, making their usage much nicer. These are | ||||
| @@ -139,11 +139,11 @@ type, making their usage much nicer. These are | ||||
| * `from_range(InputIterator from, InputIterator to)` for `IteratorGenerator<T>` | ||||
| * `from_range(Container const&)` for `IteratorGenerator<T>` | ||||
|  | ||||
| > `chunk()`, `random()` and both `range()` functions were introduced in Catch 2.7.0. | ||||
| > `chunk()`, `random()` and both `range()` functions were introduced in Catch2 2.7.0. | ||||
|  | ||||
| > `from_range` has been introduced in Catch 2.10.0 | ||||
| > `from_range` has been introduced in Catch2 2.10.0 | ||||
|  | ||||
| > `range()` for floating point numbers has been introduced in Catch 2.11.0 | ||||
| > `range()` for floating point numbers has been introduced in Catch2 2.11.0 | ||||
|  | ||||
| And can be used as shown in the example below to create a generator | ||||
| that returns 100 odd random number: | ||||
| @@ -176,7 +176,7 @@ scope and thus capturing references is dangerous. If you need to use | ||||
| variables inside the generator expression, make sure you thought through | ||||
| the lifetime implications and use `GENERATE_COPY` or `GENERATE_REF`.** | ||||
|  | ||||
| > `GENERATE_COPY` and `GENERATE_REF` were introduced in Catch 2.7.1. | ||||
| > `GENERATE_COPY` and `GENERATE_REF` were introduced in Catch2 2.7.1. | ||||
|  | ||||
| You can also override the inferred type by using `as<type>` as the first | ||||
| argument to the macro. This can be useful when dealing with string literals, | ||||
|   | ||||
| @@ -30,7 +30,7 @@ When the last `CHECK` fails in the "Bar" test case, then only one message will b | ||||
|  | ||||
| ## Logging without local scope | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch 2.7.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch2 2.7.0. | ||||
|  | ||||
| `UNSCOPED_INFO` is similar to `INFO` with two key differences: | ||||
|  | ||||
| @@ -106,7 +106,7 @@ This semicolon will be removed with next major version. It is highly advised to | ||||
|  | ||||
| **UNSCOPED_INFO(** _message expression_ **)** | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch 2.7.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1522) in Catch2 2.7.0. | ||||
|  | ||||
| Similar to `INFO`, but messages are not limited to their own scope: They are removed from the buffer after each assertion, section or test case, whichever comes first. | ||||
|  | ||||
|   | ||||
| @@ -125,7 +125,7 @@ These are | ||||
|  * `Equals` which checks whether the result is exactly equal (order matters) to a specific vector | ||||
|  * `UnorderedEquals` which checks whether the result is equal to a specific vector under a permutation | ||||
|  * `Approx` which checks whether the result is "approx-equal" (order matters, but comparison is done via `Approx`) to a specific vector | ||||
| > Approx matcher was [introduced](https://github.com/catchorg/Catch2/issues/1499) in Catch 2.7.2. | ||||
| > Approx matcher was [introduced](https://github.com/catchorg/Catch2/issues/1499) in Catch2 2.7.2. | ||||
|  | ||||
| An example usage: | ||||
| ```cpp | ||||
| @@ -148,7 +148,7 @@ are: | ||||
| * `WithinUlps(FloatingPoint target, uint64_t maxUlpDiff)`, and | ||||
| * `WithinRel(FloatingPoint target, FloatingPoint eps)`. | ||||
|  | ||||
| > `WithinRel` matcher was introduced in Catch 2.10.0 | ||||
| > `WithinRel` matcher was introduced in Catch2 2.10.0 | ||||
|  | ||||
|  | ||||
| `WithinAbs` creates a matcher that accepts floating point numbers whose | ||||
| @@ -243,7 +243,7 @@ Note that `DerivedException` in the example above has to derive from | ||||
|  | ||||
| ### Generic range Matchers | ||||
|  | ||||
| > Generic range matchers were introduced in Catch X.Y.Z | ||||
| > Generic range matchers were introduced in Catch2 X.Y.Z | ||||
|  | ||||
| Catch2 also provides some matchers that use the new style matchers | ||||
| definitions to handle generic range-like types. These are: | ||||
| @@ -345,7 +345,7 @@ style matchers arbitrarily. | ||||
|  | ||||
| ## Writing custom matchers (new style) | ||||
|  | ||||
| > New style matchers were introduced in Catch X.Y.Z | ||||
| > New style matchers were introduced in Catch2 X.Y.Z | ||||
|  | ||||
| To create a new-style matcher, you have to create your own type that | ||||
| derives from `Catch::Matchers::MatcherGenericBase`. Your type has to | ||||
|   | ||||
| @@ -59,7 +59,7 @@ TEST_CASE( "SUCCEED showcase" ) { | ||||
|  | ||||
| * `STATIC_REQUIRE` | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1362) in Catch 2.4.2. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1362) in Catch2 2.4.2. | ||||
|  | ||||
| `STATIC_REQUIRE( expr )` is a macro that can be used the same way as a | ||||
| `static_assert`, but also registers the success with Catch2, so it is | ||||
| @@ -134,7 +134,7 @@ ANON_TEST_CASE() { | ||||
|  | ||||
| * `DYNAMIC_SECTION` | ||||
|  | ||||
| > Introduced in Catch 2.3.0. | ||||
| > Introduced in Catch2 2.3.0. | ||||
|  | ||||
| `DYNAMIC_SECTION` is a `SECTION` where the user can use `operator<<` to | ||||
| create the final name for that section. This can be useful with e.g. | ||||
|   | ||||
| @@ -98,7 +98,7 @@ These macros map onto ```SECTION```s except that the section names are the _some | ||||
|  | ||||
| Similar to ```GIVEN```, ```WHEN``` and ```THEN``` except that the prefixes start with "and ". These are used to chain ```GIVEN```s, ```WHEN```s and ```THEN```s together. | ||||
|  | ||||
| > `AND_GIVEN` was [introduced](https://github.com/catchorg/Catch2/issues/1360) in Catch 2.4.0. | ||||
| > `AND_GIVEN` was [introduced](https://github.com/catchorg/Catch2/issues/1360) in Catch2 2.4.0. | ||||
|  | ||||
| When any of these macros are used the console reporter recognises them and formats the test case header such that the Givens, Whens and Thens are aligned to aid readability. | ||||
|  | ||||
| @@ -112,7 +112,7 @@ by types, in the form of `TEMPLATE_TEST_CASE`, | ||||
|  | ||||
| * **TEMPLATE_TEST_CASE(** _test name_ , _tags_,  _type1_, _type2_, ..., _typen_ **)** | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1437) in Catch 2.5.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1437) in Catch2 2.5.0. | ||||
|  | ||||
| _test name_ and _tag_ are exactly the same as they are in `TEST_CASE`, | ||||
| with the difference that the tag string must be provided (however, it | ||||
| @@ -164,7 +164,7 @@ TEMPLATE_TEST_CASE( "vectors can be sized and resized", "[vector][template]", in | ||||
|  | ||||
| * **TEMPLATE_PRODUCT_TEST_CASE(** _test name_ , _tags_, (_template-type1_, _template-type2_, ..., _template-typen_), (_template-arg1_, _template-arg2_, ..., _template-argm_) **)** | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1468) in Catch 2.6.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1468) in Catch2 2.6.0. | ||||
|  | ||||
| _template-type1_ through _template-typen_ is list of template template | ||||
| types which should be combined with each of _template-arg1_ through | ||||
| @@ -209,7 +209,7 @@ is very high and should not be encountered in practice._ | ||||
|  | ||||
| * **TEMPLATE_LIST_TEST_CASE(** _test name_, _tags_, _type list_ **)** | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1627) in Catch 2.9.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1627) in Catch2 2.9.0. | ||||
|  | ||||
| _type list_ is a generic list of types on which test case should be instantiated. | ||||
| List can be `std::tuple`, `boost::mpl::list`, `boost::mp11::mp_list` or anything with | ||||
| @@ -229,7 +229,7 @@ TEMPLATE_LIST_TEST_CASE("Template test case with test types specified inside std | ||||
|  | ||||
| ## Signature based parametrised test cases | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch 2.8.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0. | ||||
|  | ||||
| In addition to [type parametrised test cases](#type-parametrised-test-cases) Catch2 also supports | ||||
| signature base parametrised test cases, in form of `TEMPLATE_TEST_CASE_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_SIG`. | ||||
|   | ||||
| @@ -88,7 +88,7 @@ the limit is very high and should not be encountered in practice._ | ||||
|  | ||||
| ## Signature-based parametrised test fixtures | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch 2.8.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1609) in Catch2 2.8.0. | ||||
|  | ||||
| Catch2 also provides `TEMPLATE_TEST_CASE_METHOD_SIG` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG` to support | ||||
| fixtures using non-type template parameters. These test cases work similar to `TEMPLATE_TEST_CASE_METHOD` and `TEMPLATE_PRODUCT_TEST_CASE_METHOD`, | ||||
|   | ||||
| @@ -71,7 +71,7 @@ CATCH_TRANSLATE_EXCEPTION( MyType const& ex ) { | ||||
|  | ||||
| ## Enums | ||||
|  | ||||
| > Introduced in Catch 2.8.0. | ||||
| > Introduced in Catch2 2.8.0. | ||||
|  | ||||
| Enums that already have a `<<` overload for `std::ostream` will convert to strings as expected. | ||||
| If you only need to convert enums to strings for test reporting purposes you can provide a `StringMaker` specialisations as any other type. | ||||
| @@ -110,7 +110,7 @@ TEST_CASE() { | ||||
|  | ||||
| ## Floating point precision | ||||
|  | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1614) in Catch 2.8.0. | ||||
| > [Introduced](https://github.com/catchorg/Catch2/issues/1614) in Catch2 2.8.0. | ||||
|  | ||||
| Catch provides a built-in `StringMaker` specialization for both `float` | ||||
| and `double`. By default, it uses what we think is a reasonable precision, | ||||
|   | ||||
		Reference in New Issue
	
	Block a user