diff --git a/docs/Readme.md b/docs/Readme.md index 1e59cf97..843281bb 100644 --- a/docs/Readme.md +++ b/docs/Readme.md @@ -7,6 +7,7 @@ Once you're up and running consider the following reference material. Writing tests: * [Assertion macros](assertions.md#top) * [Matchers](matchers.md#top) +* [Comparing floating point numbers](comparing-floating-point-numbers.md#top) * [Logging macros](logging.md#top) * [Test cases and sections](test-cases-and-sections.md#top) * [Test fixtures](test-fixtures.md#top) diff --git a/docs/assertions.md b/docs/assertions.md index 682eb6e7..29de3cfb 100644 --- a/docs/assertions.md +++ b/docs/assertions.md @@ -53,56 +53,8 @@ This expression is too complex because of the `||` operator. If you want to chec ### Floating point comparisons -When comparing floating point numbers - especially if at least one of them has been computed - great care must be taken to allow for rounding errors and inexact representations. - -Catch provides a way to perform tolerant comparisons of floating point values through use of a wrapper class called `Approx`. `Approx` can be used on either side of a comparison expression. It overloads the comparisons operators to take a tolerance into account. Here's a simple example: - -```cpp -REQUIRE( performComputation() == Approx( 2.1 ) ); -``` - -Catch also provides a user-defined literal for `Approx`; `_a`. It resides in -the `Catch::literals` namespace and can be used like so: -```cpp -using namespace Catch::literals; -REQUIRE( performComputation() == 2.1_a ); -``` - -`Approx` is constructed with defaults that should cover most simple cases. -For the more complex cases, `Approx` provides 3 customization points: - -* __epsilon__ - epsilon serves to set the coefficient by which a result -can differ from `Approx`'s value before it is rejected. -_By default set to `std::numeric_limits::epsilon()*100`._ -* __margin__ - margin serves to set the the absolute value by which -a result can differ from `Approx`'s value before it is rejected. -_By default set to `0.0`._ -* __scale__ - scale is used to change the magnitude of `Approx` for relative check. -_By default set to `0.0`._ - -#### epsilon example -```cpp -Approx target = Approx(100).epsilon(0.01); -100.0 == target; // Obviously true -200.0 == target; // Obviously still false -100.5 == target; // True, because we set target to allow up to 1% difference -``` - -#### margin example -```cpp -Approx target = Approx(100).margin(5); -100.0 == target; // Obviously true -200.0 == target; // Obviously still false -104.0 == target; // True, because we set target to allow absolute difference of at most 5 -``` - -#### scale -Scale can be useful if the computation leading to the result worked -on different scale than is used by the results. Since allowed difference -between Approx's value and compared value is based primarily on Approx's value -(the allowed difference is computed as -`(Approx::scale + Approx::value) * epsilon`), the resulting comparison could -need rescaling to be correct. +Comparing floating point numbers is complex, and [so it has its own +documentation page](comparing-floating-point-numbers.md#top). ## Exceptions diff --git a/docs/comparing-floating-point-numbers.md b/docs/comparing-floating-point-numbers.md new file mode 100644 index 00000000..f93c0ebb --- /dev/null +++ b/docs/comparing-floating-point-numbers.md @@ -0,0 +1,192 @@ + +# Comparing floating point numbers with Catch2 + +If you are not deeply familiar with them, floating point numbers can be +unintuitive. This also applies to comparing floating point numbers for +(in)equality. + +This page assumes that you have some understanding of both FP, and the +meaning of different kinds of comparisons, and only goes over what +functionality Catch2 provides to help you with comparing floating point +numbers. If you do not have this understanding, we recommend that you first +study up on floating point numbers and their comparisons, e.g. by [reading +this blog post](https://codingnest.com/the-little-things-comparing-floating-point-numbers/). + + +## Floating point matchers + +``` +#include `WithinRel` matcher was introduced in Catch2 2.10.0 + +As with all matchers, you can combine multiple floating point matchers +in a single assertion. For example, to check that some computation matches +a known good value within 0.1% or is close enough (no different to 5 +decimal places) to zero, we would write this assertion: + +```cpp + REQUIRE_THAT( computation(input), + Catch::Matchers::WithinRel(expected, 0.001) + || Catch::Matchers::WithinAbs(0, 0.000001) ); +``` + + +### WithinAbs + +`WithinAbs` creates a matcher that accepts floating point numbers whose +difference with `target` is less-or-equal to the `margin`. Since `float` +can be converted to `double` without losing precision, only `double` +overload exists. + +```cpp +REQUIRE_THAT(1.0, WithinAbs(1.2, 0.2)); +REQUIRE_THAT(0.f, !WithinAbs(1.0, 0.5)); +// Notice that infinity == infinity for WithinAbs +REQUIRE_THAT(INFINITY, WithinAbs(INFINITY, 0)); +``` + + +### WithinRel + +`WithinRel` creates a matcher that accepts floating point numbers that +are _approximately equal_ to the `target` with a tolerance of `eps.` +Specifically, it matches if +`|arg - target| <= eps * max(|arg|, |target|)` holds. If you do not +specify `eps`, `std::numeric_limits::epsilon * 100` +is used as the default. + +```cpp +// Notice that WithinRel comparison is symmetric, unlike Approx's. +REQUIRE_THAT(1.0, WithinRel(1.1, 0.1)); +REQUIRE_THAT(1.1, WithinRel(1.0, 0.1)); +// Notice that inifnity == infinity for WithinRel +REQUIRE_THAT(INFINITY, WithinRel(INFINITY)); +``` + + +### WithinULP + +`WithinULP` creates a matcher that accepts floating point numbers that +are no more than `maxUlpDiff` +[ULPs](https://en.wikipedia.org/wiki/Unit_in_the_last_place) +away from the `target` value. The short version of what this means +is that there is no more than `maxUlpDiff - 1` representable floating +point numbers between the argument for matching and the `target` value. + +When using the ULP matcher in Catch2, it is important to keep in mind +that Catch2 interprets ULP distance slightly differently than +e.g. `std::nextafter` does. + +Catch2's ULP calculation obeys these relations: + * `ulpDistance(-x, x) == 2 * ulpDistance(x, 0)` + * `ulpDistance(-0, 0) == 0` (due to the above) + * `ulpDistance(DBL_MAX, INFINITY) == 1` + * `ulpDistancE(NaN, x) == infinity` + + +**Important**: The WithinULP matcher requires the platform to use the +[IEEE-754](https://en.wikipedia.org/wiki/IEEE_754) representation for +floating point numbers. + +```cpp +REQUIRE_THAT( -0.f, WithinULP( 0.f, 0 ) ); +``` + + +## `Approx` + +``` +#include +``` + +**We strongly recommend against using `Approx` when writing new code.** +You should be using floating point matchers instead. + +Catch2 provides one more way to handle floating point comparisons. It is +`Approx`, a special type with overloaded comparison operators, that can +be used in standard assertions, e.g. + +```cpp +REQUIRE(0.99999 == Catch::Approx(1)); +``` + +`Approx` supports four comparison operators, `==`, `!=`, `<=`, `>=`, and can +also be used with strong typedefs over `double`s. It can be used for both +relative and margin comparisons by using its three customization points. +Note that the semantics of this is always that of an _or_, so if either +the relative or absolute margin comparison passes, then the whole comparison +passes. + +The downside to `Approx` is that it has a couple of issues that we cannot +fix without breaking backwards compatibility. Because Catch2 also provides +complete set of matchers that implement different floating point comparison +methods, `Approx` is left as-is, is considered deprecated, and should +not be used in new code. + +The issues are + * All internal computation is done in `double`s, leading to slightly + different results if the inputs were floats. + * `Approx`'s relative margin comparison is not symmetric. This means + that `Approx( 10 ).epsilon(0.1) != 11.1` but `Approx( 11.1 ).epsilon(0.1) == 10`. + * By default, `Approx` only uses relative margin comparison. This means + that `Approx(0) == X` only passes for `X == 0`. + + +### Approx details + +If you still want/need to know more about `Approx`, read on. + +Catch2 provides a UDL for `Approx`; `_a`. It resides in the `Catch::literals` +namespace, and can be used like this: + +```cpp +using namespace Catch::literals; +REQUIRE( performComputation() == 2.1_a ); +``` + +`Approx` has three customization points for the comparison: + +* **epsilon** - epsilon sets the coefficient by which a result +can differ from `Approx`'s value before it is rejected. +_Defaults to `std::numeric_limits::epsilon()*100`._ + +```cpp +Approx target = Approx(100).epsilon(0.01); +100.0 == target; // Obviously true +200.0 == target; // Obviously still false +100.5 == target; // True, because we set target to allow up to 1% difference +``` + + +* **margin** - margin sets the absolute value by which +a result can differ from `Approx`'s value before it is rejected. +_Defaults to `0.0`._ + +```cpp +Approx target = Approx(100).margin(5); +100.0 == target; // Obviously true +200.0 == target; // Obviously still false +104.0 == target; // True, because we set target to allow absolute difference of at most 5 +``` + +* **scale** - scale is used to change the magnitude of `Approx` for the relative check. +_By default, set to `0.0`._ + +Scale could be useful if the computation leading to the result worked +on a different scale than is used by the results. Approx's scale is added +to Approx's value when computing the allowed relative margin from the +Approx's value. + + +--- + +[Home](Readme.md#top) diff --git a/docs/matchers.md b/docs/matchers.md index f71f18a3..ec4728da 100644 --- a/docs/matchers.md +++ b/docs/matchers.md @@ -150,49 +150,8 @@ are: > `WithinRel` matcher was introduced in Catch2 2.10.0 - -`WithinAbs` creates a matcher that accepts floating point numbers whose -difference with `target` is less than the `margin`. - -`WithinULP` creates a matcher that accepts floating point numbers that -are no more than `maxUlpDiff` -[ULPs](https://en.wikipedia.org/wiki/Unit_in_the_last_place) -away from the `target` value. The short version of what this means -is that there is no more than `maxUlpDiff - 1` representable floating -point numbers between the argument for matching and the `target` value. - -**Important**: The WithinULP matcher requires the platform to use the -[IEEE-754](https://en.wikipedia.org/wiki/IEEE_754) representation for -floating point numbers. - - -`WithinRel` creates a matcher that accepts floating point numbers that -are _approximately equal_ with the `target` with tolerance of `eps.` -Specifically, it matches if -`|arg - target| <= eps * max(|arg|, |target|)` holds. If you do not -specify `eps`, `std::numeric_limits::epsilon * 100` -is used as the default. - - -In practice, you will often want to combine multiple of these matchers, -together for an assertion, because all 3 options have edge cases where -they behave differently than you would expect. As an example, under -the `WithinRel` matcher, a `0.` only ever matches a `0.` (or `-0.`), -regardless of the relative tolerance specified. Thus, if you want to -handle numbers that are "close enough to 0 to be 0", you have to combine -it with the `WithinAbs` matcher. - -For example, to check that our computation matches known good value -within 0.1%, or is close enough (no different to 5 decimal places) -to zero, we would write this assertion: -```cpp - REQUIRE_THAT( computation(input), - Catch::Matchers::WithinRel(expected, 0.001) - || Catch::Matchers::WithinAbs(0, 0.000001) ); -``` - - -> floating point matchers live in `catch2/matchers/catch_matchers_floating.hpp` +For more details, read [the docs on comparing floating point +numbers](comparing-floating-point-numbers.md#floating-point-matchers). ### Miscellaneous matchers