3rd-party/catch2
1
0
Fork 0
mirror of https://github.com/catchorg/Catch2.git synced 2024-11-30 01:03:30 +01:00
Code Issues Packages Projects Releases Wiki Activity
6b3c563c38
catch2/tests/SelfTest/Baselines/automake.sw.approved.txt
Martin Hořeňovský 4565b826cf
Modify generator tracking to allow GENERATEs between SECTIONs 
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
2020-07-26 11:35:06 +02:00

357 lines
21 KiB
Plaintext
Raw Blame History

:test-result: PASS # A test name that starts with a #
:test-result: PASS #1005: Comparing pointer to int and long (NULL can be either on various systems)
:test-result: PASS #1027: Bitfields can be captured
:test-result: PASS #1147
:test-result: PASS #1175 - Hidden Test
:test-result: PASS #1238
:test-result: PASS #1245
:test-result: PASS #1319: Sections can have description (even if it is not saved
:test-result: PASS #1403
:test-result: FAIL #1455 - INFO and WARN can start with a linebreak
This would not be caught previously
Nor would this
:test-result: FAIL #1514: stderr/stdout is not captured in tests aborted by an exception
:test-result: PASS #1548
:test-result: PASS #1905 -- test spec parser properly clears internal state between compound tests
:test-result: PASS #1912 -- test spec parser handles escaping
:test-result: PASS #1913 - GENERATE inside a for loop should not keep recreating the generator
:test-result: PASS #1913 - GENERATEs can share a line
:test-result: PASS #1938 - GENERATE after a section
:test-result: PASS #1938 - Section followed by flat generate
:test-result: PASS #1938 - flat generate
:test-result: PASS #1938 - mixed sections and generates
:test-result: PASS #1938 - nested generate
:test-result: PASS #1954 - 7 arg template test case sig compiles - 1, 1, 1, 1, 1, 0, 0
:test-result: PASS #1954 - 7 arg template test case sig compiles - 5, 1, 1, 1, 1, 0, 0
:test-result: PASS #1954 - 7 arg template test case sig compiles - 5, 3, 1, 1, 1, 0, 0
:test-result: XFAIL #748 - captures with unexpected exceptions
:test-result: PASS #809
:test-result: PASS #833
:test-result: XFAIL #835 -- errno should not be touched by Catch
:test-result: PASS #872
:test-result: PASS #961 -- Dynamically created sections should all be reported
:test-result: FAIL 'Not' checks that should fail
:test-result: PASS 'Not' checks that should succeed
:test-result: PASS (unimplemented) static bools can be evaluated
:test-result: PASS 3x3x3 ints
:test-result: FAIL A METHOD_AS_TEST_CASE based test run that fails
:test-result: PASS A METHOD_AS_TEST_CASE based test run that succeeds
:test-result: FAIL A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<float>
:test-result: FAIL A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - Template_Foo<int>
:test-result: FAIL A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<float>
:test-result: FAIL A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that fails - std::vector<int>
:test-result: PASS A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<float>
:test-result: PASS A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - Template_Foo<int>
:test-result: PASS A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<float>
:test-result: PASS A TEMPLATE_PRODUCT_TEST_CASE_METHOD based test run that succeeds - std::vector<int>
:test-result: FAIL A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<float, 6>
:test-result: FAIL A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - Template_Foo_2<int, 2>
:test-result: FAIL A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<float, 6>
:test-result: FAIL A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that fails - std::array<int, 2>
:test-result: PASS A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<float,6>
:test-result: PASS A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - Template_Foo_2<int,2>
:test-result: PASS A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<float,6>
:test-result: PASS A TEMPLATE_PRODUCT_TEST_CASE_METHOD_SIG based test run that succeeds - std::array<int,2>
:test-result: FAIL A TEMPLATE_TEST_CASE_METHOD based test run that fails - double
:test-result: FAIL A TEMPLATE_TEST_CASE_METHOD based test run that fails - float
:test-result: FAIL A TEMPLATE_TEST_CASE_METHOD based test run that fails - int
:test-result: PASS A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - double
:test-result: PASS A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - float
:test-result: PASS A TEMPLATE_TEST_CASE_METHOD based test run that succeeds - int
:test-result: FAIL A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 1
:test-result: FAIL A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 3
:test-result: FAIL A TEMPLATE_TEST_CASE_METHOD_SIG based test run that fails - 6
:test-result: PASS A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 1
:test-result: PASS A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 3
:test-result: PASS A TEMPLATE_TEST_CASE_METHOD_SIG based test run that succeeds - 6
:test-result: FAIL A TEST_CASE_METHOD based test run that fails
:test-result: PASS A TEST_CASE_METHOD based test run that succeeds
:test-result: PASS A Template product test case - Foo<float>
:test-result: PASS A Template product test case - Foo<int>
:test-result: PASS A Template product test case - std::vector<float>
:test-result: PASS A Template product test case - std::vector<int>
:test-result: PASS A Template product test case with array signature - Bar<float, 42>
:test-result: PASS A Template product test case with array signature - Bar<int, 9>
:test-result: PASS A Template product test case with array signature - std::array<float, 42>
:test-result: PASS A Template product test case with array signature - std::array<int, 9>
:test-result: PASS A comparison that uses literals instead of the normal constructor
:test-result: FAIL A couple of nested sections followed by a failure
:test-result: FAIL A failing expression with a non streamable type is still captured
:test-result: PASS Absolute margin
:test-result: FAIL An empty test with no assertions
:test-result: PASS An expression with side-effects should only be evaluated once
:test-result: FAIL An unchecked exception reports the line of the last assertion
:test-result: PASS Anonymous test case 1
:test-result: PASS Approx setters validate their arguments
:test-result: PASS Approx with exactly-representable margin
:test-result: PASS Approximate PI
:test-result: PASS Approximate comparisons with different epsilons
:test-result: PASS Approximate comparisons with floats
:test-result: PASS Approximate comparisons with ints
:test-result: PASS Approximate comparisons with mixed numeric types
:test-result: PASS Arbitrary predicate matcher
:test-result: PASS Assertion macros support bit operators and bool conversions
:test-result: PASS Assertions then sections
:test-result: PASS Basic use of the Contains range matcher
:test-result: PASS Basic use of the Empty range matcher
:test-result: PASS CAPTURE can deal with complex expressions
:test-result: PASS CAPTURE can deal with complex expressions involving commas
:test-result: PASS CAPTURE parses string and character constants
:test-result: PASS Capture and info messages
:test-result: PASS Character pretty printing
:test-result: PASS Combining MatchAllOfGeneric does not nest
:test-result: PASS Combining MatchAnyOfGeneric does not nest
:test-result: PASS Combining MatchNotOfGeneric does not nest
:test-result: PASS Combining concrete matchers does not use templated matchers
:test-result: PASS Combining only templated matchers
:test-result: PASS Combining templated and concrete matchers
:test-result: PASS Combining templated matchers
:test-result: PASS Commas in various macros are allowed
:test-result: PASS Comparing function pointers
:test-result: PASS Comparison ops
:test-result: PASS Comparison with explicitly convertible types
:test-result: PASS Comparisons between ints where one side is computed
:test-result: PASS Comparisons between unsigned ints and negative signed ints match c++ standard behaviour
:test-result: PASS Comparisons with int literals don't warn when mixing signed/ unsigned
:test-result: PASS Composed matchers shortcircuit
:test-result: FAIL Contains string matcher
:test-result: PASS Copy and then generate a range
:test-result: FAIL Custom exceptions can be translated when testing for nothrow
:test-result: FAIL Custom exceptions can be translated when testing for throwing as something else
:test-result: FAIL Custom std-exceptions can be custom translated
:test-result: PASS Default scale is invisible to comparison
:test-result: PASS Directly creating an EnumInfo
:test-result: FAIL EndsWith string matcher
:test-result: PASS Enums can quickly have stringification enabled using REGISTER_ENUM
:test-result: PASS Enums in namespaces can quickly have stringification enabled using REGISTER_ENUM
:test-result: PASS Epsilon only applies to Approx's value
:test-result: XFAIL Equality checks that should fail
:test-result: PASS Equality checks that should succeed
:test-result: PASS Equals
:test-result: FAIL Equals string matcher
:test-result: PASS Exception as a value (e.g. in REQUIRE_THROWS_MATCHES) can be stringified
:test-result: FAIL Exception matchers that fail
:test-result: PASS Exception matchers that succeed
:test-result: PASS Exception messages can be tested for
:test-result: PASS Exceptions matchers
:test-result: FAIL Expected exceptions that don't throw or unexpected exceptions fail the test
:test-result: FAIL FAIL aborts the test
:test-result: FAIL FAIL does not require an argument
:test-result: FAIL FAIL_CHECK does not abort the test
:test-result: PASS Factorials are computed
:test-result: PASS Floating point matchers: double
:test-result: PASS Floating point matchers: float
:test-result: PASS Generators -- adapters
:test-result: PASS Generators -- simple
:test-result: PASS Generators internals
:test-result: PASS Greater-than inequalities with different epsilons
:test-result: FAIL INFO and WARN do not abort tests
:test-result: FAIL INFO gets logged on failure
:test-result: FAIL INFO gets logged on failure, even if captured before successful assertions
:test-result: FAIL INFO is reset for each loop
:test-result: XFAIL Inequality checks that should fail
:test-result: PASS Inequality checks that should succeed
:test-result: PASS Lambdas in assertions
:test-result: PASS Less-than inequalities with different epsilons
:test-result: PASS ManuallyRegistered
:test-result: PASS Matchers can be (AllOf) composed with the && operator
:test-result: PASS Matchers can be (AnyOf) composed with the || operator
:test-result: PASS Matchers can be composed with both && and ||
:test-result: FAIL Matchers can be composed with both && and || - failing
:test-result: PASS Matchers can be negated (Not) with the ! operator
:test-result: FAIL Matchers can be negated (Not) with the ! operator - failing
:test-result: FAIL Mismatching exception messages failing the test
:test-result: PASS Nested generators and captured variables
:test-result: FAIL Nice descriptive name
:test-result: FAIL Non-std exceptions can be translated
:test-result: PASS Objects that evaluated in boolean contexts can be checked
:test-result: PASS Optionally static assertions
:test-result: FAIL Ordering comparison checks that should fail
:test-result: PASS Ordering comparison checks that should succeed
:test-result: PASS Our PCG implementation provides expected results for known seeds
:test-result: FAIL Output from all sections is reported
:test-result: PASS Overloaded comma or address-of operators are not used
:test-result: PASS Parse test names and tags
:test-result: PASS Pointers can be compared to null
:test-result: PASS Precision of floating point stringification can be set
:test-result: PASS Predicate matcher can accept const char*
:test-result: PASS Process can be configured on command line
:test-result: PASS Product with differing arities - std::tuple<int, double, float>
:test-result: PASS Product with differing arities - std::tuple<int, double>
:test-result: PASS Product with differing arities - std::tuple<int>
:test-result: FAIL Reconstruction should be based on stringification: #914
:test-result: FAIL Regex string matcher
:test-result: PASS Regression test #1
:test-result: PASS SUCCEED counts as a test pass
:test-result: PASS SUCCEED does not require an argument
:test-result: PASS Scenario: BDD tests requiring Fixtures to provide commonly-accessed data or methods
:test-result: PASS Scenario: Do that thing with the thing
:test-result: PASS Scenario: This is a really long scenario name to see how the list command deals with wrapping
:test-result: PASS Scenario: Vector resizing affects size and capacity
A string sent directly to stdout
A string sent directly to stderr
A string sent to stderr via clog
:test-result: FAIL Sends stuff to stdout and stderr
:test-result: PASS Some simple comparisons between doubles
Message from section one
Message from section two
:test-result: FAIL Standard output from all sections is reported
:test-result: FAIL StartsWith string matcher
:test-result: PASS Static arrays are convertible to string
:test-result: PASS String matchers
:test-result: PASS StringRef
:test-result: PASS StringRef at compilation time
:test-result: PASS Stringifying std::chrono::duration helpers
:test-result: PASS Stringifying std::chrono::duration with weird ratios
:test-result: PASS Stringifying std::chrono::time_point<system_clock>
:test-result: FAIL Tabs and newlines show in output
:test-result: PASS Tag alias can be registered against tag patterns
:test-result: PASS Template test case method with test types specified inside std::tuple - MyTypes - 0
:test-result: PASS Template test case method with test types specified inside std::tuple - MyTypes - 1
:test-result: PASS Template test case method with test types specified inside std::tuple - MyTypes - 2
:test-result: PASS Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 0
:test-result: PASS Template test case with test types specified inside non-copyable and non-movable std::tuple - NonCopyableAndNonMovableTypes - 1
:test-result: PASS Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 0
:test-result: PASS Template test case with test types specified inside non-default-constructible std::tuple - MyNonDefaultConstructibleTypes - 1
:test-result: PASS Template test case with test types specified inside std::tuple - MyTypes - 0
:test-result: PASS Template test case with test types specified inside std::tuple - MyTypes - 1
:test-result: PASS Template test case with test types specified inside std::tuple - MyTypes - 2
:test-result: PASS TemplateTest: vectors can be sized and resized - float
:test-result: PASS TemplateTest: vectors can be sized and resized - int
:test-result: PASS TemplateTest: vectors can be sized and resized - std::string
:test-result: PASS TemplateTest: vectors can be sized and resized - std::tuple<int,float>
:test-result: PASS TemplateTestSig: vectors can be sized and resized - (std::tuple<int, float>), 6
:test-result: PASS TemplateTestSig: vectors can be sized and resized - float,4
:test-result: PASS TemplateTestSig: vectors can be sized and resized - int,5
:test-result: PASS TemplateTestSig: vectors can be sized and resized - std::string,15
:test-result: PASS Test case with one argument
:test-result: PASS Test enum bit values
:test-result: PASS Test with special, characters "in name
:test-result: FAIL The NO_FAIL macro reports a failure but does not fail the test
:test-result: FAIL This test 'should' fail but doesn't
:test-result: FAIL Thrown string literals are translated
:test-result: PASS Tracker
:test-result: PASS Trim strings
:test-result: FAIL Unexpected exceptions can be translated
:test-result: PASS Upcasting special member functions
:test-result: PASS Usage of the SizeIs range matcher
:test-result: PASS Use a custom approx
:test-result: PASS Variadic macros
:test-result: PASS Vector Approx matcher
:test-result: FAIL Vector Approx matcher -- failing
:test-result: PASS Vector matchers
:test-result: FAIL Vector matchers that fail
:test-result: PASS When checked exceptions are thrown they can be expected or unexpected
:test-result: FAIL When unchecked exceptions are thrown directly they are always failures
:test-result: FAIL When unchecked exceptions are thrown during a CHECK the test should continue
:test-result: FAIL When unchecked exceptions are thrown during a REQUIRE the test should abort fail
:test-result: FAIL When unchecked exceptions are thrown from functions they are always failures
:test-result: FAIL When unchecked exceptions are thrown from sections they are always failures
:test-result: FAIL When unchecked exceptions are thrown, but caught, they do not affect the test
:test-result: FAIL Where the LHS is not a simple value
:test-result: FAIL Where there is more to the expression after the RHS
:test-result: PASS X/level/0/a
:test-result: PASS X/level/0/b
:test-result: PASS X/level/1/a
:test-result: PASS X/level/1/b
:test-result: PASS XmlEncode
:test-result: PASS analyse no analysis
:test-result: PASS array<int, N> -> toString
:test-result: PASS atomic if
:test-result: PASS benchmark function call
:test-result: PASS boolean member
:test-result: PASS checkedElse
:test-result: FAIL checkedElse, failing
:test-result: PASS checkedIf
:test-result: FAIL checkedIf, failing
:test-result: PASS classify_outliers
:test-result: PASS comparisons between const int variables
:test-result: PASS comparisons between int variables
:test-result: PASS erfc_inv
:test-result: PASS estimate_clock_resolution
:test-result: PASS even more nested SECTION tests
:test-result: FAIL first tag
loose text artifact
:test-result: FAIL has printf
:test-result: FAIL just failure
:test-result: FAIL just failure after unscoped info
:test-result: FAIL just info
:test-result: FAIL just unscoped info
:test-result: PASS long long
:test-result: FAIL looped SECTION tests
:test-result: FAIL looped tests
:test-result: PASS make_unique reimplementation
:test-result: PASS mean
:test-result: PASS measure
:test-result: FAIL mix info, unscoped info and warning
:test-result: FAIL more nested SECTION tests
:test-result: PASS nested SECTION tests
:test-result: PASS non streamable - with conv. op
:test-result: PASS non-copyable objects
:test-result: PASS normal_cdf
:test-result: PASS normal_quantile
:test-result: PASS not allowed
:test-result: FAIL not prints unscoped info from previous failures
:test-result: PASS null strings
:test-result: PASS null_ptr
:test-result: PASS pair<pair<int,const char *,pair<std::string,int> > -> toString
:test-result: PASS parseEnums
:test-result: PASS pointer to class
:test-result: PASS print unscoped info if passing unscoped info is printed
:test-result: FAIL prints unscoped info on failure
:test-result: FAIL prints unscoped info only for the first assertion
:test-result: PASS random SECTION tests
:test-result: PASS replaceInPlace
:test-result: PASS resolution
:test-result: PASS run_for_at_least, chronometer
:test-result: PASS run_for_at_least, int
:test-result: FAIL second tag
:test-result: FAIL send a single char to INFO
:test-result: FAIL sends information to INFO
:test-result: PASS shortened hide tags are split apart
:test-result: PASS splitString
:test-result: FAIL stacks unscoped info in loops
:test-result: PASS std::map is convertible string
:test-result: PASS std::pair<int,const std::string> -> toString
:test-result: PASS std::pair<int,std::string> -> toString
:test-result: PASS std::set is convertible string
:test-result: PASS std::vector<std::pair<std::string,int> > -> toString
:test-result: FAIL string literals of different sizes can be compared
:test-result: PASS stringify ranges
:test-result: PASS stringify( has_maker )
:test-result: PASS stringify( has_maker_and_operator )
:test-result: PASS stringify( has_neither )
:test-result: PASS stringify( has_operator )
:test-result: PASS stringify( has_template_operator )
:test-result: PASS stringify( vectors<has_maker> )
:test-result: PASS stringify( vectors<has_maker_and_operator> )
:test-result: PASS stringify( vectors<has_operator> )
:test-result: PASS strlen3
:test-result: PASS tables
:test-result: FAIL thrown std::strings are translated
:test-result: PASS toString on const wchar_t const pointer returns the string contents
:test-result: PASS toString on const wchar_t pointer returns the string contents
:test-result: PASS toString on wchar_t const pointer returns the string contents
:test-result: PASS toString on wchar_t returns the string contents
:test-result: PASS toString(enum class w/operator<<)
:test-result: PASS toString(enum class)
:test-result: PASS toString(enum w/operator<<)
:test-result: PASS toString(enum)
:test-result: PASS tuple<>
:test-result: PASS tuple<float,int>
:test-result: PASS tuple<int>
:test-result: PASS tuple<0,int,const char *>
:test-result: PASS tuple<string,string>
:test-result: PASS tuple<tuple<int>,tuple<>,float>
:test-result: PASS uniform samples
:test-result: PASS unique_ptr reimplementation: basic functionality
:test-result: PASS vec<vec<string,alloc>> -> toString
:test-result: PASS vector<bool> -> toString
:test-result: PASS vector<int,allocator> -> toString
:test-result: PASS vector<int> -> toString
:test-result: PASS vector<string> -> toString
:test-result: PASS vectors can be sized and resized
:test-result: PASS warmup
:test-result: PASS weighted_average_quantile
:test-result: PASS xmlentitycheck
Reference in New Issue View Git Blame Copy Permalink