3rd-party/catch2
1
0
Fork 0
mirror of https://github.com/catchorg/Catch2.git synced 2024-12-23 11:43:29 +01:00
Code Issues Packages Projects Releases Wiki Activity
69bd213c40
catch2/tests/CMakeLists.txt

260 lines
12 KiB
CMake
Raw Normal View History

Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
include(MiscFunctions)
Fix VS2017 approvals on AppVeyor Because of a change in VS toolset, missing option <UseFullPaths> is no longer interpreted as "don't pass /FC to the compiler", but rather as "pass /FC to the compiler". This is problematic, because /FC not only changes how much of the path is reporter by the compiler (e.g. in `__FILE__` macro), but it also lower cases the path. This lower-casing of the path broke our approval tests for VS2017 about 5 months ago. Using CMake 3.13 (not yet released) would also let us fix it, but for now we use a vcxproj.user file that is merged with the main project and explicitly disables `/FC`.
2018-08-28 10:48:05 +02:00
####
# Temporary workaround for VS toolset changes in 2017
# We need to disable <UseFullPaths> property, but CMake doesn't support it
# until 3.13 (not yet released)
####
if (MSVC)
Move misc/ to tools/misc
2019-12-06 11:40:53 +01:00
configure_file(${CATCH_DIR}/tools/misc/SelfTest.vcxproj.user
Move tests from projects/ to tests/
2019-12-05 16:00:20 +01:00
${CMAKE_BINARY_DIR}/tests
Fix VS2017 approvals on AppVeyor Because of a change in VS toolset, missing option <UseFullPaths> is no longer interpreted as "don't pass /FC to the compiler", but rather as "pass /FC to the compiler". This is problematic, because /FC not only changes how much of the path is reporter by the compiler (e.g. in `__FILE__` macro), but it also lower cases the path. This lower-casing of the path broke our approval tests for VS2017 about 5 months ago. Using CMake 3.13 (not yet released) would also let us fix it, but for now we use a vcxproj.user file that is merged with the main project and explicitly disables `/FC`.
2018-08-28 10:48:05 +02:00
COPYONLY)
endif(MSVC) #Temporary workaround
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
# define the sources of the self test
# Please keep these ordered alphabetically
set(TEST_SOURCES
${SELF_TEST_DIR}/TestMain.cpp
${SELF_TEST_DIR}/IntrospectiveTests/CmdLine.tests.cpp
Add a test for the internals
2019-08-09 10:50:53 +02:00
${SELF_TEST_DIR}/IntrospectiveTests/Details.tests.cpp
First attempt at data generator support The support is to be considered experimental, that is, the interfaces, the first party generators and helper functions can change or be removed at any point in time. Related to #850
2018-06-12 18:50:39 +02:00
${SELF_TEST_DIR}/IntrospectiveTests/GeneratorsImpl.tests.cpp
Integrate Nonius benchmark into Catch2 Changes done to Nonius: * Moved things into "Catch::Benchmark" namespace * Benchmarks were integrated with `TEST_CASE`/`SECTION`/`GENERATE` macros * Removed Nonius's parameters for benchmarks, Generators should be used instead * Added relevant methods to the reporter interface (default-implemented, to avoid breaking existing 3rd party reporters) * Async processing is guarded with `_REENTRANT` macro for GCC/Clang, used by default on MSVC * Added a macro `CATCH_CONFIG_DISABLE_BENCHMARKING` that removes all traces of benchmarking from Catch
2019-04-23 23:41:13 +02:00
${SELF_TEST_DIR}/IntrospectiveTests/InternalBenchmark.tests.cpp
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
${SELF_TEST_DIR}/IntrospectiveTests/PartTracker.tests.cpp
Introduce Catch's own RNG based on the PCG family of RNGs In the future, we will also want to introduce our own `uniform_int_distribution` and `uniform_real_distribution` to get repeatable test runs across different platforms.
2019-10-06 21:47:54 +02:00
${SELF_TEST_DIR}/IntrospectiveTests/RandomNumberGeneration.tests.cpp
Fix test tag parsing to split [.foo] into [.][foo]
2019-03-29 10:48:56 +01:00
${SELF_TEST_DIR}/IntrospectiveTests/Tag.tests.cpp
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
${SELF_TEST_DIR}/IntrospectiveTests/String.tests.cpp
Start adding tests for string manip functions
2019-09-06 18:50:57 +02:00
${SELF_TEST_DIR}/IntrospectiveTests/StringManip.tests.cpp
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
${SELF_TEST_DIR}/IntrospectiveTests/Xml.tests.cpp
Moved enum tests that depend on internals to IntrospectiveTests. - also factored out makeEnumInfo, so tests don't need to touch registry - and added usage test that involves namespace
2019-04-25 11:13:11 +02:00
${SELF_TEST_DIR}/IntrospectiveTests/ToString.tests.cpp
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
${SELF_TEST_DIR}/UsageTests/Approx.tests.cpp
${SELF_TEST_DIR}/UsageTests/BDD.tests.cpp
${SELF_TEST_DIR}/UsageTests/Benchmark.tests.cpp
${SELF_TEST_DIR}/UsageTests/Class.tests.cpp
${SELF_TEST_DIR}/UsageTests/Compilation.tests.cpp
${SELF_TEST_DIR}/UsageTests/Condition.tests.cpp
${SELF_TEST_DIR}/UsageTests/Decomposition.tests.cpp
${SELF_TEST_DIR}/UsageTests/EnumToString.tests.cpp
${SELF_TEST_DIR}/UsageTests/Exception.tests.cpp
First attempt at data generator support The support is to be considered experimental, that is, the interfaces, the first party generators and helper functions can change or be removed at any point in time. Related to #850
2018-06-12 18:50:39 +02:00
${SELF_TEST_DIR}/UsageTests/Generators.tests.cpp
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
${SELF_TEST_DIR}/UsageTests/Message.tests.cpp
${SELF_TEST_DIR}/UsageTests/Misc.tests.cpp
Add `std::byte` stringification support (#1686) * Add `std::byte` support 1. Add byte detection in Catch config 2. Add a `std::byte`specialisation for Catch2::Detail::StringMaker
2019-07-13 14:47:56 +02:00
${SELF_TEST_DIR}/UsageTests/ToStringByte.tests.cpp
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
${SELF_TEST_DIR}/UsageTests/ToStringChrono.tests.cpp
${SELF_TEST_DIR}/UsageTests/ToStringGeneral.tests.cpp
Add to string for std::optional
2019-01-25 22:05:40 +01:00
${SELF_TEST_DIR}/UsageTests/ToStringOptional.tests.cpp
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
${SELF_TEST_DIR}/UsageTests/ToStringPair.tests.cpp
${SELF_TEST_DIR}/UsageTests/ToStringTuple.tests.cpp
Add StringMaker for std::variant, std::monostate (#1380) The StringMaker is off by default and can be enabled by a new macro `CATCH_CONFIG_ENABLE_VARIANT_STRINGMAKER`, to avoid increasing the footprint of stringification machinery by default.
2018-09-20 14:13:35 +02:00
${SELF_TEST_DIR}/UsageTests/ToStringVariant.tests.cpp
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
${SELF_TEST_DIR}/UsageTests/ToStringVector.tests.cpp
${SELF_TEST_DIR}/UsageTests/ToStringWhich.tests.cpp
${SELF_TEST_DIR}/UsageTests/Tricky.tests.cpp
${SELF_TEST_DIR}/UsageTests/VariadicMacros.tests.cpp
${SELF_TEST_DIR}/UsageTests/Matchers.tests.cpp
)
CheckFileList(TEST_SOURCES ${SELF_TEST_DIR})
# A set of impl files that just #include a single header
# Please keep these ordered alphabetically
set(SURROGATE_SOURCES
${SELF_TEST_DIR}/SurrogateCpps/catch_console_colour.cpp
${SELF_TEST_DIR}/SurrogateCpps/catch_debugger.cpp
${SELF_TEST_DIR}/SurrogateCpps/catch_interfaces_reporter.cpp
${SELF_TEST_DIR}/SurrogateCpps/catch_option.cpp
${SELF_TEST_DIR}/SurrogateCpps/catch_stream.cpp
${SELF_TEST_DIR}/SurrogateCpps/catch_test_case_tracker.cpp
${SELF_TEST_DIR}/SurrogateCpps/catch_test_spec.cpp
${SELF_TEST_DIR}/SurrogateCpps/catch_xmlwriter.cpp
)
CheckFileList(SURROGATE_SOURCES ${SELF_TEST_DIR}/SurrogateCpps)
# Specify the headers, too, so CLion recognises them as project files
set(HEADERS
${TOP_LEVEL_HEADERS}
${EXTERNAL_HEADERS}
${INTERNAL_HEADERS}
${REPORTER_HEADERS}
Integrate Nonius benchmark into Catch2 Changes done to Nonius: * Moved things into "Catch::Benchmark" namespace * Benchmarks were integrated with `TEST_CASE`/`SECTION`/`GENERATE` macros * Removed Nonius's parameters for benchmarks, Generators should be used instead * Added relevant methods to the reporter interface (default-implemented, to avoid breaking existing 3rd party reporters) * Async processing is guarded with `_REENTRANT` macro for GCC/Clang, used by default on MSVC * Added a macro `CATCH_CONFIG_DISABLE_BENCHMARKING` that removes all traces of benchmarking from Catch
2019-04-23 23:41:13 +02:00
${BENCHMARK_HEADERS}
Move the async-using parts of benchmarking into a .cpp file This keeps it out of the main include path when benchmarking is enabled, somewhat reducing the compilation-time penalty. Also moved some other functions into the .cpp file, especially helpers that could be given internal linkage, and concretized some iterator-templated code that only ever used `std::vector<double>::iterator`.
2019-06-15 11:14:24 +02:00
${BENCHMARK_SOURCES}
)
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
# Provide some groupings for IDEs
Unit tests build and run (todo: projects, includes, extras)
2019-11-30 20:45:34 +01:00
#SOURCE_GROUP("benchmark" FILES ${BENCHMARK_HEADERS} ${BENCHMARK_SOURCES})
#SOURCE_GROUP("Tests" FILES ${TEST_SOURCES})
#SOURCE_GROUP("Surrogates" FILES ${SURROGATE_SOURCES})
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
include(CTest)
WIP: CMake changes to build static lib + tests separately
2019-11-30 17:41:41 +01:00
add_executable(SelfTest ${TEST_SOURCES})
target_link_libraries(SelfTest PRIVATE Catch2)
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
Default the SelfTest project to C++11 Unlike the previous approach, this will still obey `-DCMAKE_CXX_STANDARD=X` properly.
2019-07-02 20:42:10 +02:00
# It took CMake until 3.8 to abandon the doomed approach of enumerating
Set CMakeLists to default to C++14 for our internal projects
2019-11-06 11:59:50 +01:00
# required features so we just list bunch of C++11 and C++14 features.
Default the SelfTest project to C++11 Unlike the previous approach, this will still obey `-DCMAKE_CXX_STANDARD=X` properly.
2019-07-02 20:42:10 +02:00
target_compile_features(SelfTest
PRIVATE
cxx_alignas
cxx_alignof
cxx_attributes
cxx_auto_type
cxx_constexpr
cxx_defaulted_functions
cxx_deleted_functions
cxx_final
cxx_lambdas
cxx_noexcept
cxx_override
cxx_range_for
cxx_rvalue_references
cxx_static_assert
cxx_strong_enums
cxx_trailing_return_types
cxx_unicode_literals
cxx_user_literals
cxx_variadic_macros
Set CMakeLists to default to C++14 for our internal projects
2019-11-06 11:59:50 +01:00
cxx_variable_templates
Default the SelfTest project to C++11 Unlike the previous approach, this will still obey `-DCMAKE_CXX_STANDARD=X` properly.
2019-07-02 20:42:10 +02:00
)
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
if (CATCH_ENABLE_COVERAGE)
set(ENABLE_COVERAGE ON CACHE BOOL "Enable coverage build." FORCE)
find_package(codecov)
add_coverage(SelfTest)
list(APPEND LCOV_REMOVE_PATTERNS "'/usr/*'")
coverage_evaluate()
endif()
# Add per compiler options
if ( CMAKE_CXX_COMPILER_ID MATCHES "Clang|AppleClang|GNU" )
Add -Wmissing-declarations to the SelfTest project This required some clean-up in our test files
2018-07-02 11:13:07 +02:00
target_compile_options( SelfTest PRIVATE -Wall -Wextra -Wunreachable-code -Wpedantic -Wmissing-declarations )
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
if (CATCH_ENABLE_WERROR)
Move the async-using parts of benchmarking into a .cpp file This keeps it out of the main include path when benchmarking is enabled, somewhat reducing the compilation-time penalty. Also moved some other functions into the .cpp file, especially helpers that could be given internal linkage, and concretized some iterator-templated code that only ever used `std::vector<double>::iterator`.
2019-06-15 11:14:24 +02:00
target_compile_options( SelfTest PRIVATE -Werror )
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
endif()
endif()
# Clang specific options go here
if ( CMAKE_CXX_COMPILER_ID MATCHES "Clang" )
target_compile_options( SelfTest PRIVATE -Wweak-vtables -Wexit-time-destructors -Wglobal-constructors -Wmissing-noreturn )
endif()
if ( CMAKE_CXX_COMPILER_ID MATCHES "MSVC" )
STRING(REGEX REPLACE "/W[0-9]" "/W4" CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS}) # override default warning level
Add MSVC warning 5038 (Wreorder equivalent) to CMakeLists
2018-11-05 20:07:33 +01:00
target_compile_options( SelfTest PRIVATE /w44265 /w44061 /w44062 /w45038 )
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
if (CATCH_ENABLE_WERROR)
target_compile_options( SelfTest PRIVATE /WX)
endif()
# Force MSVC to consider everything as encoded in utf-8
target_compile_options( SelfTest PRIVATE /utf-8 )
endif()
# configure unit tests via CTest
add_test(NAME RunTests COMMAND $<TARGET_FILE:SelfTest>)
Add test for omitting the 'Filters' heading when there were none
2019-04-16 23:49:22 +02:00
set_tests_properties(RunTests PROPERTIES
FAIL_REGULAR_EXPRESSION "Filters:"
)
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
Make --list-* exit code be 0 Previously it returned the sum of listed things because ???. This was completely useless and in many ways actively counterproductive because of the success/failure conventions around exit codes. Closes #1410
2019-06-16 16:12:47 +02:00
# Because CTest does not allow us to check both return code _and_ expected
# output in one test, we run these commands twice. First time we check
# the output, the second time we check the exit code.
add_test(NAME List::Tests::Output COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity high)
set_tests_properties(List::Tests::Output PROPERTIES
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
PASS_REGULAR_EXPRESSION "[0-9]+ test cases"
FAIL_REGULAR_EXPRESSION "Hidden Test"
)
Make --list-* exit code be 0 Previously it returned the sum of listed things because ???. This was completely useless and in many ways actively counterproductive because of the success/failure conventions around exit codes. Closes #1410
2019-06-16 16:12:47 +02:00
add_test(NAME List::Tests::ExitCode COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity high)
Specialize XML reporter's --list output
2019-06-22 15:31:11 +02:00
add_test(NAME List::Tests::XmlOutput COMMAND $<TARGET_FILE:SelfTest> --list-tests --verbosity high -r xml)
set_tests_properties(List::Tests::XmlOutput PROPERTIES
PASS_REGULAR_EXPRESSION "<Line>[0-9]+</Line>"
FAIL_REGULAR_EXPRESSION "[0-9]+ test cases"
)
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
Make --list-* exit code be 0 Previously it returned the sum of listed things because ???. This was completely useless and in many ways actively counterproductive because of the success/failure conventions around exit codes. Closes #1410
2019-06-16 16:12:47 +02:00
add_test(NAME List::Tags::Output COMMAND $<TARGET_FILE:SelfTest> --list-tags)
set_tests_properties(List::Tags::Output PROPERTIES
PASS_REGULAR_EXPRESSION "[0-9]+ tags"
Fix CTest regex error The desired behaviour was to match a literal "[.]", so the regex has to be escaped as "\\[\\.\\]" -- double backslashes, because it has to be escaped from CMake as well as from the regex engine.
2018-09-08 20:56:31 +02:00
FAIL_REGULAR_EXPRESSION "\\[\\.\\]")
Make --list-* exit code be 0 Previously it returned the sum of listed things because ???. This was completely useless and in many ways actively counterproductive because of the success/failure conventions around exit codes. Closes #1410
2019-06-16 16:12:47 +02:00
add_test(NAME List::Tags::ExitCode COMMAND $<TARGET_FILE:SelfTest> --list-tags)
Specialize XML reporter's --list output
2019-06-22 15:31:11 +02:00
add_test(NAME List::Tags::XmlOutput COMMAND $<TARGET_FILE:SelfTest> --list-tags -r xml)
set_tests_properties(List::Tags::XmlOutput PROPERTIES
PASS_REGULAR_EXPRESSION "<Count>18</Count>"
FAIL_REGULAR_EXPRESSION "[0-9]+ tags"
)
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
Make --list-* exit code be 0 Previously it returned the sum of listed things because ???. This was completely useless and in many ways actively counterproductive because of the success/failure conventions around exit codes. Closes #1410
2019-06-16 16:12:47 +02:00
add_test(NAME List::Reporters::Output COMMAND $<TARGET_FILE:SelfTest> --list-reporters)
set_tests_properties(List::Reporters::Output PROPERTIES PASS_REGULAR_EXPRESSION "Available reporters:")
add_test(NAME List::Reporters::ExitCode COMMAND $<TARGET_FILE:SelfTest> --list-reporters)
Specialize XML reporter's --list output
2019-06-22 15:31:11 +02:00
add_test(NAME List::Reporters::XmlOutput COMMAND $<TARGET_FILE:SelfTest> --list-reporters -r xml)
set_tests_properties(List::Reporters::XmlOutput PROPERTIES
PASS_REGULAR_EXPRESSION "<Name>compact</Name>"
FAIL_REGULAR_EXPRESSION "Available reporters:"
)
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
Make --list-* exit code be 0 Previously it returned the sum of listed things because ???. This was completely useless and in many ways actively counterproductive because of the success/failure conventions around exit codes. Closes #1410
2019-06-16 16:12:47 +02:00
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
add_test(NAME NoAssertions COMMAND $<TARGET_FILE:SelfTest> -w NoAssertions)
set_tests_properties(NoAssertions PROPERTIES PASS_REGULAR_EXPRESSION "No assertions in test case")
Improve reporting of unmatched filters (#1684) This PR ultimately does 3 things * Separately tracks matched tests per each filter part (that is, a set of filters separated by an OR (`,`)), which allows Catch2 to report each of the alternative filters that don't match any tests. * Fixes `-w NoTests` to return non-zero in the process * Adds tests for `-w NoTests`.
2019-08-06 20:51:19 +02:00
add_test(NAME NoTest COMMAND $<TARGET_FILE:SelfTest> Tracker, "___nonexistent_test___")
set_tests_properties(NoTest PROPERTIES
PASS_REGULAR_EXPRESSION "No test cases matched '___nonexistent_test___'"
FAIL_REGULAR_EXPRESSION "No tests ran"
)
Move tests from projects/ to tests/
2019-12-05 16:00:20 +01:00
add_test(NAME WarnAboutNoTests COMMAND ${CMAKE_COMMAND} -P ${SELF_TEST_DIR}/WarnAboutNoTests.cmake $<TARGET_FILE:SelfTest>)
Improve reporting of unmatched filters (#1684) This PR ultimately does 3 things * Separately tracks matched tests per each filter part (that is, a set of filters separated by an OR (`,`)), which allows Catch2 to report each of the alternative filters that don't match any tests. * Fixes `-w NoTests` to return non-zero in the process * Adds tests for `-w NoTests`.
2019-08-06 20:51:19 +02:00
add_test(NAME UnmatchedOutputFilter COMMAND $<TARGET_FILE:SelfTest> [this-tag-does-not-exist] -w NoTests)
set_tests_properties(UnmatchedOutputFilter
PROPERTIES
PASS_REGULAR_EXPRESSION "No test cases matched '\\[this-tag-does-not-exist\\]'"
)
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
Integrate tests for #1394 into our test suite
2019-01-15 23:13:16 +01:00
add_test(NAME FilteredSection-1 COMMAND $<TARGET_FILE:SelfTest> \#1394 -c RunSection)
set_tests_properties(FilteredSection-1 PROPERTIES FAIL_REGULAR_EXPRESSION "No tests ran")
add_test(NAME FilteredSection-2 COMMAND $<TARGET_FILE:SelfTest> \#1394\ nested -c NestedRunSection -c s1)
set_tests_properties(FilteredSection-2 PROPERTIES FAIL_REGULAR_EXPRESSION "No tests ran")
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
# AppVeyor has a Python 2.7 in path, but doesn't have .py files as autorunnable
Moved scripts/ to tools/scripts/
2019-12-06 11:53:31 +01:00
add_test(NAME ApprovalTests COMMAND ${PYTHON_EXECUTABLE} ${CATCH_DIR}/tools/scripts/approvalTests.py $<TARGET_FILE:SelfTest>)
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
set_tests_properties(ApprovalTests PROPERTIES FAIL_REGULAR_EXPRESSION "Results differed")
Add integration test for the children completion check fix
2019-06-30 00:00:41 +02:00
add_test(NAME RegressionCheck-1670 COMMAND $<TARGET_FILE:SelfTest> "#1670 regression check" -c A -r compact)
set_tests_properties(RegressionCheck-1670 PROPERTIES PASS_REGULAR_EXPRESSION "Passed 1 test case with 2 assertions.")
Integrate tests for #1394 into our test suite
2019-01-15 23:13:16 +01:00
Add a test for Catch version output
2019-08-03 20:16:46 +02:00
add_test(NAME VersionCheck COMMAND $<TARGET_FILE:SelfTest> -h)
set_tests_properties(VersionCheck PROPERTIES PASS_REGULAR_EXPRESSION "Catch v${PROJECT_VERSION}")
Add test for --libidentify command line flag
2019-09-07 20:36:12 +02:00
add_test(NAME LibIdentityTest COMMAND $<TARGET_FILE:SelfTest> --libidentify)
set_tests_properties(LibIdentityTest PROPERTIES PASS_REGULAR_EXPRESSION "description: A Catch2 test executable")
Add test for --filename-as-tags
2019-09-07 20:43:14 +02:00
add_test(NAME FilenameAsTagsTest COMMAND $<TARGET_FILE:SelfTest> -\# --list-tags)
set_tests_properties(FilenameAsTagsTest PROPERTIES PASS_REGULAR_EXPRESSION "\\[#Approx.tests\\]")
Add tests for escaping special characters in CLI
2019-09-21 00:11:45 +02:00
add_test(NAME EscapeSpecialCharactersInTestNames COMMAND $<TARGET_FILE:SelfTest> "Test with special\\, characters \"in name")
set_tests_properties(EscapeSpecialCharactersInTestNames PROPERTIES PASS_REGULAR_EXPRESSION "1 assertion in 1 test case")
Hidden tests now require positive filter match to be selected This also required some refactoring of how the pattern matching works. This means that the concepts of include and exclude patterns are no longer unified, with exclusion patterns working as just negation of an inclusion patterns (which led to including hidden tags by default, as they did not match the exclusion), but rather both include and exclude patterns are handled separately. The new logic is that given a filter and a test case, the test case must match _all_ include patterns and _no_ exclude patterns to be included by the filter. Furthermore, if the test case is hidden, then the filter must have at least one include pattern for the test case to be used. Closes #1184
2019-06-22 20:26:03 +02:00
add_test(NAME NegativeSpecNoHiddenTests COMMAND $<TARGET_FILE:SelfTest> --list-tests ~[approval])
set_tests_properties(NegativeSpecNoHiddenTests PROPERTIES FAIL_REGULAR_EXPRESSION "\\[\\.\\]")
Move tests from projects/ to tests/
2019-12-05 16:00:20 +01:00
add_test(NAME TestsInFile::SimpleSpecs COMMAND $<TARGET_FILE:SelfTest> "-f ${SELF_TEST_DIR}/Misc/plain-old-tests.input")
Add more tests for reading test specs from file Related to #1770
2019-10-20 15:14:50 +02:00
set_tests_properties(TestsInFile::SimpleSpecs PROPERTIES PASS_REGULAR_EXPRESSION "6 assertions in 2 test cases")
Add tests for the -f/--input-file command line argument
2019-10-09 12:48:55 +02:00
Move tests from projects/ to tests/
2019-12-05 16:00:20 +01:00
add_test(NAME TestsInFile::EscapeSpecialCharacters COMMAND $<TARGET_FILE:SelfTest> "-f ${SELF_TEST_DIR}/Misc/special-characters-in-file.input")
Add more tests for reading test specs from file Related to #1770
2019-10-20 15:14:50 +02:00
set_tests_properties(TestsInFile::EscapeSpecialCharacters PROPERTIES PASS_REGULAR_EXPRESSION "1 assertion in 1 test case")
# CTest does not allow us to create an AND of required regular expressions,
# so we have to split the test into 2 parts and look for parts of the expected
# output separately.
Move tests from projects/ to tests/
2019-12-05 16:00:20 +01:00
add_test(NAME TestsInFile::InvalidTestNames-1 COMMAND $<TARGET_FILE:SelfTest> "-f ${SELF_TEST_DIR}/Misc/invalid-test-names.input")
Add more tests for reading test specs from file Related to #1770
2019-10-20 15:14:50 +02:00
set_tests_properties(TestsInFile::InvalidTestNames-1 PROPERTIES PASS_REGULAR_EXPRESSION "Invalid Filter: \"Test with special, characters in \\\\\" name\"")
Move tests from projects/ to tests/
2019-12-05 16:00:20 +01:00
add_test(NAME TestsInFile::InvalidTestNames-2 COMMAND $<TARGET_FILE:SelfTest> "-f ${SELF_TEST_DIR}/Misc/invalid-test-names.input")
Add more tests for reading test specs from file Related to #1770
2019-10-20 15:14:50 +02:00
set_tests_properties(TestsInFile::InvalidTestNames-2 PROPERTIES PASS_REGULAR_EXPRESSION "No tests ran")
Add tests for the -f/--input-file command line argument
2019-10-09 12:48:55 +02:00
Add a test for Catch version output
2019-08-03 20:16:46 +02:00
Integrate CMake with `<catch2/catch.hpp>` include paths This also goes for pkg-config installed by our CMake installation. This includes * Updating CMake version on Travis * Adding a `Catch2` subfolder to the `single_include/` folder to provide this include path both _inside_ the repository, and _outside_. * Updated examples to build with the new paths * Other general CMake cleanup
2018-06-11 10:48:10 +02:00
if (CATCH_USE_VALGRIND)
add_test(NAME ValgrindRunTests COMMAND valgrind --leak-check=full --error-exitcode=1 $<TARGET_FILE:SelfTest>)
add_test(NAME ValgrindListTests COMMAND valgrind --leak-check=full --error-exitcode=1 $<TARGET_FILE:SelfTest> --list-tests --verbosity high)
set_tests_properties(ValgrindListTests PROPERTIES PASS_REGULAR_EXPRESSION "definitely lost: 0 bytes in 0 blocks")
add_test(NAME ValgrindListTags COMMAND valgrind --leak-check=full --error-exitcode=1 $<TARGET_FILE:SelfTest> --list-tags)
set_tests_properties(ValgrindListTags PROPERTIES PASS_REGULAR_EXPRESSION "definitely lost: 0 bytes in 0 blocks")
endif()
Reference in New Issue Copy Permalink