From a6cdcd43aa24adf1e36edd5517a96feaeac18c25 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martin=20Ho=C5=99e=C5=88ovsk=C3=BD?= Date: Thu, 22 Jun 2017 18:55:17 +0200 Subject: [PATCH] Added "How to test changes in PR" section to documentation Also linked it from PR template. Closes #936 --- .github/pull_request_template.md | 3 +++ docs/contributing.md | 23 +++++++++++++++++++++-- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 95e3ccd2..368f41fb 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -2,6 +2,9 @@ Please do not submit pull requests changing the `version.hpp` or the single-include `catch.hpp` file, these are changed only when a new release is made. + +Before submitting a PR you should probably read the contributor documentation +at docs/contributing.md. It will tell you how to properly test your changes. --> diff --git a/docs/contributing.md b/docs/contributing.md index 28d0643d..1bcef99b 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -18,8 +18,8 @@ Ongoing development is currently on _master_. At some point an integration branc _Users_ of Catch primarily use the single header version. _Maintainers_ should work with the full source (which is still, primarily, in headers). This can be found in the `include` folder. There are a set of test files, currently under `projects/SelfTest`. The test app can be built via CMake from the `CMakeLists.txt` file in the root, or you can generate -project files for Visual Studio, XCode, and others (instructions in the `projects` folder). If you have access to CLion -that can work with the CMake file directly. +project files for Visual Studio, XCode, and others (instructions in the `projects` folder). If you have access to CLion, +it can work with the CMake file directly. As well as the runtime test files you'll also see a `SurrogateCpps` directory under `projects/SelfTest`. This contains a set of .cpp files that each `#include` a single header. @@ -34,6 +34,25 @@ __When submitting a pull request please do not include changes to the single inc as these are managed by the scripts!__ +## Testing your changes + +Obviously all changes to Catch's code should be tested. If you added new functionality, you should add tests covering and +showcasing it. Even if you have only made changes to Catch internals (ie you implemented some performance improvements), +you should still test your changes. + +This means 3 things + +* Compiling Catch's SelfTest project -- code that does not compile is evidently incorrect. Obviously, you are not expected to +have access to all compilers and platforms Catch supports, Catch's CI pipeline will compile your code using supported compilers +once you open a PR. +* Running the SelfTest binary. There should be no unexpected failures on simple run. +* Running Catch's approval tests. Approval tests compare current output of the SelfTest binary in various configurations against +known good output. Catch's repository provides utility scripts `approvalTests.py` to help you with this. It needs to be passed +the SelfTest binary compiled with your changes, like so: `$ ./scripts/approvalTests.py clang-build/SelfTest`. The output should +be fairly self-explanatory. + + + *this document is still in-progress...* ---