Added "How to test changes in PR" section to documentation

Also linked it from PR template.

Closes #936
This commit is contained in:
Martin Hořeňovský 2017-06-22 18:55:17 +02:00 committed by Phil Nash
parent 7bcfdf8e94
commit 9382534d59
2 changed files with 24 additions and 2 deletions

View File

@ -2,6 +2,9 @@
Please do not submit pull requests changing the `version.hpp`
or the single-include `catch.hpp` file, these are changed
only when a new release is made.
Before submitting a PR you should probably read the contributor documentation
at docs/contributing.md. It will tell you how to properly test your changes.
-->

View File

@ -18,8 +18,8 @@ Ongoing development is currently on _master_. At some point an integration branc
_Users_ of Catch primarily use the single header version. _Maintainers_ should work with the full source (which is still,
primarily, in headers). This can be found in the `include` folder. There are a set of test files, currently under
`projects/SelfTest`. The test app can be built via CMake from the `CMakeLists.txt` file in the root, or you can generate
project files for Visual Studio, XCode, and others (instructions in the `projects` folder). If you have access to CLion
that can work with the CMake file directly.
project files for Visual Studio, XCode, and others (instructions in the `projects` folder). If you have access to CLion,
it can work with the CMake file directly.
As well as the runtime test files you'll also see a `SurrogateCpps` directory under `projects/SelfTest`.
This contains a set of .cpp files that each `#include` a single header.
@ -34,6 +34,25 @@ __When submitting a pull request please do not include changes to the single inc
as these are managed by the scripts!__
## Testing your changes
Obviously all changes to Catch's code should be tested. If you added new functionality, you should add tests covering and
showcasing it. Even if you have only made changes to Catch internals (ie you implemented some performance improvements),
you should still test your changes.
This means 3 things
* Compiling Catch's SelfTest project -- code that does not compile is evidently incorrect. Obviously, you are not expected to
have access to all compilers and platforms Catch supports, Catch's CI pipeline will compile your code using supported compilers
once you open a PR.
* Running the SelfTest binary. There should be no unexpected failures on simple run.
* Running Catch's approval tests. Approval tests compare current output of the SelfTest binary in various configurations against
known good output. Catch's repository provides utility scripts `approvalTests.py` to help you with this. It needs to be passed
the SelfTest binary compiled with your changes, like so: `$ ./scripts/approvalTests.py clang-build/SelfTest`. The output should
be fairly self-explanatory.
*this document is still in-progress...*
---