Commit Graph

54 Commits

Author SHA1 Message Date
sp-dani-garcia
51b29ced1a Add SonarQube Generic Test Data reporter
It outputs reports in the `Generic Execution Test Data` format, see
https://docs.sonarqube.org/latest/analysis/generic-test/, specifically
https://docs.sonarqube.org/latest/analysis/generic-test/#header-2

Close #1738 (this is a cherry-pick and fixup of that PR)
2019-10-27 10:02:41 +01:00
Martin Hořeňovský
c424ca47f9
Revert "Do not resolve symbolic path when sanitizing filepath during approvals"
This reverts commit 52f3abadbb.

Opens #1691
2019-07-19 18:16:21 +02:00
Martin Hořeňovský
52f3abadbb
Do not resolve symbolic path when sanitizing filepath during approvals
Fixes #1691
2019-07-18 16:33:57 +02:00
MaciejPatro
1faccd601d Improve path normalization for approvalTests.py
This fixes 3 problems:

* Relative paths on Windows are now supported
* Out-of-tree (paths starting with ../) builds are now supported
* Path separator normalization no longer affects non-path components of input (problem with Compact reporter)


Fixes #1379 
Fixes #1222 
Fixes #1200 
Fixes #1194
2018-10-19 12:46:06 +02:00
Martin Hořeňovský
6b9ca0888a Add tests for #1404 2018-10-13 16:53:44 +02:00
Axel Huebl
90663b2e75 Tests: Spaces & TABs
Fix TABs and none-PEP8 spaces in approval test.
Does not yet fix overlong lines for full `flake8` compliance.
2018-08-29 18:28:27 +02:00
Phil Nash
1a63fad8d6 Seed the RNG in approval tests 2018-08-19 22:34:14 +02:00
Martin Hořeňovský
2771a8ee9a Normalize OS X specific pass/fail strings in approvals 2018-06-04 23:19:55 +02:00
Rupert Steel
90988f578c Enable console colour in the approval tests on windows. 2018-05-14 09:41:18 +02:00
Martin Hořeňovský
375f2052bd Use io.open in approvalTests.py regardless of Python version
Both Python 2.7 and 3.x support full-featured io.open, so we
can avoid using a polyfill over this.
2018-04-19 22:02:31 +02:00
Tom Hughes
dc6b83bec9 Support Python3 in approval tests 2018-04-16 21:19:13 +02:00
Martin Hořeňovský
f0909dfe02 Add yet another expansion of INF macro
Closes #1151
2018-01-14 17:04:36 +01:00
Martin Hořeňovský
673dcc16a9 Make approval tests also check compact reporter 2017-12-06 15:48:46 +01:00
Martin Hořeňovský
8dbaac61ff Final set of fixes for floating matchers approvals 2017-11-12 11:49:36 +01:00
Martin Hořeňovský
cecee3459a Add another MSVC NAN macro 2017-11-10 19:52:43 +01:00
Martin Hořeňovský
0b1f1b1003 Add ULP and margin matcher
Closes #1074
2017-11-10 18:33:00 +01:00
Martin Hořeňovský
accf9859b4 Add OSX specific INFINITE macro parsing in approval tests 2017-11-01 08:46:49 +01:00
Martin Hořeňovský
22ac9d2184 Approx cleanup: More tests, INFINITY handling, etc 2017-11-01 07:30:11 +01:00
Martin Hořeňovský
79b405fd3f Add stringification for std::chrono::{duration,time_point}
Also hides std::chrono, std::pair and std::chrono::* behind
new configuration macros, CATCH_CONFIG_ENABLE_*_STRINGMAKER
to avoid dragging in <utility>, <tuple> and <chrono> in common
path, unless requested.
2017-10-09 13:03:29 +02:00
Martin Hořeňovský
ab5d176195 Fix/disable failing approval tests 2017-09-02 10:51:19 +02:00
Phil Nash
4421672fb8 Removed benchmark tests from approval tests (for now) 2017-08-05 11:26:20 +01:00
Phil Nash
c874a99c6c Enabled c++11 tests in approvals 2017-07-13 08:52:51 +01:00
Martin Hořeňovský
98e61c31df Approval tests now see different line endings as ok
This is mostly to deal with WSL, where git will checkout file with CRLF,
but code that uses formatted output will be writing LFs.
2017-04-29 18:06:36 +02:00
Martin Hořeňovský
a3ceb8f007 Approval tests now can deal with different expansions of errno 2017-04-29 17:52:12 +02:00
Martin Hořeňovský
70ac6dbb9f Minor output improvements in approvalTests.py 2017-02-24 15:56:26 +01:00
Martin Hořeňovský
090fc74cca Scripts intended to be run should now have x bit set. 2017-02-01 21:34:45 +01:00
Phil Nash
225e90d8ba Don’t include tests that rely on varying two-phase-lookup behaviour in approval tests 2017-01-23 17:47:58 +00:00
Martin Hořeňovský
78a2866dc7 Approval tests are now mostly Windows compatible
There are some differences in output between Catch on Windows and
Catch on Linux, that indicate a minor bug, but those have to be fixed separately.
2017-01-19 23:56:31 +01:00
Martin Hořeňovský
9acc6b9673 Approval tests now use Python std lib instead of call to diff
This needed to change to let it run on Windows as well as on the Unices
2017-01-19 22:52:47 +01:00
Martin Hořeňovský
c4b5057094 Approval tests now uses path relative to cwd, not the catch folder.
This means that bash's autocompletion is actually helpful.
2017-01-19 22:08:51 +01:00
Mickey Rose
d38b9266e7 simplify output filtering in approvalTests.py 2017-01-19 22:04:29 +01:00
Martin Hořeňovský
b71a06cf98 JUnit reporter outputs timestamps now
Also extended approval tests script to support the change
2017-01-16 20:21:43 +01:00
Phil Nash
62cbde369e Exclude test name from approval tests 2017-01-09 14:37:28 +00:00
Phil Nash
1ae84897d4 Marked tests that use C++11 features with [c++11] tag and exclude them from approval tests 2017-01-09 14:37:28 +00:00
Phil Nash
976a655496 Approval tests can use Catch path fixed in env. var, and convert nullptr and __null to 0 for comparison 2017-01-09 14:37:28 +00:00
Phil Nash
79acc0504b Sort test ordering during Approval testing 2016-11-29 11:32:16 +00:00
Phil Nash
5d5ed5a283 Updated approvalTests script to allow for modified version strings 2015-07-10 08:08:37 +01:00
Phil Nash
804896cdfa Fixed approval tests script for new version number formatting 2015-06-30 18:24:59 +01:00
Phil Nash
a99e75b928 Made approvals more platform agnostic
removes root from all paths - so only relative paths remain
2014-12-11 23:40:29 +00:00
Phil Nash
878c257de7 Merge pull request #255 from johannesmoene/fix-python-print
Fix python print
2014-04-21 19:15:44 +01:00
Phil Nash
87891cf38b Updated approval tests baselines 2014-04-16 18:00:55 +01:00
Martin Moene
77c9edfcdb Fix python scripts for python 3 (print now a function rather than a keyword) 2014-03-08 11:31:38 +01:00
Martin Moene
91ea20c4cc Change <> to != 2014-03-08 11:31:11 +01:00
Phil Nash
a35ee200da Filter out version number from approval tests 2013-09-30 08:05:14 +01:00
Phil Nash
16c21ee265 No, really trim paths in approval tests down to just filenames 2013-09-30 08:01:10 +01:00
Phil Nash
1e74938538 Trim paths in approval tests down to filenames 2013-09-30 07:39:06 +01:00
Phil Nash
2f7a296897 Split approve file into multiple files 2013-09-27 19:01:14 +01:00
Phil Nash
11b5179ceb Redacted test durations from JUnit reporter output in approval test script 2013-08-15 18:50:40 +01:00
Phil Nash
b80280f428 Tidied up reporting of missing assertions in test cases
- also removed basic reporter from approval test
2013-07-26 19:19:44 +01:00
Phil Nash
a9b346b14d Removed line numbers from approval tests of xml too 2013-06-29 12:04:38 +01:00