First commit of benchmarks for Catch runtime perf.

So far its very much a WIP with some problems that are known already and
not very representative tests.
This commit is contained in:
Martin Hořeňovský
2017-01-14 21:55:37 +01:00
parent e3659cdddd
commit 3b7511e564
6 changed files with 112 additions and 0 deletions

View File

@@ -0,0 +1,2 @@
#define CATCH_CONFIG_MAIN
#include "catch.hpp"

View File

@@ -0,0 +1,39 @@
#include "catch.hpp"
#include <vector>
///////////////////////////////////////////////////////////////////////////////
TEST_CASE("Successful tests -- REQUIRE", "[Success]") {
const size_t sz = 1 * 1024 * 1024;
std::vector<size_t> vec; vec.reserve(sz);
for (size_t i = 0; i < sz; ++i){
vec.push_back(i);
REQUIRE(vec.back() == i);
}
}
///////////////////////////////////////////////////////////////////////////////
TEST_CASE("Successful tests -- CHECK", "[Success]") {
const size_t sz = 1 * 1024 * 1024;
std::vector<size_t> vec; vec.reserve(sz);
for (size_t i = 0; i < sz; ++i){
vec.push_back(i);
CHECK(vec.back() == i);
}
}
///////////////////////////////////////////////////////////////////////////////
TEST_CASE("Unsuccessful tests -- CHECK", "[Failure]") {
const size_t sz = 128 * 1024;
std::vector<size_t> vec; vec.reserve(sz);
for (size_t i = 0; i < sz; ++i){
vec.push_back(i);
CHECK(vec.size() == i);
}
}

View File

@@ -0,0 +1,4 @@
This is very much a work in progress.
The past results are standardized to a developer's machine,
the benchmarking script is basic and there are only 3 benchmarks,
but this should get better in time. For now, at least there is something to go by.

View File

@@ -0,0 +1,3 @@
Successful tests -- CHECK: median: 3.38116 (s), stddev: 0.11567366292001534 (s)
Successful tests -- REQUIRE: median: 3.479955 (s), stddev: 0.16295972890734556 (s)
Unsuccessful tests -- CHECK: median: 1.966895 (s), stddev: 0.06323488524716572 (s)