Add an experimental new way of capturing stdout/stderr

Unlike the relatively non-invasive old way of capturing stdout/stderr,
this new way is also able to capture output from C's stdlib functions
such as `printf`. This is done by redirecting stdout and stderr file
descriptors to a file, and then reading this file back.

This approach has two sizeable drawbacks:
1) Performance, obviously. Previously an installed capture made the
program run faster (as long as it was then discarded), because a call
to `std::cout` did not result in text output to the console. This new
capture method in fact forces disk IO. While it is likely that any
modern OS will keep this file in memory-cache and might never actually
issue the IO to the backing storage, it is still a possibility and
calls to the file system are not free.

2) Nonportability. While POSIX is usually assumed portable, and this
implementation relies only on a very common parts of it, it is no
longer standard C++ (or just plain C) and thus might not be available
on some obscure platforms. Different C libs might also implement the
relevant functions in a less-than-useful ways (e.g. MS's `tmpfile`
generates a temp file inside system folder, so it will not work
without elevated privileges and thus is useless).

These two drawbacks mean that, at least for now, the new capture is
opt-in. To opt-in, `CATCH_CONFIG_EXPERIMENTAL_REDIRECT` needs to be
defined in the implementation file.

Closes #1243
This commit is contained in:
Martin Hořeňovský 2018-04-29 22:14:41 +02:00
parent 88a6ff0b65
commit e92b9c07c3
5 changed files with 243 additions and 44 deletions

View File

@ -159,6 +159,7 @@ set(INTERNAL_HEADERS
${HEADER_DIR}/internal/catch_objc.hpp ${HEADER_DIR}/internal/catch_objc.hpp
${HEADER_DIR}/internal/catch_objc_arc.hpp ${HEADER_DIR}/internal/catch_objc_arc.hpp
${HEADER_DIR}/internal/catch_option.hpp ${HEADER_DIR}/internal/catch_option.hpp
${HEADER_DIR}/internal/catch_output_redirect.h
${HEADER_DIR}/internal/catch_platform.h ${HEADER_DIR}/internal/catch_platform.h
${HEADER_DIR}/internal/catch_random_number_generator.h ${HEADER_DIR}/internal/catch_random_number_generator.h
${HEADER_DIR}/internal/catch_reenable_warnings.h ${HEADER_DIR}/internal/catch_reenable_warnings.h
@ -225,6 +226,7 @@ set(IMPL_SOURCES
${HEADER_DIR}/internal/catch_matchers_generic.cpp ${HEADER_DIR}/internal/catch_matchers_generic.cpp
${HEADER_DIR}/internal/catch_matchers_string.cpp ${HEADER_DIR}/internal/catch_matchers_string.cpp
${HEADER_DIR}/internal/catch_message.cpp ${HEADER_DIR}/internal/catch_message.cpp
${HEADER_DIR}/internal/catch_output_redirect.cpp
${HEADER_DIR}/internal/catch_registry_hub.cpp ${HEADER_DIR}/internal/catch_registry_hub.cpp
${HEADER_DIR}/internal/catch_interfaces_reporter.cpp ${HEADER_DIR}/internal/catch_interfaces_reporter.cpp
${HEADER_DIR}/internal/catch_random_number_generator.cpp ${HEADER_DIR}/internal/catch_random_number_generator.cpp

View File

@ -121,6 +121,7 @@ by using `_NO_` in the macro, e.g. `CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS`.
CATCH_CONFIG_DISABLE_STRINGIFICATION // Disable stringifying the original expression CATCH_CONFIG_DISABLE_STRINGIFICATION // Disable stringifying the original expression
CATCH_CONFIG_DISABLE // Disables assertions and test case registration CATCH_CONFIG_DISABLE // Disables assertions and test case registration
CATCH_CONFIG_WCHAR // Enables use of wchart_t CATCH_CONFIG_WCHAR // Enables use of wchart_t
CATCH_CONFIG_EXPERIMENTAL_REDIRECT // Enables the new (experimental) way of capturing stdout/stderr
Currently Catch enables `CATCH_CONFIG_WINDOWS_SEH` only when compiled with MSVC, because some versions of MinGW do not have the necessary Win32 API support. Currently Catch enables `CATCH_CONFIG_WINDOWS_SEH` only when compiled with MSVC, because some versions of MinGW do not have the necessary Win32 API support.
@ -131,7 +132,9 @@ Currently Catch enables `CATCH_CONFIG_WINDOWS_SEH` only when compiled with MSVC,
`CATCH_CONFIG_WCHAR` is on by default, but can be disabled. Currently `CATCH_CONFIG_WCHAR` is on by default, but can be disabled. Currently
it is only used in support for DJGPP cross-compiler. it is only used in support for DJGPP cross-compiler.
These toggles can be disabled by using `_NO_` form of the toggle, e.g. `CATCH_CONFIG_NO_WINDOWS_SEH`. With the exception of `CATCH_CONFIG_EXPERIMENTAL_REDIRECT`,
these toggles can be disabled by using `_NO_` form of the toggle,
e.g. `CATCH_CONFIG_NO_WINDOWS_SEH`.
### `CATCH_CONFIG_FAST_COMPILE` ### `CATCH_CONFIG_FAST_COMPILE`
Defining this flag speeds up compilation of test files by ~20%, by making 2 changes: Defining this flag speeds up compilation of test files by ~20%, by making 2 changes:

View File

@ -0,0 +1,131 @@
/*
* Created by Martin on 28/04/2018.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
#include "catch_output_redirect.h"
#include <cstdio>
#include <cstring>
#include <fstream>
#include <sstream>
#if defined(CATCH_PLATFORM_WINDOWS)
#include <io.h> //_dup and _dup2
#define dup _dup
#define dup2 _dup2
#define fileno _fileno
#else
#include <unistd.h> // dup and dup2
#endif
namespace Catch {
RedirectedStream::RedirectedStream( std::ostream& originalStream, std::ostream& redirectionStream )
: m_originalStream( originalStream ),
m_redirectionStream( redirectionStream ),
m_prevBuf( m_originalStream.rdbuf() )
{
m_originalStream.rdbuf( m_redirectionStream.rdbuf() );
}
RedirectedStream::~RedirectedStream() {
m_originalStream.rdbuf( m_prevBuf );
}
RedirectedStdOut::RedirectedStdOut() : m_cout( Catch::cout(), m_rss.get() ) {}
auto RedirectedStdOut::str() const -> std::string { return m_rss.str(); }
RedirectedStdErr::RedirectedStdErr()
: m_cerr( Catch::cerr(), m_rss.get() ),
m_clog( Catch::clog(), m_rss.get() )
{}
auto RedirectedStdErr::str() const -> std::string { return m_rss.str(); }
#if defined(CATCH_PLATFORM_WINDOWS)
TempFile::TempFile() {
if (tmpnam_s(m_buffer)) {
throw std::runtime_error("Could not get a temp filename");
}
if (fopen_s(&m_file, m_buffer, "w")) {
char buffer[100];
if (strerror_s(buffer, errno)) {
throw std::runtime_error("Could not translate errno to string");
}
throw std::runtime_error("Could not open the temp file: " + std::string(m_buffer) + buffer);
}
}
#else
TempFile::TempFile() {
m_file = std::tmpfile();
if (!m_file) {
throw std::runtime_error("Could not create a temp file.");
}
}
#endif
TempFile::~TempFile() {
// TBD: What to do about errors here?
std::fclose(m_file);
// We manually create the file on Windows only, on Linux
// it will be autodeleted
#if defined(CATCH_PLATFORM_WINDOWS)
std::remove(m_buffer);
#endif
}
FILE* TempFile::getFile() {
return m_file;
}
std::string TempFile::getContents() {
std::stringstream sstr;
char buffer[100] = {};
std::rewind(m_file);
while (std::fgets(buffer, sizeof(buffer), m_file)) {
sstr << buffer;
}
return sstr.str();
}
OutputRedirect::OutputRedirect(std::string& stdout_dest, std::string& stderr_dest) :
m_originalStdout(dup(1)),
m_originalStderr(dup(2)),
m_stdoutDest(stdout_dest),
m_stderrDest(stderr_dest) {
dup2(fileno(m_stdoutFile.getFile()), 1);
dup2(fileno(m_stderrFile.getFile()), 2);
}
OutputRedirect::~OutputRedirect() {
Catch::cout() << std::flush;
fflush(stdout);
// Since we support overriding these streams, we flush cerr
// even though std::cerr is unbuffered
Catch::cerr() << std::flush;
Catch::clog() << std::flush;
fflush(stderr);
dup2(m_originalStdout, 1);
dup2(m_originalStderr, 2);
m_stdoutDest += m_stdoutFile.getContents();
m_stderrDest += m_stderrFile.getContents();
}
} // namespace Catch
#if defined(CATCH_PLATFORM_WINDOWS)
#undef dup
#undef dup2
#undef fileno
#endif

View File

@ -0,0 +1,98 @@
/*
* Created by Martin on 28/04/2018.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
#ifndef TWOBLUECUBES_CATCH_OUTPUT_REDIRECT_H
#define TWOBLUECUBES_CATCH_OUTPUT_REDIRECT_H
#include "catch_platform.h"
#include "catch_stream.h"
#include <cstdio>
#include <iosfwd>
#include <string>
namespace Catch {
class RedirectedStream {
std::ostream& m_originalStream;
std::ostream& m_redirectionStream;
std::streambuf* m_prevBuf;
public:
RedirectedStream( std::ostream& originalStream, std::ostream& redirectionStream );
~RedirectedStream();
};
class RedirectedStdOut {
ReusableStringStream m_rss;
RedirectedStream m_cout;
public:
RedirectedStdOut();
auto str() const -> std::string;
};
// StdErr has two constituent streams in C++, std::cerr and std::clog
// This means that we need to redirect 2 streams into 1 to keep proper
// order of writes
class RedirectedStdErr {
ReusableStringStream m_rss;
RedirectedStream m_cerr;
RedirectedStream m_clog;
public:
RedirectedStdErr();
auto str() const -> std::string;
};
// Windows's implementation of std::tmpfile is terrible (it tries
// to create a file inside system folder, thus requiring elevated
// privileges for the binary), so we have to use tmpnam(_s) and
// create the file ourselves there.
class TempFile {
public:
TempFile(TempFile const&) = delete;
TempFile& operator=(TempFile const&) = delete;
TempFile(TempFile&&) = delete;
TempFile& operator=(TempFile&&) = delete;
TempFile();
~TempFile();
std::FILE* getFile();
std::string getContents();
private:
std::FILE* m_file = nullptr;
#if defined(CATCH_PLATFORM_WINDOWS)
char m_buffer[L_tmpnam] = { 0 };
#endif
};
class OutputRedirect {
public:
OutputRedirect(OutputRedirect const&) = delete;
OutputRedirect& operator=(OutputRedirect const&) = delete;
OutputRedirect(OutputRedirect&&) = delete;
OutputRedirect& operator=(OutputRedirect&&) = delete;
OutputRedirect(std::string& stdout_dest, std::string& stderr_dest);
~OutputRedirect();
private:
int m_originalStdout = -1;
int m_originalStderr = -1;
TempFile m_stdoutFile;
TempFile m_stderrFile;
std::string& m_stdoutDest;
std::string& m_stderrDest;
};
} // end namespace Catch
#endif // TWOBLUECUBES_CATCH_OUTPUT_REDIRECT_H

View File

@ -3,6 +3,7 @@
#include "catch_enforce.h" #include "catch_enforce.h"
#include "catch_random_number_generator.h" #include "catch_random_number_generator.h"
#include "catch_stream.h" #include "catch_stream.h"
#include "catch_output_redirect.h"
#include <cassert> #include <cassert>
#include <algorithm> #include <algorithm>
@ -10,48 +11,6 @@
namespace Catch { namespace Catch {
class RedirectedStream {
std::ostream& m_originalStream;
std::ostream& m_redirectionStream;
std::streambuf* m_prevBuf;
public:
RedirectedStream( std::ostream& originalStream, std::ostream& redirectionStream )
: m_originalStream( originalStream ),
m_redirectionStream( redirectionStream ),
m_prevBuf( m_originalStream.rdbuf() )
{
m_originalStream.rdbuf( m_redirectionStream.rdbuf() );
}
~RedirectedStream() {
m_originalStream.rdbuf( m_prevBuf );
}
};
class RedirectedStdOut {
ReusableStringStream m_rss;
RedirectedStream m_cout;
public:
RedirectedStdOut() : m_cout( Catch::cout(), m_rss.get() ) {}
auto str() const -> std::string { return m_rss.str(); }
};
// StdErr has two constituent streams in C++, std::cerr and std::clog
// This means that we need to redirect 2 streams into 1 to keep proper
// order of writes
class RedirectedStdErr {
ReusableStringStream m_rss;
RedirectedStream m_cerr;
RedirectedStream m_clog;
public:
RedirectedStdErr()
: m_cerr( Catch::cerr(), m_rss.get() ),
m_clog( Catch::clog(), m_rss.get() )
{}
auto str() const -> std::string { return m_rss.str(); }
};
RunContext::RunContext(IConfigPtr const& _config, IStreamingReporterPtr&& reporter) RunContext::RunContext(IConfigPtr const& _config, IStreamingReporterPtr&& reporter)
: m_runInfo(_config->name()), : m_runInfo(_config->name()),
m_context(getCurrentMutableContext()), m_context(getCurrentMutableContext()),
@ -299,13 +258,19 @@ namespace Catch {
Timer timer; Timer timer;
try { try {
if (m_reporter->getPreferences().shouldRedirectStdOut) { if (m_reporter->getPreferences().shouldRedirectStdOut) {
#if !defined(CATCH_CONFIG_EXPERIMENTAL_REDIRECT)
RedirectedStdOut redirectedStdOut; RedirectedStdOut redirectedStdOut;
RedirectedStdErr redirectedStdErr; RedirectedStdErr redirectedStdErr;
timer.start(); timer.start();
invokeActiveTestCase(); invokeActiveTestCase();
redirectedCout += redirectedStdOut.str(); redirectedCout += redirectedStdOut.str();
redirectedCerr += redirectedStdErr.str(); redirectedCerr += redirectedStdErr.str();
#else
OutputRedirect r(redirectedCout, redirectedCerr);
timer.start();
invokeActiveTestCase();
#endif
} else { } else {
timer.start(); timer.start();
invokeActiveTestCase(); invokeActiveTestCase();