Skip to main content

Raymii.org Raymii.org Logo

Quis custodiet ipsos custodes?
Home | About | All pages | Cluster Status | RSS Feed | Gopher

Execute a command and get both output and exit status in C++ (Windows & Linux)

Published: 07-06-2021 | Author: Remy van Elst | Text only version of this article


Table of Contents


Recently I had to parse some command line output inside a C++ program. Executing a command and getting just the exit status is easy using std::system, but also getting output is a bit harder and OS specific. By using popen, a POSIX C function we can get both the exit status as well as the output of a given command. On Windows I'm using _popen, so the code should be cross platform, except for the exit status on Windows is alway 0, that concept does not exist there. This article starts off with a stack overflow example to get just the output of a command and builds on that to a safer version (null-byte handling) that returns both the exit status as well as the command output. It also involves a lot of detail on fread vs fgets and how to handle binary data.

Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.

You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $100 credit for 60 days.

The complete code example with usage examples can be found on github here or at the bottom of this page. A working example is compiled on github actions for different platforms (windows & linux).

Normally I would advise against parsing command line output. It is error-prone, you're dependent on the language selected by the user, different versions might have different flags (OS X vs Linux) and much more. If you have the option to use a native library, you should use that. An example could be parsing curl output to get some data from an API. There are probably a metric ton of http libraries available for your favorite programming language to use instead of parsing the curl or wget or fetch output.

In my case I have to use an old program to parse a closed-source file to get some binary output. This is a temporary situation, a native parsing library is also under development. The binary is under my control as well as the system settings, language, other tools and such, so for this specific use case the solution to parse command line output was acceptable for the time being.

Do note that in this post I'll interchange the term nullbyte, null character, null termination and null-terminated. They all mean the same, the null-byte character used to end a C string (\0, or ^@, U+0000 or 0x00, you get the gist).

If you need more features, more cross-platform or async execution, boost.Process is a great alternative. I however can't use boost on the environment this code is going to run due to compiler and size constraints.

The stackoverflow example using fgets

On stackoverflow the example given is a good base to build on, however, to get the exit code and the output, it must be modified. Because we want to also grab the exit code, we cannot use the example which uses the std::unique_ptr. Which in itself is a great example of using a unique_ptr with a custom deleter (cmd is a const char* with the command to execute:

std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);

The code is copied below:

std::string exec(const char* cmd) {
    char buffer[128];
    std::string result = "";
    FILE* pipe = popen(cmd, "r");
    if (!pipe) throw std::runtime_error("popen() failed!");
    try {
        while (fgets(buffer, sizeof buffer, pipe) != NULL) 
            result += buffer;
        }
    } catch (...) {
        pclose(pipe);
        throw;
    }
    pclose(pipe);
    return result;
}

This example does what it states, but with a few gotchas. It uses a FILE* (pointer), char buffer allocation and manually closing the FILE* when something goes wrong (catch). The unique_ptr example is more modern, due to not having to handle the exception and using a std::array<char, 128> instead of a C-style char* buffer. Exception throwing is a whole other issue, but let's not get into that today. What we are going to get into today is C style code to read from FILE* and how binary data is handled.

The stackoverflow example is probably just fine if you only need textual output into a std::string. My usecase however was a bit more complex, as you'll find out while reading the rest of this artice.

fread vs fgets

Code style aside, my biggest issue was that using fgets in this way combined with adding a const char* to a std::string stops when it encounters a nullyte (\0). For regular string output that often is not an issue, most commands just output a few strings and call it a day. My output returns a binary blob, which might include nullbytes. fread reads an amount of bytes and returns how much it has read successfully, which we can use when adding the output to our std::string including the nullbytes.

The example above does result += buffer, adding a const char* to a std::string, in this case according to cppreference on operator+= on std::string: Appends the null-terminated character string pointed to by s.

The problem therein lies that the characters after the nullbyte should be added as well in my case. fgets does not give back the amount of data it read. Using fgets and a buffer of 128, if I have a nullbyte at 10 and a newline at 40, then the first 10 bytes plus what is after 40 bytes will be returned. Effectively we're loosing everything in between the nullbyte and the newline, or until the end of the buffer (128) if there is no newline in between.

fread does return the amount of bytes it has read. Combining that with a constructor of std::string that takes a const char* and a size_t we can force the entire contents inside the string. This is safe, since a std::string knows its size, it does not rely on a null-termination character. However, other code that uses const char* will not be able to work with these nullbytes, keep that in mind.

This stackoverflow post was very helpful for me to understand fread, as well as help from a co-worker that dreams in C, he explained a lot of the inner workings.

And if, after all of this, you're wondering why I'm shoehorning binary data inside a std::string, great question. I'll probably go into that another time since that would require a longer post than this entire article.

Command execution including output and exit code

My code checks the exit status of the executed binary (for error handling) and uses the data returned for further processing. To keep this all in one handy dandy place, lets start with defining a struct to hold that data. It will hold the result of a command, so the name CommandResult sounds descriptive enough.

Below you'll find the struct code, including an equality operator as well as a stream output operator.

struct CommandResult {
    std::string output;
    int exitstatus;

    friend std::ostream &operator<<(std::ostream &os, const CommandResult &result) {
        os << "command exitstatus: " << result.exitstatus << " output: " << result.output;
        return os;
    }
    bool operator==(const CommandResult &rhs) const {
        return output == rhs.output &&
               exitstatus == rhs.exitstatus;
    }
    bool operator!=(const CommandResult &rhs) const {
        return !(rhs == *this);
    }
};

The meat and potatoes of the struct are of course the output and exitstatus. I'm using an int for the exit status because of reasons.

The next part is the Command class itself, here is that code:

class Command {

public:
    /**
     * Execute system command and get STDOUT result.
     * Like system() but gives back exit status and stdout.
     * @param command system command to execute
     * @return CommandResult containing STDOUT (not stderr) output & exitstatus
     * of command. Empty if command failed (or has no output). If you want stderr,
     * use shell redirection (2&>1).
     */
    static CommandResult exec(const std::string &command) {
        int exitcode = 255;
        std::array<char, 1048576> buffer {};
        std::string result;
#ifdef _WIN32
#define popen _popen
#define pclose _pclose
#define WEXITSTATUS
#endif
        FILE *pipe = popen(command.c_str(), "r");
        if (pipe == nullptr) {
            throw std::runtime_error("popen() failed!");
        }
        try {
            std::size_t bytesread;
            while ((bytesread = fread(buffer.data(), sizeof(buffer.at(0)), sizeof(buffer), pipe)) != 0) {
                result += std::string(buffer.data(), bytesread);
            }
        } catch (...) {
            pclose(pipe);
            throw;
        }
        exitcode = WEXITSTATUS(pclose(pipe));
        return CommandResult{result, exitcode};
    }
}

The fread command will run until there are no more bytes returned from the command output. I know the kind of output I'm working with so my buffer is 1 MiB, which is probably too large for your data. In my case I benchmarked it and between 10KiB and 1 MiB was the fastest on the target architecture. 128 or 8192 is just fine as well probably, but you should benchmark that for yourself. A rather simple test is to output some enormous file with cat and take the execution time plus cpu and memory usage. Don't print the result, just look at those three things and choose what ratio is acceptable for you.

Why not also initialize the std::string with 1 MiB of characters? std::strings cannot be allocated for a given size at construction, other than by filling them or afterwards calling .reserve(), my benchmarks did not show any meaningful speed or performance boost by doing either.

Using the above code is easy. Since it's a static function, you don't need a class instance to use it. Here is an example:

std::cout << Command::exec("echo 'Hello you absolute legends!'") << std::endl;

Which results in:

command exitstatus: 0 output: Hello you absolute legends!

Since we're going through a shell, redirection works as well. Redirecting stdout to stderr results in no output, just an exit status:

std::cout << Command::exec("echo 'Hello you absolute legends!' 1>&2") << std::endl;

The output is on stderr in my shell however, which is expected:

stderr

If you do need to capture stderr then you redirect output the other way around, like so:

std::cout << Command::exec("/bin/bash --invalid  2>&1") << std::endl;

Pipes work as well as in your shell, but do note that this is all using sh and you have no control over environment variables or the default shell. Read more on the POSIX page on popen to find out why that is.

A note on Windows

Here is an example for Windows, where we must use _popen and _pclose:

std::cout << "Windows example:" << std::endl;
std::cout << Command::exec("dir * /on /p") << std::endl;

The exit code will always be zero since that concept does not translate to windows. There is %ErrorLevel%, but that is only an environment variable for console applications, not the actual exit status.

The microsoft page also notes that _popen will not work with GUI applications, just console programs. If you need that, use Boost.process or system.

Nullbytes in output example:

In the example code on github you'll also see a execFgets function, I've left that in there to show the difference in nullbyte handling. For reference I'll show an example here as well. The relevant part of command using fgets:

while (std::fgets(buffer.data(), buffer.size(), pipe) != nullptr)
    result += buffer.data();

The part using fread:

std::size_t bytesread;
while ((bytesread = fread(buffer.data(), sizeof(buffer.at(0)), sizeof(buffer), pipe)) != 0)         
    result += std::string(buffer.data(), bytesread);

The test command, including a clang-tidy warning exclusion (// NOLINT):

int main() {
    using namespace raymii;

    std::string expectedOutput("test\000abc\n", 9); //NOLINT
    commandResult nullbyteCommand = command::exec("/usr/bin/printf 'test\\000abc\\n'"); // NOLINT(bugprone-string-literal-with-embedded-nul)
    commandResult fgetsNullbyteCommand = command::execFgets("/usr/bin/printf 'test\\000abc\\n'"); // NOLINT(bugprone-string-literal-with-embedded-nul)

    std::cout << "Expected output: " << expectedOutput << std::endl;
    std::cout << "Output using fread: " << nullbyteCommand << std::endl;
    std::cout << "Output using fgets: " << fgetsNullbyteCommand << std::endl;
    return 0;
}

Output:

Expected output: test\0abc
A command with nullbytes using fread: exitstatus: 0 output: test\0abc
A command with nullbytes using fgets: exitstatus: 0 output: test

The nullbyte character is substituted with \0 in the above output. Here is a screenshot showing how it looks in my terminal:

screenshot

Once again do note that this is safe to use with std::strings, methods that take a string_view or a const char* probably will not react very well to nullbytes. For my use case this is safe, your milage may vary.

Try playing with the buffer size and then looking at the output. If you set it to 4, the output with fgets is testbc. Funny right? I like such things.

Complete code

Below you can find the header file command.h. It is also on my github. If you want usage examples you can find them in the github project main.cpp file.

#command.h
#ifndef COMMAND_H
#define COMMAND_H
// Copyright (C) 2021 Remy van Elst
//
//     This program is free software: you can redistribute it and/or modify
//     it under the terms of the GNU General Public License as published by
//     the Free Software Foundation, either version 3 of the License, or
//     (at your option) any later version.
//
//     This program is distributed in the hope that it will be useful,
//     but WITHOUT ANY WARRANTY; without even the implied warranty of
//     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
//     GNU General Public License for more details.
//
//     You should have received a copy of the GNU General Public License
//     along with this program.  If not, see <http://www.gnu.org/licenses/>.
#include <array>
#include <ostream>
#include <string>
#ifdef _WIN32
#include <stdio.h>
#endif

namespace raymii {

    struct CommandResult {
        std::string output;
        int exitstatus;
        friend std::ostream &operator<<(std::ostream &os, const CommandResult &result) {
            os << "command exitstatus: " << result.exitstatus << " output: " << result.output;
            return os;
        }
        bool operator==(const CommandResult &rhs) const {
            return output == rhs.output &&
                   exitstatus == rhs.exitstatus;
        }
        bool operator!=(const CommandResult &rhs) const {
            return !(rhs == *this);
        }
    };

    class Command {
    public:
        /**
             * Execute system command and get STDOUT result.
             * Regular system() only gives back exit status, this gives back output as well.
             * @param command system command to execute
             * @return commandResult containing STDOUT (not stderr) output & exitstatus
             * of command. Empty if command failed (or has no output). If you want stderr,
             * use shell redirection (2&>1).
             */
        static CommandResult exec(const std::string &command) {
            int exitcode = 0;
            std::array<char, 1048576> buffer {};
            std::string result;
#ifdef _WIN32
#define popen _popen
#define pclose _pclose
#define WEXITSTATUS
#endif
            FILE *pipe = popen(command.c_str(), "r");
            if (pipe == nullptr) {
                throw std::runtime_error("popen() failed!");
            }
            try {
                std::size_t bytesread;
                while ((bytesread = std::fread(buffer.data(), sizeof(buffer.at(0)), sizeof(buffer), pipe)) != 0) {
                    result += std::string(buffer.data(), bytesread);
                }
            } catch (...) {
                pclose(pipe);
                throw;
            }
            exitcode = WEXITSTATUS(pclose(pipe));
            return CommandResult{result, exitcode};
        }

    };

}// namespace raymii
#endif//COMMAND_H
Tags: articles , c++ , commandline , cpp , development , linux , printf , system , windows