Asynchronously redirect stdout/stdin from embedded python to c++?

Question

I am essentially trying to write a console interface with input and output for an embedded python script. Following the instructions here, I was able to capture stdout:

Py_Initialize();
PyRun_SimpleString("\
class StdoutCatcher:\n\
    def __init__(self):\n\
        self.data = ''\n\
    def write(self, stuff):\n\
        self.data = self.data + stuff\n\
import sys\n\
sys.stdout = StdoutCatcher()");

PyRun_SimpleString("some script");

PyObject *sysmodule;
PyObject *pystdout;
PyObject *pystdoutdata;    
char *string;
sysmodule = PyImport_ImportModule("sys");
pystdout = PyObject_GetAttrString(sysmodule, "stdout");
pystdoutdata = PyObject_GetAttrString(pystdout, "data");    
stdoutstring = PyString_AsString(pystdoutdata);

Py_Finalize();

The problem with this is that I only recieve the stdout after the script has finished running, whereas ideally for a console the stdoutstring would update as the python script updates it. Is there a way to do this?

Also, how would I go about capturing stdin?

If it helps, I am working with a compiler that accepts Objective-C. I also have the boost libraries available.

---

I've figured out the stdout part of the question. For posterity, this works:

static PyObject*
redirection_stdoutredirect(PyObject *self, PyObject *args)
{
    const char *string;
    if(!PyArg_ParseTuple(args, "s", &string))
        return NULL;
    //pass string onto somewhere
    Py_INCREF(Py_None);
    return Py_None;
}

static PyMethodDef RedirectionMethods[] = {
    {"stdoutredirect", redirection_stdoutredirect, METH_VARARGS,
        "stdout redirection helper"},
    {NULL, NULL, 0, NULL}
};

//in main...
    Py_Initialize();
    Py_InitModule("redirection", RedirectionMethods);
    PyRun_SimpleString("\
import redirection\n\
import sys\n\
class StdoutCatcher:\n\
    def write(self, stuff):\n\
        redirection.stdoutredirect(stuff)\n\
sys.stdout = StdoutCatcher()");

    PyRun_SimpleString("some script");

    Py_Finalize();

Still having trouble with stdin...

Answer 1

To process all available input inside Python I'd recommend the fileinput module.

If you want to handle input as line-by-line commands, (such as in an interactive interpreter), you might find the python function raw_input useful.

To redirect stdin using a similar helper class such as the ones you've used above, the function to override is readline, not read. See this link for more info on that (and also raw_input).

Hope this helps, Supertwang

Answer 2

Easiest way I found so far to do this is as follows:

PyObject *sys = PyImport_ImportModule("sys");
PyObject* io_stdout = PyFile_FromFile(stdout, "stdout", "a", nullptr);
PyObject_SetAttrString(sys, "stdout", io_stdout);
PyObject* io_stderr = PyFile_FromFile(stderr, "stderr", "a", nullptr);
PyObject_SetAttrString(sys, "stderr", io_stderr);
PyObject* io_stdin = PyFile_FromFile(stdin, "stdin", "r", nullptr);
PyObject_SetAttrString(sys, "stdin", io_stdin);

you can test it with:

# for test
PyRun_SimpleString("print sys.stdin.readline()");

Answer 3

If you stick with the approach you outlined, inheriting your class from io.IOBase is probably a good idea.