Continuously exchange data using python-shell

Question

I need to run some python scripts from node. Since my python scripts use complex structures I figured would be better if I only loaded those structures once and then run some the specific scripts (tasks) using the structures.

On node I want to run a script forever (or until I say it can terminate) and keep sending on-demand messages to this script. This script would create a process, using python multiprocessing, to run the specific task and start listening again.

A use case would be:

• Node starts, and wakes python script
• Some action on the node client causes it to send a command to the python script
• The python script computes what it needs and returns the data

I think python-shell can help me with this

On node I have something like this:

var app = express();
app.listen(8000, function () {
    console.log('Example app listening on port 8000!');
});

var pyshell = new PythonShell('start.py', options);
pyshell.on('message', function (message) {
    handleAnswer(message);
});

pyshell.end(function (err, code, signal) {
    handleEnd(err, code, signal);
});

app.get('/hello', function (req, res) {
    pyshell.send('hello');
});

My start.py script is:

import sys

def listen():
    while True:
        command = sys.stdin.readline()
        command = command.split('\n')[0]
        if command:
            print("Received CMD " + command)

if __name__ == '__main__':
    data = json.loads(sys.argv[1])
    listen()

I tried the solution on this similar question but gets and error when wrapping the hello message in an endpoint.

I properly require python-shell module and I'm able to execute a python script and return data (using print). My problem is to start the script and send the message after a (random) time period.

When I open localhost:8000/hello I get Error: write after end

Full error:

    Error: write after end
    at writeAfterEnd (_stream_writable.js:236:12)
    at Socket.Writable.write (_stream_writable.js:287:5)
    at Socket.write (net.js:717:40)
    at PythonShell.send (C:\Users\leonardo.schettini\Documents\recrutai\client\node_modules\python-shell\index.js:206:16)
    at C:\Users\leonardo.schettini\Documents\recrutai\client\app.js:27:12
    at Layer.handle [as handle_request] (C:\Users\leonardo.schettini\Documents\recrutai\client\node_modules\express\lib\router\layer.js:95:5)
    at next (C:\Users\leonardo.schettini\Documents\recrutai\client\node_modules\express\lib\router\route.js:137:13)
    at Route.dispatch (C:\Users\leonardo.schettini\Documents\recrutai\client\node_modules\express\lib\router\route.js:112:3)
    at Layer.handle [as handle_request] (C:\Users\leonardo.schettini\Documents\recrutai\client\node_modules\express\lib\router\layer.js:95:5)
    at C:\Users\leonardo.schettini\Documents\recrutai\client\node_modules\express\lib\router\index.js:281:22

Answer 1

Well, I reviewed the wrapper I did for the python-shell and found a pyshell.end(callback) call right after the script creation.

I was pretty sure the end would only be executed when the python script finish, but it happens that it closes the input stream without terminating the script.

Apologies for my lack of attention.