Wildcards in child_process spawn()?

Question

I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.

I tried this

var spawn = require('child_process').spawn; 
spawn("doSomething", ["./myfiles/*.csv"]);

But then the wildcard *.csv will not interpreted.

Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?

Thanks

Torben

Answer 1

The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:

var exec = require("child_process").exec;

var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
    // Handle result
});

If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?

Update

In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:

https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589

And here's some pseudo code:

var child;
var cmd = "doSomething ./myfiles/*.csv";

if ('win32' === process.platform) {
    child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
    child = spawn('/bin/sh', ['-c', cmd]);
}

Answer 2

What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).

Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.

Answer 3

Here's the simplest solution:

spawn("doSomething", ["./myfiles/*.csv"], { shell: true });

As @JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.