ProcessBuilder: Forwarding stdout and stderr of started processes without blocking the main thread

两盒软妹~` 提交于 2019-11-26 01:35:48

问题


I\'m building a process in Java using ProcessBuilder as follows:

ProcessBuilder pb = new ProcessBuilder()
        .command(\"somecommand\", \"arg1\", \"arg2\")
        .redirectErrorStream(true);
Process p = pb.start();

InputStream stdOut = p.getInputStream();

Now my problem is the following: I would like to capture whatever is going through stdout and/or stderr of that process and redirect it to System.out asynchronously. I want the process and its output redirection to run in the background. So far, the only way I\'ve found to do this is to manually spawn a new thread that will continuously read from stdOut and then call the appropriate write() method of System.out.

new Thread(new Runnable(){
    public void run(){
        byte[] buffer = new byte[8192];
        int len = -1;
        while((len = stdOut.read(buffer)) > 0){
            System.out.write(buffer, 0, len);
        }
    }
}).start();

While that approach kind of works, it feels a bit dirty. And on top of that, it gives me one more thread to manage and terminate correctly. Is there any better way to do this?


回答1:


To only way in Java 6 or earlier is with a so called StreamGobbler (which you are started to create):

StreamGobbler errorGobbler = new StreamGobbler(p.getErrorStream(), "ERROR");

// any output?
StreamGobbler outputGobbler = new StreamGobbler(p.getInputStream(), "OUTPUT");

// start gobblers
outputGobbler.start();
errorGobbler.start();

...

private class StreamGobbler extends Thread {
    InputStream is;
    String type;

    private StreamGobbler(InputStream is, String type) {
        this.is = is;
        this.type = type;
    }

    @Override
    public void run() {
        try {
            InputStreamReader isr = new InputStreamReader(is);
            BufferedReader br = new BufferedReader(isr);
            String line = null;
            while ((line = br.readLine()) != null)
                System.out.println(type + "> " + line);
        }
        catch (IOException ioe) {
            ioe.printStackTrace();
        }
    }
}

For Java 7, see Evgeniy Dorofeev's answer.




回答2:


Use ProcessBuilder.inheritIO, it sets the source and destination for subprocess standard I/O to be the same as those of the current Java process.

Process p = new ProcessBuilder().inheritIO().command("command1").start();

If Java 7 is not an option

public static void main(String[] args) throws Exception {
    Process p = Runtime.getRuntime().exec("cmd /c dir");
    inheritIO(p.getInputStream(), System.out);
    inheritIO(p.getErrorStream(), System.err);

}

private static void inheritIO(final InputStream src, final PrintStream dest) {
    new Thread(new Runnable() {
        public void run() {
            Scanner sc = new Scanner(src);
            while (sc.hasNextLine()) {
                dest.println(sc.nextLine());
            }
        }
    }).start();
}

Threads will die automatically when subprocess finishes, because src will EOF.




回答3:


A flexible solution with Java 8 lambda that lets you provide a Consumer that will process the output (eg. log it) line by line. run() is a one-liner with no checked exceptions thrown. Alternatively to implementing Runnable, it can extend Thread instead as other answers suggest.

class StreamGobbler implements Runnable {
    private InputStream inputStream;
    private Consumer<String> consumeInputLine;

    public StreamGobbler(InputStream inputStream, Consumer<String> consumeInputLine) {
        this.inputStream = inputStream;
        this.consumeInputLine = consumeInputLine;
    }

    public void run() {
        new BufferedReader(new InputStreamReader(inputStream)).lines().forEach(consumeInputLine);
    }
}

You can then use it for example like this:

public void runProcessWithGobblers() throws IOException, InterruptedException {
    Process p = new ProcessBuilder("...").start();
    Logger logger = LoggerFactory.getLogger(getClass());

    StreamGobbler outputGobbler = new StreamGobbler(p.getInputStream(), System.out::println);
    StreamGobbler errorGobbler = new StreamGobbler(p.getErrorStream(), logger::error);

    new Thread(outputGobbler).start();
    new Thread(errorGobbler).start();
    p.waitFor();
}

Here the output stream is redirected to System.out and the error stream is logged on the error level by the logger.




回答4:


It's as simple as following:

    File logFile = new File(...);
    ProcessBuilder pb = new ProcessBuilder()
        .command("somecommand", "arg1", "arg2")
    processBuilder.redirectErrorStream(true);
    processBuilder.redirectOutput(logFile);

by .redirectErrorStream(true) you tell process to merge error and output stream and then by .redirectOutput(file) you redirect merged output to a file.

Update:

I did manage to do this as follows:

public static void main(String[] args) {
    // Async part
    Runnable r = () -> {
        ProcessBuilder pb = new ProcessBuilder().command("...");
        // Merge System.err and System.out
        pb.redirectErrorStream(true);
        // Inherit System.out as redirect output stream
        pb.redirectOutput(ProcessBuilder.Redirect.INHERIT);
        try {
            pb.start();
        } catch (IOException e) {
            e.printStackTrace();
        }
    };
    new Thread(r, "asyncOut").start();
    // here goes your main part
}

Now you're able to see both outputs from main and asyncOut threads in System.out




回答5:


I too can use only Java 6. I used @EvgeniyDorofeev's thread scanner implementation. In my code, after a process finishes, I have to immediately execute two other processes that each compare the redirected output (a diff-based unit test to ensure stdout and stderr are the same as the blessed ones).

The scanner threads don't finish soon enough, even if I waitFor() the process to complete. To make the code work correctly, I have to make sure the threads are joined after the process finishes.

public static int runRedirect (String[] args, String stdout_redirect_to, String stderr_redirect_to) throws IOException, InterruptedException {
    ProcessBuilder b = new ProcessBuilder().command(args);
    Process p = b.start();
    Thread ot = null;
    PrintStream out = null;
    if (stdout_redirect_to != null) {
        out = new PrintStream(new BufferedOutputStream(new FileOutputStream(stdout_redirect_to)));
        ot = inheritIO(p.getInputStream(), out);
        ot.start();
    }
    Thread et = null;
    PrintStream err = null;
    if (stderr_redirect_to != null) {
        err = new PrintStream(new BufferedOutputStream(new FileOutputStream(stderr_redirect_to)));
        et = inheritIO(p.getErrorStream(), err);
        et.start();
    }
    p.waitFor();    // ensure the process finishes before proceeding
    if (ot != null)
        ot.join();  // ensure the thread finishes before proceeding
    if (et != null)
        et.join();  // ensure the thread finishes before proceeding
    int rc = p.exitValue();
    return rc;
}

private static Thread inheritIO (final InputStream src, final PrintStream dest) {
    return new Thread(new Runnable() {
        public void run() {
            Scanner sc = new Scanner(src);
            while (sc.hasNextLine())
                dest.println(sc.nextLine());
            dest.flush();
        }
    });
}



回答6:


Simple java8 solution with capturing both outputs and reactive processing using CompletableFuture:

static CompletableFuture<String> readOutStream(InputStream is) {
    return CompletableFuture.supplyAsync(() -> {
        try (
                InputStreamReader isr = new InputStreamReader(is);
                BufferedReader br = new BufferedReader(isr);
        ){
            StringBuilder res = new StringBuilder();
            String inputLine;
            while ((inputLine = br.readLine()) != null) {
                res.append(inputLine).append(System.lineSeparator());
            }
            return res.toString();
        } catch (Throwable e) {
            throw new RuntimeException("problem with executing program", e);
        }
    });
}

And the usage:

Process p = Runtime.getRuntime().exec(cmd);
CompletableFuture<String> soutFut = readOutStream(p.getInputStream());
CompletableFuture<String> serrFut = readOutStream(p.getErrorStream());
CompletableFuture<String> resultFut = soutFut.thenCombine(serrFut, (stdout, stderr) -> {
         // print to current stderr the stderr of process and return the stdout
        System.err.println(stderr);
        return stdout;
        });
// get stdout once ready, blocking
String result = resultFut.get();



回答7:


Thread thread = new Thread(() -> {
      new BufferedReader(
          new InputStreamReader(inputStream, 
                                StandardCharsets.UTF_8))
              .lines().forEach(...);
    });
    thread.start();

Your custom code goes instead of the ...




回答8:


By default, the created subprocess does not have its own terminal or console. All its standard I/O (i.e. stdin, stdout, stderr) operations will be redirected to the parent process, where they can be accessed via the streams obtained using the methods getOutputStream(), getInputStream(), and getErrorStream(). The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, or even deadlock.

https://www.securecoding.cert.org/confluence/display/java/FIO07-J.+Do+not+let+external+processes+block+on+IO+buffers



来源:https://stackoverflow.com/questions/14165517/processbuilder-forwarding-stdout-and-stderr-of-started-processes-without-blocki

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!