forcing a program to flush its standard output when redirected

Question

i have a closed source program that prints output to standard output. i need to parse the output. so i redirect the output to a fifo (from which i can read in the parent process that forks and execs the binary) using dup2 and then exec the program. the problem is that the fprintf calls in the file become buffered because it is now writing to a file.

i tried calling setvbuf with _IONBF on stdout before calling exec. but the problem still exists.

why does setvbuf not help in my case?

how can i force the output to get flushed?

Answer 1

setvbuf() makes no difference because it changes the state of part of the C runtime library, not part of the OS. When the new process begins running, its C runtime library will be reinitialised (that's if it uses a CRT at all!)

The only way I have heard of for getting around this is to somehow fake a terminal to the process. That's because most CRT libraries will by default perform only line buffering if they believe they are attached to an interactive terminal (in the Unix world: if isatty() returns true on the file descriptor), whereas otherwise they will buffer in larger blocks (typically 8Kb or so).

This utility looks like a pretty good place to start. (Borrowed from a comment on Trick an application into thinking its stdin is interactive, not a pipe, which has other useful info.)

Answer 2

I guess you have something like this in your program (you can reproduce this for your tests, I'm calling it isatty here)

#include <stdio.h>
#include <unistd.h>

const char* m1 = "%d: %s a TTY\n";

void isTty(FILE* f) {
    int fno = fileno(f);
    printf(m1, fno, (isatty(fno)) ? "is" : "is NOT");
}

int main(int argc, char* argv[]) {
    isTty(stdin);
    isTty(stdout);
}

for example if you run it

$ ./isatty
0: is a TTY
1: is a TTY

$ ./isatty > isatty.out
$ cat isatty.out 
0: is a TTY
1: is NOT a TTY

$ ./isatty > isatty.out < /dev/null
$ cat isatty.out 
0: is NOT a TTY
1: is NOT a TTY

Now if you create an expect script isatty.expect (install expect for your distro if not installed)

#! /usr/bin/expect -f

spawn "./isatty"
expect

and run it

$ ./isatty.expect 
spawn ./isatty
0: is a TTY
1: is a TTY

or

$ ./isatty.expect > isatty.out 
$ cat isatty.out 
spawn ./isatty
0: is a TTY
1: is a TTY

Answer 3

The unbuffer tool can assist with this issue:

It's part of the expect-dev and can be installed in ubuntu using

sudo apt-get install expect-dev

to use it type:

unbuffer ls > log.txt