C reading (from stdin) stops at 0x1a character

Question

currently I\'m implementing the Burrows-Wheeler transform (and inverse transform) for raw data (like jpg etc.). When testing on normal data like textfiles no problems occur.

Answer 1

As you've noticed, you're reading from stdin in ASCII mode and it is hitting the SUB character (substitute, aka CTRL+Z, aka DOS End-of-File).

You have to change the mode to binary with setmode while on Windows:

#if defined(WIN32)
#include <io.h>
#include <fcntl.h>
#endif /* defined(WIN32) */

/* ... */

#if defined(WIN32)
_setmode(_fileno(stdin), _O_BINARY);
#endif /* defined(WIN32) */

On platforms other than Windows you don't run into this distinction in modes.

Answer 2

You must open the file as a binary file.

Use something similar to

fopen("file", "rb");

Answer 3

You can use _setmode to convert stdin to binary mode.

There is also freopen -- see this SO question

Answer 4

Use read() to read in the data.
Since you are interested in getting data from the stdin, use

fd = fcntl(STDIN_FILENO, F_DUPFD, 0);

to obtain the fd of stdin.

More info here.

The issue has something to do with the fact that windows treats 0x1a a.k.a. CTRL+Z as the EOF. As Earlz pointed out, opening it in binary mode fixes this on windows and works on linux too.

Answer 5

You cannot do this without an OS dependency. The C language specification says (7.19.3)

At program startup, three text streams are predefined...

stdin is a text stream. Depending on your OS, there may be ways to change the mode of an existing stream or access the low-level stream data, but you claim that you do not want any OS-specific code.