Determine the size of a pipe without calling read()

Question

I need a function called SizeOfPipe() which should return the size of a pipe - I only want to know how much data is in the pipe and not actually read data off t

Answer 1

Unfortunately the system cannot always know the size of a pipe - for example if you are piping a long-running process into another command, the source process may not have finished running yet. In this case there is no possible way (even in theory) to know how much more data is going to come out of it.

If you want to know the amount of data currently available to read out of the pipe that might be possible, but it will depend on OS buffering and other factors which are hard to control. The most common approach here is just to keep reading until there's nothing left to come (if you don't get an EOF then the source process hasn't finished yet). However I don't think this is what you are looking for.

So I'm afraid there is no general solution.

Answer 2

As many have answered, you cannot portably tell how many bytes there is to read, OTOH what you can do is poll the pipe for data to be read. First be sure to open the pipe with O_RDWR|O_NONBLOCK - it's mandated by POSIX that a pipe be open for both read and write to be able poll it.

Whenever you want to know if there is data available, just select/poll for data to read. You can also know if the pipe is full by checking for write.

You won't know how many data there is but keep in mind writes up to PIPE_BUF bytes are guaranteed to be atomic, so if you're concerned about having a full message on the pipe, just make sure they fit within that or split them up.

When you select for write, though, even if you can write to the pipe a write <= PIPE_BUF will return EAGAIN - I have no ideas how to tell there's enough room to write though that is what I was looking for (I may end padding with \0's to PIPE_BUF... it's just for testing anyway).

I have an old example app Perl that can read one or more pipes in non-blocking mode, OCP_Daemon. The code is pretty close to what you would do in C using an event loop.

Answer 3

Depending on your unix implementation ioctl/FIONREAD might do the trick

err = ioctl(pipedesc, FIONREAD, &bytesAvailable);

Unless this returns the error code for "invalid argument" (or any other error) bytesAvailable contains the amount of data available for unblocking read operations at that time.

Answer 4

There is no generic, portable way to tell how much data is available in a pipe without reading it. At least not under POSIX specifications.

Pipes are not seekable, and neither is it possible to put the data back into the reading end of a pipe.

Platform-specific tricks might be possible, though. If your question is platform-specific, editing your question to say so might improve your chances to get a working answer.

Answer 5

It's not in general possible to know the amount of data you can read from a pipe just from the pipe handle alone. The data may be coming in across a network, or being dynamically generated by another process. If you need to know up front, you should arrange for the information to be sent to you - through the pipe, or out of band - by whatever process is at the other end of the pipe.

Answer 6

If you want to know the amount of data that it's expected to arrive, you could always write at the begining of every msg sent by the pipes the size of the msg. So write for example 4 bytes at the start of every msg with the length of your data, and then only read the first 4 bytes.