When a non-blocking send() only transfers partial data, can we assume it would return EWOULDBLOCK the next call?

Question

Two cases are well-documented in the man pages for non-blocking sockets:

• If send() returns the same length as the transfer buffer, the entire transfer finished successfully, and the socket may or may not be in a state of returning EAGAIN/EWOULDBLOCK the next call with >0 bytes to transfer.
• If send() returns -1 and errno is EAGAIN/EWOULDBLOCK, none of the transfer finished, and the program needs to wait until the socket is ready for more data (EPOLLOUT in the epoll case).

What's not documented for nonblocking sockets is:

• If send() returns a positive value smaller than the buffer size.

Is it safe to assume that the send() would return EAGAIN/EWOULDBLOCK on even one more byte of data? Or should a non-blocking program try to send() one more time to get a conclusive EAGAIN/EWOULDBLOCK? I'm worried about putting an EPOLLOUT watcher on the socket if it's not actually in a "would block" state to respond to it coming out of.

Obviously, the latter strategy (trying again to get something conclusive) has well-defined behavior, but it's more verbose and puts a hit on performance.

Answer 1

A call to send has three possible outcomes:

1. There is at least one byte available in the send buffer →send succeeds and returns the number of bytes accepted (possibly fewer than you asked for).
2. The send buffer is completely full at the time you call send.
   →if the socket is blocking, send blocks
   →if the socket is non-blocking, send fails with EWOULDBLOCK/EAGAIN
3. An error occurred (e.g. user pulled network cable, connection reset by peer) →send fails with another error

If the number of bytes accepted by send is smaller than the amount you asked for, then this consequently means that the send buffer is now completely full. However, this is purely circumstantial and non-authorative in respect of any future calls to send.
The information returned by send is merely a "snapshot" of the current state at the time you called send. By the time send has returned or by the time you call send again, this information may already be outdated. The network card might put a datagram on the wire while your program is inside send, or a nanosecond later, or at any other time -- there is no way of knowing. You'll know when the next call succeeds (or when it doesn't).

In other words, this does not imply that the next call to send will return EWOULDBLOCK/EAGAIN (or would block if the socket wasn't non-blocking). Trying until what you called "getting a conclusive EWOULDBLOCK" is the correct thing to do.

Answer 2

If send() returns the same length as the transfer buffer, the entire transfer finished successfully, and the socket may or may not be in a blocking state.

No. The socket remains in the mode it was in: in this case, non-blocking mode, assumed below throughout.

If send() returns -1 and errno is EAGAIN/EWOULDBLOCK, none of the transfer finished, and the program needs to wait until the socket is isn't blocking anymore.

Until the send buffer isn't full any more. The socket remains in non-blocking mode.

If send() returns a positive value smaller than the buffer size.

There was only that much room in the socket send buffer.

Is it safe to assume that the send() would block on even one more byte of data?

It isn't 'safe' to 'assume [it] would block' at all. It won't. It's in non-blocking mode. EWOULDBLOCK means it would have blocked in blocking mode.

Or should a non-blocking program try to send() one more time to get a conclusive EAGAIN/EWOULDBLOCK?

That's up to you. The API works whichever you decide.

I'm worried about putting an EPOLLOUT watcher on the socket if it's not actually blocking on that.

It isn't 'blocking on that'. It isn't blocking on anything. It's in non-blocking mode. The send buffer got filled at that instant. It might be completely empty a moment later.

I don't see what you're worried about. If you have pending data and the last write didn't send it all, select for writability, and write when you get it. If such a write sends everything, don't select for writability next time.

Sockets are usually writable, unless their send buffer is full, so don't select for writability all the time, as you just get a spin loop.