I have been reading about out of memory conditions on Linux, and the following paragraph from the man pages got me thinking:
By default, Linux follows
I think the malloc can still return NULL. The reason why is that there is a difference between the system memory available (RAM + swap) and the amount in your process's address space.
For example, if you ask for 3GB of memory from malloc on a standard x86 linux, it will surely return NULL since this is impossible given the amount of memory given to user space apps.