mmap fails when length is larger than 4GB

Question

(The correct code is in \'Update 5\')

I tried to map a range of memory from 0x100000000 to 0x200000000 in this example C code:

#include 

        

Answer 1

Just ran your code on Fedora 13 and it produces result 2.

Check errno when mmap() returns MAP_FAILED (-1). You can also stick the following line before and after mmap call to see if you've got space in the virtual address space of the process for a 4GB region:

system("pmap -x $$");

Update: The above actually prints the map of the child process. Correct code:

char buf[0x100];
snprintf(buf, sizeof buf, "pmap -x %u", (unsigned)getpid());
system(buf);

Answer 2

How much physical memory is there available? Linux has two distinct modes for address space allocation: Memory allocation on write (i.e. overcommit mode) or memory allocation on address space allocation. You can check by reading two files in procfs:

cat /proc/sys/vm/overcommit_memory
cat /proc/sys/vm/overcommit_ratio

If overcommit_memory is not 0, then every address space allocation must be backed by physical memory (RAM + swap space), if overcommit_memory is 0, then memory is overcommited, i.e. the kernel will happily hand out address space, but the memory will be only allocated if data is writen to the allocated address space. And then memory is not allocated for the full reserved address space, but only for those pages that are touched. This is kinda like booking a flight ticket: Airlines usually sell more tickets than there are seats on a flight, expecting not all booked passengers will actually show up. Now you may wonder, what happens if all programs make use of the full space… Well then some nasty thing kicks in: The Linux Out Of Memory Killer will wreak havoc on your system and very likely kill those processes you need the most, due to it's arcane heuristics.

overcommit_ratio tells the kernel

• in overcommit mode to which ratio physical memory may be overcommited, i.e. how much more address space may be handed out, than there is physical memory.

• in non-overcommit-mode how much spare memory to keep

So maybe the overcommit mode just differs between the systems.

Answer 3

Since you try to map to a specific address, it will depend on the current memory layout for your process when you call mmap. The strategy at which address the request is fulfill is system dependent, the linux man page says something of a "hint".

So maybe in the first case there simply not enough room in the virtual address space of your process to fulfill the request, since there is already another mapping in the way in that range.

A good idea to check if this is related to that would be to check if you succeed when you don't give the addr hint.

Answer 4

If you do not actually have significantly more than 8G of swap configured, then that large mapping is likely to fail.

You can add the MAP_NORESERVE flag to mmap() to tell it not to reserve any swap space for the mapping up front.

Answer 5

Maybe you are running into resource limits? Try adding system("ulimit -m -v"); to print out the amount of memory and address space that may be allocated.

EDIT: Well, I'm out of ideas. Sorry. After cleaning up the errors and warnings in the code, I have this source:

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <sys/mman.h>

int main(void)
{
    uint64_t* rr_addr = 0;
    uint64_t i = 17179869184;

    printf("\nsizeof(size_t): %lu\n", sizeof(size_t));

    printf("(uint64_t)0x100000000: %lx\n", (uint64_t)0x100000000);
    printf("1L << 33: %lx\n", 1L << 33);

    char cmd[20];

    sprintf(cmd, "pmap -x %i", getpid());
    printf("%s\n", cmd);
    system(cmd);

    rr_addr = mmap((void*)i, (1UL << 33), PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, -1, 0);

    printf("%s\n", cmd);
    system(cmd);


    printf("rr_addr: %p, %lu \n", rr_addr, rr_addr);
    if (rr_addr == MAP_FAILED) {
        perror("mmap error");
    }

    return 0;
}

and this output:

sizeof(size_t): 8
(uint64_t)0x100000000: 100000000
1L << 33: 200000000
pmap -x 23819
23819:   ./zhiqiang
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000000400000       0       4       0 r-x--  zhiqiang
0000000000600000       0       4       4 r----  zhiqiang
0000000000601000       0       4       4 rw---  zhiqiang
00007f37b3c27000       0     260       0 r-x--  libc-2.12.1.so
00007f37b3da1000       0       0       0 -----  libc-2.12.1.so
00007f37b3fa0000       0      16      16 r----  libc-2.12.1.so
00007f37b3fa4000       0       4       4 rw---  libc-2.12.1.so
00007f37b3fa5000       0      12      12 rw---    [ anon ]
00007f37b3faa000       0     108       0 r-x--  ld-2.12.1.so
00007f37b41aa000       0      12      12 rw---    [ anon ]
00007f37b41c7000       0      12      12 rw---    [ anon ]
00007f37b41ca000       0       4       4 r----  ld-2.12.1.so
00007f37b41cb000       0       4       4 rw---  ld-2.12.1.so
00007f37b41cc000       0       4       4 rw---    [ anon ]
00007fff70cf8000       0      12      12 rw---    [ stack ]
00007fff70dff000       0       4       0 r-x--    [ anon ]
ffffffffff600000       0       0       0 r-x--    [ anon ]
----------------  ------  ------  ------
total kB            3912     464      88
pmap -x 23819
23819:   ./zhiqiang
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000000400000       0       4       0 r-x--  zhiqiang
0000000000600000       0       4       4 r----  zhiqiang
0000000000601000       0       4       4 rw---  zhiqiang   
0000000400000000       0       0       0 rw---    [ anon ]
00007f37b3c27000       0     260       0 r-x--  libc-2.12.1.so
00007f37b3da1000       0       0       0 -----  libc-2.12.1.so
00007f37b3fa0000       0      16      16 r----  libc-2.12.1.so
00007f37b3fa4000       0       4       4 rw---  libc-2.12.1.so
00007f37b3fa5000       0      12      12 rw---    [ anon ]
00007f37b3faa000       0     108       0 r-x--  ld-2.12.1.so
00007f37b41aa000       0      12      12 rw---    [ anon ]
00007f37b41c7000       0      12      12 rw---    [ anon ]
00007f37b41ca000       0       4       4 r----  ld-2.12.1.so
00007f37b41cb000       0       4       4 rw---  ld-2.12.1.so
00007f37b41cc000       0       4       4 rw---    [ anon ]
00007fff70cf8000       0      12      12 rw---    [ stack ]
00007fff70dff000       0       4       0 r-x--    [ anon ]
ffffffffff600000       0       0       0 r-x--    [ anon ]
----------------  ------  ------  ------
total kB         8392520     464      88
rr_addr: 0x400000000, 17179869184

And details of my system:

Linux haig 2.6.35-24-generic #42-Ubuntu SMP Thu Dec 2 02:41:37 UTC 2010 x86_64 GNU/Linux
gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
GNU C Library (Ubuntu EGLIBC 2.12.1-0ubuntu10.1) stable release version 2.12.1, by Roland McGrath et al.