C/C++ - Memory map file using mmap

问题

I'm new to memory mapping files and am a bit confused. Is it possible to map files larger than the total amount of memory because as I understood memory mapping uses demand paging and will hold only the currently relevant pages in memory. Is this right or will my application crash when using more space than the actual memory.

Thanks

EDIT OS: Ubuntu 14.04 LTS x86_64 App-Bitness: 64bit (I guess: pointers are 8 byte)

I'm trying to allocate memory from the mapped file in order to store the tree inside the mapped file.

#define MEMORY_SIZE 300000000

unsigned char *mem_buffer;
void *start_ptr;

void *my_malloc(int size) {
    unsigned char *ptr = mem_buffer;
    mem_buffer += size;

    return ptr;
}

void *my_calloc(int size, int object_size) {
    unsigned char *ptr = mem_buffer;
    mem_buffer += (size * object_size);

    return ptr;
}

void init(const char *file_path) {
    int fd = open(file_path, O_RDWR, S_IREAD | S_IWRITE);

    if (fd < 0) {
        perror("Could not open file for memory mapping");
        exit(1);
    }

    start_ptr = mmap(NULL, MEMORY_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
    mem_buffer = (unsigned char *) start_ptr;

    if (mem_buffer == MAP_FAILED) {
        perror("Could not memory map file");
        exit(1);
    }

    printf("Successfully mapped file.\n");
}

void unmap() {
    if (munmap(start_ptr, MEMORY_SIZE) < 0) {
        perror("Could not unmap file");
        exit(1);
    }

    printf("Successfully unmapped file.\n");
}

main method:

int main(int argc, char **argv) {

    init(argv[1]);

    unsigned char *arr = (unsigned char *) my_malloc(6);
    arr[0] = 'H';
    arr[1] = 'E';
    arr[2] = 'L';
    arr[3] = 'L';
    arr[4] = 'O';
    arr[5] = '\0';

    unsigned char *arr2 = (unsigned char *) my_malloc(5);
    arr2[0] = 'M';
    arr2[1] = 'I';
    arr2[2] = 'A';
    arr2[3] = 'U';
    arr2[4] = '\0';

    printf("Memory mapped string1: %s\n", arr);
    printf("Memory mapped string2: %s\n", arr2);

    struct my_btree_node *root = NULL;

    insert(&root, arr, 10);
    insert(&root, arr2, 20);

    print_tree(root, 0, false);

//  cin.ignore();

    unmap();

    return EXIT_SUCCESS;
}

回答1:

Here's an excerpt from The GNU C Library: Memory-mapped I/O

Since mmapped pages can be stored back to their file when physical memory is low, it is possible to mmap files orders of magnitude larger than both the physical memory and swap space. The only limit is address space. The theoretical limit is 4GB on a 32-bit machine - however, the actual limit will be smaller since some areas will be reserved for other purposes.

回答2:

It is possible (and very normal) to map files larger than the total amount of memory but your app must be able to address it. You have a limited address range if your system/app is 32-bit. Your system & app must be 64-bit to address bigger than several gigabyte sized files...

Note that: Even if your file is small, you may still not be able to map it. Maybe you consumed your address space by simply allocating memory for other purposes (stack space for threads, memory for application buffers/lists, etc.)

来源：https://stackoverflow.com/questions/29210851/c-c-memory-map-file-using-mmap

标签

mmap