mmap() vs. reading blocks

前端 未结 12 1534
醉酒成梦
醉酒成梦 2020-11-22 16:59

I\'m working on a program that will be processing files that could potentially be 100GB or more in size. The files contain sets of variable length records. I\'ve got a first

12条回答
  •  野性不改
    2020-11-22 17:24

    I think the greatest thing about mmap is potential for asynchronous reading with:

        addr1 = NULL;
        while( size_left > 0 ) {
            r = min(MMAP_SIZE, size_left);
            addr2 = mmap(NULL, r,
                PROT_READ, MAP_FLAGS,
                0, pos);
            if (addr1 != NULL)
            {
                /* process mmap from prev cycle */
                feed_data(ctx, addr1, MMAP_SIZE);
                munmap(addr1, MMAP_SIZE);
            }
            addr1 = addr2;
            size_left -= r;
            pos += r;
        }
        feed_data(ctx, addr1, r);
        munmap(addr1, r);

    Problem is that I can't find the right MAP_FLAGS to give a hint that this memory should be synced from file asap. I hope that MAP_POPULATE gives the right hint for mmap (i.e. it will not try to load all contents before return from call, but will do that in async. with feed_data). At least it gives better results with this flag even that manual states that it does nothing without MAP_PRIVATE since 2.6.23.

提交回复
热议问题