Why I can create a shared memory bigger than the size mounted on /dev/shm using POSIX?

问题

I am trying to handle errors using shared memory IPC in a Ubuntu 16.04. First, I checked the available memory in /dev/shm using df -h, having 500M availables, so I coded something quickly in order to check what happens if I try to create a shared mem bigger than the mounted size. The code is the following (it has been modified several times so I know that is not very tidy):

#include <iostream>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <cstring>
#include <stdint.h>
#include <stddef.h>
#include <cerrno>

//static const size_t size = 4000000L;
static const size_t size = 701000000L;
//static const size_t size = 999999999L;

extern int errno;

static int open(const char* name, int oflag, mode_t mode)
{
   int shm = -1;

   /* Create/Open a shared memory zone */
    shm = shm_open(name, oflag, mode);
    if(shm == -1)
    {/* Print error */
        std::cout << "!!!Error getting file descriptor while opening!!!" << std::endl;
        std::cout << "ERROR:"<< strerror(errno) << std::endl;
    }
   return shm;
}

static void write_shm(void *addr, size_t size)
{
    size_t i = 0;
    uint32_t *shm_index = (uint32_t *)addr;

    /* 4 bytes to be written in memory */
    const char *test = "DEAD";

    /* Maximum allowed memory address*/
    ptrdiff_t max = (size+(ptrdiff_t)addr);

    for (i = 0; (ptrdiff_t)(shm_index + i) < max; i++)
    {
        std:memcpy(&shm_index[i], (uint32_t*)test, sizeof(uint32_t));
    }
}
static int adjust (int fd, size_t size)
{
     std::cout<<__func__<<": The size of the shared memory is: "<<size<< std::endl;
     int result = ftruncate(fd, size);
     std::cout<<__func__<< "ftruncate return: " <<result<< std::endl;
     errno = 0;
     std::cout << "errno: "<< std::strerror(errno) <<std::endl;
     if (result)
     {/* Print error */;
        std::cout << "FUNCION!!!Error in ftruncate!!!" << std::endl;
     }
     return result;
}

int main()
{
    const char *name = "vardb";
    int fd = -1;
    int oflag = O_CREAT | O_EXCL | O_RDWR;
    mode_t mode = S_IRWXU | S_IRWXG | S_IRWXO; // POSIX 1003.1 (Realtime Extensions)
    size_t sizeToUse = (size/sizeof(uint32_t)* sizeof(uint32_t));

    /* Let's try to get a file descriptor related to the shared memory*/
    fd = open(name, oflag, mode);
    if (fd == -1)
        return fd;

    /* Adjust the size of the shared memory to the expected one */
    int result = adjust(fd, sizeToUse);
    if (result)
        return -1;

    int prot = PROT_READ | PROT_WRITE;
    int flags = MAP_SHARED;

    /* Map the memory */
    void *addr = mmap(NULL, size, prot, flags, fd, 0);
    std::cout<<__func__<< "mmap return: " <<*(int *)addr<< std::endl;
    std::cout<<__func__<< "mmap mapped to this address: " <<addr<< std::endl;
    errno = 0;
    std::cout << "mmap errno: "<< std::strerror(errno) <<std::endl;

    struct stat fileStat;
    if(fstat(fd, &fileStat) < 0)    
        return 1;

    std::cout<<__func__<< "File Size: " <<fileStat.st_size<<" bytes"<<std::endl;

    /* Write all the shared memory previously reserved for the test */
    write_shm(addr, sizeToUse);


    /* Release the memory */
    munmap(addr, size);

    return 0;
}

I do not unlink the shared mem in order to hexdump it, so this requires to remove it manually before relaunching the program.

Well, what I am issuing is that I do not get any error while I am creating a bigger shared mem than the /dev/shm mounted size... Obviously, I get a Bus Error as I try to write out of the available range of memory but I need to control the creation of the shared memory... I cannot understand how the system let me create something like this without reporting me any error.

Thanks in advance.

Best regards,

Iván.

回答1:

The short (and unsatisfying answer): you can't force shm_open to fail if there is insufficient space on /dev/shm. (You can force it to fail by explicitly setting the process filesize limit using setrlimit to modify RLIMIT_FSIZE, but that's a global setting not applicable to a single filesystem, so it is almost certainly not what you want it to do.)

When Posix standardised shared memory, various implementation options were considered and an attempt was made to allow considerable flexibility to implementations as long as it did not complicate the interface. In particular, many Unix implementations already had mechanisms for mapping file objects directly to process memory, and for memory-based filesystems, a combination ideally suited to the implementation of shared memory:

Simple shared memory is clearly a special case of the more general file mapping capability. In addition, there is relatively widespread agreement and implementation of the file mapping interface. In these systems, many different types of objects can be mapped (for example, files, memory, devices, and so on) using the same mapping interfaces. This approach both minimizes interface proliferation and maximizes the generality of programs using the mapping interfaces. (from Posix rationale: Mapped file functions)

In particular, "…the above requirements do not preclude: [t]he sharable memory object from being implemented using actual files on an actual file system." (Posix rationale: Shared memory objects). Although I don't believe the Linux library does this, Posix even allows shm_open() to be implemented as a macro wrapping an ordinary open() call; in the case of implementations (like Linux) which simply map shared memory onto the filesystem, nothing requires special handling of the ftruncate() system interface.

It's important to highlight one aspect of the ftruncate() call (emphasis added):

If the file size is increased, the extended area shall appear as if it were zero-filled.

Many filesystems allow for "sparse files". In a sparse file, file blocks entirely filled with zeros are simply not mapped onto physical storage; if an application reads one of these blocks, it receives a page of zeros. If a block is modified and committed to disk, then -- and only then -- does the filesystem allocate storage for the block. [Note 1]

Lazy allocation of zero-filled blocks means that in the case where ftruncate() is expanding a file, it only needs to update the file's metadata, allowing it to return very rapidly. Unless the desired size exceeds the process's file size limit (or the filesystem limit in the case of filesystems which don't use large enough integer types for file sizes), no error will be produced by ftruncate(). The error will occur when it becomes impossible to allocate physical storage (or a dedicate memory buffer, in the case of a memory-mapped file).

This is entirely consistent with Linux's optimistic approach to memory allocation: mmap always succeeds (as long as address space is available) and the error is only noted when the memory is actually used. (But this particular approach to shared memory implementation is not limited to those using optimistic memory allocation.)

Notes

I used to demonstrate this by storing a 2GB file on a floppy disk, but I suppose that many readers today will not even know what a floppy disk is, much less what its actual capacity was.

来源：https://stackoverflow.com/questions/50642571/why-i-can-create-a-shared-memory-bigger-than-the-size-mounted-on-dev-shm-using

标签

c++

posix

ipc

shared-memory