mmap

问题 After advises on a question I made to close mmapped files (c++ close a open() file read with mmap) I made some comparisons and I noticed that, as suggested by some users, std::cin buffer approach is performing similar to my mmapped approach. I decided to make a performance comparison: every script opens a file that contains other files path (about 3500), read the file and takes 10 random paths and opening all these 10 files (of 500 lines every, about 700 characters each line) and read the

问题 I'm using ctypes.from_buffer() to map a ctypes structure to a memory mapped file for some tasks. Typically, these files contain a concatenation of structured headers and binary data. The ctypes structure allows for a stable binary representation and easy pythonic access of the fields - a real winning team in this respect. These memory mapped files grow dynamically over time. Apart from the complication of accepting growth in mmap.PAGESIZE granularity only, the mmap responds with allergic

问题 I have a program that reads a file line by line whose size varies, i would like use mmap but how use it to read a file line by line? Thank you for your answers! 回答1: Once you have mmap() ed the file, you can make the file available to a suitable stream buffer reading data from existing memory and then use std::getline() : #include <streambuf> #include <string> #include <istream> struct membuf std::streambuf { membuf(char* start, size_t size) { this->setg(start, start, start + size); } }; int

问题 I am writing a C program that makes use of mmap system call, running on Linux 3.12 64-bit. If I have two processes mmaping the same region of a disk file with read/write access, and then modify the region content from both processes at the same time... Can one process see (read) changed data made by the other process, before or after msync? If update can be seen by the other process - is there a happens-before guarantee made by Linux mmap implementation? 回答1: Yes, that's one of the purposes

问题 If I use mmap to write uint32_t 's, will I run into issues with big endian/little endian conventions? In particular, if I write some data mmap 'ed on a big-endian machine, will I run into issues when I try to read that data on a little-endian machine? 回答1: If you're using mmap, your probably concerned about speed and efficiency. You basically have a few choices. Wrap all your reads and writes with htonl, htons, ntohl, ntohs functions. Calling htonl (host to network) order on Windows will

问题 Is the order of page flushes with msync(MS_ASYNC) on linux guaranteed to be the same as the order the pages where written to? If it depends on circumstances, is there a way for me (full server access) to make sure they are in the same order? Background I'm currently using OpenLDAP Symas MDB as a persistent key/value storage and without MDB_MAPASYNC - which results in using msync(MS_ASYNC) (I looked through the source code) - the writes are so slow, that even while processing data a single

问题 I'm trying to exceed the shared memory object after shm_open and ftruncate successfully at fisrt. Here is the code, char *uuid = GenerateUUID(); int fd = shm_open(uuid, O_RDWR|O_CREAT|O_EXCL, S_IRUSR|S_IWUSR); if(fd == -1) perror("shm_open"); size_t shmSize = sizeof(container); int ret = ftruncate(fd, shmSize); perror("ftruncate first"); ret = ftruncate(fd, shmSize * 2); perror("ftruncate second"); It could pass the first ftruncate, but for the second ftruncate, it exceeds failed with errno

问题 I have very large binary files with x number of int16 data points for y sensors, along with headers with some basic info. The binary file is written as y values for each sample time up to x samples, then another set of readings and so on. If I want all of the data, I am using numpy.fromfile() which works really nice and fast. However, if I only want a subset of sensor data or only specific sensors, I currently have a hideous double for loop, using file.seek() , file.read() , and struct.unpack

问题 Is there any way to map a file's content into memory in Windows that does not hold a lock on the file (in particular, such that the file can be deleted while still mmap'd)? The Java NIO libraries mmap files in Windows in such a way that the mapped file cannot be deleted while there is any non garbage collected MappedByteBuffer reference left in the heap. The JDK team claim that this is a limitation of Windows, but only when files are mmap'd, not when they are opened as regular files: https:/

问题 I'm looking to retrieve just the last line of a file quickly in Haskell---starting from the end, not the beginning---and having some difficulties using hSeek correctly. It seems the SeekFromEnd N behaves differently than finding the length of the file sz , and using AbsoluteSeek to go (sz - N) bytes. outh <- openFile "test.csv" ReadMode λ> hIsSeekable outh True λ> hFileSize outh 81619956 λ> hSeek outh AbsoluteSeek 1000 λ> hTell outh 1000 λ> hSeek outh SeekFromEnd 1000 λ> hTell outh 81620956 λ