mmap

shared memory after exec()

生来就可爱ヽ(ⅴ<●) 提交于 2019-12-10 10:02:02
问题 How can I share memory between the parent and the child if the child has run exec() to load another program? Is it possible using mmap ? By now parent and child share memory properly using mmap, but not after exec is done 回答1: You can use shm_open to open a "named" shared memory block which is identified by a file on the filesystem. Example: In the parent: int memFd = shm_open("example_memory", O_CREAT | O_RDWR, S_IRWXU); if (memFd == -1) { perror("Can't open file"); return 1; } int res =

mmap vs O_DIRECT for random reads (what are the buffers involved?)

纵饮孤独 提交于 2019-12-10 06:28:44
问题 I am implementing a disk based hashtable supporting large amount of keys (26+ million). The value is deserialized. Reads are essentially random throughout the file, values are less than the page size, and I am optimising for SSDs. Safety/consistency are not such huge issues (performance matters). My current solution involves using a mmap() file with MADV_RANDOM | MADV_DONTNEED set to disable prefetching by the kernel and only load data as needed on-demand. I get the idea that the kernel reads

19、共享内存mmap

不问归期 提交于 2019-12-10 05:33:12
1、特点: ① 进程相关的 ② 与 XSI 共享内存一样,需要与同步原语一起使用 ③ 只能是有共同祖先的进程才能使用 2、使用 系统调用 mmap() 用于共享内存的两种方式: ( 1 )使用普通文件提供的内存映射: 适用于任何进程之间。此时,需要打开或创建一个文件,然后再调用 mmap() 典型调用代码如下: fd=open(name, flag, mode); if(fd<0) ... ptr=mmap(NULL, len , PROT_READ|PROT_WRITE, MAP_SHARED , fd , 0); 通过 mmap() 实现共享内存的通信方式有许多特点和要注意的地方,可以参看 UNIX 网络编程第二卷。【 3 】 ( 2 )使用特殊文件提供匿名内存映射: 适用于具有亲缘关系的进程之间。由于父子进程特殊的亲缘关系,在父进程中先调用 mmap() ,然后调用 fork() 。那么在调用 fork() 之后,子进程继承父进程匿名映射后的地址空间,同样也继承 mmap() 返回的地址,这样,父子进程就可以通过映射区域进行通信了。一般来说,子进程单独维护从父进程继承下来的一些变量。而 mmap() 返回的地址,却由父子进程共同维护。 对于具有亲缘关系的进程实现共享内存最好的方式应该是采用匿名内存映射的方式 。此时,不必指定具体的文件,只要设置相应的标志即可。 3、说明 (1

python学习日志

牧云@^-^@ 提交于 2019-12-10 04:18:18
python中if __name__ == '__main__': 的解析 当你打开一个.py文件时,经常会在代码的最下面看到if __name__ == '__main__':,现在就来介 绍一下它的作用. 模块是对象,并且所有的模块都有一个内置属性 __name__。一个模块的 __name__ 的值取决于您如何应用模块。如果 import 一个模块,那么模块__name__ 的值通常为模块文件名,不带路径或者文件扩展名。但是您也可以像一个标准的程序样直接运行模块,在这 种情况下, __name__ 的值将是一个特别缺省"__main__"。 /////////////////////////////////////////////////////////////////////////////////////////////////// 在cmd 中直接运行.py文件,则__name__的值是'__main__'; 而在import 一个.py文件后,__name__的值就不是'__main__'了; 从而用if __name__ == '__main__'来判断是否是在直接运行该.py文件 Python 获得命令行参数的方法 需要模块:sys 参数个数:len(sys.argv) 脚本名: sys.argv[0] 参数1: sys.argv[1] 参数2: sys.argv

Java map / nio / NFS issue causing a VM fault: “a fault occurred in a recent unsafe memory access operation in compiled Java code”

给你一囗甜甜゛ 提交于 2019-12-10 02:45:03
问题 I have written a parser class for a particular binary format (nfdump if anyone is interested) which uses java.nio's MappedByteBuffer to read through files of a few GB each. The binary format is just a series of headers and mostly fixed-size binary records, which are fed out to the called by calling nextRecord(), which pushes on the state machine, returning null when it's done. It performs well. It works on a development machine. On my production host, it can run for a few minutes or hours,

What is the most reliable / portable way to allocate memory at low addresses on 64-bit systems?

血红的双手。 提交于 2019-12-09 18:01:06
问题 I need to allocate large blocks of memory (to be used by my custom allocator) that fall within the first 32GB of virtual address space. I imagine that if I needed, say, 1MB blocks, I could iterate using mmap and MAP_FIXED_NOREPLACE (or VirtualAlloc) from low addresses onwards in increments of, say, 1MB, until the call succeeds. Continue from the last successful block for the next one. This sounds clumsy, but at least it will be somewhat robust against OS address space layout changes, and ASLR

python 基于mmap模块的jsonmmap实现本地多进程内存共享

喜夏-厌秋 提交于 2019-12-09 16:24:48
python 基于mmap模块的jsonmmap实现本地多进程内存共享 ###1.概述 共享内存可以说是最有用的进程间通信方式.两个不用的进程共享内存的意思是:同一块物理内存被映射到两个进程的各自的进程地址空间.一个进程可以及时看到另一个进程对共享内存的更新,反之亦然.采用共享内存通信的一个显而易见的好处效率高,因为进程可以直接读写内存,而不需要任何数据的复制.对于向管道和消息队列等通信等方式,则需要在内核和用户空间进行四次的数据复制,而共享内存则只需要两次数据复制:一次从输入文件到共享内存区,另一个从共享内存区到输出文件.实际上,进程之间在共享内存时,并不总是读写少量数据后就解除映射,有新的通信时,再重新建立共享内存区域.而是保持共享区域,知道通信完毕为止,这样,数据内容就一直保存在共享内存中,并没有写回文件.共享内存中的内容往往是在解除映射时才写回文件的.因此,采用共享内存的通信方式效率非常高. mmap系统调用是的是的进程间通过映射同一个普通文件实现共享内存.普通文件被映射到进程地址空间后,进程可以向像访问普通内存一样对文件进行访问,不必再调用read,write等操作.与mmap系统调用配合使用的系统调用还有munmap,msync等. 实际上,mmap系统调用并不是完全为了用于共享内存而设计的.它本身提供了不同于一般对普通文件的访问方式

What is the fastest way to read 10 GB file from the disk?

♀尐吖头ヾ 提交于 2019-12-09 14:54:45
问题 We need to read and count different types of messages/run some statistics on a 10 GB text file, e.g a FIX engine log. We use Linux, 32-bit, 4 CPUs, Intel, coding in Perl but the language doesn't really matter. I have found some interesting tips in Tim Bray's WideFinder project. However, we've found that using memory mapping is inherently limited by the 32 bit architecture. We tried using multiple processes, which seems to work faster if we process the file in parallel using 4 processes on 4

Mapping a physical device to a pointer in User space

淺唱寂寞╮ 提交于 2019-12-09 12:13:40
问题 We have an embedded system where a memory mapped device is connected, and an ARM CPU runs Linux. The device is located at address 0x40400000 and occupies a megabyte (most of it is not backed by an actual memory, but the address space is mapped to the device anyway). We currently don't have a device driver for this device. In the device there is a special read-only register (called CID) at address 0x404f0704 . This register contains the value CID = 0x404 . I am trying to read this register

Why mmap a 4GB file on 32-bit armv7l succeeded?

笑着哭i 提交于 2019-12-08 16:39:00
问题 I had the impression from the mmap(2) man page and search results, that mmap is only limited to system's available address spaces, minus the system reserved address spaces. So on 32-bit armv7l, I assume it's around 3GB = (4GB - 1GB). But it seemed like I could actually mmap a 5 GB file without any problem: int main(int argc, char** argv) { // stats char * path = argv[1]; struct stat sb; stat(path, &sb); std::cout << "File size: " << sb.st_size << std::endl; // open int fd = open(path, O