mmap

深入了解Netty【二】零拷贝

别来无恙 提交于 2019-12-06 03:39:13
引言 以下翻译自: Zero Copy I: User-Mode Perspective 零拷贝是什么? 为了更好地理解问题的解决方案,我们首先需要理解问题本身。让我们来看看什么是参与网络服务器的简单过程dæmon服务数据存储在一个文件通过网络客户端。下面是一些示例代码: read(file, tmp_buf, len); write(socket, tmp_buf, len); 看起来很简单;您会认为只有这两个系统调用不会带来太多开销。事实上,这与事实相去甚远。在这两个调用之后,数据至少复制了四次,并且几乎执行了相同数量的用户/内核上下文切换。(实际上这个过程要复杂得多,但我想让它保持简单)。为了更好地了解所涉及的流程,请看图1。顶部显示上下文切换,底部显示复制操作。 图1。复制两个示例系统调用 第一步:read系统调用导致上下文从用户模式切换到内核模式。第一个副本由DMA引擎执行,它从磁盘读取文件内容并将其存储到内核地址空间缓冲区中。 第二步:将数据从内核缓冲区复制到用户缓冲区,read系统调用返回。调用的返回导致上下文从内核切换回用户模式。现在数据存储在用户地址空间缓冲区中,它可以再次开始向下移动。 第三步:write系统调用导致上下文从用户模式切换到内核模式。执行第三次复制,再次将数据放入内核地址空间缓冲区。不过,这一次,数据被放入一个不同的缓冲区

Avoid copying of data between user and kernel space and vice-versa

心不动则不痛 提交于 2019-12-06 02:37:13
I am developing a active messaging protocol for parallel computation that replaces TCP/IP. My goal is to decrease the latency of a packet. Since the environment is a LAN, i can replace TCP/IP with simpler protocol to reduce the packet latency. I am not writing any device driver and i am just trying to replace the TCP/IP stack with something simpler. Now I wanted to avoid copying of a packet's data from user space to kernel space and vice-versa . I heard of the mmap(). Is it the best way to do this? If yes, it will be nice if you can give links to some examples. I am a linux newbie and i really

How to Disable Copy-on-write and zero filled on demand for mmap()

不打扰是莪最后的温柔 提交于 2019-12-06 01:11:18
问题 I am implementing cp(file copy) command using mmap(). For that I mapped the source file in MAP_PRIVATE (As I just want to read)mode and destination file in MAP_SHARED mode(As I have to writeback the changed content of destination file). While doing this I have observed performance penalty due to lots of minor page faults that occurs due to 2 reason. 1) Zero fill on demand while calling mmap(MAP_PRIVATE) for source file. 2) Copy on write while calling mmap(MAP_SHARED) for destination file. Is

How GPIO is mapped in memory?

。_饼干妹妹 提交于 2019-12-06 00:36:42
问题 I am recently browsing GPIO driver for pi2, I found user space pi2 GPIO lib (like RPi.GPIO 0.5.11 of python) use /dev/mem for BCM2708 (begins at 0x20000000,and GPIO begins at 0x200000 relatively) to mmap a user space memory region in order to handler GPIO. But I found drivers/gpio in linux source tree is designed to be handled by /sys/class/gpio/* . I found nothing like I/O ports mapping like request_io_region and __io_remap . My question is How GPIO for BCM2708 mapped in memory ? Is there

shared memory after exec()

匆匆过客 提交于 2019-12-05 21:53:36
How can I share memory between the parent and the child if the child has run exec() to load another program? Is it possible using mmap ? By now parent and child share memory properly using mmap, but not after exec is done You can use shm_open to open a "named" shared memory block which is identified by a file on the filesystem. Example: In the parent: int memFd = shm_open("example_memory", O_CREAT | O_RDWR, S_IRWXU); if (memFd == -1) { perror("Can't open file"); return 1; } int res = ftruncate(memFd, /*size of the memory block you want*/); if (res == -1) { perror("Can't truncate file"); return

SIGBUS while doing memcpy from mmap ed buffer which is in RAM as identified by mincore

丶灬走出姿态 提交于 2019-12-05 21:45:18
I am mmapping a block as: mapAddr = mmap((void*) 0, curMapSize, PROT_NONE, MAP_LOCKED|MAP_SHARED, fd, curMapOffset); if this does not fail (mapAddr != MAP_FAILED) I query mincore as: err = mincore((char*) mapAddr, pageSize, &mincoreRet); to find out whether it is in RAM. In case it is in RAM (err == 0 && mincoreRet & 0x01) I mmap it again for reading as: copyAddr = mmap((void*) 0, curMapSize, PROT_READ, MAP_LOCKED|MAP_SHARED, fd, curMapOffset); and then I try to copy it out to my buffer as: memcpy(data, copyAddr, pageSize); everything works fine except in the last memcpy once in a while I get

Can mmap and gzip collaborate?

不想你离开。 提交于 2019-12-05 20:47:12
问题 I'm trying to figure how to use mmap with a gzip compressed file. Is that even possible ? import mmap import os import gzip filename = r'C:\temp\data.gz' file = gzip.open(filename, "rb+") size = os.path.getsize(filename) file = mmap.mmap(file.fileno(), size) print file.read(8) The output data is compressed. 回答1: Well, not the way you want. mmap() can be used to access the gzipped file if the compressed data is what you want. mmap() is a system call for mapping disk blocks into RAM almost as

Does mmap share memory with all processes?

风流意气都作罢 提交于 2019-12-05 20:38:34
When I do this : myProgram.h myProgram.c struct PipeShm { // all my fields // more // ... }; struct PipeShm myPipe = { /* initialization for all fields */ }; struct PipeShm * sharedPipe = &myPipe; void func() { sharedPipe = mmap (NULL, sizeof * sharedPipe, PROT_READ | PROT_WRITE,MAP_SHARED | MAP_ANONYMOUS, -1, 0); } When I mmap the pointer sharedPipe , if I invoke from main() any methods from myProgram code, would all processes share the exact shared memory that I shared with myPipe struct? Or would each new child that's created, would have a new myPipe of his own? Regards EDIT: This is after

using numpy.memmap to map a device file

帅比萌擦擦* 提交于 2019-12-05 18:32:00
Is there a reason that opening a device file (rather than a regular file) using numpy's memmap shouldn't work? self.surface = np.memmap('/dev/fb1', dtype=np.uint16, mode='r+', shape=(320,240)) I'm working with a custom kernel module that adds a framebuffer device, which works fine with python's regular mmap module. But using numpy seems to hang the kernel's mutex on accessing the filesystem or something (I'm really not sure exactly what's happening). My question here is specifically is this something that numpy's memmap can't handle and I should go a different way? I've asked another question

How do I choose a fixed address for mmap?

微笑、不失礼 提交于 2019-12-05 17:42:28
问题 mmap() can be optionally supplied with a fixed location to place the map. I would like to mmap a file and then have it available to a few different programs at the same virtual address in each program. I don't care what the address is, just as long as they all use the same address. If need be, the address can be chosen by one of them at run time (and communicated with the others via some other means). Is there an area of memory that Linux guarantees to be unused (by the application and by the