Does Python support zero-copy I/O?

被刻印的时光 ゝ 提交于 2021-01-21 03:45:41

问题


I have two open file objects, dest and src. File object dest is opened for writing, with the seek position placed at some offset within the file, and file object src is opened for reading. What I need to do is simply read from the current position in src to EOF and transfer the contents to dest as quickly as possible.

If I were programming in Java, I could utilize the FileChannel#transferTo() method to perform zero-copy file I/O.

Does Python also support zero-copy?


回答1:


Since version 3.3, Python has os.sendfile, which interfaces to various Unix variants' sendfile(2) zero-copy I/O interfaces. It operates on file descriptors, not general file-like objects. For older Pythons, there's py-sendfile.




回答2:


Since Python 3.8, you can use shutil.copyfile (and others from shutil) which will internally use zero-copy if possible, such as os.sendfile, and if not possible, fall back to a simple read-write loop.

See the shutil docs for details. Or issue 33671 (Efficient zero-copy for shutil.copy* functions (Linux, OSX and Win)). And the corresponding (merged) pull request.

You might also be interested in copy-on-write support or server-side copy support. See here, here. The os.copy_file_range (since Python 3.8) would do that. See issue 37159 (Use copy_file_range() in shutil.copyfile()) (maybe Python 3.9 or 3.10).



来源:https://stackoverflow.com/questions/7456181/does-python-support-zero-copy-i-o

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!