file fragmentation caused by file mapping

喜夏-厌秋 提交于 2020-01-25 05:37:08

问题


I have a program (written in c) that creates 200+ files simultaneously and writes to them all simultaneously by using mmap to map them into the programs address space. Everything works fine, but when I come to back up the directory, files that are created and written in one go by the same program copy off the same disk much faster (5-10x) than those that are mapped and written a few bytes at a time. I can only imagine that this is due to some kind of file fragmentation, though I didn't think this was even possible on a ufs file system.

Does anyone have any ideas on:

1) is it even a good idea to try to solve this, given under normal use the files will likely be read back in the same order that they were written in?

2) if there is a good solution to this, what to do about it?

The file system of the disk is ufs


回答1:


Eventually I found in the Free BSD man pages

"WARNING! Extending a file with ftruncate(2), thus creating a big hole, and then filling the hole by modifying a shared mmap() can lead to severe file frag- mentation. In order to avoid such fragmentation you should always pre-allocate the file's backing store by write()ing zero's into the newly extended area prior to modifying the area via your mmap(). The fragmentation problem is especially sensitive to MAP_NOSYNC pages, because pages may be flushed to disk in a totally random order."

I have come to the conclusion that although the Linux or Solaris man pages have no such warning, the problem must exist just the same. What to do about it is another question - writing zeros over the whole file is not a good idea if the final size of the files are unknown at the beginning. Possibly forcing the use of large pages may help, but any solution of this nature is going to be platform specific.



来源:https://stackoverflow.com/questions/19469496/file-fragmentation-caused-by-file-mapping

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!