How to reduce the number of files in the anaconda directory?

纵然是瞬间 提交于 2019-12-08 15:58:05

问题


I'm running conda environments on a compute cluster where the total number of files per "project" is restricted (200k files max). I've only created a couple of conda environments (anaconda for Python 2.7; ~200 python & R packages installed in each environment; high package overlap between environments) and already hit that file number limit. Even when using conda clean -a only a small fraction of the files are removed. Some python packages in my conda environments (e.g., boost) contain >10k files, and clean does not reduce this.

Is there any way to greatly reduce the number of files stored as part of a conda environment?


回答1:


Anaconda uses hard links to reduce the consumed disk space. But if a limit is imposed on the number of files, each hard link counts.

As discussed in the comments, using Miniconda instead of Anaconda, and installing only the packages you actually need, might help.

If this isn't enough, I'd recommend to merge several of your environments into one. Then you'll have fewer hardlinks for the packages that overlap. Of course that is the opposite of what environments are there for, but such is the nature of workarounds.



来源:https://stackoverflow.com/questions/40241133/how-to-reduce-the-number-of-files-in-the-anaconda-directory

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!