In vim, there is this amazing plugin called command-t, that lets you fuzzy-search through all the files in your project. We have a fairly large rails app with a ton of files
Try https://github.com/technomancy/find-file-in-project which use GNU Find or BSD Find to find files. Nothing can beat the speed of C!
M-x find-file-in-project-by-selected is the only command you need use.
I tested with 50000+ files at some stone-aged netbook without any issue.
By default it uses efficient ivy-mode to filter the candidate and ido-mode as fallback. I tested ivy-mode with three million candidates. Now you get the idea that Emacs Lisp itself is quick enough.
At *nix, the kernel provides cache for find, so if you search the files with same input, the response is instant.
Besides, the ivy-mode automatically cache the last search results in Emacs lisp variable ivy-last, so you can M-x ivy-resume to get the previous candidates without bothering Find.
You can (setq my-cached-result ivy-last) to store ivy-last into another variable. Then you can M-x my-ivy-resume:
(defun my-ivy-resume ()
(interactive)
(let* ((ivy-last (if my-cached-result my-cached-result ivy-last))
(default-directory (ffip-get-project-root-directory)))
(ivy-resume)))
In this way you can store last 1000 results and re-use them ;)