I'll also mention filecache.el. It basically provides a specialized completion (which you can bind to a key for use when entering file names) which first completes on "filenames, regardless of directory", and once that is complete, cycles through the few directories where such files are found.