Taking Peter Norvig\'s advice, I am pondering on the question:
How much time does it take to fetch one word from memory, with and without a cache miss
Fair summary here, with some imprecision. When written (2+ years ago) and for a mid-range PC of the time, it estimated: memory access, 60ns; L1 cache, 10ns; L2 cache, 20-30ns (no estimates for L3 cache access times). It all varies a lot of course, depending on contention and access patterns (since cache layers are typically filled "by lines" from slower memory, if you access address X then address X+1 the second access may be a bit faster as the cache line filling was started by the first access).
And, of course, a high-end, well-tuned server will be much faster (relative differences between such machines in memory access latency are typically much larger than ones in "raw" CPU speeds).