What is caching?

Question

I\'m constantly hearing about person y had performance issue x which they solved through caching.

Or, how doing x,y,z in your programs code can hurt your caching abil

Answer 1

The cache concept is an overloaded term here. I'm not familiar with the nuts and bolts of database caching.

In applications there are two uses of the term.

When someone says that they found a piece of code that would hurt caching and after they fixed it, it improved the speed of their app, what are they talking about?

In this case they're making reference to the CPU cache.

The CPU cache is on-CPU memory that's a lot quicker than RAM, but it doesn't have random access. What the CPU decides to load into cache can get a little complicated. See Ulrich Dreppers What every programmer should know about memory for lots of details.

Being mindful of the CPU cache can speed things up pretty well - you just have to pay a little more attention to where things are going to placed relative to each other in physical memory and when they're likely to be used.

One example (also probably an anti-pattern for maintainability) is that is you have an array of structures and you do a lot of looping over the members of the structure you might be better served with a structure where the fields are all arrays. If the data you're looping over is contiguous in memory you have a better chance at non upsetting the cache.

All kinds of things can effect the efficiency of your cache usage - branch prediction for code loaded into the cache, size and alignment of data structures and access patterns, where and when to declare local variables that are going to be put onto the stack.

The other common use of the term for application programming can be done by something called memoization. The factorial example on that wikipedia page explains things better than I would have done.

Answer 2

It's probably easier than you could imagine--and that's why people are trying to close it.

It just means to store the values in your memory rather than go back to the database for them every time.

There are lots of ways to do so, but the concept itself is trivial.

Edit: It can be done at ANY level too--anything that takes a long time can be cached somewhere that you can get to more quickly.

Answer 3

caching is taking the result of a long or cpu intensive algorithm and saving the answer so that you do not have to run the algorithm again, you just reuse the result.