I always thought that Haskell would do some sort of automatic intelligent memoizing. E.g., the naive Fibonacci implementation
fib 0 = 0
fib 1 = 1
fib n = fib
No, Haskell does not do automatic memoisation of functions. What it does is store values, so if you have
x = somethingVeryLong
and somewhere else in the same scope you have
y = f x
z = g x
then x will only be computed once.
This package shows how memoised values can be stored using a variety of keys and look-up tables. The memoising is typically used within a single invocation of a larger function, so that the memoised values don't hang around forever (which as you say would be a problem). If you want a memoiser that also forgets old values using LRU or something then I think you probably need to put it in a state monad or something; you can't get Haskell to behave like that using the conventional memoising approach.