I am using a Dictionary to store data, and will be caching it. I would like to avoid server memory issues, and have good performance by limiting the size of the Dictionary&
You don't need to limit the size of the Dictionary to archieve good performance.
As the documentation says:
Retrieving a value by using its key is very fast, close to O(1)