If you really need better performance, you're going to have to give up something major - like generics, dynamic memory allocation, etc. All those features sacrifice some performance.
I would avoid using Contains if at all possible and look at TryGetValue etc.