Check out the python Scipy library. It's an open source fast n-dimensional array manipulation library in python. It has all the basic image processing tools:
- linear and rank filters
- FFT, convolution
- morphological operation
It doesn't have all the more advanced functions of Matlab's Image Processing Toolbox. However, as Vereb suggested, a lot of those can be found in ITK (also available in python flavor).
Like you I wanted to get away from matlab to a dynamic language like python. And like you, I was disappointed by PIL, when I realized it was just an ImageMagick in python. You'll still need to use PIL for reading/writing images.
To get a Matlab-like image processing experience with python get Numpy/Scipy, Matplotlib and Spyder. All of which and more is conveniently packaged in Python(X,Y) for windows only.