If you want to focus on the actual sound processing (as opposed to all the crud that GUI applications are saddled with), I'd check out CSound. That gives you a textfile based langugae that allows you to build almost any audio device virtually from what they call opcodes, but also you have the source code which you can delve into to see the actual algorithms that the opcodes represent.
