i am a engineering student and i have to do a lot of numerical processing, plots, simulations etc. The tool that i use currently is Matlab. I use it in my university compute
Try Sage - it is designed as an open source replacement for Matlab, Mathematica etc. It is implemented in Python and can be scripted with Python, but it also adds a lot of maths-specific features. There is an installer for the Mac, so you will not need to download lots of individual packages.
There is also GNU Octave - another open source alternative to Mathematica/Matab that has its own programming language. However I have not found any information on a Mac version (though I have not looked very hard).