set go=c
Use console dialogs instead of popup dialogs for simple choices.
I think this will turn it off, not just hiding it.
http://vimdoc.sourceforge.net/htmldoc/options.html#%27go-c%27
Edit: Just to clarify, this turning off the menus and toolbars in a GUI as well.