I think the first question is: what sort of program would it be interesting to create? One of the things that got me started with programming as a kid (in BBC basic and then QBasic) was the ease of writing graphical programs. I could write a couple of lines of code and see my program draw a line on the screen straight away.
The closest I've seen to that sort of simplicity recently are the pygame library for python and Processing, a set of java libraries with an IDE.
I imagine that hacking on web pages would be another good way to get started: that would entail HTML, Javascript (using a library like jQuery), perhaps PHP or something along those lines.
Whatever tools you provide, the crucial thing is for it to be easy to get started straight away. If you have to write twenty lines of correct code and figure out how to invoke the compiler before you see any tangible results, progress is going to be slow.
