GZip every file separately

Question

How can we GZip every file separately?

I don\'t want to have all of the files in a big tar.

Answer 1

After seven years, this highly upvoted comment still doesn't have its own full-fledged answer, so I'm promoting it now:

gzip -r .

This has two advantages over the currently accepted answer: it works recursively if there are any subdirectories, and it won't fail from Argument list too long if the number of files is very large.