This is an opportunity for the 'git clean' command. If you don't care about the untracked files ... git clean -n to see what will be removed, and git clean -f to go ahead and rm untracked files.
Add -d to the commands to also operate on directories:
git clean -dn
git clean -df