.gitignore will only ignore files that you haven't already added to your repository.
If you did a git add ., and the file got added to the index, .gitignore won't help you. You'll need to do git rm sites/default/settings.php to remove it, and then it will be ignored.