I have a structure similar to the following:
/root/ /root/data/ /root/data/script.php /root/data/some.json /root/data/feature/one.json /root/data/feature/tw
I've written a post about such problem recently. See here.
Basically what you need is to put one .gitignore with *.json in the /data/ directory.
.gitignore
*.json
/data/
UPD: Since git 1.8.4 (1.8.2 if you're using msysgit) it is possible to use double-star patterns, like /data/**/*.json
/data/**/*.json