Real-life case (from caff) to exemplify the short question subject:
$CONFIG{\'owner\'} = q{Peter Palfrader};
$CONFIG{\'email\'} = q{peter@palfrader.org};
$CO
The main issue here is reusability in an environment where multiple languages are possible. If your config file is in language A, then you want to share this configuration with language B, you will have to do some rewriting.
This is even more complicated if you have more complex configurations (example the apache config files) and are trying to figure out how to handle potential differences in data structures. If you use something like JSON, YAML, etc., parsers in the language will be aware of how to map things with regards to the data structures of the language.
The one major drawback of not having them in a language, is that you lose the potential of utilizing setting config values to dynamic data.