Python configuration file: Any file format recommendation? INI format still appropriate? Seems quite old school

Question

I need to store configurations (key/value) for a Python application and I am searching for the best way to store these configurations in a file.

I run into Python\'s

Answer 1

For completeness, you can also use a shell-style configuration format with the help of the "shlex" module. If you have a fixed set of configuration parameters then you can combine it with the "optparse" module.

from optparse import OptionParser
_o = OptionParser("%prog [options] configfiles...")
_o.add_option("--hostname", metavar="HOSTNAME", default="10.0.0.1")
_o.add_option("--username", metavar="USERNAME", default="admin")
_o.add_option("--password", metavar="PASSWORD", default="admin")

import shlex
def parse(filename, defaults):
    opt, args = _o.parse_args(shlex.split(open(filename).read()), defaults)
    return opt

if __name__ == "__main__":
    import sys
    values, args = _o.parse_args()
    for arg in args:
       values = parse(arg, values)
    values, args = _o.parse_args(values = values)
    for name in _o.defaults:
        print name, "=", getattr(values, name)

The example shows how you can chain ini files for having a default set of values and user-defined redefinitions. So assume you have two files containing

file1.ini:

--hostname 10.2.3.4
--password admin-sc

file2.ini:

--username "foo bar"
--password "special key"

Then you can run ./configtest.py file1.ini file2.ini --password other and the resulting values will have hostname as 10.2.3.4 and username as "foo bar" and password as "other". This variation for configuration settings comes in handy if you do already have an optparse-definition for your program parameters -> just reuse it and you can chain the values from the command line with the values from a config file and possibly some global config settings.

As an incentive, your configuration parameters are always documented and mistyped configuration parameters will come up early as an error, just as you can use the optparse-instance to precheck your default settings file (schema check). As a downside, there are no comments allowed in the ini and configuration elements are not easily substructured.Still your parser is essentially a one-liner.

Answer 2

It depends on how the config file will be used.

One of the advantages of INI files is that they are really easy to read and understand. It's much easier to make a mistake in JSON or XML file if you edit config by hand. PHP still uses INI files.

However, if you your config is not meant to be edited by hand, go with any format you like, because INI is not the easiest one to parse.

Answer 3

Consider using plain Python files as configuration files.

An example (config.py):

# use normal python comments

value1 = 32
value2 = "A string value"

value3 = ["lists", "are", "handy"]
value4 = {"and": "so", "are": "dictionaries"}

In your program, load the config file using exec (docs):

from pathlib import Path

if __name__ == "__main__":
    config = {}
    exec(Path("config.py").read_text(encoding="utf8"), {}, config)
    
    print config["value1"]
    print config["value4"]

I like this approach, for the following reasons:

• In the simple case, the format is as easy to author as an INI-style config file. It also shares an important characteristic with INI files: it is very suitable for version control (this is less true for XML and maybe also for JSON)
• I like the flexibility that comes with having the config file in an actual programming language.

The approach is widely used, a few examples:

• A Django site's settings lives inside settings.py. Django does not use execfile, it uses import to read/execute settings.py AFAIK, but the end result is the same: the code inside the settings file is executed.
• The bash shell reads and executes ~/.bashrc on startup.
• The Python interpreter imports site.py on startup.

Answer 4

I ran into same problem more than once. I love the idea of python file(s) as settings files, it's just simple, elegant and all fellow python devs like it. Same time not really convinced with execfile idea.

So I went ahead and created Converge.

It supports some advanced options but at it's heart it simple python module as settings file.

It's as simple as

• creating default_settings.py, prod_settings.py
• pip install converge
• import settings

Answer 5

Dictionaries are pretty popular as well. Basically a hash table.

{"one": 1, "two": 2} is an example, kind of looks like json.

Then you can call it up like mydict["one"], which would return 1.

Then you can use shelve to save the dictionary to a file:

mydict = shelve.open(filename)
# then you can call it from there, like
mydict["one"]

So, it's somewhat easier then an ini file. You can add stuff just like a list or change options pretty easily and then once you close it, it will write it back out.

Heres a simple example of what i mean:

import shelve

def main():
    mydict = shelve.open("testfile")
    mydict["newKey"] = value("some comment", 5)
    print(mydict["newKey"].value)
    print(mydict["newKey"].comment)
    mydict.close()


class value():
    def __init__(self, comment, value):
        self.comment = comment
        self.value = value



if __name__ == '__main__':
    main()

Answer 6

INI is till totally OK and as other said, the format of your config file really depends from how you are going to use it.

Personally I am a fan of YAML: concise, readable, flexible.

Google seems to share my enthusiasm, as they use it too in the Google App Engine. The python parser is here.