how to “source” file into python script

Question

I have a text file /etc/default/foo which contains one line:

FOO=\"/path/to/foo\"

In my python script, I need to reference the

Answer 1

The Solution

Here is my approach: parse the bash file myself and process only variable assignment lines such as:

FOO="/path/to/foo"

Here is the code:

import shlex

def parse_shell_var(line):
    """
    Parse such lines as:
        FOO="My variable foo"

    :return: a tuple of var name and var value, such as
        ('FOO', 'My variable foo')
    """
    return shlex.split(line, posix=True)[0].split('=', 1)

if __name__ == '__main__':
    with open('shell_vars.sh') as f:
        shell_vars = dict(parse_shell_var(line) for line in f if '=' in line)
    print(shell_vars)

How It Works

Take a look at this snippet:

        shell_vars = dict(parse_shell_var(line) for line in f if '=' in line)

This line iterates through the lines in the shell script, only process those lines that has the equal sign (not a fool-proof way to detect variable assignment, but the simplest). Next, run those lines into the function parse_shell_var which uses shlex.split to correctly handle the quotes (or the lack thereof). Finally, the pieces are assembled into a dictionary. The output of this script is:

{'MOO': '/dont/have/a/cow', 'FOO': 'my variable foo', 'BAR': 'My variable bar'}

Here is the contents of shell_vars.sh:

FOO='my variable foo'
BAR="My variable bar"
MOO=/dont/have/a/cow
echo $FOO

Discussion

This approach has a couple of advantages:

• It does not execute the shell (either in bash or in Python), which avoids any side-effect
• Consequently, it is safe to use, even if the origin of the shell script is unknown
• It correctly handles values with or without quotes

This approach is not perfect, it has a few limitations:

• The method of detecting variable assignment (by looking for the presence of the equal sign) is primitive and not accurate. There are ways to better detect these lines but that is the topic for another day

• It does not correctly parse values which are built upon other variables or commands. That means, it will fail for lines such as:

  FOO=$BAR
  FOO=$(pwd)