Reading file using relative path in python project

Question

Say I have a python project that is structured as follows:

project
    /data
        test.csv
    /package
        __init__.py
        module.py
    main.py

__init__.py:

from .module import test

module.py:

import csv

with open("..data/test.csv") as f:
    test = [line for line in csv.reader(f)]

main.py:

import package

print(package.test)

When I run main.py I get the following error:

 C:\Users\Patrick\Desktop\project>python main.py
Traceback (most recent call last):
  File "main.py", line 1, in <module>
    import package
  File "C:\Users\Patrick\Desktop\project\package\__init__.py", line 1, in <module>
    from .module import test
  File "C:\Users\Patrick\Desktop\project\package\module.py", line 3, in <module>
    with open("../data/test.csv") as f:
FileNotFoundError: [Errno 2] No such file or directory: '../data/test.csv'

However, if I run module.py from the package directory I get no errors. So it seems that the relative path used in open(...) is only relative to where the originating file is being run from (i.e __name__ == "__main__")? I don't want to use absolute paths. What are some ways to deal with this?

Answer 1

Relative paths are relative to current working directory. If you do not your want your path to be, it must be absolute.

But there is an often used trick to build an absolute path from current script: use its __file__ special attribute:

import csv
import os.path

my_path = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(my_path, "../data/test.csv")
with open(path) as f:
    test = list(csv.reader(f))

Note, from python 3.4, __file__ is always absolute for imported modules and you can drop the os.path.abspath part in this example. Not that it is strictly necessary, but it avoids surprises if you change the current working directory at some point and your module was imported using a relative path.

Answer 2

For Python 3.4+:

import csv
from pathlib import Path

base_path = Path(__file__).parent
file_path = (base_path / "../data/test.csv").resolve()

with open(file_path) as f:
    test = [line for line in csv.reader(f)]