Python - reading files from directory file not found in subdirectory (which is there)

Question

I am convinced it is something simply syntactic - I however can not figure out why my code:

import os
from collections import Counter
d = {}
for filename in          


        

Answer 1

Change:

f = open(filename,'r')

To:

f = open(os.path.join('testfilefolder',filename),'r')

Which is effectively what you are doing in:

f = open("testfilefolder/file2.txt",'r')

Reason: you are listing the files in 'testfilefolder' (a subdirectory of your current directory) but then trying to open the file in your current directory.

Answer 2

As isedev pointed out, listdir() returns just the file names, not the full path (or relative paths). Another way to deal with this problem is to os.chdir() into the directory in question, then os.listdir('.').

Secondly, it seems your goal is to count frequency of words, not letters (characters). For that, you will need to break up the contents of the files into words. I prefer to use regular expression for this.

Thirdly, your solution counts words frequencies for each files separately. If you ever need to do it for all files, create a Counter() object in the beginning, then call the update() method to tally the counts.

Without further ado, my solution:

import collections
import re
import os

all_files_frequency = collections.Counter()

previous_dir = os.getcwd()
os.chdir('testfilefolder')
for filename in os.listdir('.'):
    with open(filename) as f:
        file_contents = f.read().lower()

    words = re.findall(r"[a-zA-Z0-9']+", file_contents) # Breaks up into words
    frequency = collections.Counter(words)              # For this file only
    all_files_frequency.update(words)                   # For all files
    print(frequency)

os.chdir(previous_dir)

print ''
print all_files_frequency