Python move files from directories that match given criteria to new directory

Question

I have a directory that looks something like this:

.
├── files.py
├── homework
├── hw1
│   └── hw1.pdf
├── hw10
│   └── hw10.pdf
├── hw13
│   └── hw13.pdf
├── hw2
│   └── hw2.pdf
├── hw3
│   └── hw3.pdf
├── hw4
│   └── hw4.pdf
├── hw7
│   └── hw7.pdf
├── IntroductionToAlgorithms.pdf
├── p157
│   └── Makefile
├── p164
│   └── project
├── p171
│   ├── project
├── p18
│   └── project
├── p246
│   ├── project
├── p257
│   ├── project
├── p307
│   ├── project
├── p34
│   └── project
├── p363
│   ├── project
├── p431
│   ├── bit_buffer.h
├── p565
│   ├── project
├── p72
│   └── project
├── README.md
└── tree.txt

I want to move all the files inside the hwN folders into homework. Example homework will contain hw1.pdf -> hw13.pdf and not keep any of the folders named hwN Where N is one of the numbered homework folders.

I have a python script that is very nearly working:

files.py:

import os
import shutil

if not os.path.exists("homework"):
    os.makedirs("homework")
    print("created hw directory")

source='/home/kalenpw/Documents/School/2017Spring/CS3385/homework/'

files = os.listdir()

for f in files:
    if f.startswith("hw") and len(f) > 2:
        #This line works but it keeps the subfolders where I want the files directly in ./homework
        shutil.move(f, source)
#        for eachFile in os.listdir(f):
#           #Ideally this would move all the files within the hw folders and move just the file not the folder to my source
#            shutil.move(eachFile, source)

However, the commented out code which I am trying to use to move just the files not the folders results in this error:

Traceback (most recent call last):
  File "/usr/lib/python3.5/shutil.py", line 538, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: 'hw13.pdf' -> '/home/kalenpw/Documents/School/2017Spring/CS3385/homework/hw13.pdf'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "files.py", line 17, in <module>
    shutil.move(eachFile, source)
  File "/usr/lib/python3.5/shutil.py", line 552, in move
    copy_function(src, real_dst)
  File "/usr/lib/python3.5/shutil.py", line 251, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.5/shutil.py", line 114, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'hw13.pdf'

To summarize, how can I move all the files contained in hw1, hw2, etc. to ./homework without moving the folders themselves? If this is xy problem and there is in fact an easier way to do this please point me in that direction. Also yes I realize that in the amount of time I've spent debugging and writing this I could easily have done it by hand, but that isn't the point.

Thanks.

Answer 1

You're almost there. When you get to shutil.move(eachFile, source), 'eachFile' here is only the name of the file you want. For example, 'hw13.pdf'. So it will try to search for it in the root path, but there is no 'hw13.pdf' in the root (as the Exception message points out).

What you need to do is just join the name of the folder you're in to the name of the file you want to move:

for f in files:
    if f.startswith("hw") and len(f) > 2:
        for eachFile in os.listdir(f):
            filePath = os.path.join(f, eachFile)
            shutil.move(filePath, source)

Answer 2

try this:

from os import walk, path

source='/home/kalenpw/Documents/School/2017Spring/CS3385/homework/'

for (dirpath, dirnames, filenames) in walk(source):
    for file in filenames:
        shutil.move(path.join(dirpath,file), source)