Splitting a large directory into smaller ones in Linux

问题

I have a large directory named as application_pdf which contains 93k files. My use-case is to split the directory into 3 smaller subdirectories (to a different location that the original large directory) containing around 30k files each. Can this be done directly from the commandline. Thanks!

回答1:

Using bash:

x=("path/to/dir1" "path/to/dir2" "path/to/dir3")
c=0
for f in *
do
    mv "$f" "${x[c]}"
    c=$(( (c+1)%3 ))
done

回答2:

If you have the rename command from Perl, you could try it like this:

rename --dry-run -pe 'my @d=("dirA","dirB","dirC"); $_=$d[$N%3] . "/$_"' *.pdf

In case you are not that familiar with the syntax:

• -p says to create output directories, à la mkdir -p
• -e says to execute the following Perl snippet
• $d[$N%3] selects one of the directories in array @d as a function of the serially incremented counter $N provided to the snippet by rename
• The output value is passed back to rename by setting $_

Remove the --dry-run if it looks good. Please run on a small directory with a copy of 8-10 files first, and make a backup before trying on all your 93k files.

Test

touch {0,1,2,3,4,5,6}.pdf

rename --dry-run -pe 'my @d=("dirA","dirB","dirC"); $_=$d[$N%3] . "/$_"' *.pdf
'0.pdf' would be renamed to 'dirB/0.pdf'
'1.pdf' would be renamed to 'dirC/1.pdf'
'2.pdf' would be renamed to 'dirA/2.pdf'
'3.pdf' would be renamed to 'dirB/3.pdf'
'4.pdf' would be renamed to 'dirC/4.pdf'
'5.pdf' would be renamed to 'dirA/5.pdf'
'6.pdf' would be renamed to 'dirB/6.pdf'

---

More for my own reference, but if you don't have the Perl rename command, you could do it just in Perl:

perl -e 'use File::Copy qw(move);my @d=("dirA","dirB","dirC"); my $N=0; @files = glob("*.pdf"); foreach $f (@files){my $t=$d[$N++%3] . "/$f"; print "Moving $f to $t\n"; move $f,$t}'

回答3:

Something like this might work:

for x in $(ls -1 originPath/*.pdf | head -30000); do
    mv originPath/$x destinationPath/
done

来源：https://stackoverflow.com/questions/45930989/splitting-a-large-directory-into-smaller-ones-in-linux