recursively “normalize” filenames

问题

i mean getting rid of special chars in filenames, etc.

i have made a script, that can recursively rename files [http://pastebin.com/raw.php?i=kXeHbDQw]:

e.g.: before:

THIS i.s my file (1).txt

after running the script:

This-i-s-my-file-1.txt

Ok. here it is:

But: when i wanted to test it "fully", with filenames like this:

¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÂÃÄÅÆÇÈÊËÌÎÏÐÑÒÔÕ×ØÙUÛUÝÞßàâãäåæçèêëìîïðñòôõ÷øùûýþÿ.txt
áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&'()*+,:;<=>?@[\]^_`{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£.txt

it fails [http://pastebin.com/raw.php?i=iu8Pwrnr]:

$ sh renamer.sh directorythathasthefiles
mv: cannot stat `./áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&\'()*+,:;<=>?@[]^_`{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£': No such file or directory
mv: cannot stat `./áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&\'()*+,:;<=>?@[]^_`{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£': No such file or directory
mv: cannot stat `./áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&\'()*+,:;<=>?@[]^_`{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£': No such file or directory
mv: cannot stat `./áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&\'()*+,:;<=>?@[]^_`{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£': No such file or directory
mv: cannot stat `./áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&\'()*+,:;<=>?@[]^_`{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£': No such file or directory
mv: cannot stat `./áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&\'()*+,:;<=>?@[]^_`{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£': No such file or directory
mv: cannot stat `./áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&\'()*+,:;<=>?@[]^_`{|}~€‚ƒ„…†....and so on
$

so "mv" can't handle special chars.. :\

i worked on it for many hours..

does anyone has a working one? [that can handle chars [filenames] in that 2 lines too?]

回答1:

mv handles special characters just fine. Your script doesn't.

In no particular order:

You are using find to find all directories, and ls each directory separately.
1. Why use for DEPTH in... if you can do exactly the same with one command?
```
find -maxdepth 100 -type d
```
2. Which makes the arbitrary depth limit unnecessary
```
find -type d
```
3. Don't ever parse the output of ls, especially if you can let find handle that, too
```
find -not -type d
```
4. Make sure it works in the worst possible case:
```
find -not -type d -print0 | while read -r -d '' FILENAME; do
```
  This stops read from eating certain escapes and choking on filenames with new-line characters.
You are repeating the entire ls | replace cycle for every single character. Don't - it kills performance. Loop over ~~each directory~~ all files once, and just use multiple sed's, or multiple replacements in one sed command.
```
sed 's/á/a/g; s/í/i/g; ...'
```
(I was going to suggest sed 'y/áí/ai/', but unfortunately that doesn't seem to work with Unicode. Perhaps perl -CS -Mutf8 -pe 'y/áí/ai/' would.)
You're still thinking in ASCII: "other special chars - ASCII Codes 33.. ..255". Don't.
1. These days, most systems use Unicode in UTF-8 encoding, which has a much wider range of "special" characters - so big that listing them out one by one becomes pointless. (It is even multibyte - "e" is one byte, "ė" is three bytes.)
2. True ASCII has 128 characters. What you currently have in mind are the ISO 8859 character sets (sometimes called "ANSI") - in particular, ISO 8859-1. But they go all the way up to 8859-16, and only the "ASCII" part stays the same.
echo -n $(command) is rather useless.

There are much easier ways to find the directory and basename given a path. For example, you can do

directory=$(dirname "$path")
oldnname=$(basename "$path")
# filter $oldname
mv "$path" "$directory/$newname"

Do not use egrep to check for errors. Check the program's return code. (Like you already do with cd.)

And instead of filtering out other errors, do...

if [[ -e $directory/$newname ]]; then
    echo "target already exists, skipping: $oldname -> $newname"
    continue
else
    mv "$path" "$directory/$newname"
fi

The ton of sed 's/------------/-/g' calls can be changed to a single regexp:
```
sed -r 's/-{2,}/-/g'
```
The [ ]s in tr [foo] [bar] are unnecessary. They just cause tr to replace [ to [, and ] to ].

Seriously?

echo "$FOLDERNAME" | sed "s/$/\//g"

How about this instead?

echo "$FOLDERNAME/"

And finally, use detox.

回答2:

Try something like:

find . -print0 -type f | awk 'BEGIN {RS="\x00"} { printf "%s\x00", $0; gsub("[^[:alnum:]]", "-"); printf "%s\0", $0 }' | xargs -0 -L 2 mv

Use of xargs(1) will ensure that each filename passed exactly as one parameter. awk(1) is used to add new filename right after old one.

One more trick: sed -e 's/-+/-/g' will replace groups of more than one "-" with exactly one.

回答3:

Assuming the rest of your script is right, your problem is that you are using read but you should use read -r. Notice how the backslash disappeared:

áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&'()*+,:;<=>?@[\]^_`{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£.txt
áíüűúöőóéÁÍÜŰÚÖŐÓÉ!"#$%&\'()*+,:;<=>?@[]^_`{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£

回答4:

Ugh...

Some tips to clean up your script:

** Use sed to do translation on multiple characters at once, that'll clean things up and make it easier to manage:

dev:~$ echo 'áàaieeé!.txt' | sed -e 's/[áàã]/a/g; s/[éè]/e/g'
aaaieee!.txt

** rather than renaming the file for each change, run all your filters then do one move

$ NEWNAME='áàaieeé!.txt'
$ NEWNAME="$(echo "$NEWNAME" | sed -e 's/[áàã]/a/g; s/[éè]/e/g')"
$ NEWNAME="$(echo "$NEWNAME" | sed -e 's/aa*/a/g')"
$ echo $NEWNAME
aieee!.txt

** rather than doing a ls | read ... loop, use:

for OLDNAME in $DIR/*; do
  blah
  blah
  blah
done

** separate out your path traversal and renaming logic into two scripts. One script finds the files which need to be renamed, one script handles the normalization of a single file. Once you learn the 'find' command, you'll realize you can toss the first script :)

来源：https://stackoverflow.com/questions/4721249/recursively-normalize-filenames

标签

Linux

bash