Argument list too long error for rm, cp, mv commands

Question

I have several hundred PDFs under a directory in UNIX. The names of the PDFs are really long (approx. 60 chars).

When I try to delete all PDFs together using the fol

Answer 1

tl;dr

It's a kernel limitation on the size of the command line argument. Use a for loop instead.

Origin of problem

This is a system issue, related to execve and ARG_MAX constant. There is plenty of documentation about that (see man execve, debian's wiki).

Basically, the expansion produce a command (with its parameters) that exceeds the ARG_MAX limit. On kernel 2.6.23, the limit was set at 128 kB. This constant has been increased and you can get its value by executing:

getconf ARG_MAX
# 2097152 # on 3.5.0-40-generic

Solution: Using for Loop

Use a for loop as it's recommended on BashFAQ/095 and there is no limit except for RAM/memory space:

Dry run to ascertain it will delete what you expect:

for f in *.pdf; do echo rm "$f"; done

And execute it:

for f in *.pdf; do rm "$f"; done

Also this is a portable approach as glob have strong and consistant behavior among shells (part of POSIX spec).

Note: As noted by several comments, this is indeed slower but more maintainable as it can adapt more complex scenarios, e.g. where one want to do more than just one action.

Solution: Using find

If you insist, you can use find but really don't use xargs as it "is dangerous (broken, exploitable, etc.) when reading non-NUL-delimited input":

find . -maxdepth 1 -name '*.pdf' -delete 

Using -maxdepth 1 ... -delete instead of -exec rm {} + allows find to simply execute the required system calls itself without using an external process, hence faster (thanks to @chepner comment).

References

• I'm getting "Argument list too long". How can I process a large list in chunks? @ wooledge
• execve(2) - Linux man page (search for ARG_MAX) ;
• Error: Argument list too long @ Debian's wiki ;
• Why do I get “/bin/sh: Argument list too long” when passing quoted arguments? @ SuperUser