How to use > in an xargs command?

Question

I want to find a bash command that will let me grep every file in a directory and write the output of that grep to a separate file. My guess would have been to do something

Answer 1

I assume your example is just an example and that you may need > for other things. GNU Parallel http://www.gnu.org/software/parallel/ may be your rescue. It does not need additional quoting as long as your filenames do not contain \n:

ls | parallel "grep ABC {} > {}.out"

If you have filenames with \n in it:

find . -print0 | parallel -0 "grep ABC {} > {}.out"

As an added bonus you get the jobs run in parallel.

Watch the intro videos to learn more: http://pi.dk/1

The 10 seconds installation will try to do a full installation; if that fails, a personal installation; if that fails, a minimal installation:

$ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
   fetch -o - http://pi.dk/3 ) > install.sh
$ sha1sum install.sh | grep 67bd7bc7dc20aff99eb8f1266574dadb
12345678 67bd7bc7 dc20aff9 9eb8f126 6574dadb
$ md5sum install.sh | grep b7a15cdbb07fb6e11b0338577bc1780f
b7a15cdb b07fb6e1 1b033857 7bc1780f
$ sha512sum install.sh | grep 186000b62b66969d7506ca4f885e0c80e02a22444
6f25960b d4b90cf6 ba5b76de c1acdf39 f3d24249 72930394 a4164351 93a7668d
21ff9839 6f920be5 186000b6 2b66969d 7506ca4f 885e0c80 e02a2244 40e8a43f
$ bash install.sh

If you need to move it to a server, that does not have GNU Parallel installed, try parallel --embed.

Answer 2

A solution without xargs is the following:

find . -mindepth 1 -maxdepth 1 -type f -exec sh -c "grep ABC '{}' > '{}.out'" \;

...and the same can be done with xargs, it turns out:

ls -1 | xargs -I {} sh -c "grep ABC '{}' > '{}.out'"

---

Edit: single quotes added after remark by lhunath.

Answer 3

Do not make the mistake of doing this:

sh -c "grep ABC {} > {}.out"

This will break under a lot of conditions, including funky filenames and is impossible to quote right. Your {} must always be a single completely separate argument to the command to avoid code injection bugs. What you need to do, is this:

xargs -I{} sh -c 'grep ABC "$1" > "$1.out"' -- {}

Applies to xargs as well as find.

By the way, never use xargs without the -0 option (unless for very rare and controlled one-time interactive use where you aren't worried about destroying your data).

Also don't parse ls. Ever. Use globbing or find instead: http://mywiki.wooledge.org/ParsingLs

Use find for everything that needs recursion and a simple loop with a glob for everything else:

find /foo -exec sh -c 'grep "$1" > "$1.out"' -- {} \;

or non-recursive:

for file in *; do grep "$file" > "$file.out"; done

Notice the proper use of quotes.