atomic create file if not exists from bash script

Question

In system call open(), if I open with O_CREAT | O_EXCL, the system call ensures that the file will only be created if it does not exist. The atomic

Answer 1

Just to be clear, ensuring the file will only be created if it doesn't exist is not the same thing as atomicity. The operation is atomic if and only if, when two or more separate threads attempt to do the same thing at the same time, exactly one will succeed and all others will fail.

The best way I know of to create a file atomically in a shell script follows this pattern (and it's not perfect):

1. create a file that has an extremely high chance of not existing (using a decent random number selection or something in the file name), and place some unique content in it (something that no other thread would have - again, a random number or something)
2. verify that the file exists and contains the contents you expect it to
3. create a hard link from that file to the desired file
4. verify that the desired file contains the expected contents

In particular, touch is not atomic, since it will create the file if it's not there, or simply update the timestamp. You might be able to play games with different timestamps, but reading and parsing a timestamp to see if you "won" the race is harder than the above. mkdir can be atomic, but you would have to check the return code, because otherwise, you can only tell that "yes, the directory was created, but I don't know which thread won". If you're on a file system that doesn't support hard links, you might have to settle for a less ideal solution.

Answer 2

You could create it under a randomly-generated name, then rename (mv -n random desired) it into place with the desired name. The rename will fail if the file already exists.

Like this:

#!/bin/bash

touch randomFileName
mv -n randomFileName lockFile

if [ -e randomFileName ] ; then
    echo "Failed to acquired lock"
else
    echo "Acquired lock"
fi

Answer 3

A 100% pure bash solution:

set -o noclobber
{ > file ; } &> /dev/null

This command creates a file named file if there's no existent file named file. If there's a file named file, then do nothing (but return a non-zero return code).

Pros wrt the touch command:

• Doesn't update timestamp if file already existed
• 100% bash builtin
• Return code as expected: fail if file already existed or if file couldn't be created; success if file didn't exist and was created.

Cons:

• need to set the noclobber option (but it's okay in a script, if you're careful with redirections, or unset it afterwards).

I guess this solution is really the bash counterpart of the open system call with O_CREAT | O_EXCL.

Answer 4

Another way to do this is to use umask to try to create the file and open it for writing, without creating it with write permissions, like this:

LOCK_FILE=only_one_at_a_time_please
UMASK=$(umask)
umask 777
echo "$$" > "$LOCK_FILE"
umask "$UMASK"
trap "rm '$LOCK_FILE'" EXIT

If the file is missing, the script will succeed at creating and opening it for writing, despite the file being created without writing permissions. If it already exists, the script won't be able to open the file for writing. It would be possible to use exec to open the file and keep the file descriptor around.

rm requires you to have write permissions to the directory itself, without regards to file permissions.

Answer 5

touch is the command you are looking for. It updates timestamps of the provided file if the file exists or creates it if it doesn't.

Answer 6

Here's a bash function using the mv -n trick:

function mkatomic() {
  f="$(mktemp)"
  mv -n "$f" "$1"
  if [ -e "$f" ]; then
    rm "$f"
    echo "ERROR: file exists:" "$1" >&2
    return 1
  fi
}

Examples:

$ mkatomic foo
$ wc -c foo
0 foo
$ mkatomic foo
ERROR: file exists: foo