Detecting that log file has been deleted or truncated on POSIX systems?

Question

Suppose a long-running process writes to a log file. Suppose that log file is kept open indefinitely. Suppose that a careless system administrator deletes that log file. Can the program detect that this has happened?

Is it safe to assume that fstat() will report a link count of zero for a deleted file?

Truncation, it seems to me, is slightly trickier. In part, it depends on whether the file descriptor is running in O_APPEND mode. If the log file is not running with O_APPEND, then the current write position of the program's log descriptor doesn't change, and the truncation removes the leading bytes, but the program continues to write at 'the end', leaving a gap of phantasmal zero bytes (they read as zeroes, but don't necessarily occupy space on disk).

If the program runs with O_APPEND, then it will write at the end of the file as it currently exists. The only way to observe the truncation is to note that the file position is not where the program expected it - which in turn means tracking that position explicitly.

On the whole, I'm not so worried about truncation as deletion, but any thoughts would be welcome.

Answer 1

Checking that fstat() returns a link count of zero would fail if the file was hard linked or renamed. I would probably instead periodically compare stat()'s inode number against fstat()'s.

I'm not sure about truncation.

tail -F checks for deletion and maybe truncation, so I'd check its source to see how it implements it.

Answer 2

Suppose the careless system administrator kills the process. Do you really want to protect against the admin doing random things? I guess you're just looking for a way to start a new logfile from time to time, like using logrotate. There it is enough to provide a way to manually let the program reopen the logfile. The standard way to do this is to listen for the HUP-Signal in the program, and reopen the logfile if it arrives:

#include <signal.h>

volatile int f_sighup;

void sighup_handler() {
  f_sighup = 1;
}

void trap_sighup() {
  struct sigaction sa;
  int rv;

  memset(&sa, 0, sizeof(struct sigaction));
  sa.sa_handler = &sighup_handler;
  rv = sigaction(SIGHUP, &sa, NULL);
  if (-1 == rv) {
    fprintf(stderr, "warning: setting SIGHUP signal handler failed");
  }
}

int main() {
  f_sighup = 0;
  trap_sighup();
  ...
}

Then regularly check the f_sighup flag in the main program to see if the logfile should be reopened. This plays nice with tools like logrotate, which can rename the old logfile and then call kill -s HUP $PID. And the careless sysadmin can do this manually, after deleting (or better renaming) the old logfile.

Answer 3

You could use inotify to watch your log file, monitoring it for file system events.

Answer 4

In response to søren-holm's answer

When a file is closed the modification time is changed.

that doesn't appear to be correct:

import os
from time import sleep

TMPF = '/tmp/f'

def print_stats():
    print("%s, %s" % (os.stat(TMPF).st_mtime, os.stat(TMPF).st_ctime))
    sleep(1.1)

print("Opening...")
with open(TMPF, 'w') as f:
    print_stats()
    print("Writing...")
    os.write(f.fileno(), 'apple')
    print_stats()
    print("Flushing...")
    f.flush()
    print_stats()
    print("Closing...")

print_stats()

Produces:

Opening...
1483052647.08, 1483052647.08
Writing...
1483052648.18, 1483052648.18
Flushing...
1483052648.18, 1483052648.18
Closing...
1483052648.18, 1483052648.18

Admittedly, there's a bit of Python magic going on in there; that write() is not reasonably guaranteed to be flushed automatically, but the point stands that mtime is updated when the file is modified, not when the file is closed. The behavior of ctime is going to depend on your filesystem and its mount options.

Answer 5

When a file is closed the modification time is changed. So check periodically on the mtime using stat() will work.