fortran: wait to open a file until closed by another application

Question

I have a fortran code which needs to read a series of ascii data files (which all together are about 25 Gb). Basically the code opens a given ascii file, reads the informati

Answer 1

I know this is an old thread but I've been struggling with the same issue for my own code.

My first attempt was creating a variable on a certain process (e.g. the master) and accessing this variable exclusively using one-sided passive MPI. This is fancy and works well, but only with newer versions of MPI.

Also, my code seemed happy to open (with READWRITE status) files that were also open in other processes.

Therefore, the easiest workaround, if your program has file access, is to make use of an external lock file, as described here. In your case, the code might look something like this:

• A process checks whether the lock file exists using the NEW statement, which fails if a file already exists. It will look something like:

  file_exists = .true.
  do while (file_exists)
      open(STATUS='NEW',unit=11,file=lock_file_name,iostat=open_stat)
      if (open_stat.eq.0) then
          file_exists = .false.
          open(STATUS='OLD',ACTION=READWRITE',unit=12,file=data_file_name,iostat=ierr)
          if (ierr.ne.0) stop
      else
          call sleep(1)
      end if
  end do
  

• The file is now opened exclusively by the current process. Do the operations you need to do, such as reading, writing.

• When you are done, close the data file and finally the lock file

  close(12,iostat=ierr)
  if (ierr.ne.0) stop
  close(11,status='DELETE',iostat=ierr)
  if (ierr.ne.0) stop
  

• The data file is now again unlocked for the other processes.

I hope this may be useful for other people who have the same problem.

Answer 2

Two processes should be able to read the same file. Perhaps action="read" on the open statement might help. Must the files be human readable? The I/O would very likely be much faster with unformatted (sometimes call binary) files.

P.S. If your OS doesn't support multiple-read access, you might have to create your own lock system. Create a master file that a process opens to check which files are in use or not, and to update said list. Immediately closing after a check or update. To handle collisions on this read/write file, use iostat on the open statement and retry after a delay if there is an error.