what is posix compliance for filesystem?

Question

Posix compliance is a standard that is been followed by many a companies. I have few question around this area, 1. does all the file systems need to be posix compliant? 2. a

Answer 1

In the area of "requires POSIX filesystem semantics" what is typically meant is:

• allows hierarchical file names and resolution (., .., ...)
• supports at least close-to-open semantics
• umask/unix permissions, 3 filetimes
• 8bit byte support
• supports atomic renames on same filesystem
• fsync()/dirfsync() durability gurantee/limitation
• supports multi-user protection (resizing file returns 0 bytes not previous content)
• rename and delete open files (Windows does not do that)
• file names supporting all bytes beside '/' and \0

Sometimes it also means symlink/hardlink support as well as file names and 32bit file pointers (minimum). In some cases it is also used to refer specific API features like fcntl() locking, mmap() or truncate() or AIO.

Answer 2

Answering your questions in a very objective way:

1. does all the file systems need to be posix compliant? Actually not. In fact POSIX defines some standards for operational systems in general. Good to have, but no really required.

2. are applications also required to be posix compliant? No.

3. are there any non posix filesystems? HDFS (hadoop file system)

Answer 3

When I think about POSIX compliance for distributed file systems, I use the general standard that a distributed file system is POSIX compliant if multiple processes running on different nodes see the same behavior as if they were running on the same node using a local file system. This basically has two implications:

1. If the system has multiple buffer-caches, it needs to ensure cache consistency.
   • Various mechanisms to do so include locks and leases. An example of incorrect behavior in this case would be a writer who writes successfully on one node but then a reader on a different node receives old data.
   • Note however that if the writer/reader are independently racing one another that there is no correct defined behavior because they do not know which operation will occur first. But if they are coordinating with each other via some mechanism like messaging, then it would be incorrect if the writer completes (especially if it issues a sync call), sends a message to the reader which is successfully received by the reader, and then the reader reads and gets stale data.
2. If data is striped across multiple data servers, reads and writes that span multiple stripes must be atomic.
   • For example, when a reader reads across stripes at the same time as a writer writes across those same stripes, then the reader should either receive all stripes as they were before the write or all stripes as they were after the write. Incorrect behavior would be for the reader to receive some old and some new.
   • Contrary to the above, this behavior must work correctly even when the writer/reader are racing.

Although my examples were reads/writes to a single file, correct behavior also includes write/writes to a single file as well as read/writes and write/writes to the hierarchical namespace via calls such as stat/readdir/mkdir/unlink/etc.