Is C++11 atomic<T> usable with mmap?

Question

I want to add network control of a handful of parameters used by a service (daemon) running on a Linux embedded system. There's no need for procedure calls, each parameter can be polled in a very natural way. Shared memory seems a nice way to keep networking code out of the daemon, and limit shared access to a carefully controlled set of variables.

Since I don't want partial writes to cause visibility of values never written, I was thinking of using std::atomic<bool> and std::atomic<int>. However, I'm worried that std::atomic<T> might be implemented in a way that only works with C++11 threads and not with multiple processes (potentially, not even with OS threads). Specifically, if the implementation uses any data structures stored outside the shared memory block, in a multi-process scenario this would fail.

I do see some requirements which suggest to be that std::atomic won't hold an embedded lock object or pointer to additional data:

The atomic integral specializations and the specialization atomic<bool> shall have standard layout. They shall each have a trivial default constructor and a trivial destructor. They shall each support aggregate initialization syntax.

There shall be pointer partial specializations of the atomic class template. These specializations shall have standard layout, trivial default constructors, and trivial destructors. They shall each support aggregate initialization syntax.

Trivial default construction and destruction seems to me to exclude associated per-object data, whether stored inside the object, via a pointer member variable, or via an external mapping.

However, I see nothing that excludes an implementation from using a single global mutex / critical section (or even a global collection, as long as the collection elements aren't associated with individual atomic objects -- something along the lines of a cache association scheme could be used to reduce false conflicts). Obviously, access from multiple processes would fail on an implementation using a global mutex, because the users would have independent mutexes and not actually synchronize with each other.

Is an implementation of atomic<T> allowed to do things that are incompatible with inter-process shared memory, or are there other rules that make it safe?

---

I just noticed that trivial default construction leaves the object in a not-ready state, and a call to atomic_init is required. And the Standard mentions initialization of locks. If these are stored inside the object (and dynamic memory allocation seems impossible, since the destructor remains trivial) then they would be shared between processes. But I'm still concerned about the possibility of a global mutex.

In any case, guaranteeing a single call to atomic_init for each variable in a shared region seems difficult... so I suppose I'll have to steer away from the C++11 atomic types.

Answer 1

I'm two months late, but I'm having the exact same problem right now and I think I've found some sort of an answer. The short version is that it should work, but I'm not sure if I'd depend on it.

Here's what I found:

• The C++11 standard defines a new memory model, but it has no notion of OS-level "process", so anything multiprocessing-related is non-standard.

• However, section 29.4 "Lock-free property" of the standard (or at least the draft I have, N3337) ends with this note:

  [ Note: Operations that are lock-free should also be address-free. That is, atomic operations on the same memory location via two different addresses will communicate atomically. The implementation should not depend on any per-process state. This restriction enables communication by memory that is mapped into a process more than once and by memory that is shared between two processes. — end note ]

  This sounds very promising. :)

• That note appears to come from N2427, which is even more explicit:

  To facilitate inter-process communication via shared memory, it is our intent that lock-free operations also be address-free. That is, atomic operations on the same memory location via two different addresses will communicate atomically. The implementation shall not depend on any per-process state. While such a definition is beyond the scope of the standard, a clear statement of our intent will enable a portable expression of class of a programs already extant.

  So it appears that yes, all lock-free operations are supposed to work in this exact scenario.

• Now, operations on std::atomic<type> are atomic but they may or may not be lock-free for particular type, depending on capabilities of the platform. And We can check any variable x by calling x.is_lock_free().

• So why did I write that I would not depend on this? I can't find any kind of documentation for gcc, llvm or anyone else that's explicit about this.

Answer 2

Until C++11, the standard did not specify how multiple threads share memory, so we wrote programs with multiple threads that relied on implementation-specific behavior. The standard still doesn't specify how processes with shared memory - or if you prefer, threads that only partially share memory - interact. Whatever you end up doing you will be relying on implementation-specific guarantees.

That said, I think an implementation that supports process-shared memory will try to make its thread synchronization mechanisms like atomics usable in process-shared memory for process synchronization. At the very least, I think it would be hard to devise a lock-free implementation of a std::atomic specialization that does not work correctly cross-process.