It's an Actor Model World, Really...
My view is that async / await are simply a way of dressing up software systems so as to avoid having to concede that, really, a lot of systems (especially those with a lot of network comms) are better seen as Actor model (or better yet, Communicating Sequential Process) systems.
With both of these the whole point is that you wait for one of several things to become complete-able, take the necessary action when one does, and then return to waiting. Specifically you're waiting for a message to arrive from somewhere else, reading it, and acting on the content. In *nix, the waiting is generally done with a call to epoll() or select().
Using await / async is simply a way of pretending that your system is still kinda synchronous method calls (and therefore familiar), whilst making it difficult to efficiently cope with things not consistently completing in the same order every time.
However, once you get over the idea that you're no longer calling methods but simply passing messages to and fro it all becomes very natural. It's very much a "please do this", "sure, here's the answer" thing, with many such interactions intertwined. Wrapping it up with a big WaitForLotsOfThings() call at the top of a loop is merely an explicit acknowledgement that your program will wait until it has something to do in response to many other programs communicating with it.
How Windows Makes it Hard
Unfortunately, Windows makes it very hard to implement a proactor system ("if you read that message now, you'll get it"). Windows is reactor ("that message you asked me to read? It's now been read."). It's an important distinction.
With the former, a message (or indeed a timeout) that means "stop listening to that other actor" is easily dealt with - you simply exclude that other actor from the list you'll listen to next time you wait.
With a reactor, it's a lot harder. How does one honour a "stop listening to that other actor" message when the read has already been started with some sort of async call, and won't complete until something is read, a doubtful outcome given the instruction recently received?
I'm nit-picking to some extent. Proactor is very useful in systems with dynamic connectivity, Actors dropping into the system, dropping out again. Reactor is fine if you have a fixed population of actors with comms links that'll never go away. Nevertheless, given that a reactor system is easily implemented in a proactor platform, but a proactor system cannot easily be implemented on a reactor platform (time won't go backwards), I find Window's approach particularly irritating.
So one way or other, async / await are definitely still in reactor land.
Knock on Impact
This has infected many other libraries.
C++'s Boost asio is also reactor, even on *nix, largely it seems because they wanted to have a Windows implementation.
ZeroMQ, which is an proactor framework, is limited to some extent on Windows being based on a call to select() (which in Windows works on only sockets).
For the cygwin family of POSIX runtimes on Windows, they had to implement select(), epoll(), etc. by having a thread per file descriptor polling (yes, polling!!!!) the underlying socket / serial port / pipe for incoming data in order to recreate POSIX's routines. Yeurk! The comments on the cygwin dev's mailing lists dating back to the time when they were implementing that part make for amusing reading.
Actor Isn't Necessarily Slow
It's worth noting that the phrase "passing messages" doesn't necessarily mean passing copies around - there's plenty of formulations of the Actor Model where you're merely passing ownership of references to messages around (e.g. Dataflow, part of the Task Parallel library in C#). This makes it fast. I've not yet got round to looking at the Dataflow library, but it doesn't really make Windows proactor all of a sudden. It doesn't give you an actor model proactor system working on all sorts of data bearers like sockets, pipes, queues, etc.
Windows 10's Linux Runtime
So having just blasted Windows and it's inferior reactor architecture, one intriguing point is that Windows 10 now runs Linux binaries. How, I'd very much like to know, has Microsoft implemented the system call that underlies select(), epoll() given that it has to function on sockets, serial ports, pipes and everything else in the land of POSIX that is a file descriptor, when everything else on Windows can't? I'd give my hind teeth to know the answer to that question.
