async and await: are they bad?

Question

We recently developed a site based on SOA but this site ended up having terrible load and performance issues when it went under load. I posted a question related this issue

Answer 1

There is a lot of stuff to discuss.

First of all, async/await can help you naturally when your application has almost no business logic. I mean the point of async/await is to do not have many threads in sleep mode waiting for something, mostly some IO, e.g. database queries (and fetching). If your application does huge business logic using cpu for 100%, async/await does not help you.

The problem of 900 threads is that they are inefficient - if they run concurrently. The point is that it's better to have such number of "business" threads as you server has cores/processors. The reason is thread context switching, lock contention and so on. There is a lot of systems like LMAX distruptor pattern or Redis which process data in one thread (or one thread per core). It's just better as you do not have to handle locking.

How to reach described approach? Look at disruptor, queue incoming requests and processed them one by one instead of parallel.

Opposite approach, when there is almost no business logic, and many threads just waits for IO is good place where to put async/await into work.

How it mostly works: there is a thread which reads bytes from network - mostly only one. Once some some request arrive, this thread reads the data. There is also limited thread pool of workers which processes requests. The point of async is that once one processing thread is waiting for some thing, mostly io, db, the thread is returned in poll and can be used for another request. Once IO response is ready, some thread from pool is used to finish the processing. This is the way how you can use few threads to server thousand request in a second.

I would suggest that you should draw some picture how your site is working, what each thread does and how concurrently it works. Note that it's necessary to decide whether throughput or latency is important for you.