How do you make good use of multicore CPUs in your PHP/MySQL applications?

Question

I maintain a custom built CMS-like application.

Whenever a document is submitted, several tasks are performed that can be roughly grouped into the following categori

Answer 1

PHP is not quite oriented towards multi-threading : as you already noticed, each page is served by one PHP process -- that does one thing at a time, including just "waiting" while an SQL query is executed on the database server.

There is not much you can do about that, unfortunately : it's the way PHP works.

Still, here's a couple of thoughts :

• First of all, you'll probably have more that 1 user at a time on your server, which means you'll serve several pages at the same time, which, in turn, means you'll have several PHP processes and SQL queries running at the same time... which means several cores of your server will be used.
  • Each PHP process will run on one core, in response to the request of one user, but there are several sub-processes of Apache running in parallel (one for each request, up to a couple of dozens or hundreds, depending on your configuration)
  • The MySQL server is multi-threaded, which means it can use several distinct cores to answer several concurrent requests -- even if each request cannot be served by more that one core.

So, in fact, your server's 8 core will end up being used ;-)

And, if you think your pages are taking too long to generate, a possible solution is to separate your calculations in two groups :

• On one hand, the things that have to be done to generate the page : for those, there is not much you can do
• On the other hand, the things that have to be run sometimes, but not necessarily immediately
  • For instance, I am think about some statistics calculations : you want them to be quite up to date, but if they lag a couple of minutes behind, that's generally quite OK.
  • Same for e-mail sending : anyway, several minutes will pass before your users receive/read their mail, so there is no need to send them immediately.

For the kind of situations in my second point, as you don't need those things done immediately... Well, just don't do them immediately ;-)
A solution that I often use is some queuing mechanism :

• The web application store things in a "todo-list"
• And that "todo-list" is de-queued by some batches that are run frequently via a cronjob

And for some other manipulations, you just want them run every X minutes -- and, here too, a cronjob is the perfect tool.