The thing I've been thinking about, is aren't most divide-and-conquer algorithms massively parallelizable? Every split should be able to be run in two separate threads...
Anyway, I'm concerned when I need to be concerned. When my program starts getting slow, then I'll look for ways to speed it up. Unfortunately, this is a problem in my line of work.
