Multi-Core and Concurrency - Languages, Libraries and Development Techniques [closed]

Answer 1

You mentioned Java, but you only mention threads. Have you looked at Java's concurrent library? It comes bundled with Java 5 and above.

It's a very nice library containing ThreadPools, CopyOnWriteCollections to name a very few. Check out the documentation at the Java Tutorial. Or if you prefer, the Java docs.

Answer 2

Java has an actors library too you know. And did you know that Java is a functional language? ;)

Answer 3

C++0x will provide std::lock functions for locking more than one mutex together. This will help alleviate deadlock due to out-of-order locking. Also, the C++0x thread library will have promises, futures and packaged tasks, which allow a thread to wait for the result of an operation performed on another thread without any user-level locks.

Answer 4

I've used processing for Python. It mimicks the API of the threading module and is thus quite easy to use.

If you happen to use map/imap or a generator/list comprehension, converting your code to use processing is straightforward:

def do_something(x):
    return x**(x*x)

results = [do_something(n) for n in range(10000)]

can be parallelized with

import processing
pool = processing.Pool(processing.cpuCount())
results = pool.map(do_something, range(10000))

which will use however many processors you have to calculate the results. There are also lazy (Pool.imap) and asynchronous variants (Pool.map_async).

There is a queue class which implements Queue.Queue, and workers that are similar to threads.

Gotchas

processing is based on fork(), which has to be emulated on Windows. Objects are transferred via pickle/unpickle, so you have to make sure that this works. Forking a process that has acquired resources already might not be what you want (think database connections), but in general it works. It works so well that it has been added to Python 2.6 on the fast track (cf. PEP-317).

Answer 5

multiprocessing is a python library that simplifies multi-core programming, as mentionned in another answer.

Program written with python's multiprocessing can easily be modified to ship work on the cloud, instead of to local cores. piCloud takes advantage of that to provide large, on-demand processing power on the cloud: you just need to modify 2 lines of your code.

So, here is the take-away: when selecting a library for multi-core, one may want to ask whether a cloud approach would also make sense.

Answer 6

Intel's Threading Building Blocks for C++ looks very interesting to me. It offers a much higher level of abstraction than raw threads. O'Reilly has a very nice book if you like dead tree documentation. See, also, Any experiences with Intel’s Threading Building Blocks?.