multicore

So I just finished watching this talk on the Python Global Interpreter Lock (GIL) http://blip.tv/file/2232410 . The gist of it is that the GIL is a pretty good design for single core systems (Python essentially leaves the thread handling/scheduling up to the operating system). But that this can seriously backfire on multi-core systems and you end up with IO intensive threads being heavily blocked by CPU intensive threads, the expense of context switching, the ctrl-C problem[*] and so on. So since the GIL limits us to basically executing a Python program on one CPU my thought is why not accept

I maintain a custom built CMS-like application. Whenever a document is submitted, several tasks are performed that can be roughly grouped into the following categories: MySQL queries. HTML content parsing. Search index updating. Category 1 includes updates to various MySQL tables relating to a document's content. Category 2 includes parsing of HTML content stored in MySQL LONGTEXT fields to perform some automatic anchor tag transformations. I suspect that a great deal of computation time is spent in this task. Category 3 includes updates to a simple MySQL-based search index using just a

I calculated vector using numpy. How can I calculate vector using multicore and numpy? import numpy as np num_row, num_col = 6000, 13572 ss = np.ones((num_row, num_col), dtype=np.complex128) ph = np.random.standard_normal(num_row) fre = np.random.standard_normal(num_row) tau = np.random.standard_normal(num_col) for idx in range(num_row): ss[idx, :] *= np.exp(1j*(ph[idx] + fre[idx]*tau)) We could leverage broadcasting to have a NumPy based solution - ss = np.exp(1j*(ph[:,None] + fre[:,None]*tau)) Porting this over to numexpr to leverage fast transcendental operations alongwith multi-core

I'm using Python 3 on Windows and trying to construct a toy example, demonstrating how using multiple CPU cores can speed up computation. The toy example is rendering of the Mandelbrot fractal. So far: I have avoided threading, since the Global Interpreter Lock prohibits multicores in this context I'm ditching example code that won't work on Windows because it lacks the forking capability of Linux Trying to use the "multiprocessing" package. I declare p=Pool(8) (8 is my number of cores) and using p.starmap(..) to delegate work. This is supposed to produce multiple "subprocesses" which windows