multicore

We've just bought a 32-core Opteron machine, and the speedups we get are a little disappointing: beyond about 24 threads we see no speedup at all (actually gets slower overall) and after about 6 threads it becomes significantly sub-linear. Our application is very thread-friendly: our job breaks down into about 170,000 little tasks which can each be executed separately, each taking 5-10 seconds. They all read from the same memory-mapped file of size about 4Gb. They make occasional writes to it, but it might be 10,000 reads to each write - we just write a little bit of data at the end of each of

I have read a lot about Hadoop and Map-Reduce running on clusters of machines. Does some one know if the Apache distribution can be run on an SMP with several cores. In particular, can multiple Map-Reduce processes be run on the same machine. The scheduler will take care of spreading them across multiple cores. Thanks. - KG Yes. You have multiple map and reduce slots in each machine which are determined by the RAM and CPU (each JVM instance needs 1GB by default so a 8GB machine with 16 cores should still have 7 task slots) from hadoop wiki Use the configuration knob: mapred.tasktracker.map

Are there any packages specifically to let R run faster via parallel computing? I have made a very large IP that needs to run for a while, so I was wondering if there was a specific package in R that could help me run my IP. Currently, I have a function that returns the solution of an IP and the primary line that R gets stuck on (for a very...very long time) is when I use lp (....all.int = TRUE). My CPU is around 12.5% (8 cores) on my Windows computer, and I want it to near 100 Edit: I tried using the doParallel package, library('doParallel') cl <- makeCluster(8) registerDoParallel(cl) But my

Consider a case where we have multiple processor/cores and two threads. Is it possible to force the linux scheduler to always schedule the specific thread(both) to a specific processor at every instance of its execution. Is setting processor affinity to the threads, while creation, sufficient for this purpose If you look at the man page for taskset you can see the following statement: The Linux scheduler will honor the given CPU affinity and the process will not run on any other CPUs. This means that setting the CPU affinity for a particular process will make sure that it's always run on that

Any multi-core sorting implementation in .NET? Here's a multi-threaded QuickSort I put together a while back using async / await . Under a certain sort size, it "reverts" back to an elementary sort known as Double-Ended Selection Sort: public static class SortExtensions { /// <summary> /// Sorts the list. /// </summary> /// <typeparam name="T"> /// The IComparable element type of the list. /// </typeparam> /// <param name="list">The list to sort.</param> public static async Task SortIt<T>(this IList<T> list) where T : IComparable<T> => await SortIt(list, 0, list.Count - 1); /// <summary> ///