问题
We have a recent performance bench mark that I am trying to understand. We have a large script that performance appears 50% slower on a Redhat Linux machine than a Windows 7 laptop where the specs are comparable. The linux machine is virtualized using kvm and has 4 cores assigned to it along with 16GB of memory. The script is not io intensive but has quite a few for loops. Mainly I am wondering if there are any R compile options that I can use to optimize or any kernel compiler options that might help to make this more comparable. Any pointers would be appreciated. I will try to get another machine and test it in using raw metal also for a better comparison.
These are the configure flags that I am using to compile R on the linux machine. I have experimented quite a bit and this seems to cut 12 seconds off the execution time in the green for larger data sets. Essentially I went from 2.087 to 1.48 seconds with these options.
./configure CFLAGS="-O3 -g -std=gnu99" CXXFLAGS="-O3 -g" FFLAGS="-O2 -g" LDFLAGS="-Bdirect,--hash-stype=both,-Wl,-O1" --enable-R-shlib --without-x --with-cairo --with-libpng --with-jpeglib
Update 1
The script has not been optimized yet. Another group is actually working on the script and we have put in requests to use the apply function but not sure how this explains the disparity in the times.
The top of the profile looks like this. Most of these functions will later be optimized using the apply functions but right now it is bench marked apples to apples on both machines.
"eval.with.vis" 8.66 100.00 0.12 1.39
"source" 8.66 100.00 0.00 0.00
"[" 5.38 62.12 0.16 1.85
"GenerateQCC" 5.30 61.20 0.00 0.00
"[.data.frame" 5.20 60.05 2.40 27.71
"ZoneCalculation" 5.12 59.12 0.02 0.23
"cbind" 3.22 37.18 0.34 3.93
"[[" 2.26 26.10 0.38 4.39
"[[.data.frame" 1.88 21.71 0.82 9.47
My first suspicion and I will be testing shortly and updating with my findings is that KVM linux virtualization is to blame. This script is very memory intensive and due to the large number of array operations and R being pass by copy ( which of course has to malloc ) this may be causing the problem. Since the VM does not have direct access to the memory controller and must share it with it's other VM's this could very likely cause the problem. I will be getting a raw machine later on today and will update with my findings.
Thank you all for the quick updates.
Update 2
We originally thought the cause of the performance problem was caused by hyper threading with a VM, but this turned out to be incorrect and performance was the same on a bare metal machine comparatively.
We later realized that the windows laptop is using a 32 bit version of R for computations. This led us to try the 64 bit version of R and the result was ~140% slower than 32 bit on the same exact same script. This leads me to the question of how is it possible that the 64 bit could be ~140% slower than the 32 bit version of R?
What we are seeing is that the 32
Windows 32 bit execution time 48 seconds Windows 64 bit execution time 2.33 seconds.
Linux 64 bit execution time 2.15 seconds. Linux 32 bit execution time < in progress > ( Built a 32 bit version on RHEL 6.3 x86_64 but did not see performance improvement am going to reload with 32 bit version of RHEL 6.3 )
I found this link but it only explains a 15-20% hit on some 64 bit machines.
http://www.hep.by/gnu/r-patched/r-admin/R-admin_51.html
Sorry I cannot legally post the script.
回答1:
Have a look at the sections on "profiling" in the Writing R Extensions manual.
From 30,000 feet, you can't say much else -- you will need profiling information. "General consensus" (and I am putting this is in quotes as you can't really generalize these things) is that Linux tends to do better on memory management and file access, so I am a little astonished by your results.
回答2:
Building R with --enable-R-shlib can cause a performance penalty. This is discussed in R Installation and Administration, in Appendix B, Section 1. That alone could explain 10-20% of the variation. Other sources could be from the differences of the "comparable specs".
回答3:
The issue was resolved and it was caused by a non optimized BLAS library.
This article was a great help. Using ATLAS was a great help.
http://www.cybaea.net/Blogs/Data/Faster-R-through-better-BLAS.html
来源：https://stackoverflow.com/questions/12692049/how-to-optimize-r-performance