benchmarking

Spark: Inconsistent performance number in scaling number of cores

孤者浪人 提交于 2019-11-26 08:32:33
I am doing a simple scaling test on Spark using sort benchmark -- from 1 core, up to 8 cores. I notice that 8 cores is slower than 1 core. //run spark using 1 core spark-submit --master local[1] --class john.sort sort.jar data_800MB.txt data_800MB_output //run spark using 8 cores spark-submit --master local[8] --class john.sort sort.jar data_800MB.txt data_800MB_output The input and output directories in each case, are in HDFS. 1 core: 80 secs 8 cores: 160 secs I would expect 8 cores performance to have x amount of speedup. user6910411 Theoretical limitations I assume you are familiar Amdahl's

How to benchmark efficiency of PHP script

怎甘沉沦 提交于 2019-11-26 05:48:09
I want to know what is the best way to benchmark my PHP scripts. Does not matter if a cron job, or webpage or web service. I know i can use microtime but is it really giving me the real time of a PHP script? I want to test and benchmark different functions in PHP that do the same thing. For example, preg_match vs strpos or domdocument vs preg_match or preg_replace vs str_replace` Example of a webpage: <?php // login.php $start_time = microtime(TRUE); session_start(); // do all my logic etc... $end_time = microtime(TRUE); echo $end_time - $start_time; This will output: 0.0146126717 (varies all

Is MATLAB OOP slow or am I doing something wrong?

别说谁变了你拦得住时间么 提交于 2019-11-26 03:14:36
问题 I\'m experimenting with MATLAB OOP, as a start I mimicked my C++\'s Logger classes and I\'m putting all my string helper functions in a String class, thinking it would be great to be able to do things like a + b , a == b , a.find( b ) instead of strcat( a b ) , strcmp( a, b ) , retrieve first element of strfind( a, b ) , etc. The problem: slowdown I put the above things to use and immediately noticed a drastic slowdown. Am I doing it wrong (which is certainly possible as I have rather limited

What is microbenchmarking?

我怕爱的太早我们不能终老 提交于 2019-11-26 02:51:57
问题 I\'ve heard this term used, but I\'m not entirely sure what it means, so: What DOES it mean and what DOESN\'T it mean? What are some examples of what IS and ISN\'T microbenchmarking? What are the dangers of microbenchmarking and how do you avoid it? (or is it a good thing?) 回答1: It means exactly what it says on the tin can - it's measuring the performance of something "small", like a system call to the kernel of an operating system. The danger is that people may use whatever results they

Which is faster: multiple single INSERTs or one multiple-row INSERT?

China☆狼群 提交于 2019-11-26 02:51:53
I am trying to optimize one part of my code that inserts data into MySQL. Should I chain INSERTs to make one huge multiple-row INSERT or are multiple separate INSERTs faster? https://dev.mysql.com/doc/refman/8.0/en/insert-optimization.html The time required for inserting a row is determined by the following factors, where the numbers indicate approximate proportions: Connecting: (3) Sending query to server: (2) Parsing query: (2) Inserting row: (1 × size of row) Inserting indexes: (1 × number of indexes) Closing: (1) From this it should be obvious, that sending one large statement will save

Spark: Inconsistent performance number in scaling number of cores

扶醉桌前 提交于 2019-11-26 02:00:01
问题 I am doing a simple scaling test on Spark using sort benchmark -- from 1 core, up to 8 cores. I notice that 8 cores is slower than 1 core. //run spark using 1 core spark-submit --master local[1] --class john.sort sort.jar data_800MB.txt data_800MB_output //run spark using 8 cores spark-submit --master local[8] --class john.sort sort.jar data_800MB.txt data_800MB_output The input and output directories in each case, are in HDFS. 1 core: 80 secs 8 cores: 160 secs I would expect 8 cores

Why is Skylake so much better than Broadwell-E for single-threaded memory throughput?

ぃ、小莉子 提交于 2019-11-26 01:54:31
问题 We\'ve got a simple memory throughput benchmark. All it does is memcpy repeatedly for a large block of memory. Looking at the results (compiled for 64-bit) on a few different machines, Skylake machines do significantly better than Broadwell-E, keeping OS (Win10-64), processor speed, and RAM speed (DDR4-2133) the same. We\'re not talking a few percentage points, but rather a factor of about 2 . Skylake is configured dual-channel, and the results for Broadwell-E don\'t vary for dual/triple/quad

Execution time of C program

送分小仙女□ 提交于 2019-11-26 01:48:16
问题 I have a C program that aims to be run in parallel on several processors. I need to be able to record the execution time (which could be anywhere from 1 second to several minutes). I have searched for answers, but they all seem to suggest using the clock() function, which then involves calculating the number of clocks the program took divided by the Clocks_per_second value. I\'m not sure how the Clocks_per_second value is calculated? In Java, I just take the current time in milliseconds

How to Calculate Execution Time of a Code Snippet in C++

偶尔善良 提交于 2019-11-26 01:46:26
问题 I have to compute execution time of a C++ code snippet in seconds. It must be working either on Windows or Unix machines. I use code the following code to do this. (import before) clock_t startTime = clock(); // some code here // to compute its execution duration in runtime cout << double( clock() - startTime ) / (double)CLOCKS_PER_SEC<< \" seconds.\" << endl; However for small inputs or short statements such as a = a + 1, I get \"0 seconds\" result. I think it must be something like 0

Which is faster: multiple single INSERTs or one multiple-row INSERT?

安稳与你 提交于 2019-11-26 01:05:33
问题 I am trying to optimize one part of my code that inserts data into MySQL. Should I chain INSERTs to make one huge multiple-row INSERT or are multiple separate INSERTs faster? 回答1: https://dev.mysql.com/doc/refman/8.0/en/insert-optimization.html The time required for inserting a row is determined by the following factors, where the numbers indicate approximate proportions: Connecting: (3) Sending query to server: (2) Parsing query: (2) Inserting row: (1 × size of row) Inserting indexes: (1 ×