printf slows down my program

Question

I have a small C program to calculate hashes (for hash tables). The code looks quite clean I hope, but there\'s something unrelated to it that\'s bugging me.

I can e

Answer 1

Unbuffered output is very slow.

By default stdout is fully-buffered, however when attached to terminal, stdout is either unbuffered or line-buffered.

Try to switch on buffering for stdout using setvbuf(), like this:

char buffer[8192];

setvbuf(stdout, buffer, _IOFBF, sizeof(buffer));

Answer 2

I guess the terminal type is using some buffered output operations, so when you do a printf it does not happen to output in split micro-seconds, it is stored in the buffer memory of the terminal subsystem.

This could be impacted by other things that could cause a slow down, perhaps there's a memory intensive operation running on it other than your program. In short there's far too many things that could all be happening at the same time, paging, swapping, heavy i/o by another process, configuration of memory utilized, maybe memory upgrade, and so on.

It might be better to concatenate the strings until a certain limit is reached, then when it is, write it all out at once. Or even using pthreads to carry out the desired process execution.

Edited: As for 2,3 it is beyond me. For 4, I am not familiar with Sun, but do know of and have messed with Solaris, There may be a kernel option to use a virtual tty.. i'll admit its been a while since messing with the kernel configs and recompiling it. As such my memory may not be great on this, have a root around with the options to see.

user@host:/usr/src/linux $ make; make menuconfig **OR kconfig if from X**

This will fire up the kernel menu, have a dig around in to see the video settings section under the devices sub-tree..

Edited: but there's a tweak you put into the kernel by adding a file into the proc filesystem (if a such thing does exist), or possibly a switch passed into the kernel, something like this (this is imaginative and does not imply it actually exists), fastio

Hope this helps, Best regards, Tom.

Answer 3

If you are printf()ing to the console it's usually extremely slow. I'm not sure why but I believe it doesn't return until the console graphically shows the outputted string. Additionally you can't mmap() to stdout.

Writing to a file should be much faster (but still orders of magnitude slower than computing a hash, all I/O is slow).

Answer 4

As I/O is always much slower than CPU computation, you might store all values in fastest possible I/O first. So use RAM if you have enough, use Files if not, but it is much slower than RAM.

Printing out the values can now be done afterwards or in parallel by another thread. So the calculation thread(s) may not need to wait until printf has returned.

Answer 5

You can try to redirect output in shell from console to a file. Using this, logs with gigabytes in size can be created in just seconds.

Answer 6

1. I/O is always slow in comparison to straight computation. The system has to wait for more components to be available in order to use them. It then has to wait for the response before it can carry on. Conversely if it's simply computing, then it's only really moving data between the RAM and CPU registers.

2. I've not tested this, but it may be quicker to append your hashes onto a string, and then just print the string at the end. Although if you're using C, not C++, this may prove to be a pain!

3 and 4 are beyond me I'm afraid.