openmp - while loop for text file reading and using a pipeline

Question

I discovered that openmp doesn\'t support while loops( or at least doesn\'t like them too much). And also doesn\'t like the \' != \' operator.

I have this bit of cod

Answer 1

First, even though it is very close, but openmp doesn't magically make your code parallel. It works with for because for has lower and upper bounds that it can understand. Openmp uses those bounds to divide work among different threads.

There is no such thing possible with a while loop.

Second, how do you expect your task to be parallelized? You are reading from a file, where probably sequential access is going to give you better performance than parallel access. You might parallelize segment_read (based on its implementation).

Alternatively, you may want to overlap file read with processing. For that, you need to use more low level functions such as Unix's open and read functions. Then, do asynchronous reads, meaning you send a read request, process the last read block and then wait for the read request to finish. Search for "linux asynchronous io" for example to read more on this.

Using a pipe might not actually help you much. That would depend on many internals of the pipe that I'm not very familiar with. However, if you have a big enough memory, you may also want to consider loading the whole data first, then processing it. That way, loading the data is done as fast as possible (sequentially) and then you can parallelize its processing.