prefetch

I am working on data prefetch in CUDA (Fermi GPU) through C code. Cuda reference manual talks about the prefetching at ptx level code not at C level code. Can anyone connect me with some documents or something regarding prefetching through cuda code (cu file). Any help would be appreciated. According to PTX manual here is how prefetch works in PTX: You can embed the PTX instructions into the CUDA kernel. Here is a tiny sample from NVIDIA's documentation : __device__ int cube (int x) { int y; asm("{\n\t" // use braces for local scope " .reg .u32 t1;\n\t" // temp reg t1, " mul.lo.u32 t1, %1, %1;

I want to prefetch some code into the instruction cache. The code path is used infrequently but I need it to be in the instruction cache or at least in L2 for the rare cases that it is used. I have some advance notice of these rare cases. Does _mm_prefetch work for code? Is there a way to get this infrequently used code in cache? For this problem I don't care about portability so even asm would do. The answer depends on your CPU architecture. That said, if you are using gcc or clang, you can use the __builtin_prefetch instruction to try to generate a prefetch instruction. On Pentium 3 and

In my application, at one point I need to perform calculations on a large contiguous block of memory data (100s of MBs). What I was thinking was to keep prefetching the part of the block my program will touch in future, so that when I perform calculations on that portion, the data is already in the cache. Can someone give me a simple example of how to achieve this with gcc? I read _mm_prefetch somewhere, but don't know how to properly use it. Also note that I have a multicore system, but each core will be working on a different region of memory in parallel. gcc uses builtin functions as an

I am trying to vectorize a loop, computing dot product of a large float vectors. I am computing it in parallel, utilizing the fact that CPU has large amount of XMM registers, like this: __m128* A, B; __m128 dot0, dot1, dot2, dot3 = _mm_set_ps1(0); for(size_t i=0; i<1048576;i+=4) { dot0 = _mm_add_ps( dot0, _mm_mul_ps( A[i+0], B[i+0]); dot1 = _mm_add_ps( dot1, _mm_mul_ps( A[i+1], B[i+1]); dot2 = _mm_add_ps( dot2, _mm_mul_ps( A[i+2], B[i+2]); dot3 = _mm_add_ps( dot3, _mm_mul_ps( A[i+3], B[i+3]); } ... // add dots, then shuffle/hadd result. I heard that using prefetch instructions could help

It appears the general logic for prefetch usage is that prefetch can be added, provided the code is busy in processing until the prefetch instruction completes its operation. But, it seems that if too much of prefetch instructions are used, then it would impact the performance of the system. I find that we need to first have the working code without prefetch instruction. Later we need to various combination of prefetch instruction in various locations of code and do analysis to determine the code locations that could actually improve because of prefetch. Is there any better way to determine

The GCC doc here specifies the usage of _buitin_prefetch. Third argument is perfect. If it is 0, compiler generates prefetchtnta (%rax) instruction If it is 1, compiler generates prefetcht2 (%rax) instruction If it is 2, compiler generates prefetcht1 (%rax) instruction If it is 3 (default), compiler generates prefetcht0 (%rax) instruction. If we vary third argument the opcode already changed accordingly. But second argument do not seem to have any effect. __builtin_prefetch(&x,1,2); __builtin_prefetch(&x,0,2); __builtin_prefetch(&x,0,1); __builtin_prefetch(&x,0,0); The above is the sample