why do we need cudaDeviceSynchronize(); in kernels with device-printf?

Question

__global__ void helloCUDA(float f)
{
    printf("Hello thread %d, f=%f\n", threadIdx.x, f);
}

int main()
{
    helloCUDA<<<1, 5>>>(1.2345f);
    cudaDeviceSynchronize();
    return 0;
}

Why is cudaDeviceSynchronize(); at many places for example here it is not required after kernel call?

Answer 1

A kernel launch is asynchronous. This means it returns control to the CPU thread immediately after starting up the GPU process, before the kernel has finished executing.

So what is the next thing in the CPU thread here? Application exit.

At application exit, it's ability to send output to the standard output is terminated by the OS.

Thus the output that is generated later by the kernel has nowhere to go, and you won't see it.

On the other hand, if you use cudaDeviceSynchronize(), then the kernel is guaranteed to finish (and the output from the kernel will find a waiting standard output queue), before the application is allowed to exit.