I installed the nvidia-cuda-toolkit package:
$ sudo apt install nvidia-cuda-toolkit
and it worked.
I did not find the solution nor on the tensorflow website nor on the nvidia installation page. I found it by luck while looking for a way to get the cuda version with a command line: How to get the cuda version?