In my case the MIME type was correct but my server was not allowing the client to request partial files with the Range HTTP header.
So the server must send "Accept-Ranges: bytes" when you request that mp4 file, the iPad will refuse to download that file if it is forced to download it entirely.