Before you say this is an existing question, or that all I need to use is ms-interpolation-mode, or that I just need to have a pre-scaled version of the image on the server
There seems to be two questions.
The first is about downscaled and the second is about downscaling image in canvas.
The first happens only on IE but the later happens to other browsers too.
You are doing well, but you can clarify your question more next time; both has been answered on SO and each has different solutions.
For , as discussed in other answers there is nothing you can do, except to provide a properly downscaled image.
For canvas, the question boils down to downscale quality with drawImage, and has already been answered:
HTML5 Canvas Resize (Downscale) Image High Quality?
Here is a sample, using the algorithm in that answer, that produce a nicely scaled image on IE 11: