Ater compiling libjpeg and then using it, I stumbled upon the problem:
JPEG parameter struct mismatch: library thinks size is 512, caller expects 492
In my case, the problem was that the library and my program were compiled with different values of HAVE_BOOLEAN, so boolean type had different size.
Library was compiled with HAVE_BOOLEAN not defined, so it was using int as defined in libjpeg. Program was compiled with HAVE_BOOLEAN defined, and because some other headers were included before libjpeg, it was using unsigned char as defined in Windows RTL.
Reason I used HAVE_BOOLEAN was to fix compile error about this, but it then caused a linker error. Instead, the proper solution was to not include Windows RTL (or any file that includes it) in the file that uses libjpeg.