EDIT - SOLUTION:
I ended up figuring out a way to solve this issue. Because manually changing the height of the ImageView removes the extra padding,
Here are a few things you can try. You try variations of this to see if it fixes the view ratio. Description
imageView.setScaleType(ScaleType.FIT_CENTER);
or possibly manually setting the ImageView size manually will help.
LinearLayout.LayoutParams layoutParams = new LinearLayout.LayoutParams(bitmapWidth, bitmapHeight);
image.setLayoutParams(layoutParams);
If not, there are a few other ways, these were just the first that came to mind.