EDIT - SOLUTION:
I ended up figuring out a way to solve this issue. Because manually changing the height of the ImageView removes the extra padding,
Both your layout and image view have layout_width="match_parent". This tells the view (either the layout or the image view) to stretch itself so that it fills the parent container. Try changing this property to wrap_content. You can find detailed explanation of the property in the official documentation - here
Setting it to wrap_content should have the same effect as on the height - it should be just wide enough to draw the content (the image in this case) instead of fixing its size (to that of the parent/container)