I solved this by putting two CardViews in a RelativeLayout. One with background of the border color and the other one with the image. (or whatever you wish to use)
Note the margin added to top and start for the second CardView. In my case I decided to use a 2dp thick border.