That title might not be very accurate but here is the situation:
I solved this issue by adding clearfix elements where they should be. I wanted 3 columns on md and 2 columns on sm and this is how I did it:
So i used clearfix visible-sm after every second div and clearfix visible-md after every third div. I don't find this solution ideal, but it works rather well.
EDIT:
As of Bootstrap v3.2.0 .visible-* classes are deprecated.
http://getbootstrap.com/css/#responsive-utilities:
The classes .visible-xs, .visible-sm, .visible-md, and .visible-lg also exist, but are deprecated as of v3.2.0. They are approximately equivalent to .visible-*-block, except with additional special cases for toggling -related elements.
EDIT 2: This solution works as long as you do not want to edit CSS, if you have the option to do so, I recommend you use Jonas's answer as it is much simpler in my opinion.