You could put the two floated divs into another one that's got "overflow: hidden" set:
Main Data
edit — To add a bit to this 5-year-old answer: I think the cause of the confusing behavior is the somewhat complicated process of margin collapse. A good trick with the original HTML from the OP is to add a CSS rule like this:
div { border: 1px solid transparent; }
Poof! Now (without my additional edit again — For the complete (and, I think, completely accurate) story, see Mark Amery's excellent answer. The details have some complexity that this answer glosses over.clear: both works and the margin collapse rules that result in the unexpected layout from the code in the OP.