In this question someone was having a layout problem because they had two floated divs inside a non-floated div. I suggested adding float: left to their outer
Overflow of anything but visible creates a new block formatting context which causes floats to be contained. It's standard behaviour.
Floats, absolutely positioned elements, inline-blocks, table-cells, table-captions, and elements with 'overflow' other than 'visible' (except when that value has been propagated to the viewport) establish new block formatting contexts.
In a block formatting context, boxes are laid out one after the other, vertically, beginning at the top of a containing block. The vertical distance between two sibling boxes is determined by the 'margin' properties. Vertical margins between adjacent block boxes in a block formatting context collapse.
In a block formatting context, each box's left outer edge touches the left edge of the containing block (for right-to-left formatting, right edges touch). This is true even in the presence of floats (although a box's line boxes may shrink due to the floats), unless the box establishes a new block formatting context (in which case the box itself may become narrower due to the floats).
Floating items removes them from the normal layout in many cases. It's not exactly like or unlike position: absolute; in that. Block items tend to ignore floated items (including block items that contain the item), but unlike position: absolute; items, floated ones are recognized and wrapped around by inline items, like text.
Having the wrapping item (div or otherwise) also be floated causes it to behave differently related to floated items it contains. Having the wrapping item set to overflow: hidden; forces it to consider the items it contains differently, as well. My guess is it's just a happy coincidence that the end result here appears to be the same. Not a quirk or bug... just how it works out.