Why is the floated element overlapping the element below and not the other way round?

Answer 1

Why so ?

Because that is how float works.

When you are putting float on an element, you are not actually saying much about how that element itself should displayed - but rather how/that the following content should flow around it.

To be more precise, the following inline content.

Since your paragraphs are block elements, and you have also fixed their width and height to 50px, this is the result you get - the inline content of the #p2 paragraph does flow around the #p1 paragraph; but width and height restrictions of the parent paragraph don't allow that parent to grow accordingly.

---

Edit:
If you're having trouble understanding what I am trying to say, consider the following simpler example:

#floated {
  float:left;
  width: 50px;
  height: 50px;
  border:1px solid red;
}

#unfloated {
  border:1px solid blue;
  background: green;
}

Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged.

As you can easily see, without a background for the floated element, the second, non-floated div element stretches across the whole width, "under" the floated element, and only its inline content, the Lorem Ipsum text, floats around the floated element.

That is how float works.

In your example, you see the "B" show up in the second square - because the parent block element of that B is limited to 50px in width and height, so it can not grow to encompass the "B", the "B" text simply flows out of its parent element dimensions as is standard with the default overflow:visible.