flatMap that shit! (as they say):
scala> val ys = xs flatMap {
| case _: B | _: C => None
| case other => Some(other)
| }
ys: List[A] = List(D@7ecdc97b, E@2ce07e6b, E@468bb9d1)
In your case you were getting a List[ScalaObject] because ScalaObject is the least upper bound of None, D, and E.