This has a lot to do with covariance, e.g., generic types are considered as parameters, and if the parameters do not resolve properly to a more specific type then the operation fails. The implication of such is that you really cannot cast to a more general type like object. And as stated by Rex, the List object won't convert each object for you.
You might want to try the ff code instead:
List sl = new List();
//populate sl
List
or:
List ol = new List();
ol.AddRange(sl);
ol will (theoretically) copy all the contents of sl without problems.