Generally templates (or generics) are more type safe than casts. In that respect, i would say that an issue with casting is type-safety. However, there is another more subtle issue associated especially with downcasting: design. From my perspective at least, downcasting is a code smell, an indication that something might be wrong with my desing and i should investigate further. Why is simple: if you "get" the abstractions right, you simply don't need it! Nice question by the way...
Cheers!