I'm not a DBA, but from what I recall learning from our DBA, the reasoning (at least with SQL Server) is that the DB caching algorithms don't cache '*' queries well, but if you are running the same query with exact columns specified multiple times, it will cache that well.
I'm sure a more knowledgeable DBA could go into the exact details of how the caching mechanism works, but that's why there is a performance hit.
NOTE: Caching performance only works if the query is going to be run multiple times, especially in a small time frame, so otherwise you would see no performance difference.
