Dunno if you've seen this thread in the GWT Contributor's forum...
Basically, it starts with the same problem you've identified, proposes some new compiler flags, and goes on to show how to use some JSNI to get around the casts.
Edit In the GWT trunk there's a new compiler flag. See the wiki...