I have put all my images for my admin theme in the assets folder within a folder called admin. Then I link to it like normal ie.
# Ruby
image_tag \"admin
In 3.1 you just get rid of the 'images' part of the path. So an image that lives in /assets/images/example.png will actually be accessible in a get request at this url - /assets/example.png
Because the assets/images folder gets generated along with a new 3.1 app, this is the convention that they probably want you to follow. I think that's where image_tag will look for it, but I haven't tested that yet.
Also, during the RailsConf keynote, I remember D2h saying the the public folder should not have much in it anymore, mostly just error pages and a favicon.