I was wondering if I could pop up JOptionPanes or other Swing components from within a browser using JSP.
Assuming you're familiar with Swing, you may want to introduce yourself to Apache Wicket which is very similar to the way you build web pages as Swing is to building GUI:s. That is, in Wicket you add new instancef of Label, Button, DataTable etc. etc. to page and link those to bean property data which is then ultimately transformed to fully functioning web page.
As for the actual question, you really can't do it as-is. If choosing a new/different/another web framework to help you isn't possible, the only proper way I can think of is doing what sblundy suggested.