I searched through many posts and figured out that JLabel supports HTML. So I can do
JLabel
JLabel search = new JLabel(\"Search
Slightly simpler HTML than seen in @MadProgrammer's answer:
new JLabel("SearchBy:");