By using a good builder the development will be a lot faster.
The anxiety you have of not being able to modify the code may be due to you have a lot of logic in the view where it does not belong. Actually this change may help you to move logic from the view which is good precisely for the ability to change the visual elements with out breaking the code.
As per the "unreadable" you should consider use a better GUI generator. I've heard only positive things about NetBeans, and I have used IntelliJ GUI builder whose code is pretty clean.
On both the source code is readable and editable, but only for minor tweaks. For major ones, well use the GUI builder.
I think for a small number of forms, there is no problem in proceeding by hand, but since you have "many of them" as you just said, using this tools will be most beneficial.
