A good reference on a development process that keeps trunk stable and does all work in branches is Divmod's Ultimate Quality Development System. A quick summary:
- All work done must have a ticket associated with it
- A new branch is created for each ticket where the work for that ticket is done
- Changes from that branch are not merged back into the mainline trunk without being reviewed by another project member
They use SVN for this, but this could easily be done with any of the distributed version control systems.
