What I am trying to achieve is to have two control modes, a free \"flying\" one, and an object-centered one (trackball) and with a button press seamlessly switch between the
Making a new camera is the easiest way to unbind all the keyboard/mouse events that were setup with the previous controls. If you don't do this you'll have orbitcontrols events fire with fly mode on.