The golden rule is to set debuggable option to off prior to releasing your Android application to the public.
What would happen if I leave (forget to turn off) this
It's also possible that a malicious user could learn more about the inner-workings of your app then you'd like them to.
One good practice is to link debugging mode specifically to your unique device id.