The golden rule is to set debuggable option to off prior to releasing your Android application to the public.
What would happen if I leave (forget to turn off) this
@askmo: you can use some tools in the SDK to check if an APK has the debug value. Check the following link:
http://lulachronicles.blogspot.com/2011/04/how-to-check-if-apk-has-flag.html
BR,
Ignacio