Another possible source of this issue is USB tethering. If you have used USB tethering, turn it off, then unplug the device from USB, plug it back in, then do
adb kill-server
adb devices
That did the trick in my case (Ubuntu 12.04, Nexus S, SDK in home dir, never needed root to get it running). Depending on your device, you may need to run adb devices as root, though.
