The output of ls -al /usr/bin/adb should show that it is owned by user root and group root. You can use Linux ACL (Access Control Lists) to give your local user permissions for adb as follows:
setfacl -m "u:userName:rwx" /usr/bin/adb
This is preferable to setting the SUID bit on /usr/bin/adb and also limits the users who can use adb to userName and root.
