I have a Nexus 4 with Android 4.3 and I am trying to connect the device to a computer with Windows 7 64bit.
I installed the latest drivers and the latest adb>
Had the same issue. Not sure if these are the same steps for Windows as I'm using an OS X device but you can try:
Open the terminal and type:
cd ~/.android
adb push adbkey.pub /data/misc/adb/adb_keys
All done! Just adb shell reboot
and feel the power!