We are having issues with NuGet package updates and TFS source control integration lately. This is causing allot of hassles with our team and making us hesitate in adopting
What's happening here (likely) is that you had a previous version (or the same version) of the .dll as a reference, and at SOME point (whether you did it or VS did it), it removed the reference. While you're not seeing the reference in the folder, the TFS / GIT checkin process is still waiting to "check in" the fact that you've removed it. So if you haven't checked in this fact, VS thinks it's still there, when in reality it's not. You can verify this in the Team Explorer area where you should be able to see the removed .dll (indicated by a line through it).
Kinda stupid, but that's how it goes.