I am trying to understand what the object.GetHashCode() is used for. I read that it is used by collections to uniquely identify keys. But I wanted to test this
GetHashCode is only the first check, used to determine non-equality and possible equality. After that, Equals is checked. Which for objects defaults to reference-equality, and for structs is a memberwise compare. Override Equals to give an appropriate implementation (paired with the hash-code), and it should give the results you expect (duplicate key).
btw, the IDE is probably already giving you a warning that GetHashCode and Equals should always be treated together...