Move semantics replace copy semantics in situations where copying is inefficient. Copy semantics deals fully with copyable objects, including const objects.
Already,
Sorry, but the premise is flawed.
An unordered_set doesn't actually hold const objects. It's just not giving you write access to the contained elements. That's a property of the accessors only.
It would be possible to add an key erase(iterator) function which just moves the element out to a temporary. I'm not sure why you'd want a key&& there.
As for const MyClass&& ~MyClass() const, that doesn't make sense for three reasons: dtors have neither return types nor CV classification, nor is overload resolution done for them.