I need a HashSet that preserves insertion ordering, are there any implementations of this in the framework?
If you need constant complexity of Add, Remove, Contains and order preservation, then there's no such collection in .NET Framework 4.5.
If you're okay with 3rd party code, take a look at my repository (permissive MIT license): https://github.com/OndrejPetrzilka/Rock.Collections
There's OrderedHashSet collection:
HashSet source code (from .NET Core)HashSetAdd and Remove operations are 20% slower compared to HashSet