The system might be generating a serializer for the whole of mscorlib, which could be very slow.
You could probably avoid this by wrapping the system type in your own type and serialising that instead - then you'd get a serializer for your own assembly.
You might be able to build the serializer for mscorlib with sgen.exe, which was the old way of building serializer dlls before it got integrated into VS.