First off - good on you for thinking about this problem!
Is it possible to instead generate a private key, communicate with your Certificate Authority and have it sign the key (and manage a CRL as well)?
As an alternative, if this is going to be running on Windows, you can use the Crypto API to securely store a private key that is marked as not-exportable. How you distribute that key securely can be another challenge though.
