In our application we have a lot of sensitive configuration settings, which we are storing in a xml file which is again encrypted.
This secure file has to be decrypt
You should use the Machine Keystore, it's a secure storage especially for this purpose. For example:
CspParameters cspParams = new CspParameters(PROV_RSA_FULL, null, KEYNAME);
cspParams.Flags = CspProviderFlags.UseMachineKeyStore;
RSACryptoServiceProvider rsa = new RSACryptoServiceProvider(cspParams);
Where KEYNAME is a custom string that can be used to retrieve the key later on.
For more examples, see this question: How to store a public key in a machine-level RSA key container