I know that RSACryptoServiceProvider can encrypt with the public key, then it can be decrypted with the private key.
Is it possible to encrypt with the private key a
No. That's not how any public/private key encryption works. You can only encrypt with the public key, and only decrypt with the private key.
If you want to apply the private key to a message, maybe you're looking for a signature, rather than encryption? This is a different cryptographic scheme that can also use RSA keys.