From the NodeJS documentation on TLS/SSL for Node v10.9.0 (2018-AUG)
https://nodejs.org/api/tls.html#tls_tls_ssl_concepts
openssl genrs
The problem here is DH keys are not RSA keys and not fully compatible.
Unfortunately, node does not have the ability to produce real RSA pairs via the crypto module either which is a bit disappointing. You'll need to either interact with your local openssl library to do this, or a third party module depending on your requirements.
As far as third party modules, keypair is a simple library which will work in your given situation
const keypair = require('keypair');
const keys: { private: string, public: string } = keypair({ bits : 2056 }); // 2056 is the default but added for example
I've also found good results with openpgpjs which is much more featured while also focusing on being a platform agnostic module. If you're looking at doing crypto in the browser as well as node, then this might be a good option.
As of Node.js v10.12.0, you can use crypto.generateKeyPair and crypto.generateKeyPairSync.
I have provided an example from the Node.js docs below (with added comments):
const { generateKeyPairSync } = require('crypto');
const { publicKey, privateKey } = generateKeyPairSync('rsa', {
modulusLength: 4096, // the length of your key in bits
publicKeyEncoding: {
type: 'spki', // recommended to be 'spki' by the Node.js docs
format: 'pem'
},
privateKeyEncoding: {
type: 'pkcs8', // recommended to be 'pkcs8' by the Node.js docs
format: 'pem',
cipher: 'aes-256-cbc', // *optional*
passphrase: 'top secret' // *optional*
}
});