Decrypting file in C++, which was encrypted with openssl -aes-128-cbc

Question

I'm trying to decrypt a file in C++. This file is encrypted with the following command:

openssl enc -nosalt -aes-128-cbc -pass pass:test -in "test.txt" -out "test_enc.txt" -p

The console shows the key=098F6BCD4621D373CADE4E832627B4F6 and iv=0A9172716AE6428409885B8B829CCB05.

In C++ I have included the #include openssl/aes.h line and try to decrypt with the following code:

const char *indata = string.toAscii().constData();

unsigned char outdata[strlen(indata)];

unsigned char ckey[] = "098F6BCD4621D373CADE4E832627B4F6";
    unsigned char ivec[] = "0A9172716AE6428409885B8B829CCB05";

    /* data structure that contains the key itself */
    AES_KEY key;

    /* set the encryption key */
    AES_set_decrypt_key(ckey, 256, &key);

    AES_cbc_encrypt((unsigned char*) indata, outdata, strlen(indata), &key, ivec, AES_DECRYPT);

    QString result = QString((const char*) outdata);

    return result;

The variable outdata contains different value than before encryption with OpenSSL.

Answer 1

You specify -aes-128-cbc as an option on OpenSSL so the key and initialization vector will be 128 bits long. openssl prints these out as hex strings, as they would be obfuscated on the console if printed binary.

Therefor you should initialize your ckey[] and ivec[] as the binary value of the hex strings like this:

unsigned char ckey[] = "\x09\x8F\x6B\xCD\x46\x21\xD3\x73\xCA\xDE\x4E\x83\x26\x27\xB4\xF6";
unsigned char ivec[] = "\x0A\x91\x72\x71\x6A\xE6\x42\x84\x09\x88\x5B\x8B\x82\x9C\xCB\x05";

and also, use key length 128 instead of 256 in:

AES_set_decrypt_key(ckey, 128, &key);

Answer 2

OpenSSL creates the key using the password you offer, and also the vector you specified is related to decryption and encryption. Make sure you have the same key and vector while decrypting the text.