How do you use the crypt library in C for DES encryption? (setkey, encrypt, crypt, etc.)

Question

I need to do some simple DES encryption in C to interface with some old code. From what I understand you can use the "crypt" library for this, with the functions setkey, encrypt, crypt, etc. I have been messing with it and can't get it right. The example on the man page for setkey/encrypt is lacking.

I want to get the same output as I would be able to get with some java code (see below).

So let's say I have two character arrays in C.

char *message = "hellothe";
char *key = "iamakey0";

Can someone give an example of how to encrypt these with setkey/encrypt and get the same result as I would from the java code? I realize you have to put message and key into a 64-byte array where each char represents a bit, but some of that is confusing too. Apparently you have to get the bit parity right on that too or something?

public static byte[] encryptDES(byte[] message, byte[] key) {
    byte[] encrypted = new byte[0];
    try{
        Cipher c = Cipher.getInstance("DES");
        c.init(Cipher.ENCRYPT_MODE,new SecretKeySpec(key,"DES"));
        encrypted = c.doFinal(message);
    }
    catch (Exception e) {
        e.printStackTrace();
    }
    return encrypted;
}

Answer 1

Because you're calling Cipher.getInstance with just the string "DES", you're not specifying a cipher mode or padding method. This means you get the defaults, which depends on what Java cryptography provider you're using - you need to know exactly what they are to write compatible C. (You really should be specifying them rather than relying on defaults).

If you're using the SunJCE provider, then the defaults for DES are ECB mode and PKCS #5 padding. The best way to do this is probably to use OpenSSL or another meaty crypto library - but if you want to use functions that are usually found in the standard C library on UNIX-type platforms, the ecb_crypt family of functions is going to be a lot easier to work with than the setkey / encrypt family.

You will need to add PKCS#5 padding when encrypting, and check it (and discard it) when decrypting). The following ecb_pkcs5_encrypt function should do the rough equivalent of the above Java code using those functions.

/* Returns a newly-allocated buffer containing the contents of `data',
 * padded out to a multiple of 8 bytes using PKCS #5 style padding.
 *
 * If `padded_len` is non-NULL, the value it points to is updated to
 * the size of the padded output data.
 *
 * Returns NULL on error.
 */
char *pad_pkcs5(const char *data, size_t data_len, size_t *padded_len)
{
    char *padded_data;
    unsigned padding_len = 8 - (data_len % 8);
    const char padding = padding_len;
    char *pad_ptr;

    /* check for length overflow */
    if (data_len + padding_len < data_len)
        return NULL;

    /* Copy data into a new buffer and pad it out */
    padded_data = malloc(data_len + padding_len);

    if (!padded_data)
        return NULL;

    memcpy(padded_data, data, data_len);

    if (*padded_len)
    {
        *padded_len = data_len + padding_len;
    }

    /* Add the padding bytes */
    pad_ptr = padded_data + data_len;
    while (padding_len--)
    {
        *pad_ptr++ = padding;
    }

    return padded_data;
}

/* Returns a newly-allocated buffer containing the contents of `data',
 * encrypted with `key' using DES/ECB/PKCS5.
 *
 * If `out_len` is non-NULL, the value it points to is updated to
 * the size of the encrypted output data (which will always be a
 * multiple of 8).
 *
 * Returns NULL on error.
 */
char *ecb_pkcs5_encrypt(const char *key, const char *data, size_t data_len, size_t *out_len)
{
    char des_key[8];
    char *padded_data;
    size_t padded_len;
    int status;

    /* One of the few cases where strncpy() is exactly what we want! */
    strncpy(des_key, key, sizeof des_key);
    des_setparity(des_key);

    padded_data = pad_pkcs5(data, data_len, &padded_len);

    if (!padded_data)
        return NULL;

    status = ecb_crypt(des_key, padded_data, padded_len, DES_ENCRYPT);

    if (DES_FAILED(status))
        return NULL;

    if (out_len)
        *out_len = padded_len;

    return padded_data;
}

Answer 2

Don't use crypt(). It uses some non-standard algorithm so it will be very difficult to interoperate with other systems. Besides, DES is not secure anyway.

I suggest you use OpenSSL in C. Most of its ciphers are compatible with JCE.

If you really have to use crypt, Sun's JRE comes with a class to handle crypt,

   com.sun.security.auth.module.Crypt

This is internal class so the documentation isn't there. Just read the source code.