android verify signature of file with .der public key

Question

I'm trying to verify the signature of a file. I followed these instruction to generate a certificate:

// generate a private key with size of 2048 bits
openssl genrsa -out private_key.pem 2048

// derive a public key from the above private key
openssl rsa -in private_key.pem -out public_key.pem -outform PEM -pubout

// iOS will not import PEM encoded data so it needs to be converted to DER encoded data
openssl rsa -pubin -inform PEM -outform DER -in public_key.pem -out public_key.der

// generate a self-signed certificate for testing
openssl req -new -x509 -key private_key.pem -out test_cert.pem -days 1095

// show the content of the original certificate
openssl x509 -in test_cert.pem -text -noout
// convert the certificate to DER format
openssl x509 -in test_cert.pem -outform der -out test_cert.der
// show the content of the new certificate
openssl x509 -in test_cert.der -inform der -text -noout

I used the example at this link to hash some data and verify that everything was all good (I did steps 2 - 5).

Now I'm trying to put the .der file, the signature file, and the data file into an Android application and basically verify it's all good again. I'm not getting any errors, but I'm not getting false.. Below is the code I've written:

import android.app.Activity;
import android.os.Bundle;
import android.os.Environment;
import android.util.Base64;
import android.util.Log;
import android.widget.TextView;

import java.io.*;
import java.security.*;
import java.security.cert.CertificateFactory;
import java.security.cert.Certificate;

// Most of the below is taken from:
// http://www.herongyang.com/JDK/Digital-Signature-JcaVerify-Signature-Verification-Program.html
public class MyActivity extends Activity {

String input = Environment.getExternalStorageDirectory() + "/data.txt";
String signFile = Environment.getExternalStorageDirectory() + "/signature";
String signAlgo = "SHA1withRSA";
int keyFile = R.raw.test_cert_josh;
String TAG = "VERIFY";

@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

    TextView textView = (TextView) findViewById(R.id.textView);

    try {
        PublicKey pubKey = readPublicKey(keyFile);
        byte[] sign = readSignature(signFile);
        textView.setText(verify(input, signAlgo, sign, pubKey) + "");
    } catch (Exception e) {
        e.printStackTrace();
    }
}

PublicKey readPublicKey(int cert_id) throws Exception {
    InputStream in = MyActivity.this.getResources().openRawResource(cert_id);
    byte[] buff = new byte[4000];
    int bytesRead;
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    while((bytesRead = in.read(buff)) != -1) {
        out.write(buff, 0, bytesRead);
        Log.i(TAG, "bytes read: " + bytesRead);
    }

    byte[] publicKeyBytes = out.toByteArray();

    CertificateFactory cf = CertificateFactory.getInstance("X509");
    Certificate cert = cf.generateCertificate(new ByteArrayInputStream(publicKeyBytes));

    PublicKey pubKey = cert.getPublicKey();
    Log.i(TAG, "Public Key Info: ");
    Log.i(TAG, "Algorithm = " + pubKey.getAlgorithm());
    Log.i(TAG, "toString = " + pubKey.toString());
    return pubKey;
}

byte[] readSignature(String input) throws Exception {
    FileInputStream signStream = new FileInputStream(input);
    byte[] signBytes = new byte[signStream.available()];
    signStream.read(signBytes);
    signStream.close();
    return  signBytes;
}

boolean verify(String input, String algorithm, byte[] sign, PublicKey pubKey) throws Exception {
    Signature sg = Signature.getInstance(algorithm);
    sg.initVerify(pubKey);
    Log.i(TAG, "Signature Object Info: ");
    Log.i(TAG, "Algorithm = "+sg.getAlgorithm());
    Log.i(TAG, "Provider = "+sg.getProvider());

    FileInputStream in = new FileInputStream(input);
    byte[] buff = new byte[in.available()];
    in.read(buff);

    sg.update(buff);

    boolean ok = sg.verify(sign);
    Log.i(TAG, "Verify Processing Info: ");
    Log.i(TAG, "Verification result = "+ok);
    return ok;
}
}

Everything I've read says my code is right. I think the part about getting the public key out is right because the modulus matches the one in the actual .der file.

Please help!

EDIT: I think it has to do with either how I'm reading the signature file or data file in. I'm pretty sure I'm getting everything about the public key correctly because i printed out additional information about it and it matches up to the openssl output

EDIT per Nikolay's suggestion below, I tried to sign the same data inside of the Android app that I was signing in openssl using this code:

// testing code to sign it myself
// read the data file in
FileInputStream dataStream = new FileInputStream(input);
byte[] dataBytes = new byte[dataStream.available()];
dataStream.read(dataBytes);
dataStream.close();

// hash the data file, like i do with openssl
// per Nikolay's comments, this is not needed
MessageDigest digest = MessageDigest.getInstance("SHA-1");
// digest.update(dataBytes, 0, dataBytes.length);
// dataBytes = digest.digest();
Log.i(TAG, "data from file: " + new String(dataBytes));

// read the private key in
FileInputStream fis = new FileInputStream(Environment.getExternalStorageDirectory() + "/phaero/private_key.pem");
byte[] keyBytes = new byte[fis.available()];
fis.read(keyBytes);
fis.close();

// clean up the private key and decode it
String temp = new String(keyBytes);
Log.i(TAG, "private key: " + temp);
String privKeyPEM = temp.replace("-----BEGIN RSA PRIVATE KEY-----\n", "");
privKeyPEM = privKeyPEM.replace("-----END RSA PRIVATE KEY-----", "");
byte[] decoded = Base64.decode(privKeyPEM.getBytes(), Base64.DEFAULT);

// create the private key object from the private key data
PKCS8EncodedKeySpec private_spec = new PKCS8EncodedKeySpec(decoded);
KeyFactory kf = KeyFactory.getInstance("RSA");
PrivateKey privateKey =  kf.generatePrivate(private_spec);

// set up for signing
Signature signer = Signature.getInstance(signAlgo);
signer.initSign(privateKey);
signer.update(dataBytes, 0, dataBytes.length);

// sign, and hash again so i can compare against openssl output
byte[] signed = signer.sign();
digest.update(signed, 0, signed.length);
Log.i(TAG, new BigInteger(1, digest.digest()).toString(16));

Quick note: using the above code in Jelly Bean, you have to specify "BC" when creating the KeyFactory.

Then I hashed the signature result to see if it would match the hash of the signature file generated with openssl, and they don't match.. so i don't know what that means, but I know it means something! I'm assuming I need to specify some kind of additional option when creating the signature file in openssl?

EDIT:

per Nikolay's comment below, I updated my test code to not sign, but the hash of the signature created in java still doesnt match the hash of the signature file created with openssl... not sure what's wrong, Java, OpenSSL, or me.

Answer 1

After A LOT of help from @NikolayElenkov, I finally figured out what was wrong. Trying a different google search, I stumbled upon this stackoverflow question, where the guy says there's two different signature commands you can run. When I was creating all my signature, I was using the stuff I linked to above:

// create a hash
echo 'data to sign' > data.txt
openssl dgst -sha1 < data.txt > hash
// sign it
openssl rsautl -sign -inkey private.pem -keyform PEM -in hash  > signature
// verify it
openssl rsautl -verify -inkey public.pem -keyform PEM -pubin -in signature > verified
diff -s verified hash

and from the post I found today, I tried:

openssl dgst -sha1 -sign privateKey.pem -out signature1 someInputFile

which, as the guy says, creates a different signature file. This is the one my Android code needed! So, the answer to get this to verify is I needed to change how I was generating my signature file! (I wouldn't have gotten this far without @NikolayElenkov, thanks a lot!)