Sounds like you want to set the OnCancelListener when you create the Dialog. It looks like this:
dialog.setOnCancelListener(new DialogInterface.OnCancelListener() {
@Override
public void onCancel(DialogInterface dialog) {
//do whatever you want the back key to do
}
});