When you use ternary operator,
flag ? type1 : type2
Type1 and type2 must be of same type while conversion. First it realises type1 and then type2.
Now look at your cases
final Integer b = false ? 0 : a;
Since type1 is 0 and it takes as a primitive and since a is trying to convert it as a primitive. Hence the null pointer.
where as same tricky test5
final Integer b = false ? a : 0;
Since a is of type Integer 0 boxed to wrapper integer and assigned to the LHS.