I think this is one of the writing or printing mistake. Because if we can not compile this code as it holds a compiletime error. And here this is inside if clause, the return type must be boolean. So how can it possible. The answer is simply a mistake or as Mike said obfuscated way of writing.
And only possible if both variables a and b are boolean which will be compiled as a is equal to (not)b i.e (a = !b) .