The result of expression !x has type bool (in C++) and equal to false if x is not equal to 0 and true otherwise.
In your case !x will be equal to false because x == 5. Then this value (false) is converted to type int in the assignment statement and x becomes equal to 0.
Maybe you meant the operator ~? That is
x = ~x;
I was speaking about C++. In C the value of operator is either 0 or 1 corresponding to false and true in C++ and the result has type int.
