In C++ the modulus operator is defined so that the following is true for all integers except for b == 0:
(a/b)*b + a%b == a
So it is forced to be consistent with the integer division, which from C++ 11 onwards truncates to zero even for negative numbers. Hence everything is well defined even for negative numbers.
However, in your case you have an signed / unsigned division (because .size() returns unsigned) and the usual signed/unsigned rules apply. This means that in this case all arguments are converted to unsigned before the operation is carried out (see also Ruslan's comment).
So -4 is converted to unsigned (and becomes a very large number) and then modulo is carried out.
You can also see this as 5 is not a correct answer for -4 modulo 7 with any definition of integer division (3 would be correct).
Arithmetic rules with C and C++ are not intuitive.