Yes, it's entirely possible. On some machines, a pointer to a byte contains two values: A pointer to the WORD address of the memory word containing the byte, and a "byte index" that gives the position of the byte within the word. E.g. on a 32-bit machine, the "byte index" is 0..3.
This would require more storage space than a "int *", which is just a pointer to the relevant word.