Add a color factor to a grouped ggplot without regrouping

Question

I have a dataset with multiple groups that I would like to leave with colors by group, but from some points, based on a separate factor meeting a condition, I would like to