I usually use ggplot2, but in this case I am using the regular image() function to plot a heatmap of a large data set. I can label all the labels as re
Accepting @thelatemail's answer as the most flexible, but it also turns out to be pretty simple using text() if you add xpd = TRUE to allow plotting outside the frame. Using mtext() can also work, but it doesn't allow you to rotate labels.
grid = structure(c(1:20),.Dim = c(4,5))
labs = c("A","B","C","D","E")
redlabs = c("B","D")
colorlist = c("black","red")
# one of many ways to generate the color labels
axiscolor = colorlist[labs %in% redlabs +1 ]
image(1:4,1:5,grid,axes=FALSE, xlab="", ylab = "")
axis(2,at=1:length(labs),labels=FALSE)
# This would work for sideways labels
# mtext(text=labs, side=2,at=1:length(labs),col=axiscolor,adj=.5)
text(labels=labs, col=axiscolor, x=rep(.45,length(labs)), y=1:length(labs), srt = 0, pos = 2, xpd = TRUE)

UPDATE for ggplot2: You can use theme() and element_text to set the colors and other parameters. Something like this...
p + theme(axis.text.y = element_text(color=axiscolor))