Detecting hair in a portrait image?

What would be the best approach for detecting and removing a person's hair in a simple portrait image ? Any useful libraries of algorithms ? I have been looking over openCV which looks like it could be of some use

You're dealing with two different problems here:

detecting if a face in a portrait has hair
"removing" the hair

The first is solvable fairly easily:

Separate the face from the background (as you've mentioned a "simple portrait image", this shouldn't be too hard).
Convert your image to the Y'CbCr color space
Human skin has a fairly narrow range of chrominance values, regardless of race. Check out this paper for the details.
The approach above will help you separate skin areas of the face from non-skin areas
Assume that the non-skin areas consist of hair. Note that facial hair will get picked up as a non-skin area, too.

As far as the second problem goes, you need to clarify exactly what you mean by "removing":

Are you simply cutting out the part of the portrait that has hair? In this case, the solution follows directly from the detection method above.
Are you trying to make it look like the person has no hair at all (e.g. is bald, clean-shaven?) In this case, things will be a lot harder -- there's a reason why professional photo manipulators get paid well.

I think this is hard problem - consider for detecting and removing hair in this case: