Actually this is a classic problem as SO user Victor put it (in another SO question regarding which tasks to ask during an interview).
I couldn\'t do it in an hour
This is called the "Point in the Triangle" test.
Here is an article with several solutions to this problem: Point in the Triangle Test.
A common way to check if a point is in a triangle is to find the vectors connecting the point to each of the triangle's three vertices and sum the angles between those vectors. If the sum of the angles is 2*pi (360-degrees) then the point is inside the triangle, otherwise it is not.