Method to detect intersection between a rectangle and a polygon?

Question

What is the best method to detect whether the red rectangle overlaps the black polygon? Please refer to this image:

Answer 1

Simply to find whether there is an intersection, I think you may be able to combine two algorithms.

1) The ray casting algorithm. Using the vertices of each polygon, determine if one of the vertices is in the other. Assuming you aren't worried about the actual intersection region, but just the existence of it. http://en.wikipedia.org/wiki/Point_in_polygon

2) Line intersection. If step 1 produces nothing, check line intersection.

I'm not certain this is 100% correct or optimal.

If you actually need to determine the region of the intersection, that is more complex, see previous SO answer:

A simple algorithm for polygon intersection