What is the best method to detect whether the red rectangle overlaps the black polygon? Please refer to this image:
If you use axis-aligned rectangles and polygons consist of rectangles only, templatetypedef's answer is what you need.
If you use arbitrary polygons, it's a much more complex problem. First, you need to subdivide polygons into convex parts, then perform collision detection using, for example, the SAT algorithm