How could I match two detected rectangles from image?

问题

I want to calculate the ratio of detected rectangle from many images by two detector based on deep learning. for example with below picture, firstly, the first detector detects big circles (c1 and c2), and the second detector detects small circles (d1 and d2). Now, first detector returns the coordinates of c1 and c2, second detector also those of d1 and d2.

So I want to make the function and code that can calculate the ratio d1/ c1 and d2 / c2 automatically. so, my idea is that if coordinate of small one(e.g. d1, d2) is included in corresponding big one (e.g. c1, c2), they are matched for calculation of ratio.

c1 coordinate (xmin, ymin, xmax, ymax) : (10, 10, 30, 30) d1 coorinate : (13, 15, 20, 23)

c2 coordinate : (20, 20, 40, 40) d2 coordinate : (25, 24, 32, 33)

But, I don't know many function of OpenCV and python. Could you please make some code, function or recommend library?

Thank you

回答1:

The metric which is generally used for this task is "Intersection over Union" i.e. IOU.

With inputs for bounding box (rectangle) in this format [50, 60, 200, 150], you can write a custom function for it like this -

def intersection_over_union(box1, box2):
    # Get coordinates of the intersection 
    x1 = max(box1[0], box2[0])
    y1 = max(box1[1], box2[1])
    x2 = min(box1[2], box2[2])
    y2 = min(box1[3], box2[3])

    # Get the area of intersection rectangle
    intersection = max(0, x2 - x1 + 1) * max(0, y2 - y1 + 1)

    # Get the area of both rectangles
    box1Area = (box1[2] - box1[0] + 1) * (box1[3] - box1[1] + 1)
    box2Area = (box2[2] - box2[0] + 1) * (box2[3] - box2[1] + 1)

    iou = intersection / float(box1Area + box2Area - intersection)

    return iou

Note that this needs the bounding boxes are returned in the standard format where following conditions are correct:

assert box1['x1'] <= box1['x2']
assert box1['y1'] <= box1['y2']
assert box2['x1'] <= box2['x2']
assert box2['y1'] <= box2['y2']

More details in this answer.

If bounding box pair does not hold that condition, its best to use jaccard_score (another name for IOU) from sklearn

from sklearn.metrics import jaccard_score
import numpy as np

box1 = [180, 400, 450, 450]
box2 = [200, 450, 425, 425]

img = np.zeros((800, 800, 3), np.uint8)  # use your image shape here or directly below

img1 = cv2.rectangle(np.zeros(img.shape), (box1[0], box1[1]), (box1[2], box1[3]), (1, 1, 1), -1) 
img2 = cv2.rectangle(np.zeros(img.shape), (box2[0], box2[1]), (box2[2], box2[3]), (1, 1, 1), -1)

jaccard_score(img1.ravel(),img2.ravel())

来源：https://stackoverflow.com/questions/64832706/how-could-i-match-two-detected-rectangles-from-image

标签

python

OpenCV

computer-vision