Understanding COCO evaluation “maximum detections”

Question

I started using the cocoapi to evaluate a model trained using the Object Detection API. After reading various sources that explain mean average precision (mAP) and recall, I

Answer 1

You can change the maxDets parameter and define a new summarize() instance method.

Let's create a COCOeval object:

cocoEval = COCOeval(cocoGt,cocoDt,annType)
cocoEval.params.maxDets = [200]
cocoEval.params.imgIds  = imgIdsDt
cocoEval.evaluate()
cocoEval.accumulate()
cocoEval.summarize_2() # instead of calling cocoEval.summarize()

Now, define summarize_2() method in cocoeval.py module in the following way:

def summarize_2(self):
    # Copy everything from `summarize` method here except
    # the function `_summarizeDets()`.
    def _summarizeDets():
        stats = np.zeros((12,))
        stats[0] = _summarize(1, maxDets=self.params.maxDets[0])
        stats[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[0])
        stats[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[0])
        stats[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[0])
        stats[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[0])
        stats[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[0])
        stats[6] = _summarize(0, maxDets=self.params.maxDets[0])
        stats[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[0])
        stats[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[0])
        stats[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[0])
        return stats
    # Copy other things which are left from `summarize()` here.

If you run the above method over your dataset, you will get an output similar to this:

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=200 ] = 0.507
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=200 ] = 0.699
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=200 ] = 0.575
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=200 ] = 0.586
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=200 ] = 0.519
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=200 ] = 0.501
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=200 ] = 0.598
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=200 ] = 0.640
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=200 ] = 0.566
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=200 ] = 0.564