tf object detection api - extract feature vector for each detection bbox

Question

I\'m using Tensorflow object detection API and working on pretrainedd ssd-mobilenet model. is there a way to extact the last global pooling of the mobilenet for each bbox as a f

Answer 1

This is admittedly not a perfect answer but I've done a lot of digging into Faster-RCNN with the TF-OD API and made some progress on this problem. I'll explain what I've come to understand from digging into the Faster-RCNN version and hopefully you can translate it to SSD. You're best bet is to dig through the graph on TensorBoard and sift through the tensor names in the detection graph.

First, there isn't always a simple one to one correspondence between the features and the boxes/scores. That is there isn't a simple tensor that you can pull from the network that will provide this, at least not by default.

Here is the code to get the features from a Faster-RCNN network:

https://gist.github.com/markdtw/02ece6b90e75832bd44787c03a664e8d

Though this provides with something that looks like the feature vectors, you can see that there are a few other people that have run into trouble with this solution. The fundamental issue is that the feature vector is pulled before the SecondStagePostprocessor which does several operations before the detection_boxes tensor, and similar tensors, are created.

Before the SecondStagePostprocessor, the class scores and boxes are created and the feature vector is left behind never to be seen from again. In the post-processor, there's a multiclass NMS stage and a sorting stage. The end results is MaxProposalsFromSecondStage whereas the feature vector is populated for [MaxProposalsFromFirstStage, NumberOfFeatureVectors]. So there is a decimation and a sorting operation that makes it difficult to pair final output with the feature vector indices.

My current solution is to pull the feature vector and the boxes from the before the second stage and do the rest by hand. There's undoubtedly a better solution than this but it's hard to follow a graph and to find the proper tensors for the sort operation.

I hope this helps you out! Sorry that I couldn't offer you an end-to-end solution but I hope this gets you over your current road block.