How to reduce a fully-connected (`“InnerProduct”`) layer using truncated SVD

Question

In the paper Girshick, R Fast-RCNN (ICCV 2015), section \"3.1 Truncated SVD for faster detection\", the author proposes to use SVD trick to reduce the size and computation t

Answer 1

Actually, Ross Girshick's py-faster-rcnn repo includes an implementation for the SVD step: compress_net.py.

BTW, you usually need to fine-tune the compressed model to recover the accuracy (or to compress in a more sophisticated way, see for example "Accelerating Very Deep Convolutional Networks for Classification and Detection", Zhang et al).

Also, for me scipy.linalg.svd worked faster than numpy's svd.