问题
Caffe have reshape layer implemented, but say I want to first reshape a blob of (1, n, k, p) to (1, a, b, k, p), where n= a*b and then transpose it to shape (1, b, a, k, p), how to implement this operation, I know I can write a seperate python layer and do all this with numpy.reshape and numpy.transpose, but that would be not efficient, is it?
回答1:
transpose and reshape are two fundamentally different operations:
While reshape only changes the shape of a blob, it does not affect its internal structure (and thus can be execute very efficiently). On the other hand, transpose re-arrange the blob's data.
Let's look at a simple example.
Suppose you have a 2x2 blob with values
[[0, 1], [2, 3]]
In memory the values are stored in a 1D contiguous way (row-major):
[0, 1, 2, 3]
If you reshape the blob to 4x1
[[0], [1], [2], [3]]
The underlying arrangement of the elements in memory is not changed.
However, if you transpose the blob to get
[[0, 2], [1, 3]]
The underlying arrangement is also changed to
[0, 2, 1, 3]
Therefore, you cannot use "Reshape" layer to transpose a blob.
Caffe SSD branch (by Weilu) has a "Permute" layer which is equivalent to transpose.
A note about performance:
While reshape only changes the blob's header (O(1) runtime and space), transpose needs to re-arrange elements in memory thus taking O(n) time and space.
To make things worse, if you use numpy.transpose to perform the task it means you transpose in CPU (host memory) thus adding two sync operations between CPU and GPU memory (sync GPU->CPU, transpose in CPU, sync CPU->GPU).
So, if you have no alternative but to transpose (aka "Permute") make sure you have a GPU implementation.
来源：https://stackoverflow.com/questions/48396224/can-caffe-reshape-layer-do-transpose