Broadcasting np.dot vs tf.matmul for tensor-matrix multiplication (Shape must be rank 2 but is rank 3 error)

Question

Let's say I have the following tensors:

X = np.zeros((3,201, 340))
Y = np.zeros((340, 28))

Making a dot product of X, Y is successful with numpy, and yields a tensor of shape (3, 201, 28). However with tensorflow I get the following error: Shape must be rank 2 but is rank 3 error ...

minimal code example:

X = np.zeros((3,201, 340))
Y = np.zeros((340, 28))
print(np.dot(X,Y).shape) # successful (3, 201, 28)
tf.matmul(X, Y) # errornous

Any idea how to achieve the same result with tensorflow?

Answer 1

Since, you are working with tensors, it would be better (for performance) to use tensordot there than np.dot. NumPy allows it (numpy.dot) to work on tensors through lowered performance and it seems tensorflow simply doesn't allow it.

So, for NumPy, we would use np.tensordot -

np.tensordot(X, Y, axes=((2,),(0,)))

For tensorflow, it would be with tf.tensordot -

tf.tensordot(X, Y, axes=((2,),(0,)))

Related post to understand tensordot.

Answer 2

Tensorflow doesn't allow for multiplication of matrices with different ranks as numpy does.

To cope with this, you can reshape the matrix. This essentially casts a matrix of, say, rank 3 to one with rank 2 by "stacking the matrices" one on top of the other.

You can use this: tf.reshape(tf.matmul(tf.reshape(Aijk,[i*j,k]),Bkl),[i,j,l])

where i, j and k are the dimensions of matrix one and k and l are the dimensions of matrix 2.

Taken from here.