How to concatenate word vectors to form sentence vector

Question

I have learned in some essays (Tomas Mikolov...) that a better way of forming the vector for a sentence is to concatenate the word-vector.

but due to my clumsy in mathematics, I am still not sure about the details.

for example,

supposing that the dimension of word vector is m; and that a sentence has n words.

what will be the correct result of concatenating operation?

is it a row vector of 1 x m*n ? or a matrix of m x n ?

please advise

thanks

Answer 1

There are at least three common ways to combine embedding vectors; (a) summing, (b) summing & averaging or (c) concatenating. So in your case, with concatenating, that would give you a 1 x m*a vector, where a is the number of sentences. In the other cases, the vector length stays the same. See gensim.models.doc2vec.Doc2Vec, dm_concat and dm_mean - it allows you to use any of those three options [1,2].

[1] http://radimrehurek.com/gensim/models/doc2vec.html#gensim.models.doc2vec.LabeledLineSentence

[2] https://github.com/piskvorky/gensim/blob/develop/gensim/models/doc2vec.py