What is an Embedding in Keras?

Question

Keras documentation isn\'t clear what this actually is. I understand we can use this to compress the input feature space into a smaller one. But how is this done from a neur

Answer 1

In Keras, the Embedding layer is NOT a simple matrix multiplication layer, but a look-up table layer (see call function below or the original definition).

def call(self, inputs):
    if K.dtype(inputs) != 'int32':
        inputs = K.cast(inputs, 'int32')
    out = K.gather(self.embeddings, inputs)
    return out

What it does is to map each a known integer n in inputs to a trainable feature vector W[n], whose dimension is the so-called embedded feature length.