Computing cosine_proximity loss between two outputs of the network

问题

I'm using Keras 2.0.2 Functional API (Tensorflow 1.0.1) to implement a network that takes several inputs and produces two outputs a and b. I need to train the network using the cosine_proximity loss, such that b is the label for a. How do I do this?

Sharing my code here. The last line model.fit(..) is the problematic part because I don't have labeled data per se. The label is produced by the model itself.

from keras.models import Model
from keras.layers import Input, LSTM
from keras import losses

shared_lstm = LSTM(dim)

q1 = Input(shape=(..,.. ), name='q1')
q2 = Input(shape=(..,.. ), name='q2')
a = shared_lstm(q1)
b = shared_lstm(q2)
model = Model(inputs=[q1,q2], outputs=[a, b])
model.compile(optimizer='adam', loss=losses.cosine_proximity)

model.fit([testq1, testq2], [?????])

回答1:

You can define a fake true label first. For example, define it as a 1-D array of ones of the size of your input data.

Now comes the loss function. You can write it as follows.

def my_cosine_proximity(y_true, y_pred):
    a = y_pred[0]
    b = y_pred[1]
    # depends on whether you want to normalize
    a = K.l2_normalize(a, axis=-1)
    b = K.l2_normalize(b, axis=-1)        
    return -K.mean(a * b, axis=-1) + 0 * y_true

I have multiplied y_true by zero and added it just so that Theano does give not missing input warning/error.

You should call your fit function normally i.e. by including your fake ground-truth labels.

model.compile('adam', my_cosine_proximity) # 'adam' used as an example optimizer 
model.fit([testq1, testq2], fake_y_true)

来源：https://stackoverflow.com/questions/43299274/computing-cosine-proximity-loss-between-two-outputs-of-the-network

标签

deep-learning

keras

loss