How to keep the weight value to zero in a particular location using theano or lasagne?

问题

I'm a theano and lasagne user.

I have a problem dealing with the variable length of the input matrix.

i.e)

x1 = [0, 1, 3]
x2 = [1, 2]

matrix_embedding = [ [ 0.1, 0.2, 0.3],
                     [ 0.4, 0.5, 0.6],
                     [ 0.2, 0.3, 0.5],
                     [ 0.5, 0.6, 0.7],    ]

matrix_embedding[x1] = [
                     [ 0.1, 0.2, 0.3],
                     [ 0.4, 0.5, 0.6],
                     [ 0.5, 0.6, 0.7]
                             ]

matrix_embedding[x2] = [
                     [ 0.4, 0.5, 0.6],
                     [ 0.2, 0.3, 0.5],
                             ]

So, I try to use the padding.

matrix_padding_embedding = [ [ 0.1, 0.2, 0.3],
                           [ 0.4, 0.5, 0.6],
                           [ 0.2, 0.3, 0.5],
                           [ 0.5, 0.6, 0.7],
                           [ 0.0, 0.0, 0.0] ]

x1 = [0, 1, 3]
x2 = [1, 2, -1]

matrix_embedding[x1] = [
                     [ 0.1, 0.2, 0.3],
                     [ 0.4, 0.5, 0.6],
                     [ 0.5, 0.6, 0.7]
                             ]

 matrix_embedding[x2] = [
                     [ 0.4, 0.5, 0.6],
                     [ 0.2, 0.3, 0.5],
                     [ 0.0, 0.0, 0.0]       ]

But, after processing, theano updates the parameters matrix_padding_embedding, so, matrix_padding_embedding[-1] no longer a 0.

How to keep the weight value to zero in matrix_padding_embedding[-1]?

Or, whether there are other ways of dealing with variable length?

回答1:

you can create the padded matrix by concatenating two matrices, like,

import theano as the
import theano.tensor as ten
import numpy as np    
matrix_embedding = the.shared(np.asarray([[0.1, 0.2, 0.3],
                                          [0.4, 0.5, 0.6],
                                          [0.2, 0.3, 0.5],
                                          [0.5, 0.6, 0.7]]))
matrix_padding_embedding = ten.concatenate((matrix_embedding, ten.zeros((1, 3))))

x = ten.lvector()
y = ten.sum(matrix_padding_embedding[x])
grad = the.grad(y, matrix_embedding)
fn = the.function([x], [matrix_padding_embedding, grad])

x2 = [1, 2, -1]
p, g = fn(x2)
print p
print g

results are

# [[ 0.1  0.2  0.3]
#  [ 0.4  0.5  0.6]
#  [ 0.2  0.3  0.5]
#  [ 0.5  0.6  0.7]
#  [ 0.   0.   0. ]]
# 
# [[ 0.  0.  0.]
#  [ 1.  1.  1.]
#  [ 1.  1.  1.]
#  [ 0.  0.  0.]]

来源：https://stackoverflow.com/questions/36501853/how-to-keep-the-weight-value-to-zero-in-a-particular-location-using-theano-or-la

标签

python

neural-network

theano

lasagne