Custom weight initialization tensorflow tf.layers.dense

Question

I\'m trying to set up custom initializer to tf.layers.dense where I initialize kernel_initializer with a weight matrix I already have.

Answer 1

There are at least two ways to achieve this:

1 Create your own layer

  W1 = tf.Variable(YOUR_WEIGHT_MATRIX, name='Weights')
  b1 = tf.Variable(tf.zeros([YOUR_LAYER_SIZE]), name='Biases') #or pass your own
  h1 = tf.add(tf.matmul(X, W1), b1)

2 Use the tf.constant_initializer

init = tf.constant_initializer(YOUR_WEIGHT_MATRIX)
l1 = tf.layers.dense(X, o, kernel_initializer=init)