How to add regularizations in TensorFlow?

Question

I found in many available neural network code implemented using TensorFlow that regularization terms are often implemented by manually adding an additional term to loss valu

Answer 1

I'll provide a simple correct answer since I didn't find one. You need two simple steps, the rest is done by tensorflow magic:

1. Add regularizers when creating variables or layers:

   tf.layers.dense(x, kernel_regularizer=tf.contrib.layers.l2_regularizer(0.001))
   # or
   tf.get_variable('a', regularizer=tf.contrib.layers.l2_regularizer(0.001))
   

2. Add the regularization term when defining loss:

   loss = ordinary_loss + tf.losses.get_regularization_loss()