Bias only Layer in Keras

问题

How could one build a layer in Keras which maps an input x to an output of the form x+b where b is a trainable weight of the same dimension? (Also the activation function here would be the identity).

回答1:

You can always build a custom layer by extending tf.keras.layers.Layer class, here is how I'd do it

import tensorflow as tf
print('TensorFlow:', tf.__version__)

class BiasLayer(tf.keras.layers.Layer):
    def __init__(self, *args, **kwargs):
        super(BiasLayer, self).__init__(*args, **kwargs)

    def build(self, input_shape):
        self.bias = self.add_weight('bias',
                                    shape=input_shape[1:],
                                    initializer='zeros',
                                    trainable=True)
    def call(self, x):
        return x + self.bias

input_layer = tf.keras.Input(shape=[5])
x  = BiasLayer()(input_layer)
model = tf.keras.Model(inputs=[input_layer], outputs=[x])

model.summary()

TensorFlow: 2.1.0
Model: "model_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_7 (InputLayer)         [(None, 5)]               0         
_________________________________________________________________
bias_layer_3 (BiasLayer)     (None, 5)                 5         
=================================================================
Total params: 5
Trainable params: 5
Non-trainable params: 0
_________________________________________________________________

来源：https://stackoverflow.com/questions/60644157/bias-only-layer-in-keras