How to use maxout activation function in tensorflow?

问题:

I want to use maxout activation function in tensorflow, but I don't know which function should use.

回答1:

I sent a pull request for maxout, here is the link:

https://github.com/tensorflow/tensorflow/pull/5528

Code is as follows:

def maxout(inputs, num_units, axis=None):     shape = inputs.get_shape().as_list()     if axis is None:         # Assume that channel is the last dimension         axis = -1     num_channels = shape[axis]     if num_channels % num_units:         raise ValueError('number of features({}) is not a multiple of num_units({})'              .format(num_channels, num_units))     shape[axis] = -1     shape += [num_channels // num_units]     outputs = tf.reduce_max(tf.reshape(inputs, shape), -1, keep_dims=False)     return outputs 

Here is how it works:

回答2:

I don't think there is a maxout activation but there is nothing stopping yourself from making it yourself. You could do something like the following.

with tf.variable_scope('maxout'):   layer_input = ...   layer_output = None   for i in range(n_maxouts):     W = tf.get_variable('W_%d' % d, (n_input, n_output))     b = tf.get_variable('b_%d' % i, (n_output,))     y = tf.matmul(layer_input, W) + b     if layer_output is None:       layer_output = y     else:       layer_output = tf.maximum(layer_output, y) 

Note that this is code I just wrote in my browser so there may be syntax errors but you should get the general idea. You simply perform a number of linear transforms and take the maximum across all the transforms.

回答3:

How about this code? This seems to work in my test.

def max_out(input_tensor,output_size): shape = input_tensor.get_shape().as_list() if shape[1] % output_size == 0:     return tf.transpose(tf.reduce_max(tf.split(input_tensor,output_size,1),axis=2)) else:     raise ValueError("Output size or input tensor size is not fine. Please check it. Reminder need be zero.") 

I refer the diagram in the following page.

回答4:

From version 1.4 on you can use tf.contrib.layers.maxout.

文章来源: How to use maxout activation function in tensorflow?