How does mask_zero in Keras Embedding layer work?

Question

I thought mask_zero=True will output 0\'s when the input value is 0, so the following layers could skip computation or something.

How does mask_ze

Answer 1

The process of informing the Model that some part of the Data is actually Padding and should be ignored is called Masking.

There are three ways to introduce input masks in Keras models:

1. Add a keras.layers.Masking layer.
2. Configure a keras.layers.Embedding layer with mask_zero=True.
3. Pass a mask argument manually when calling layers that support this argument (e.g. RNN layers).

Given below is the code to introduce Input Masks using keras.layers.Embedding

import numpy as np

import tensorflow as tf

from tensorflow.keras import layers

raw_inputs = [[83, 91, 1, 645, 1253, 927],[73, 8, 3215, 55, 927],[711, 632, 71]]
padded_inputs = tf.keras.preprocessing.sequence.pad_sequences(raw_inputs,
                                                              padding='post')

print(padded_inputs)

embedding = layers.Embedding(input_dim=5000, output_dim=16, mask_zero=True)
masked_output = embedding(padded_inputs)

print(masked_output._keras_mask)

Output of the above code is shown below:

[[  83   91    1  645 1253  927]
 [  73    8 3215   55  927    0]
 [ 711  632   71    0    0    0]]

tf.Tensor(
[[ True  True  True  True  True  True]
 [ True  True  True  True  True False]
 [ True  True  True False False False]], shape=(3, 6), dtype=bool)

For more information, refer this Tensorflow Tutorial.