Sparse Matrix from a dense one Tensorflow

Question

I am creating a convolutional sparse autoencoder and I need to convert a 4D matrix full of values (whose shape is [samples, N, N, D]) into a sparse matrix.

For each sample, I have D NxN feature maps. I want to convert each NxN feature map to a sparse matrix, with the maximum value mapped to 1 and all the others to 0.

I do not want to do this at run time but during the Graph declaration (because I need to use the resulting sparse matrix as an input to other graph operations), but I do not understand how to get the indices to build the sparse matrix.

Answer 1

You can use tf.where and tf.gather_nd to do that:

import numpy as np
import tensorflow as tf

# Make a tensor from a constant
a = np.reshape(np.arange(24), (3, 4, 2))
a_t = tf.constant(a)
# Find indices where the tensor is not zero
idx = tf.where(tf.not_equal(a_t, 0))
# Make the sparse tensor
# Use tf.shape(a_t, out_type=tf.int64) instead of a_t.get_shape()
# if tensor shape is dynamic
sparse = tf.SparseTensor(idx, tf.gather_nd(a_t, idx), a_t.get_shape())
# Make a dense tensor back from the sparse one, only to check result is correct
dense = tf.sparse_tensor_to_dense(sparse)
# Check result
with tf.Session() as sess:
    b = sess.run(dense)
np.all(a == b)
>>> True

Answer 2

Simple code to convert dense numpy array to tf.SparseTensor:

def denseNDArrayToSparseTensor(arr):
  idx  = np.where(arr != 0.0)
  return tf.SparseTensor(np.vstack(idx).T, arr[idx], arr.shape)