Shearing image in tensorflow

问题

I am using tf.keras to build my network. And I am doing all the augmentation in tensor_wise level since my data in tfrecords file. Then I needed to do shearing and zca for augmentation but couldn't find a proper implementation in tensor flow. And I can't use the DataImageGenerator that did both operation I needed because as I said my data doesn't fit in memory and it is in tfrecord format. So all my augmentations process should be tesnorwise.

@fchollet here suggested a way to use ImgaeDataGenerator with large dataset.

My first questino is if I use @fchollet way, which is basically using X-sample of the large data to run the ImageDataGenerator then using train_on_batch to train the network , how I can feed my validation data to the network.

My Second question is there any tensor-wise implementation for shear and zca operations. Some people like here suggested using tf.contrib.image.transform but couldn't understand how. If some one have the idea on how to do it, I will appreciate that.

Update:

This is my trial to construct the transformation matrix through ski_image

from skimage import io
from skimage import transform as trans
import tensor flow as tf 

def augment()
  afine_tf = trans.AffineTransform(shear=0.2)
  transform = tf.contrib.image.matrices_to_flat_transforms(tf.linalg.inv(afine_tf.params))
  transform= tf.cast(transform, tf.float32)
  image = tf.contrib.image.transform(image, transform)  # Image here is a tensor 
  return image


dataset_train = tf.data.TFRecordDataset(training_files, num_parallel_reads=calls)

dataset_train = dataset_train.apply(tf.contrib.data.shuffle_and_repeat(buffer_size=1000+ 4 * batch_size))
dataset_train = dataset_train.map(decode_train, num_parallel_calls= calls)  
dataset_train = dataset_train.map(augment,num_parallel_calls=calls )  
dataset_train = dataset_train.batch(batch_size)    
dataset_train = dataset_train.prefetch(tf.contrib.data.AUTOTUNE)

回答1:

I will answer the second question.

Today one of my old questions was commented by a user, but the comments have been deleted when I was adding more details on how to use tf.contrib.image.transform. I guess it's you, right?

So, I have edited my question and added an example, check it here.

TL;DR:

def transformImg(imgIn,forward_transform):
    t = tf.contrib.image.matrices_to_flat_transforms(tf.linalg.inv(forward_transform))
    # please notice that forward_transform must be a float matrix,
    # e.g. [[2.0,0,0],[0,1.0,0],[0,0,1]] will work
    # but [[2,0,0],[0,1,0],[0,0,1]] will not
    imgOut = tf.contrib.image.transform(imgIn, t, interpolation="BILINEAR",name=None)
    return imgOut
def shear_transform_example(filename,shear_lambda):
    image_string = tf.read_file(filename)
    image_decoded = tf.image.decode_jpeg(image_string, channels=3)
    img = transformImg(image_decoded, [[1.0,shear_lambda,0],[0,1.0,0],[0,0,1.0]])
    # Notice that this is a shear transformation parallel to the x axis
    # If you want a y axis version, use this:
    # img = transformImg(image_decoded, [[1.0,0,0],[shear_lambda,1.0,0],[0,0,1.0]])
    return img
img = shear_transform_example("white_square.jpg",0.1)

来源：https://stackoverflow.com/questions/52938836/shearing-image-in-tensorflow