Keras Image Preprocessing

Question

My training images are downscaled versions of their associated HR image. Thus, the input and the output images aren\'t the same dimension. For now, I\'m using a hand-crafted

Answer 1

Christof Henkel's suggestion is very clean and nice. I would just like to offer another way to do it using imgaug, a convenient way to augment images in lots of different ways. It's usefull if you want more implemented augmentations or if you ever need to use some ML library other than Keras.

It unfortunatly doesn't have a way to make crops that way but it allows implementing custom functions. Here is an example function for generating random crops of a set size from an image that's at least as big as the chosen crop size:

from imgaug import augmenters as iaa

def random_crop(images, random_state, parents, hooks):
    crop_h, crop_w = 128, 128
    new_images = []
    for img in images:
        if (img.shape[0] >= crop_h) and (img.shape[1] >= crop_w):
            rand_h = np.random.randint(0, img.shape[0]-crop_h)
            rand_w = np.random.randint(0, img.shape[1]-crop_w)
            new_images.append(img[rand_h:rand_h+crop_h, rand_w:rand_w+crop_w])
        else:
             new_images.append(np.zeros((crop_h, crop_w, 3)))
    return np.array(new_images)

def keypoints_dummy(keypoints_on_images, random_state, parents, hooks):
    return keypoints_on_images

cropper = iaa.Lambda(func_images=random_crop, func_keypoints=keypoints_dummy)

You can then combine this function with any other builtin imgaug function, for example the flip functions that you're already using like this:

seq = iaa.Sequential([cropper, iaa.Fliplr(0.5), iaa.Flipud(0.5)])

This function could then generate lots of different crops from each image. An example image with some possible results (note that it would result in actual (128, 128, 3) images, they are just merged into one image here for visualization):

Your image set could then be generated by:

crops_per_image = 10
images = [skimage.io.imread(path) for path in glob.glob('train_data/*.jpg')]
augs = np.array([seq.augment_image(img)/255 for img in images for _ in range(crops_per_image)])

It would also be simple to add new functions to be applied to the images, for example the remove mean functions you mentioned.