Keras: Create a custom generator for two input model using flow_from _directory() function

问题

I was trying to train my siamese network with fit_generator(),I learned from this answer: Keras: How to use fit_generator with multiple inputs that the best way to do this was to create your own generator that yield the multiple data points, my problem was that I retrieve my data with flow_from_directory() function and I didn't know if that was possible.

This is my attempt to readapt a generator for my problem:

from keras.models import load_model
from keras import optimizers
from keras.preprocessing.image import ImageDataGenerator
import numpy as np

model = load_model("siamese_model.h5")

train_datagen = ImageDataGenerator(rescale = 1./255)

def generator():
    t1 = train_datagen.flow_from_directory(base_dir,target_size = (150, 150), batch_size = 20, class_mode = 'categorical',shuffle = True)
    t2 = train_datagen.flow_from_directory(base_dir,target_size = (150, 150), batch_size = 20, class_mode = 'categorical', shuffle = True)
    while True:
        d1,y = t1.next()
        d2 = t2.next()
        yield ([d1[0], d2[0]],y)

model.compile(loss = 'categorical_crossentropy',optimizer= optimizers.RMSprop(lr=2e-5),metrics=['acc'])

history = model.fit_generator(generator(),
                              steps_per_epoch = 10,
                              epochs = 5)

My code give me the exact same error as when I tried to fit my model without the custom generator:

ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays: [array([[[[0.14509805, 0.15686275, 0.16862746],
         [0.14509805, 0.15686275, 0.16862746],
         [0.14509805, 0.15686275, 0.16862746],
         ...,
         [0.14117648, 0.15294118, 0.16862746...

What am I doing wrong?

回答1:

Try this instead :

while True:
    d1 = t1.next()
    d2 = t2.next()
    yield ([d1[0], d2[0]], d1[1])

Also , your input will be shuffle in a different way so they will lose their link if you put them in a certain order in your folder.

I would suggest :

t1 = train_datagen.flow_from_directory(base_dir,target_size = (150, 150), batch_size = 20, class_mode = 'categorical', shuffle = False, seed='13')

t2 = train_datagen.flow_from_directory(base_dir,target_size = (150, 150), batch_size = 20, class_mode = 'categorical', shuffle = False, seed='13')

or with the same seed for shuffle

t1 = train_datagen.flow_from_directory(base_dir,target_size = (150, 150), batch_size = 20, class_mode = 'categorical', shuffle = True, seed='13')

t2 = train_datagen.flow_from_directory(base_dir,target_size = (150, 150), batch_size = 20, class_mode = 'categorical', shuffle = True, seed='13')

来源：https://stackoverflow.com/questions/57930476/keras-create-a-custom-generator-for-two-input-model-using-flow-from-directory