问题
I was just hoping to clarify some information about a previous post that discusses how a Keras pre-trained model like VGG or InceptionV3 can be extended to different sized images. My problem is that I have some satellite images that come in 8-bands. So an image might be 650x650x8 instead of the usual RBG 3 band images. I wanted to know if I can use a Keras pre-trained model on images with 8 bands instead of 3 bands.
Now there is an original post and deals with something similar to this. The reference post was about applying a Keras VGG pre-trained model to an image of different size. So VGG was trained on 224x224x3 and the user wanted to use this model against an image of 160x320x3.
Here is the original post: Change input tensor shape for VGG16 application
Here is the code from the original post:
from keras.models import Model
from keras.layers import Dense,Flatten
from keras.applications import vgg16
from keras import backend as K
model = vgg16.VGG16(weights='imagenet', include_top=False, input_shape=(160,320,3))
model.summary(line_length=150)
flatten = Flatten()
new_layer2 = Dense(10, activation='softmax', name='my_dense_2')
inp2 = model.input
out2 = new_layer2(flatten(model.output))
model2 = Model(inp2, out2)
model2.summary(line_length=150)
So if I was to substitute the 6th line with something like:
model = vgg16.VGG16(weights='imagenet', include_top=False, input_shape=(650,650,8))
Would that work, or would the pre-trained model not accept an increase in channels as it would accept a change in image height or width?
Also, I would have to conduct additional training on the model with the extra channels. But I as not clear how the pre-trained model actually implement this extension. Are the weights for the added nodes just set to 0 or some sort of initializer determined weights? I am trying to guage just how much extra training I would need to do.
Thanks for any tips or suggestions.
回答1:
From the Keras docs:
input_shape: optional shape tuple, only to be specified if include_top is False (otherwise the input shape has to be (224, 224, 3) (with 'channels_last' data format) or (3, 224, 224) (with 'channels_first' data format). It should have exactly 3 inputs channels, and width and height should be no smaller than 48. E.g. (200, 200, 3) would be one valid value.
You will not be able to use this VGG implementation for multispectral imagery. As you already mention, this would introduce additional weights to the model that have not been pretrained.
Neural nets for multispectral images are quite an active research topic, but I'm afraid that there are few out of the box solutions such as the networks pretrained on imagenet. You could try a dimensionality reduction technique such as PCA to compress your images to three channels. Or train a custom architecture altogether that takes 8-channel images as inputs.
来源：https://stackoverflow.com/questions/51432652/extending-a-keras-pre-trained-model-to-images-with-additional-channels-or-bands