python - How to handle variable sized input in CNN with Keras? -

i trying perform usual classification on mnist database randomly cropped digits. images cropped following way : removed randomly first/last and/or row/column.

i use convolutional neural network using keras (and tensorflow backend) perform convolution , usual classification.

inputs of variable size , can't manage work.

here how cropped digits

import numpy np keras.utils import to_categorical sklearn.datasets import load_digits  digits = load_digits()  x = digits.images x = np.expand_dims(x, axis=3)  x_crop = list() index in range(len(x)):     x_crop.append(x[index, np.random.randint(0,2):np.random.randint(7,9), np.random.randint(0,2):np.random.randint(7,9), :]) x_crop = np.array(x_crop)  y = to_categorical(digits.target)  sklearn.model_selection import train_test_split  x_train, x_test, y_train, y_test = train_test_split(x_crop, y, train_size=0.8, test_size=0.2) 

and here architecture of model want use

from keras.layers import dense, dropout keras.layers.convolutional import conv2d keras.models import sequential  model = sequential()  model.add(conv2d(filters=10,                   kernel_size=(3,3),                   input_shape=(none, none, 1),                   data_format='channels_last'))  model.add(dense(128, activation='relu')) model.add(dropout(0.2))  model.add(dense(10, activation='softmax'))   model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])  model.summary()  model.fit(x_train, y_train, epochs=100, batch_size=16, validation_data=(x_test, y_test)) 

1. does have idea on how handle variable sized input in neural network?

2. and how perform classification?

tl/dr - go point 4

so - before point - let's fix problems network:

1. your network not work because of activation: categorical_crossentropy need have softmax activation:

   model.add(dense(10, activation='softmax')) 

2. vectorize spatial tensors: daniel mentioned - need to, @ stage, switch vectors spatial (images) vectorized (vectors). - applying dense output conv2d equivalent (1, 1) convolution. - output network spatial - not vectorized causes dimensionality mismatch (you can check running network or checking model.summary(). in order change need use either globalmaxpooling2d or globalaveragepooling2d. e.g.:

   model.add(conv2d(filters=10,               kernel_size=(3, 3),               input_shape=(none, none, 1),              padding="same",              data_format='channels_last')) model.add(globalmaxpooling2d()) model.add(dense(128, activation='relu')) model.add(dropout(0.2))  model.add(dense(10, activation='softmax')) 

3. concatenated numpy arrays need have same shape: if check shape of x_crop you'll see it's not spatial matrix. it's because concatenated matrices different shapes. sadly - it's impossible overcome issue numpy.array need have fixed shape.

4. how make network train on examples of different shape: important thing in doing understand 2 things. first - in single batch every image should have same size. second - calling fit multiple times bad idea - reset inner model states. here needs done:

   a. write function crops single batch - e.g. get_cropped_batches_generator given matrix cuts batch out of , crops randomly.

   b. use train_on_batch method. here example code:

   from 6 import next  batches_generator = get_cropped_batches_generator(x, batch_size=16) losses = list() epoch_nb in range(nb_of_epochs):     epoch_losses = list()     batch_nb in range(nb_of_batches):         # cropped_x has different shape different batches (in general)         cropped_x, cropped_y = next(batches_generator)          current_loss = model.train_on_batch(cropped_x, cropped_y)         epoch_losses.append(current_loss)     losses.append(epoch_losses.sum() / (1.0 * len(epoch_losses)) final_loss = losses.sum() / (1.0 * len(losses)) 

so - few comments code above: first, train_on_batch doesn't use nice keras progress bar. returns single loss value (for given batch) - that's why added logic compute loss. use progbar callback also. second - need implement get_cropped_batches_generator - haven't written code keep answer little bit more clear. ask question on how implement it. last thing - use six keep compatibility between python 2 , python 3.