基础_cifar10_序贯-白红宇

今天的基础研究主要是在cifar10数据集上解决一下几个问题：

1、从头开始，从最简单的序贯开始，尝试model的构造；

2、要将模型打印出来。最好是能够打印出图片，否则也要summary;

3、尝试对例子的参数进行分析，得出初步修改意见。

1、构建模型

'''Train a simple deep CNN on the CIFAR10 small images dataset.

It gets to 75% validation accuracy in 25 epochs, and 79% after 50 epochs.

(it's still underfitting at that point, though).

'''

from
__future__
import print_function 

#!apt-get -qq install -y graphviz && pip install -q pydot

import pydot 

import keras 

import cv2 

from keras.datasets 
import cifar10 

from keras.preprocessing.image 
import ImageDataGenerator 

from keras.models 
import Sequential 

from keras.layers 
import Dense, Dropout, Activation, Flatten 

from keras.layers 
import Conv2D, MaxPooling2D 

from keras.utils.vis_utils 
import plot_model 

import matplotlib.image 
as image 
# image 用于读取图片

import matplotlib.pyplot 
as plt 

import os 

       %matplotlib inline 

       %config InlineBackend.figure_format = 
'retina'

       batch_size = 
32

       num_classes = 
10

#epochs = 100

       epochs = 
3

       data_augmentation = 
True

       num_predictions = 
20

       save_dir = os.path.join(os.getcwd(), 
'saved_models') 

       model_name = 
'keras_cifar10_trained_model.h5'

# The data, split between train and test sets:

       (x_train, y_train), (x_test, y_test) = cifar10.load_data() 

print( 
'x_train shape:', x_train.shape) 

print(x_train.shape[ 
0], 
'train samples') 

print(x_test.shape[ 
0], 
'test samples') 

# Convert class vectors to binary class matrices.

       y_train = keras.utils.to_categorical(y_train, num_classes) 

       y_test = keras.utils.to_categorical(y_test, num_classes) 

       model = Sequential() 

model.add(Conv2D(32, (3, 3), padding='same', input_shape=x_train.shape[1:]))model.add(Activation('relu'))model.add(Conv2D(32, (3, 3)))model.add(Activation('relu'))model.add(MaxPooling2D(pool_size=(2, 2)))model.add(Dropout(0.25))

#显示模型

       model.summary() 

       plot_model(model, 
to_file= 
'model1111.png', 
show_shapes= 
True) 

       files.download( 
'model1111.png') 

       img = image.imread( 
'model1111.png') 

print(img.shape) 

       plt.imshow(img) 
# 显示图片

       plt.axis( 
'off') 
# 不显示坐标轴

       plt.show() 

# initiate RMSprop optimizer

       opt = keras.optimizers.rmsprop( 
lr= 
0.0001, 
decay= 
1e-6) 

# Let's train the model using RMSprop

       model.compile( 
loss= 
'categorical_crossentropy', 

optimizer=opt, 

metrics=[ 
'accuracy']) 

       x_train = x_train.astype( 
'float32') 

       x_test = x_test.astype( 
'float32') 

       x_train /= 
255

       x_test /= 
255

if
not data_augmentation: 

print( 
'Not using data augmentation.') 

       model.fit(x_train, y_train, 

batch_size=batch_size, 

epochs=epochs, 

validation_data=(x_test, y_test), 

shuffle= 
True) 

else: 

print( 
'Using real-time data augmentation.') 

# This will do preprocessing and realtime data augmentation:

       datagen = ImageDataGenerator( 

featurewise_center= 
False, 
# set input mean to 0 over the dataset

samplewise_center= 
False, 
# set each sample mean to 0

featurewise_std_normalization= 
False, 
# divide inputs by std of the dataset

samplewise_std_normalization= 
False, 
# divide each input by its std

zca_whitening= 
False, 
# apply ZCA whitening

rotation_range= 
0, 
# randomly rotate images in the range (degrees, 0 to 180)

width_shift_range= 
0.1, 
# randomly shift images horizontally (fraction of total width)

height_shift_range= 
0.1, 
# randomly shift images vertically (fraction of total height)

horizontal_flip= 
True, 
# randomly flip images

vertical_flip= 
False) 
# randomly flip images

# Compute quantities required for feature-wise normalization

# (std, mean, and principal components if ZCA whitening is applied).

       datagen.fit(x_train) 

# Fit the model on the batches generated by datagen.flow().

       model.fit_generator(datagen.flow(x_train, y_train, 

batch_size=batch_size), 

epochs=epochs, 

validation_data=(x_test, y_test), 

workers= 
4) 

# Save model and weights

if
not os.path.isdir(save_dir): 

       os.makedirs(save_dir) 

       model_path = os.path.join(save_dir, model_name) 

       model.save(model_path) 

print( 
'Saved trained model at 
%s
 ' % model_path) 

# Score trained model.

       scores = model.evaluate(x_test, y_test, 
verbose= 
1) 

print( 
'Test loss:', scores[ 
0]) 

print( 
'Test accuracy:', scores[ 
1]) 

2、要将模型打印出来，目前只有本地才有图片。这个图片也可以本地看。

Using TensorFlow backend.

x_train shape: (50000, 32, 32, 3)

50000 train samples

10000 test samples

_________________________________________________________________

Layer (type) Output Shape Param #

=================================================================

conv2d_1 (Conv2D) (None, 32, 32, 32) 896

_________________________________________________________________

activation_1 (Activation) (None, 32, 32, 32) 0

_________________________________________________________________

conv2d_2 (Conv2D) (None, 30, 30, 32) 9248

_________________________________________________________________

activation_2 (Activation) (None, 30, 30, 32) 0

_________________________________________________________________

max_pooling2d_1 (MaxPooling2 (None, 15, 15, 32) 0

_________________________________________________________________

dropout_1 (Dropout) (None, 15, 15, 32) 0

_________________________________________________________________

conv2d_3 (Conv2D) (None, 15, 15, 64) 18496

_________________________________________________________________

activation_3 (Activation) (None, 15, 15, 64) 0

_________________________________________________________________

conv2d_4 (Conv2D) (None, 13, 13, 64) 36928

_________________________________________________________________

activation_4 (Activation) (None, 13, 13, 64) 0

_________________________________________________________________

max_pooling2d_2 (MaxPooling2 (None, 6, 6, 64) 0

_________________________________________________________________

dropout_2 (Dropout) (None, 6, 6, 64) 0

_________________________________________________________________

flatten_1 (Flatten) (None, 2304) 0

_________________________________________________________________

dense_1 (Dense) (None, 512) 1180160

_________________________________________________________________

activation_5 (Activation) (None, 512) 0

_________________________________________________________________

dropout_3 (Dropout) (None, 512) 0

_________________________________________________________________

dense_2 (Dense) (None, 10) 5130

_________________________________________________________________

activation_6 (Activation) (None, 10) 0

=================================================================

Total params: 1,250,858

Trainable params: 1,250,858

Non-trainable params: 0

_________________________________________________________________

(2065, 635, 4)

Using real-time data augmentation.

WARNING:tensorflow:Variable *= will be deprecated. Use variable.assign_mul if you want assignment to the variable value or 'x = x * y' if you want a new python Tensor object.

Epoch 1/3

138/1563 [=>........

大图：

3、尝试对例子的参数进行分析，得出初步修改意见。

从这个序贯模型的建立过程中，其模型大概是这样的：

第一段是

model.add(Conv2D(32, (3, 3), padding='same',

input_shape=x_train.shape[1:]))

model.add(Activation('relu'))

model.add(Conv2D(32, (3, 3)))

model.add(Activation('relu'))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Dropout(0.25))

基本上相当于卷积->激活->卷积->激活->maxPooling->dropout

然后

model.add(Conv2D(64, (3, 3), padding='same'))

model.add(Activation('relu'))

model.add(Conv2D(64, (3, 3)))

model.add(Activation('relu'))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Dropout(0.25))

几乎是原样的来了一遍，唯一不同的是变成了64个一组。

model.add(Flatten())

model.add(Dense(512))

model.add(Activation('relu'))

model.add(Dropout(0.5))

model.add(Dense(num_classes))

model.add(Activation('softmax'))

最后，到输出阶段了，应该是要准备输出了。

在这个地方，应该触及DL这门技术的核心了，

就是我应该构造增益的网络？又怎样根据生成的结果来调整网络。迁移我在图像处理方面的知识，我首先是知道了基础的工具，然后有了很多实际的经验，这样才能够在拿到问题的第一时间，有初步的设想。

更简单的网络代表可以更快地训练，在我的研究过程中，需要寻找的并不是我们的网络能够复杂到什么程度—而是怎样简单的网络就可以完成目标，达到既定的acc。首先可能是90%到95%，逐渐地去接触更多东西。在cifar-10上要起码达到这个结果。

当然我知道增加epoch，一般时候能够提高准确率，当然也会过拟合；另一个方向，如果我缩小数据，比如在上面的例子中，不添加64位层，结果是这样：

model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same',
                 input_shape=x_train.shape[1:]))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Activation('softmax'))
model2 = Sequential()
model2.add(Conv2D(32, (3, 3), padding='same',
                 input_shape=x_train.shape[1:]))
model2.add(Activation('relu'))
model2.add(Conv2D(32, (3, 3)))
model2.add(Activation('relu'))
model2.add(MaxPooling2D(pool_size=(2, 2)))
model2.add(Dropout(0.25))
model2.add(Flatten())
model2.add(Dense(512))
model2.add(Activation('relu'))
model2.add(Dropout(0.5))
model2.add(Dense(num_classes))
model2.add(Activation('softmax'))

Test loss: 0.8056231224060059

Test accuracy: 0.7182

10000/10000 [==============================] - 2s 161us/step

Test loss2: 0.9484411451339722

Test accuracy2: 0.6764

最后，在《NN&DL》中反复被提及的一点，我也实际体会到了：训练需要时间，你可以先去做其它的事情。

到此，我认为《

基础_cifar10_序贯》可以结束。

目前方向：图像拼接融合、图像识别联系方式：jsxyhelu@foxmail.com