CNN——Flame and nonflame background classifcation with some special function parts

Intro

Today I want to share some of my understanding about how to gain our pixel dataset(big data) from the originally labelled pictures.
Besides I will introduce how to create a function to batch the dataset for the benefit of passing data in tensorflow.
Finally, I will show the results of the most basic CNN when classifying flame and nonflame pictures.

Pixels from picture dataset to matrix dataset.

1. os order statement

path = r'C:\whether_fire_100'
os.chdir(path)
def read_img(path):
    cate = ['fire','nofire']
    imgs = []
    labels = []
    for idx, folder in enumerate(cate):
        for im in glob.glob(folder + '/*.jpg'):
            #print('reading the images:%s' % (im))
            img = io.imread(im)
            img = transform.resize(img, (w, h))
            imgs.append(img)
            labels.append(idx)
    ret

There are two loops in the main part. The outer one can help us regulate the type label(flame and nonflame) by the different number of outer folders. The inner one then will mainly be went through for all its pictures, transfering pictures to matrixes.

for idx,folder in enumerate(cate):
    print(folder)
>>>
(0, 'fire')
(1, 'nofire')

This function ‘enumerate’ can transfer a traversable data object (such as a list, tuple, or string) into an index sequence, listing both data and data subscripts, typically used in a for loop.

The whole process can be displayed as follows.

Build a minibatch function for ‘feed_dict’ in tensorflow

def minibatches(inputs=None, targets=None, batch_size=None, shuffle=False):
    assert len(inputs) == len(targets)
    if shuffle:
        indices = np.arange(len(inputs))
        np.random.shuffle(indices)
    for start_idx in range(0, len(inputs) - batch_size + 1, batch_size):
        if shuffle:
            excerpt = indices[start_idx:start_idx + batch_size]
        else:
            excerpt = slice(start_idx, start_idx + batch_size)
        yield inputs[excerpt], targets[excerpt]

1. Function parameters:
Inputs：network inputs
Targets：network standard outputs
batch_size：the length of every batch in CNN
Shuffle : ‘True or False’ means whether to disorder the indices

2. np.random.shuffle(indices):
Creat a random order tag of the indices. And here the word ‘indices’ stores the tag order of the inputs and targets.

3. Function statement ——assert & yield:
‘Asssert’ here equals to ‘if’. It will check the correctnes of the statement behind it. Once the sentence shows ‘false’ the programme will pause instantly and export ‘AssertionError’
‘yield’ this key work is a little similiar with ‘return’.

minibatches(x_train, y_train, batch_size, shuffle=True)
<generator object minibatches at 0x000001DDA8A3EE58>

Here we can see that this key word ‘yield’ will generate a iteration space, more likely to be a pointer to return the location as ordered.

Pass two types of dataset (Flame and Nonflame background) to our CNN

The holistic frame and parameters haven’t changed a lot and The differencers just lie in the Nonflame dataset(Last time I used parts of the background while this time I used them all)

1. Network training results
   I trained for twice with 12 echos each time.
   Lets jump to my traning results:
   
   
   As can be seen, the second picture above illustrated a downward accuracy in both test and train dataset even though the loss kept falling indicating that CNN is progessing.

2. visualized results—using the fisrt well-trained CNN to test pictures.
   This part of code has been added appart from the whole programme code last time I posted.

import matplotlib.pyplot as plt
figure, ax = plt.subplots()
import random
os.chdir(r'C:\whether_fire_100\fire')
for i in range(16):
    I = '%d' % (random.randint(1, 5841))
    filename = "%s%s" %("%s%s" % ('fire_only', I),'.jpg')
    img_test = io.imread(filename)
    test = np.asarray([img_test])
    err, ac = sess.run([loss, acc], feed_dict={x: test, y_: np.asarray([0])})
    plt.subplot(4, 4, i + 1)
#    img_test = img_test[:, :, [2, 1, 0]]
    plt.imshow(img_test)
    if ac == 1.0:
        title = "%s%s" % ('correctly classify ', I)
        plt.title(title, color='green')
    else:
        title = "%s%s" % ('incorrectly classify ', I)
        plt.title(title, color='red')
figure.suptitle('Flame')
plt.show()

Here let’s jump to the picture results.

It is probably because either our nonflame background dataset didn’t show the distinguished features or more nonflame background pictures should be passed to CNN to make it more comprehensive. However, the flame recognition between flame and nonflame displayed a wonderful result of nearly 100% .

TAG

Next step, I will focus on some mobilenet of CNN to simplify the entire caculation process as such a process is pretty time-consuming. And commence to learn some improvements about CNN.

The complete code shows as follows.

from skimage import io, transform
import glob
import os
import tensorflow as tf
import numpy as np
import time
import cv2



path = r'C:\whether_fire_100'
os.chdir(path)
# 将所有的图片resize成100*100
w = 100
h = 100
c = 3


# 读取图片
def read_img(path):
    cate = ['fire','nofire']
    imgs = []
    labels = []
    for idx, folder in enumerate(cate):
        for im in glob.glob(folder + '/*.jpg'):
            #print('reading the images:%s' % (im))
            img = io.imread(im)
            img = transform.resize(img, (w, h))
            imgs.append(img)
            labels.append(idx)
    return np.asarray(imgs, np.float32), np.asarray(labels, np.int32)

data, label = read_img(path)

# 打乱顺序
num_example = data.shape[0]
arr = np.arange(num_example)
np.random.shuffle(arr)
data = data[arr]
label = label[arr]

# 将所有数据分为训练集和验证集
ratio = 0.8
s = np.int(num_example * ratio)
x_train = data[:s]
y_train = label[:s]
x_val = data[s:]
y_val = label[s:]

# -----------------构建网络----------------------
# 占位符
x = tf.placeholder(tf.float32, shape=[None, w, h, c], name='x')
y_ = tf.placeholder(tf.int32, shape=[None, ], name='y_')

# 第一个卷积层（100——>50)
conv1 = tf.layers.conv2d(
    inputs=x,
    filters=32,
    kernel_size=[5, 5],
    padding="same",
    activation=tf.nn.relu,
    kernel_initializer=tf.truncated_normal_initializer(stddev=0.01))
pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2)

# 第二个卷积层(50->25)
conv2 = tf.layers.conv2d(
    inputs=pool1,
    filters=64,
    kernel_size=[5, 5],
    padding="same",
    activation=tf.nn.relu,
    kernel_initializer=tf.truncated_normal_initializer(stddev=0.01))
pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2)

# 第三个卷积层(25->12)
conv3 = tf.layers.conv2d(
    inputs=pool2,
    filters=128,
    kernel_size=[3, 3],
    padding="same",
    activation=tf.nn.relu,
    kernel_initializer=tf.truncated_normal_initializer(stddev=0.01))
pool3 = tf.layers.max_pooling2d(inputs=conv3, pool_size=[2, 2], strides=2)

# 第四个卷积层(12->6)
conv4 = tf.layers.conv2d(
    inputs=pool3,
    filters=128,
    kernel_size=[3, 3],
    padding="same",
    activation=tf.nn.relu,
    kernel_initializer=tf.truncated_normal_initializer(stddev=0.01))
pool4 = tf.layers.max_pooling2d(inputs=conv4, pool_size=[2, 2], strides=2)

re1 = tf.reshape(pool4, [-1, 6 * 6 * 128])

# 全连接层
dense1 = tf.layers.dense(inputs=re1,
                         units=1024,
                         activation=tf.nn.relu,
                         kernel_initializer=tf.truncated_normal_initializer(stddev=0.01),
                         kernel_regularizer=tf.contrib.layers.l2_regularizer(0.003))
dense2 = tf.layers.dense(inputs=dense1,
                         units=512,
                         activation=tf.nn.relu,
                         kernel_initializer=tf.truncated_normal_initializer(stddev=0.01),
                         kernel_regularizer=tf.contrib.layers.l2_regularizer(0.003))
logits = tf.layers.dense(inputs=dense2,
                         units=2,
                         activation=None,
                         kernel_initializer=tf.truncated_normal_initializer(stddev=0.01),
                         kernel_regularizer=tf.contrib.layers.l2_regularizer(0.003))
# ---------------------------网络结束---------------------------

loss = tf.losses.sparse_softmax_cross_entropy(labels=y_, logits=logits)
train_op = tf.train.AdamOptimizer(learning_rate=0.001).minimize(loss)
correct_prediction = tf.equal(tf.cast(tf.argmax(tf.abs(logits), 1), tf.int32), y_)
acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


# 定义一个函数，按批次取数据
def minibatches(inputs=None, targets=None, batch_size=None, shuffle=False):
    assert len(inputs) == len(targets)
    if shuffle:
        indices = np.arange(len(inputs))
        np.random.shuffle(indices)
    for start_idx in range(0, len(inputs) - batch_size + 1, batch_size):
        if shuffle:
            excerpt = indices[start_idx: start_idx + batch_size]
        else:
            excerpt = slice(start_idx, start_idx + batch_size)
        yield inputs[excerpt], targets[excerpt]


# 训练和测试数据，可将n_epoch设置更大一些

n_epoch = 1
batch_size = 64
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
for epoch in range(n_epoch):
    start_time = time.time()

    # training
    train_loss, train_acc, n_batch = 0, 0, 0
    for x_train_a, y_train_a in minibatches(x_train, y_train, batch_size, shuffle=True):
        _, err, ac = sess.run([train_op, loss, acc], feed_dict={x: x_train_a, y_: y_train_a})
        train_loss += err;
        train_acc += ac;
        n_batch += 1
    print("   train loss: %f" % (train_loss / n_batch))
    print("   train acc: %f" % (train_acc / n_batch))

    # validation
    val_loss, val_acc, n_batch = 0, 0, 0
    for x_val_a, y_val_a in minibatches(x_val, y_val, batch_size, shuffle=False):
        err, ac = sess.run([loss, acc], feed_dict={x: x_val_a, y_: y_val_a})
        val_loss += err;
        val_acc += ac;
        n_batch += 1
    print("   validation loss: %f" % (val_loss / n_batch))
    print("   validation acc: %f" % (val_acc / n_batch))


import matplotlib.pyplot as plt
figure, ax = plt.subplots()
import random
os.chdir(r'C:\whether_fire_100\fire')
for i in range(16):
    I = '%d' % (random.randint(1, 5841))
    filename = "%s%s" %("%s%s" % ('fire_only', I),'.jpg')
    img_test = io.imread(filename)
    test = np.asarray([img_test])
    err, ac = sess.run([loss, acc], feed_dict={x: test, y_: np.asarray([1])})
    plt.subplot(4, 4, i + 1)
#    img_test = img_test[:, :, [2, 1, 0]]
    plt.imshow(img_test)
    if ac == 1.0:
        title = "%s%s" % ('correctly classify ', I)
        plt.title(title, color='green')
    else:
        title = "%s%s" % ('incorrectly classify ', I)
        plt.title(title, color='red')
figure.suptitle('Flame')
plt.show()

import matplotlib.pyplot as plt
figure, ax = plt.subplots()
os.chdir(r'C:\whether_fire_100\nofire')
for i in range(16):
    I = '%d' % (random.randint(1, 2995))
    filename = "%s%s" % ("%s%s" % ('nonfire', I), '.jpg')
    img_test = io.imread(filename)
    test = np.asarray([img_test])
    err, ac = sess.run([loss, acc], feed_dict={x: test, y_: np.asarray([0])})
    plt.subplot(4, 4, i + 1)
    #    img_test = img_test[:, :, [2, 1, 0]]
    plt.imshow(img_test)
    if ac == 1.0:
        title = "%s%s" % ('correctly classify ', I)
        plt.title(title, color='green')
    else:
        title = "%s%s" % ('incorrectly classify ', I)
        plt.title(title, color='red')
figure.suptitle('Nonflame')
plt.show()