【深度学习】TensorFlow实现LeNet5

Implementation of the neural network model called LeNet5.

1. ---

   Theory studies

   1. The article proposing the model at first: Gradient-Based Learning Applied to Document Recognition
   2. The fundamental theory of CNN: Convolutional Neural Network

2. ---

   Installation of TensorFlow in Windows 10 with the use of Anaconda

   1. Create the environment in anaconda
   2. Configuration of ipython kernel
   3. Test of convolution.py on the dataset MNIST

3. ---

   Studies of the codes in convolution.py in TensorFlow source codes

After creating the environment for tensorflow, we can find a folder in

/anaconda/envs/tensorflow35/Lib/site-packages/tensorflow/models/image/mnist/convolutional.py

Extract data

def extract_data(filename, num_images):
 """Extract the images into a 4D tensor [image index, y, x, channels].

 Values are rescaled from [0, 255] down to [-0.5, 0.5].
 """
 print('Extracting', filename)
 with gzip.open(filename) as bytestream:
 bytestream.read(16)
 buf = bytestream.read(IMAGE_SIZE * IMAGE_SIZE * num_images * NUM_CHANNELS)
 data = numpy.frombuffer(buf, dtype=numpy.uint8).astype(numpy.float32)
 data = (data - (PIXEL_DEPTH / 2.0)) / PIXEL_DEPTH
 data = data.reshape(num_images, IMAGE_SIZE, IMAGE_SIZE, NUM_CHANNELS)
 return data

def extract_labels(filename, num_images):
 """Extract the labels into a vector of int64 label IDs."""
 print('Extracting', filename)
 with gzip.open(filename) as bytestream:
 bytestream.read(8)
 buf = bytestream.read(1 * num_images)
 labels = numpy.frombuffer(buf, dtype=numpy.uint8).astype(numpy.int64)
 return labels

Training step

There are three types of layers used to build ConvNets:

• Convolutional Layer
• Pooling Layer
• Fully-Connected Layer

Conv Layer 1

The CONV layer’s parameters consist of a set of learnable filters, here is 5*5 filter. It connects the input layer(generally using the receive fields 5*5) with the outputs of neurons

conv1_weights = tf.Variable(
    tf.truncated_normal(
       [5, 5, NUM_CHANNELS, 32], # 5x5 filter, depth 32.
       stddev=0.1, 
       seed=SEED, 
       dtype=data_type()
    )
)

conv1_biases = tf.Variable(tf.zeros([32], dtype=data_type()))