MNIST

it provides methods that facilitate the creation of dense (fully connected) layers and convolutional layers, adding activation functions, and applying dropout regularization


CNNs apply a series of filters to the raw pixel data of an image to extract and learn higher-level features, which the model can then use for classification. CNNs contains three components:
(1)convolutional layers, which apply a specified number of convolution filters to the image. for each subregion, the layer performs a set of mathematical operations to produce a single value in the output feature map. Convolutional layers then typically apply a ReLU activation function to the output to introduce nonlinearities into the model
(2)pooling layers, which downsample the image data extracted by the convolutional layers to reduce the dimensionality of the feature map in order to decrease processing time. a commonly used pooling algorithm is max pooling, which extracts subregions of the feature map, keeps their maximum value, and discards all other values
(3)dense layers, which perform classification on the features extracted by the convolutional layers and downsampled by the pooling layers. in a dense layer, every node in the layer is connected to every node in the preceding layer


a CNN is composed of a stack of convolutional modules that perform feature extraction. each module consists of a convolutional layer followed by a pooling layer. the last convolutional module is followed by one or more dense layers that perform classification. the final dense layer in a CNN contains a single node for each target class in the model (all the possible classes the model may predict), with a softmax activation function to generate a value between 0-1 for each node (the sum of all these softmax values is equal to 1). we can interpret the softmax values for a given image as relative measurements of how likely it is that the image falls into each target class


the tf.layers module contains methods to create each of the three layer types above:
(1)conv2d(). constructs a two-dimensional convolutional layer. takes number of filters, filter kernel size, padding, and activation function as arguments
(2)max_pooling2d(). constructs a two-dimensional pooling layer using the max-pooling algorithm. takes pooling filter size and stride as arguments
(3)dense(). constructs a dense layer. takes number of neurons and activation function as arguments


###
input_layer = tf.reshape(features["x"], [-1, 28, 28, 1])


this allows us to treat batch_size as a hyperparameter that we can tune


###
conv1 = tf.layers.conv2d(
inputs = input_layer,
filters = 32,
kernel_size = [5, 5],
padding = "same",
activation = tf.nn.relu
)


if filter height and width have the same value, you can instead specify a single integer for kernel_size--kernel_size=5


for both training and evaluation, we need to define a loss function that measures how closely the model's predictions match the target classes. for multiclass classification problems like MNIST, cross entropy is typically used as the loss metric
###
onehot_labels = tf.one_hot(indices=tf.cast(labels, tf,int32), depth=10)
loss = tf.losses.softmax_cross_entropy(
onehot_labels=onehot_labels, logits=logits
)