Logistic regression is good at classification but when complexity(non linearity) increases, the accuracy of model decreases.
Therefore, we need to increase complexity of model.
In order to increase complexity of model, we need to add more non linear functions as hidden layer.
*break point:
Create ANN Model:
We add 3 hidden layers.
We use ReLU, Tanh and ELU activation functions for diversity
# Create ANN ModelclassANNModel(nn.Module):def__init__(self, input_dim, hidden_dim, output_dim):super(ANNModel, self).__init__()# Linear function 1: 784 --> 150
self.fc1 = nn.Linear(input_dim, hidden_dim)# Non-linearity 1
self.relu1 = nn.ReLU()# Linear function 2: 150 --> 150
self.fc2 = nn.Linear(hidden_dim, hidden_dim)# Non-linearity 2
self.tanh2 = nn.Tanh()# Linear function 3: 150 --> 150
self.fc3 = nn.Linear(hidden_dim, hidden_dim)# Non-linearity 3
self.elu3 = nn.ELU()# Linear function 4 (readout): 150 --> 10
self.fc4 = nn.Linear(hidden_dim, output_dim)defforward(self, x):# Linear function 1
out = self.fc1(x)# Non-linearity 1
out = self.relu1(out)# Linear function 2
out = self.fc2(out)# Non-linearity 2
out = self.tanh2(out)# Linear function 2
out = self.fc3(out)# Non-linearity 2
out = self.elu3(out)# Linear function 4 (readout)
out = self.fc4(out)return out
## instantiate ANN
input_dim =28*28
hidden_dim =150#hidden layer dim is one of the hyper parameter and it should be chosen and tuned. For now I only say 150 there is no reason.
output_dim =10# Create ANN
model = ANNModel(input_dim, hidden_dim, output_dim)# SGD Optimizer
learning_rate =0.02# visualization loss
plt.plot(iteration_list,loss_list)
plt.xlabel("Number of iteration")
plt.ylabel("Loss")
plt.title("ANN: Loss vs Number of iteration")
plt.show()# visualization accuracy
plt.plot(iteration_list,accuracy_list,color ="red")
plt.xlabel("Number of iteration")
plt.ylabel("Accuracy")
plt.title("ANN: Accuracy vs Number of iteration")
plt.show()
Logistic regression is good at classification but when complexity(non linearity) increases, the accuracy of model decreases.Therefore, we need to increase complexity of model.In order to increase co...