机器学习 - 提高模型 (代码）

如果模型出现了 underfitting 问题，就得提高模型了。

Model improvement technique	What does it do?
Add more layers	Each layer potentially increases the learning capabilities of the model with each layer being able to learn some kind of new pattern in the data, more layers is often referred to as making your neural network deeper.
Add more hidden units	More hidden units per layer means a potential increase in learning capabilities of the model, more hidden units is often referred to as making your neural network wider.
Fitting for longer (more epochs)	Your model might learn more if it had more opportunities to look at the data.
Changing the activation functions	Some data just can’t be fit with only straight lines, using non-linear activation functions can help with this.
Change the learning rate	Less model specific, but still related, the learning rate of the optimizer decides how much a model should change its parameter each step, too much and the model overcorrects, too little and it doesn’t learn enough.
Change the loss function	Less model specific but still important, different problems require different loss functions. For example, a binary cross entropy loss function won’t work with a multi-class classification problem.
Use transfer learning	Take a pretrained model from a problem domain similar to yours and adjust it to your own problem.

举个例子，代码如下：

class CircleModelV1(nn.Module):
  def __init__(self):
    super().__init__()
    self.layer_1 = nn.Linear(in_features = 2, out_features = 10)
    self.layer_2 = nn.Linear(in_features = 10, out_features = 10)
    self.layer_3 = nn.Linear(in_features = 10, out_features = 1)
  
  def forward(self, x):
    return self.layer_3(self.layer_2(self.layer_1(x)))

model_1 = CircleModelV1().to("cpu")
print(model_1)

loss_fn = nn.BCEWithLogitsLoss()
optimizer = torch.optim.SGD(model_1.parameters(), lr=0.1)

torch.manual_seed(42)

epochs = 1000

X_train, y_train = X_train.to("cpu"), y_train.to("cpu")
X_test, y_test = X_test.to("cpu"), y_test.to("cpu")

for epoch in range(epochs):
  ### Training
  # 1. Forward pass 
  y_logits = model_1(X_train).squeeze()
  y_pred = torch.round(torch.sigmoid(y_logits))  # logits -> probabilities -> prediction labels 

  # 2. Calculate loss/accuracy 
  loss = loss_fn(y_logits, y_train)
  acc = accuracy_fn(y_true = y_train, y_pred = y_pred)

  # 3. Optimizer zero grad 
  optimizer.zero_grad()

  # 4. Loss backwards 
  loss.backward()

  # 5. Optimizer step 
  optimizer.step() 

  ### Testing 
  model_1.eval()
  with torch.inference_mode():
    # 1. Forward pass 
    test_logits = model_1(X_test).squeeze()
    test_pred = torch.round(torch.sigmoid(test_logits))
    # 2. Calculate loss/accuracy 
    test_loss = loss_fn(test_logits, y_test)
    test_acc = accuracy_fn(y_true = y_test, y_pred = test_pred)
  
  if epoch % 100 == 0:
    print(f"Epoch: {epoch} | Loss: {loss:.5f}, Accuracy: {acc:.2f}%")

# 结果如下
CircleModelV1(
  (layer_1): Linear(in_features=2, out_features=10, bias=True)
  (layer_2): Linear(in_features=10, out_features=10, bias=True)
  (layer_3): Linear(in_features=10, out_features=1, bias=True)
)
Epoch: 0 | Loss: 0.69528, Accuracy: 51.38%
Epoch: 100 | Loss: 0.69325, Accuracy: 47.88%
Epoch: 200 | Loss: 0.69309, Accuracy: 49.88%
Epoch: 300 | Loss: 0.69303, Accuracy: 50.50%
Epoch: 400 | Loss: 0.69300, Accuracy: 51.38%
Epoch: 500 | Loss: 0.69299, Accuracy: 51.12%
Epoch: 600 | Loss: 0.69298, Accuracy: 51.50%
Epoch: 700 | Loss: 0.69298, Accuracy: 51.38%
Epoch: 800 | Loss: 0.69298, Accuracy: 51.50%
Epoch: 900 | Loss: 0.69298, Accuracy: 51.38%

都看到这了，点个赞呗~