Pytorch深度学习(八):用Softmax和CrossEntroyLoss解决多分类问题(Minst数据集)

本文详细介绍使用Softmax和CrossEntropyLoss解决多分类问题的方法,通过Minst数据集实例演示了从模型构建到训练测试的全过程,包括数据预处理、模型搭建、损失函数与优化器配置、训练与测试流程。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

一.SoftMax分类器

1.函数原型:

在这里插入图片描述

这是我们常见的一个神经网络模型。在输出的时候我们常常会遇到多分类问题,这时就需要我们应用Softmax函数来发挥作用了。
在这里插入图片描述
如图就是一个简单的Softmax函数,它只是求了加权平均值,然后进行分类。
在这里插入图片描述

2.上述例子用Numpy实现:

在这里插入图片描述

import numpy as np
y = np.array([1, 0, 0])
z = np.array([0.2, 0.1, -0.1])
y_pred = np.exp(z) / np.exp(z).sum()
loss = (- y * np.log(y_pred)).sum()
print(loss)

运行结果:
在这里插入图片描述

3.用pytorch实现

在这里插入图片描述

import torch
y = torch.LongTensor([0])
#print(y)
z = torch.Tensor([[0.2, 0.1, -0.1]])
#print(z)
criterion = torch.nn.CrossEntropyLoss()
loss = criterion(z, y)
print(loss)

运行结果:
在这里插入图片描述

4.设置batch_size为3时做的一个简单分类模型:

import torch
criterion = torch.nn.CrossEntropyLoss()
y = torch.LongTensor([2,0,1])
y_pred1 = torch.Tensor([[0.1, 0.2, 0.9],
[1.1, 0.1, 0.2],
[0.2, 2.1, 0.1]])
y_pred2 = torch.Tensor([[0.8, 0.2, 0.3],
[0.2, 0.3, 0.5],
[0.2, 0.2, 0.5]])
l1 = criterion(y_pred1,y)
l2 = criterion(y_pred2,y)
print("Batch Loss1",l1.data,"\nBatch Loss2",l2.data)

在这里插入图片描述

二、完成MNIST识别手写体数字

在这里插入图片描述
每个图片是28*28
在这里插入图片描述

1.准备数据集

import torch
from torchvision import transforms
from torchvision import datasets
from torch.utils.data import DataLoader
import torch.nn.functional as F
import torch.optim as optim

batch_size = 64
transform = transforms.Compose([
    transforms.ToTensor(),   #先将图像变成一个张量tensor
    transforms.Normalize((0.1307,),(0.3081,))  #其中的0.1307是MNIST数据集的均值,0.3081是MNIST数据集的标准差
])

train_dataset = datasets.MNIST(root='./dataset/mnist/',
                               train=True,
                               download=True,
                               transform=transform)
train_loader = DataLoader(train_dataset,
                          shuffle=True,
                          batch_size=batch_size)
                          
test_dataset = datasets.MNIST(root='./dataset/mnist/',
                              train=False,
                              download=True,
                              transform=transform)
test_loader = DataLoader(test_dataset,
                         shuffle=False,
                         batch_size=batch_size)                          

2.构建训练模型class

class Net(torch.nn.Module):
    def __init__(self):
        super(Net,self).__init__()
        self.l1 = torch.nn.Linear(784,512)
        self.l2 = torch.nn.Linear(512,256)
        self.l3 = torch.nn.Linear(256,128)
        self.l4 = torch.nn.Linear(128,64)
        self.l5 = torch.nn.Linear(64,10) 
        
        '''
        这里-1表示一个不确定的数,就是你如果不确定你想要reshape成几行,但是你很肯定要reshape成4列,那不确定的地方就可以写成-1
        例如一个长度的16向量x,
        x.view(-1, 4)等价于x.view(4, 4)
        x.view(-1, 2)等价于x.view(8,2)
        以此类推
        '''
    def forward(self,x):
        x = x.view(-1,784)   #784 = 28 * 28,即将N *1*28*28转化成 N *1*784
        x = F.relu(self.l1(x))
        x = F.relu(self.l2(x))
        x = F.relu(self.l3(x))
        x = F.relu(self.l4(x))
        return self.l5(x)
model = Net()

3.构造损失函数和优化器

criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(),lr=0.01,momentum=0.5) #momentum 是带有优化的一个训练过程参数

4.训练和测试

def train(epoch):
    running_loss = 0.0
    for batch_idx,data in enumerate(train_loader,0):
        inputs,target = data
        optimizer.zero_grad()
        
        #forward
        outputs = model(inputs)
        loss = criterion(outputs,target)
        
        #backward
        loss.backward()
        
        #update
        optimizer.step()
        
        running_loss += loss.item()
        if batch_idx % 300 ==299 :  #每训练300次就打印一次结果
            print('[%d,%5d] loss: %.3f' % (epoch+1,batch_idx+1,running_loss/300))
            running_loss = 0.0
'''
在分类问题中,通常需要使用max()函数对softmax函数的输出值进行操作,求出预测值索引。下面讲解一下torch.max()函数的输入及输出值都是什么。
1. torch.max(input, dim) 函数
output = torch.max(input, dim)
输入:input是softmax函数输出的一个tensor,dim是max函数索引的维度0/1,0是每列的最大值,1是每行的最大值
输出:会返回两个tensor,第一个tensor是每行的最大值,softmax的输出中最大的是1,所以第一个tensor是全1的tensor;第二个tensor是每行最大值的索引。
'''            
def test():
    correct = 0
    total = 0 
    with torch.no_grad():   #不需要计算梯度
        for data in test_loader:
            images,labels = data
            outputs = model(images)
            _,predicted = torch.max(outputs.data,dim=1)  #用max 来找最大值的下标
            total += labels.size(0)
            correct += (predicted == labels).sum().item()
    print("Accuracy on test set:%d %%" % (100*correct/total))
    

if __name__=='__main__':
    for epoch in range(10):
        train(epoch)
        test()      

5.运行结果如下:

在这里插入图片描述

参考链接:
https://blog.csdn.net/weixin_44425647/article/details/107901381

https://www.bilibili.com/video/av97078147?p=9

以下是使用PyTorch实现的胶囊网络模型,以及训练测试代码,包括准确率、损失率混淆矩阵热图的计算可视化: ```python import torch import torch.nn as nn import torch.optim as optim import torch.nn.functional as F from torchvision import datasets, transforms from torch.utils.data import DataLoader from sklearn.metrics import confusion_matrix import matplotlib.pyplot as plt import numpy as np # 定义胶囊网络模型 class CapsuleNet(nn.Module): def __init__(self): super(CapsuleNet, self).__init__() self.conv1 = nn.Conv2d(1, 256, kernel_size=9) self.primary_capsules = PrimaryCapsules() self.digit_capsules = DigitCapsules() def forward(self, x): x = self.conv1(x) x = F.relu(x) x = self.primary_capsules(x) x = self.digit_capsules(x) return x class PrimaryCapsules(nn.Module): def __init__(self): super(PrimaryCapsules, self).__init__() self.conv2 = nn.Conv2d(256, 32*8, kernel_size=9, stride=2, padding=0) def forward(self, x): x = self.conv2(x) x = x.view(x.size(0), 32, -1) x = self.squash(x) return x def squash(self, x): norm = x.norm(dim=-1, keepdim=True) x = x / (1 + norm**2) * norm return x class DigitCapsules(nn.Module): def __init__(self): super(DigitCapsules, self).__init__() self.W = nn.Parameter(torch.randn(10, 32, 16*6*6)) self.num_iterations = 3 def forward(self, x): x = x[:, :, None, :] u_hat = torch.matmul(self.W, x) b = torch.zeros(u_hat.size(0), 10, 16, 1) if x.is_cuda: b = b.cuda() for i in range(self.num_iterations): c = F.softmax(b, dim=1) s = (c * u_hat).sum(dim=-1, keepdim=True) v = self.squash(s) if i < self.num_iterations - 1: b = b + (u_hat * v).sum(dim=-1, keepdim=True) return v.squeeze() def squash(self, x): norm = x.norm(dim=-1, keepdim=True) x = x / (1 + norm**2) * norm return x # 加载数据集 transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]) train_dataset = datasets.MNIST(root='./data', train=True, transform=transform, download=True) train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True) test_dataset = datasets.MNIST(root='./data', train=False, transform=transform, download=True) test_loader = DataLoader(test_dataset, batch_size=64, shuffle=True) # 定义损失函数优化器 criterion = nn.CrossEntropyLoss() optimizer = optim.Adam(model.parameters(), lr=0.001) # 训练模型 def train(model, train_loader, criterion, optimizer, epoch): model.train() train_loss = 0 correct = 0 for batch_idx, (data, target) in enumerate(train_loader): optimizer.zero_grad() output = model(data) loss = criterion(output, target) loss.backward() optimizer.step() train_loss += loss.item() pred = output.argmax(dim=1, keepdim=True) correct += pred.eq(target.view_as(pred)).sum().item() train_loss /= len(train_loader.dataset) accuracy = 100. * correct / len(train_loader.dataset) print('Epoch: {} Train set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)'.format( epoch, train_loss, correct, len(train_loader.dataset), accuracy)) return train_loss, accuracy # 测试模型 def test(model, test_loader, criterion): model.eval() test_loss = 0 correct = 0 y_true = [] y_pred = [] with torch.no_grad(): for data, target in test_loader: output = model(data) test_loss += criterion(output, target).item() pred = output.argmax(dim=1, keepdim=True) correct += pred.eq(target.view_as(pred)).sum().item() y_true.extend(target.tolist()) y_pred.extend(pred.view(-1).tolist()) test_loss /= len(test_loader.dataset) accuracy = 100. * correct / len(test_loader.dataset) print('Test set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)'.format( test_loss, correct, len(test_loader.dataset), accuracy)) conf_mat = confusion_matrix(y_true, y_pred) return test_loss, accuracy, conf_mat # 训练测试模型 model = CapsuleNet() train_losses = [] train_accuracies = [] test_losses = [] test_accuracies = [] conf_matrices = [] for epoch in range(1, 11): train_loss, train_accuracy = train(model, train_loader, criterion, optimizer, epoch) test_loss, test_accuracy, conf_mat = test(model, test_loader, criterion) train_losses.append(train_loss) train_accuracies.append(train_accuracy) test_losses.append(test_loss) test_accuracies.append(test_accuracy) conf_matrices.append(conf_mat) # 绘制准确率损失率曲线 plt.figure() plt.plot(range(1, 11), train_losses, 'bo-', label='Training loss') plt.plot(range(1, 11), test_losses, 'ro-', label='Test loss') plt.legend() plt.xlabel('Epoch') plt.ylabel('Loss') plt.title('Training and Test Loss') plt.show() plt.figure() plt.plot(range(1, 11), train_accuracies, 'bo-', label='Training accuracy') plt.plot(range(1, 11), test_accuracies, 'ro-', label='Test accuracy') plt.legend() plt.xlabel('Epoch') plt.ylabel('Accuracy') plt.title('Training and Test Accuracy') plt.show() # 绘制混淆矩阵热图 for i in range(10): conf_matrix = conf_matrices[i] conf_matrix = conf_matrix / conf_matrix.sum(axis=1, keepdims=True) plt.figure() plt.imshow(conf_matrix, cmap='Blues', vmin=0, vmax=1) plt.colorbar() plt.xticks(range(10)) plt.yticks(range(10)) plt.xlabel('Predicted label') plt.ylabel('True label') plt.title('Confusion Matrix (Epoch {})'.format(i+1)) plt.show() ``` 上述代码实现了胶囊网络模型的训练测试,包括准确率、损失率混淆矩阵热图的计算可视化。其中,胶囊网络模型使用PyTorch实现,数据集使用了MNIST,并且使用了Adam优化器交叉熵损失函数。训练过程中,每个epoch都会输出训练集的平均损失准确率,测试集的平均损失、准确率混淆矩阵,然后绘制准确率损失率曲线以及混淆矩阵热图。
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值