softmax 回归分类

最新推荐文章于 2024-02-08 21:27:57 发布

古风子

最新推荐文章于 2024-02-08 21:27:57 发布

阅读量923

点赞数

分类专栏： pytorch实践文章标签：深度学习 pytorch 机器学习

本文链接：https://blog.csdn.net/jiadongfengyahoo/article/details/122311720

版权

pytorch实践专栏收录该内容

26 篇文章 1 订阅

订阅专栏

获取数据集

Fashion-MNIST中⼀一共包括了了10个类别，分别为t-shirt（T恤）、 trouser（裤⼦子）、 pullover（套衫）、
dress（连⾐衣裙）、 coat（外套）、 sandal（凉鞋）、 shirt（衬衫）、 sneaker（运动鞋）、
bag（包）和ankle boot（短靴）。

import torch
import torchvision
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
import time
import sys
sys.path.append('..')

mnist_train =torchvision.datasets.FashionMNIST(root='./Datasets/FashionMNIST',train=True, download=True, transform=transforms.ToTensor())
mnist_test =torchvision.datasets.FashionMNIST(root='./Datasets/FashionMNIST',train=False, download=True, transform=transforms.ToTensor())

FashionMNIST返回一个Dataset对象， Dataset是 PyTorch 中用来表示数据集的一个抽象类。如果数据集用这个类来表示，至少覆写下面两个方法

len：数据集大小
getitem：实现这个方法后，可以通过下标的方式（ dataset[i] ）的来取得第 i 个数据

mnist_train

Dataset FashionMNIST
    Number of datapoints: 60000
    Root location: ./Datasets/FashionMNIST
    Split: Train
    StandardTransform
Transform: ToTensor()

因此，我们可以通过下标的方式返回单个数据数据进行查看

features, labels = mnist_train[0]

features.shape, labels#第一个图片对应的尺寸和标签

(torch.Size([1, 28, 28]), 9)

数据中存储的是各个图片对应的标签，为了展示，转化成对应的类名

def get_fashion_mnist_labels(labels):
    text_labels = ['t-shirt', 'trouser', 'pullover', 'dress','coat','sandal', 'shirt', 'sneaker', 'bag', 'ankleboot']
    return [text_labels[int(i)] for i in labels]

显示图片

def show_fashion_mnist(images, labels):
    _, figs = plt.subplots(1, len(images), figsize=(12, 12))
    for f, img, lbl in zip(figs, images, labels):
        f.imshow(img.view((28, 28)).numpy())
        f.set_title(lbl)
        f.axes.get_xaxis().set_visible(False)
        f.axes.get_yaxis().set_visible(False)
    plt.show()
X, y = [], []
for i in range(10):
    X.append(mnist_train[i][0])
    y.append(mnist_train[i][1])
show_fashion_mnist(X, get_fashion_mnist_labels(y))

请添加图片描述

为了批量化训练和测试，我们将数据改成批量化形式，提高训练和测试速度

batch_size = 256
num_workers = 4
train_iter = torch.utils.data.DataLoader(mnist_train,batch_size=batch_size, shuffle=True, num_workers=num_workers)
test_iter = torch.utils.data.DataLoader(mnist_test,batch_size=batch_size, shuffle=False, num_workers=num_workers)

初始化模型权重参数

使用回归的方式预测图片大小，每个图片28x28个像素，表示网络的输入是28个值，输出是10个值，根据矩阵相乘的概念的，W的形状应该是[784, 10],以保证X*W的输出是10

import numpy as np
num_inputs = 784 #单个图片像素个数 28x28
num_outputs = 10 #分类格式
W  = torch.tensor(np.random.normal(0,0.01, (num_inputs, num_outputs)), dtype = torch.float)
b = torch.zeros(num_outputs, dtype = torch.float)
W.requires_grad_(requires_grad=True)
b.requires_grad_(requires_grad=True)

tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], requires_grad=True)

X[0].view(-1, num_inputs).shape, W.shape,b.shape

(torch.Size([1, 784]), torch.Size([784, 10]), torch.Size([10]))

xw = torch.mm(X[0].view(-1, num_inputs), W) #矩阵相乘,得到10个输出（因为有10个分类）
xw.shape

torch.Size([1, 10])

softmax运算

softmax函数，又称归一化指数函数。
它是二分类函数sigmoid在多分类上的推广，目的是将多分类的结果以概率的形式展现出来。函数定义方式如下：

def softmax(X):
    X_exp = X.exp()
    partition = X_exp.sum(dim=1, keepdim=True)
    return X_exp / partition # 这⾥里里应⽤用了了⼴广播机制

output = softmax(xw)#转程分类概率输出格式
output

tensor([[0.1119, 0.1234, 0.1014, 0.1312, 0.0873, 0.0978, 0.0950, 0.0897, 0.0783,
         0.0840]], grad_fn=<DivBackward0>)

每个图片的网络输出经过softmax处理，得到10个范围在[0,1]的输出，且各个输出的和为1

定义模型

根据以上步骤的验证，很容易定义网络的结构

def net(X):
    xw = torch.mm(X.view(-1, num_inputs), W) +b # y = X.W+B
    return softmax(xw)#输出分类概率值

output = net(X[0])
output

tensor([[0.1119, 0.1234, 0.1014, 0.1312, 0.0873, 0.0978, 0.0950, 0.0897, 0.0783,
         0.0840]], grad_fn=<DivBackward0>)

定义损失函数

使用gather函数

y_hat = output #两个图片预测的对应的3个分类的阈值
y = torch.LongTensor([9])#图片0对应的真实标签id
y_hat.gather(1, y.view(-1, 1))#根据真实标签，获取标签位置出的阈值

tensor([[0.0840]], grad_fn=<GatherBackward>)

交叉熵损失函数

def cross_entropy(y_hat, y):
    return - torch.log(y_hat.gather(1, y.view(-1, 1)))

def accuracy(y_hat, y):
    return (y_hat.argmax(dim=1) == y).float().mean().item()

y_hat

tensor([[0.1119, 0.1234, 0.1014, 0.1312, 0.0873, 0.0978, 0.0950, 0.0897, 0.0783,
         0.0840]], grad_fn=<DivBackward0>)

y_hat.argmax(dim=1)#输出的10个分类阈值，最大值对应的标签id，通过改方法，可以得到预测值对应的预测标签

tensor([3])

def sgd(params, lr, batch_size): 
    for param in params:
        param.data -= lr * param.grad / batch_size # 注意这⾥里里更更改
        
def evaluate_accuracy(data_iter, net):
    acc_sum, n = 0.0, 0
    for X, y in data_iter:
        acc_sum += (net(X).argmax(dim=1) == y).float().sum().item()
        n += y.shape[0]
    return acc_sum / n


def train_ch3(net, train_iter, test_iter, loss, num_epochs, batch_size, params=None, lr=None, optimizer=None):
    for epoch in range(num_epochs):
        train_l_sum, train_acc_sum, n = 0.0, 0.0, 0
        for X, y in train_iter:
            y_hat = net(X)
            l = loss(y_hat, y).sum()
            # 梯度清零
            if optimizer is not None:
                optimizer.zero_grad()
            elif params is not None and params[0].grad is not None:
                for param in params:
                    param.grad.data.zero_()
            l.backward()
            if optimizer is None:
                sgd(params, lr, batch_size)
            else:
                optimizer.step() # “softmax回归的简洁实现”⼀一节将⽤用到
            train_l_sum += l.item()
            train_acc_sum += (y_hat.argmax(dim=1) ==y).sum().item()
            n += y.shape[0]
        test_acc = evaluate_accuracy(test_iter, net)
        print('epoch %d, loss %.4f, train acc %.3f, test acc %.3f'% (epoch + 1, train_l_sum / n, train_acc_sum / n,test_acc))

num_epochs = 30
lr = 0.01
train_ch3(net, train_iter, test_iter, cross_entropy, num_epochs, batch_size, [W, b], lr)

epoch 1, loss 1.3667, train acc 0.640, test acc 0.682
epoch 2, loss 0.9165, train acc 0.716, test acc 0.725
epoch 3, loss 0.8031, train acc 0.748, test acc 0.744
epoch 4, loss 0.7431, train acc 0.767, test acc 0.761
epoch 5, loss 0.7034, train acc 0.780, test acc 0.770
epoch 6, loss 0.6746, train acc 0.788, test acc 0.777
epoch 7, loss 0.6523, train acc 0.794, test acc 0.784
epoch 8, loss 0.6343, train acc 0.799, test acc 0.787
epoch 9, loss 0.6193, train acc 0.804, test acc 0.792
epoch 10, loss 0.6067, train acc 0.807, test acc 0.794
epoch 11, loss 0.5958, train acc 0.809, test acc 0.799
epoch 12, loss 0.5863, train acc 0.812, test acc 0.800
epoch 13, loss 0.5778, train acc 0.815, test acc 0.804
epoch 14, loss 0.5703, train acc 0.816, test acc 0.804
epoch 15, loss 0.5635, train acc 0.819, test acc 0.808
epoch 16, loss 0.5573, train acc 0.819, test acc 0.809
epoch 17, loss 0.5517, train acc 0.821, test acc 0.810
epoch 18, loss 0.5465, train acc 0.822, test acc 0.810
epoch 19, loss 0.5418, train acc 0.824, test acc 0.812
epoch 20, loss 0.5374, train acc 0.825, test acc 0.813
epoch 21, loss 0.5333, train acc 0.827, test acc 0.813
epoch 22, loss 0.5294, train acc 0.827, test acc 0.815
epoch 23, loss 0.5258, train acc 0.829, test acc 0.816
epoch 24, loss 0.5225, train acc 0.829, test acc 0.816
epoch 25, loss 0.5194, train acc 0.830, test acc 0.817
epoch 26, loss 0.5164, train acc 0.830, test acc 0.818
epoch 27, loss 0.5135, train acc 0.831, test acc 0.818
epoch 28, loss 0.5110, train acc 0.832, test acc 0.820
epoch 29, loss 0.5083, train acc 0.832, test acc 0.821
epoch 30, loss 0.5060, train acc 0.833, test acc 0.821

整体的准确率在82%左右

整合代码

from utils.SlowFast import SlowFast1
from utils.SlowFast import get_fashion_data

slowfast1 = SlowFast1(image_size = 28*28, class_nums = 10)
batch_size = 256
train_iter, test_iter = get_fashion_data(img_dir='./Datasets/FashionMNIST', batch_size = batch_size)
num_epochs = 30
params = [slowfast1.W, slowfast1.b]
lr = 0.01
slowfast1.train(slowfast1.net, train_iter, test_iter, slowfast1.cross_entropy, num_epochs, batch_size, params, lr)

epoch 1, loss 1.3701, train acc 0.643, test acc 0.679
epoch 2, loss 0.9185, train acc 0.716, test acc 0.724
epoch 3, loss 0.8041, train acc 0.749, test acc 0.743
epoch 4, loss 0.7436, train acc 0.768, test acc 0.756
epoch 5, loss 0.7040, train acc 0.779, test acc 0.772
epoch 6, loss 0.6748, train acc 0.788, test acc 0.778
epoch 7, loss 0.6526, train acc 0.795, test acc 0.783
epoch 8, loss 0.6344, train acc 0.799, test acc 0.788
epoch 9, loss 0.6193, train acc 0.803, test acc 0.792
epoch 10, loss 0.6067, train acc 0.806, test acc 0.795
epoch 11, loss 0.5958, train acc 0.810, test acc 0.799
epoch 12, loss 0.5862, train acc 0.812, test acc 0.801
epoch 13, loss 0.5778, train acc 0.815, test acc 0.804
epoch 14, loss 0.5702, train acc 0.817, test acc 0.806
epoch 15, loss 0.5633, train acc 0.818, test acc 0.809
epoch 16, loss 0.5572, train acc 0.820, test acc 0.810
epoch 17, loss 0.5516, train acc 0.821, test acc 0.811
epoch 18, loss 0.5464, train acc 0.822, test acc 0.813
epoch 19, loss 0.5417, train acc 0.824, test acc 0.812
epoch 20, loss 0.5371, train acc 0.826, test acc 0.814
epoch 21, loss 0.5331, train acc 0.826, test acc 0.813
epoch 22, loss 0.5294, train acc 0.827, test acc 0.815
epoch 23, loss 0.5258, train acc 0.828, test acc 0.816
epoch 24, loss 0.5224, train acc 0.829, test acc 0.816
epoch 25, loss 0.5192, train acc 0.830, test acc 0.817
epoch 26, loss 0.5162, train acc 0.831, test acc 0.819
epoch 27, loss 0.5135, train acc 0.831, test acc 0.819
epoch 28, loss 0.5107, train acc 0.832, test acc 0.819
epoch 29, loss 0.5083, train acc 0.832, test acc 0.820
epoch 30, loss 0.5058, train acc 0.833, test acc 0.821

完成代码地址：SlowFast1