从0到1，AI我来了- （1）从AI手写数字识别开始

最新推荐文章于 2024-07-27 17:06:15 发布

zhm6422107

最新推荐文章于 2024-07-27 17:06:15 发布

阅读量412

点赞数 3

分类专栏： AI 文章标签：人工智能 pytorch MNIST 深度学习

本文链接：https://blog.csdn.net/zhm6422107/article/details/140675520

版权

AI 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

前两篇我们我们把控制台、Python环境Anaconda 搞定了，接下来，我们快速进入主题，把AI 界的“Hello World” 实现一下，有个感觉，再逐步了解一些AI的概念。

1、Pytorch 安装

1) 什么是Pytorch?

一个深度学习框架，封装了很多深度学习相关函数，目前是pytorch已成为最受欢迎的深度学习框架之一，除了它，目前还有一些在用的TensorFlow、Keras、MXNet、Caffe 等。

2) 为什么用Pytorch?

我们公司在用它，行业主流也在用它，我没必要一开始学冷门，出了问题，找不到“知音”

3）如何用上Pytorch?

官网介绍在这里：Start Locally | PyTorch

macOS 通过Anaconda 安装pytorch

conda install pytorch torchvision -c pytorch

正常情况下，你会遇到我一样的错误

❯ conda install pytorch torchvision -c pytorch

Channels:
 - pytorch
 - https://mirrors.ustc.edu.cn/anaconda/pkgs/free
 - https://mirrors.ustc.edu.cn/anaconda/pkgs/main
 - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
 - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
 - defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): failed

CondaHTTPError: HTTP 429 TOO MANY REQUESTS for url <https://mirrors.ustc.edu.cn/anaconda/pkgs/free/osx-arm64/repodata.json>
Elapsed: 00:26.256248

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
'https//mirrors.ustc.edu.cn/anaconda/pkgs/free/osx-arm64'

解决办法我放到下方的附录部分了：

重新执行安装

❯ conda install pytorch torchvision -c pytorch

Channels:
 - pytorch
 - defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /opt/anaconda3

  added / updated specs:
    - pytorch
    - torchvision


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    libjpeg-turbo-2.0.0        |       h1a28f6b_0         386 KB  defaults
    pytorch-2.4.0              |         py3.12_0        57.6 MB  pytorch
    torchvision-0.19.0         |        py312_cpu         6.8 MB  pytorch
    ------------------------------------------------------------
                                           Total:        64.8 MB

The following NEW packages will be INSTALLED:

  libjpeg-turbo      anaconda/pkgs/main/osx-arm64::libjpeg-turbo-2.0.0-h1a28f6b_0
  pytorch            pytorch/osx-arm64::pytorch-2.4.0-py3.12_0
  torchvision        pytorch/osx-arm64::torchvision-0.19.0-py312_cpu


Proceed ([y]/n)? y


Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

验证一下，没有报错，则安装成功了

❯ python
Python 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 10:07:17) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>>

2、开始写程序

先有个体感，从MNIST 手写的一批数字中，训练出一个模型，然后再拿一部分MNIST数据区验证模型准确率。下面的程序打印了一些识别错误的反例。

先跑跑，下篇我们逐行分析一下。

import torch  
import torch.nn as nn  
import torch.optim as optim  
from torch.utils.data import DataLoader  
from torchvision import datasets, transforms  
from torch.utils.tensorboard import SummaryWriter  
import matplotlib.pyplot as plt  

# 数据加载与预处理  
transform = transforms.Compose([  
    transforms.ToTensor(),  
    transforms.Normalize((0.5,), (0.5,))  
])  

train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)  
test_dataset = datasets.MNIST(root='./data', train=False, download=True, transform=transform)  
print("load success")
train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)  
test_loader = DataLoader(test_dataset, batch_size=64, shuffle=False)  

# 模型定义  
class SimpleCNN(nn.Module):  
    def __init__(self):  
        super(SimpleCNN, self).__init__()  
        self.conv1 = nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1)  
        self.conv2 = nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1)  
        self.pool = nn.MaxPool2d(2, 2)  
        self.fc1 = nn.Linear(64 * 7 * 7, 128)  
        self.fc2 = nn.Linear(128, 10)  

    def forward(self, x):  
        x = self.pool(nn.ReLU()(self.conv1(x)))  
        x = self.pool(nn.ReLU()(self.conv2(x)))  
        x = x.view(-1, 64 * 7 * 7)  
        x = nn.ReLU()(self.fc1(x))  
        x = self.fc2(x)  
        return x  

# 训练设置  
model = SimpleCNN()  
criterion = nn.CrossEntropyLoss()  
optimizer = optim.Adam(model.parameters(), lr=0.001)  
num_epochs = 10  

# TensorBoard 初始化  
writer = SummaryWriter()  

# 记录损失和准确率  
loss_values = []  
accuracy_values = []  

# 训练循环  
for epoch in range(num_epochs):  
    running_loss = 0.0  
    correct = 0  
    total = 0  
    
    for images, labels in train_loader:  
        optimizer.zero_grad()  
        outputs = model(images)  
        loss = criterion(outputs, labels)  
        loss.backward()  
        optimizer.step()  

        running_loss += loss.item()  

        _, predicted = torch.max(outputs.data, 1)  
        total += labels.size(0)  
        correct += (predicted == labels).sum().item()  
    
    avg_loss = running_loss / len(train_loader)  
    accuracy = 100 * correct / total  

    # 记录到 TensorBoard  
    writer.add_scalar('Loss/train', avg_loss, epoch)  
    writer.add_scalar('Accuracy/train', accuracy, epoch)  

    # 记录到列表  
    loss_values.append(avg_loss)  
    accuracy_values.append(accuracy)  

    print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {avg_loss:.4f}, Accuracy: {accuracy:.2f}%')  

writer.close()  

# 测试模型  
model.eval()  
test_loss = 0.0  
correct = 0  
total = 0  
i=0

with torch.no_grad():  
    for images, labels in test_loader:  
        outputs = model(images)  
        loss = criterion(outputs, labels)  
        test_loss += loss.item()  

        _, predicted = torch.max(outputs.data, 1)  
        total += labels.size(0)  
        correct += (predicted == labels).sum().item() 
        
        if (predicted[i] != labels[i] ):
            print("我猜是",predicted[i],"实际是",labels[i])
            plt.imshow(images[i].reshape(28,28),cmap='Greys', interpolation='nearest')
            plt.show()
        
        

test_loss /= len(test_loader)  
test_accuracy = 100 * correct / total  
print(f'Test Loss: {test_loss:.4f}, Test Accuracy: {test_accuracy:.2f}%')  

# 使用 Matplotlib 绘制损失和准确率  
plt.figure(figsize=(12, 4))  

# 绘制损失图  
plt.subplot(1, 2, 1)  
plt.plot(range(1, num_epochs + 1), loss_values, label='Loss')  
plt.title('Training Loss')  
plt.xlabel('Epochs')  
plt.ylabel('Loss')  
plt.legend()  

# 绘制准确率图  
plt.subplot(1, 2, 2)  
plt.plot(range(1, num_epochs + 1), accuracy_values, label='Accuracy', color='orange')  
plt.title('Training Accuracy')  
plt.xlabel('Epochs')  
plt.ylabel('Accuracy (%)')  
plt.legend()  

plt.tight_layout()  
plt.show()

load success
Epoch [1/10], Loss: 0.1447, Accuracy: 95.53%
Epoch [2/10], Loss: 0.0447, Accuracy: 98.61%
Epoch [3/10], Loss: 0.0303, Accuracy: 99.06%
Epoch [4/10], Loss: 0.0218, Accuracy: 99.28%
Epoch [5/10], Loss: 0.0164, Accuracy: 99.47%
Epoch [6/10], Loss: 0.0136, Accuracy: 99.52%
Epoch [7/10], Loss: 0.0105, Accuracy: 99.65%
Epoch [8/10], Loss: 0.0079, Accuracy: 99.75%
Epoch [9/10], Loss: 0.0076, Accuracy: 99.75%
Epoch [10/10], Loss: 0.0078, Accuracy: 99.72%

附录

1、Http 429 限制问题解决，找到用户目录的 .condarc 文件

vim ~/.condarc

vim 编辑后，替换文件内容一下为下方的内容

channels:
  - defaults
show_channel_urls: true
default_channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:
  conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  menpo: https://mirrors.tuna.tsinghua.edu

再执行安装，即可