PyTorch1.5训练LeNet
文档描述
本样例使用成都人工智能计算中心(以下简称智算中心)ModelArts平台,通过PyTorch自定义镜像训练LeNet,为大家学习ModelArts提供参考。
前置条件
请检查以下条件要求是否满足,如不满足请按照备注进行相应处理。
条件 | 要求 | 备注 |
---|---|---|
PyTorch镜像版本 | >=21.0.3 | PyTorch镜像为Ascend Hub镜像,请参考容器镜像服务指南下载并上传镜像至智算中心镜像服务 |
OBS Browser+ | 安装 | 请参考对象存储服务指南完成下载安装并登录,方便后续上传代码、数据集等 |
模型迁移 | 迁移完成 | 请参考PyTorch网络模型移植&训练指南完成NPU版PyTorch的网络模型迁移,可在modelarts调试 |
训练环境
- 训练平台:成都人工智能计算中心-ModelArts服务
- 底层硬件资源:Atlas 900 集群
- 训练镜像:ascend-pytorch-arm-modelarts
- 训练框架:PyTorch1.5
- CANN版本:5.0.3
LeNet描述
LeNet是1998年提出的一种典型的卷积神经网络。它被用于数字识别并取得了巨大的成功。
论文: Y.Lecun, L.Bottou, Y.Bengio, P.Haffner.Gradient-Based Learning Applied to Document Recognition.Proceedings of the IEEE.1998.
模型架构
LeNet非常简单,包含5层,由2个卷积层和3个全连接层组成。
数据集
使用的数据集:FashionMNIST(使用torchvision.datasets下载)
- 数据集大小:52.4M,共10个类,6万张 28*28图像
- 训练集:6万张图像
- 测试集:1万张图像
- 数据格式:二进制文件
- 目录结构如下:
└─data
├──FashionMNIST
├──raw
├──t10k-images-idx3-ubyte
├──t10k-labels.idx1-ubyte
├──train-images.idx3-ubyte
├──train-labels.idx1-ubyte
数据拷贝
自定义镜像训练过程中,OBS桶中的训练代码会自动上传至ModelArts分配的环境中,但数据集需手动拷贝(当前版本训练日志和保存的模型文件也需手动拷贝至OBS桶中),具体文档参考MoXing相关API。
# modelarts dataset path
data_path = '/cache/data'
# copy dataset to modelarts from ob,useage:mox.file.copy_parallel('obs://[bucket_name]/[data_dir]', [modelarts_data_dir])
mox.file.copy_parallel('obs://wesley/models/lenet_torch1.5/data', data_path)
# model save path
weight_path = '/cache/weights'
# mkdir on modelarts
os.mkdir(weight_path)
# copy train log to obs from modelarts
mox.file.copy('/tmp/log/train.log',
'obs://wesley/models/lenet_torch1.5/train.log')
# copy models to obs from modelarts
mox.file.copy_parallel(weight_path, 'obs://wesley/models/lenet_torch1.5/weights')
注:ModelArts会挂载硬盘至“/cache”目录,Ascend规格资源该目录大小为3T,用户可以使用此目录来储存临时文件,如数据集和保存的模型文件等。
完整代码
代码基于PyTorch官网教程中的Quickstart修改,代码中的路径请自行修改
import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision import datasets
from torchvision.transforms import ToTensor
import os
import moxing as mox
# Define model
class LeNet5(nn.Module):
def __init__(self):
super().__init__()
# input_shape 28*28
self.conv1 = nn.Sequential(
nn.Conv2d(in_channels=1, out_channels=6,
kernel_size=5, stride=1),
nn.MaxPool2d(kernel_size=2)
)
self.conv2 = nn.Sequential(
nn.Conv2d(in_channels=6, out_channels=16,
kernel_size=5, stride=1),
nn.MaxPool2d(kernel_size=2)
)
self.fc1 = nn.Sequential(
nn.Linear(in_features=4 * 4 * 16, out_features=120)
)
self.fc2 = nn.Sequential(
nn.Linear(in_features=120, out_features=84)
)
self.fc3 = nn.Sequential(
nn.Linear(in_features=84, out_features=10)
)
def forward(self, input):
conv1_output = self.conv1(input) # [28,28,1]
conv2_output = self.conv2(conv1_output)
conv2_output = conv2_output.view(-1, 4 * 4 * 16)
fc1_output = self.fc1(conv2_output)
fc2_output = self.fc2(fc1_output)
output = self.fc3(fc2_output)
return output
data_path = '/cache/data'
weight_path = '/cache/weights'
batch_size = 64
epochs = 5
# if dataset in obs, use mox.file.copy() to copy dataset to modelarts
mox.file.copy_parallel('obs://wesley/models/lenet_torch1.5/data', data_path)
os.mkdir(weight_path)
# Download training data from open datasets.
training_data = datasets.FashionMNIST(
root=data_path,
train=True,
download=False,
transform=ToTensor(),
)
# Download test data from open datasets.
test_data = datasets.FashionMNIST(
root=data_path,
train=False,
download=False,
transform=ToTensor(),
)
# Create data loaders.
train_dataloader = DataLoader(training_data, batch_size=batch_size)
test_dataloader = DataLoader(test_data, batch_size=batch_size)
for X, y in test_dataloader:
print(f"Shape of X [N, C, H, W]: {X.shape}")
print(f"Shape of y: {y.shape} {y.dtype}")
break
# using ascend device 1p
device = "cpu" if not torch.npu.is_available() else f"npu:{torch.npu.current_device()}"
torch.npu.set_device(device)
print(f"Using {device} device")
model = LeNet5().to(device)
print(model)
# loss function
loss_fn = nn.CrossEntropyLoss()
# optimizer
optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)
# define train function
def train(dataloader, model, loss_fn, optimizer):
size = len(dataloader.dataset)
model.train()
for batch, (X, y) in enumerate(dataloader):
X, y = X.to(device), y.to(device)
# Compute prediction error
pred = model(X)
loss = loss_fn(pred, y)
# Backpropagation
optimizer.zero_grad()
loss.backward()
optimizer.step()
if batch % 100 == 0:
loss, current = loss.item(), batch * len(X)
print(f"loss: {loss:>7f} [{current:>5d}/{size:>5d}]")
# define test function
def test(dataloader, model, loss_fn):
size = len(dataloader.dataset)
num_batches = len(dataloader)
model.eval()
test_loss, correct = 0, 0
with torch.no_grad():
for X, y in dataloader:
X, y = X.to(device), y.to(device)
pred = model(X)
test_loss += loss_fn(pred, y).item()
correct += (pred.argmax(1) == y).type(torch.float).sum().item()
test_loss /= num_batches
correct /= size
print(
f"Test Error: \n Accuracy: {(100 * correct):>0.1f}%, Avg loss: {test_loss:>8f} \n")
# training
try:
for t in range(epochs):
print(f"Epoch {t + 1}\n-------------------------------")
train(train_dataloader, model, loss_fn, optimizer)
torch.save(model.state_dict(), f"{weight_path}/lenet_{t + 1}.pth")
print(f"saved model lenet_{t + 1}.pth success")
# copy models to obs from modelarts
mox.file.copy_parallel(weight_path, 'obs://wesley/models/lenet_torch1.5/weights')
test(test_dataloader, model, loss_fn)
print("Done!")
finally:
# copy train log to obs from modelarts
mox.file.copy('/tmp/log/train.log', 'obs://wesley/models/lenet_torch1.5/train.log')
训练过程
创建训练作业
当前新版训练作业无法使用自定义镜像(非华为云ModelArts),请使用旧版训练作业,步骤如下:
1.算法来源:
选择自定义
2.镜像地址:
先择前置条件中的PyTorch21.0.3镜像
3.代码目录:
选择OBS桶中的训练代码目录
4.运行命令:
默认启动命令如下,可在本地运行PyTorch21.0.3镜像,在/home/work/run_train.sh
中查看或修改。
/bin/bash /home/work/run_train.sh ${obs-code-path} ${the-base-name-of-obs-code-path}/${boot-file} '/tmp/log/train.log' ${python_file_parameter}
前两个参数不可修改,前五个参数的顺序不可修改,示例:
/bin/bash /home/work/run_train.sh 'obs://training-bucket/ascend-tf-1.15/resnet50/' 'resnet50/train.py' '/tmp/log/train.log' --'data_url'='obs://training-bucket/cifar-10/'
5.数据来源
数据集通过MoXing手动拷贝,此处可选择OBS桶中训练代码目录下的空文件夹
6.训练输出位置
训练日志和模型文件通过MoXing手动拷贝,此处可选择OBS桶中训练代码目录下的空文件夹
7.环境变量
训练中如需添加环境变量,点击增加即可
8.作业日志路径
该日志为训练过程中Ascend产生的plog日志,选择OBS桶中指定的文件夹即可
9.资源池
按照所需资源选择后点击创建作业即可
其他Python依赖
如果环境中没有训练代码所需Python依赖,在OBS桶中训练代码目录下上传requirements.txt
,在其中添加所需依赖及版本即可
查看训练结果
查看训练日志
...
...
Epoch 5
-------------------------------
loss: 1.047490 [ 0/60000]
loss: 1.055037 [ 6400/60000]
loss: 0.821756 [12800/60000]
loss: 0.984526 [19200/60000]
loss: 0.920349 [25600/60000]
loss: 0.918822 [32000/60000]
loss: 0.933400 [38400/60000]
loss: 0.944582 [44800/60000]
loss: 0.886811 [51200/60000]
loss: 0.894010 [57600/60000]
saved model lenet_5.pth success
Test Error:
Accuracy: 67.5%, Avg loss: 0.883350
Done!
THPModule_npu_shutdown success.
[Modelarts Service Log]2022-05-16 09:51:58,597 - INFO - Begin destroy training processes
[Modelarts Service Log]2022-05-16 09:51:58,598 - INFO - proc-rank-0-device-0 (pid: 121) has exited
[Modelarts Service Log]2022-05-16 09:51:58,598 - INFO - End destroy training processes
...
...
[ModelArts Service Log][INFO][2022/05/16 09:51:59]: env MA_OUTPUTS is not found, skip the outputs handler
[ModelArts Service Log]modelarts-pipe: total length: 184
[Modelarts Service Log]Training end at 2022-05-16-09:52:00
[Modelarts Service Log]Training completed.
查看拷贝文件
训练结束后,查看OBS训练代码目录下模型、日志等是否拷贝成功