文章目录
- 关于Anaconda的一些命令
- Dataset
- tensorboard
- transforms
- torchvision中的数据集使用
- DataLoader中的数据集使用
- torch.nn
- nn.Module的使用
- torch.nn 是 torch.nn.functional封装
- torch.nn.CONV2D
- Pooling layers的nn.MaxPool2d
- Padding layer
- Non-linear Activations
- Normalization Layers
- Recurrent Layers
- Transformer Layers
- Linear Layers
- Dropout Layers
- Sparse Layers
- Distance Functions
- Loss Functions
- Vision Layers
- Shuffle Layers
- DataParallel Layers (multi-GPU, distributed)
- Utilities
- Quantized Functions
- Lazy Modules Initialization
- TORCH.OPTIM
- 使用和修改现有的网络模型
- 网络模型的保存和读取
- 模型训练的流程
- Question & Answer
- Python中with的使用
- RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor
- 使用writer.add_images发生AssertionError
- AssertionError: size of input tensor and input format are different. tensor shape: (64, 3, 32, 32), input_format: CHW
- urllib.error.URLError:
关于Anaconda的一些命令
原文
https://blog.csdn.net/qq_42257666/article/details/121361983
https://blog.csdn.net/ITLearnHall/article/details/81708148?spm=1001.2101.3001.6650.1&utm_medium=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7Edefault-1.pc_relevant_default&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7Edefault-1.pc_relevant_default&utm_relevant_index=2
cmd命令行
查看CUDA版本
nvidia-smi
进入Anaconda
conda
activate
在Anaconda安装 pytorch
conda install pytorch torchvision cudatoolkit=10.1
创建conda虚拟环境
conda create --name pytorch1 python=3.8
进入创建的conda
conda activate pytorch1
移除创建的conda
conda remove -n pytorch1 --all
展示在Anaconda中创建的环境
conda env list
Dataset
类继承Dataset
# MyData 类
class MyData(Dataset):
def __init__(self, root_dir, label_dir):
self.root_dir = root_dir
self.label_dir = label_dir
self.path = os.path.join(self.root_dir, self.label_dir)
self.img_path_list = os.listdir(self.path)
def __getitem__(self, index):
# 得到dataset图片
img_name = self.img_path_list[index]
img_item_path = os.path.join(self.root_dir,self.label_dir,img_name)
img = Image.open(img_item_path)
label = self.label_dir
return img,label
def __len__(self):
return len(self.img_path_list)
使用
# 实例
root_dir = "dataset1/train"
# 获得ants数据集
ants_label_dir = "ants_image"
ant_dataset = MyData(root_dir=root_dir, label_dir=ants_label_dir)
# 获得bees数据集
bees_label_dir = "bees_image"
bee_dataset = MyData(root_dir=root_dir, label_dir=bees_label_dir)
# 整合bees和anys 的数据集
train_dataset = ant_dataset + bee_dataset
tensorboard
使用tensorboard
from torch.utils.tensorboard import SummaryWriter
import numpy as np
from PIL import Image
writer = SummaryWriter("logs")
img_path = "dataset1/train/bees_image/92663402_37f379e57a.jpg"
img = Image.open(img_path)
img_array = np.array(img)
# 图像数据类型 img_tensor
writer.add_image("img_array1",img_array,2,dataformats='HWC')
for i in range(100):
# Args:
# tag (string): Data title
# scalar_value : y Value
# global_step (int): x Value
writer.add_scalar("y=x",3*i,i) # 添加标量
writer.close()
打开tensorboard生成的事务文件
Terminal
tensorboard --logdir=logs
-port=6007
将图片的格式PIL.JpegImagePlugin.JpegImageFile转成add_image接受的格式
pip install opencv-python
numpy.array
img_array = np.array(img)
transforms
transforms.ToTensor
作用
将数据转换成神经网络需要的格式
将数据转换成为 Tensor
使用
img_path = "dataset1/train/bees_image/92663402_37f379e57a.jpg"
img = Image.open(img_path)
# img_array = np.array(img)
tensor_trans = transforms.ToTensor()
tensor_img = tensor_trans(img)
transforms.Normalize
output[channel] = (input[channel] - mean[channel]) / std[channel]
例子
0 < input[channel] < 1
mean =[0.5,0.5,0.5] #len(mean) = channel
std =[0.5,0.5,0.5]
output[channel=0] = input[channel=0] * 2 - 1
-1 < output[channel=0] < 1
transforms.Resize & transforms.Compose
# 使用Resize 对图片进行任意的缩放 512,512
print(img.size)
trans_resize = transforms.Resize((512, 512))
img_resize1 = trans_resize(img)
print(img_resize1)
# PIL To TOTENSOR
img_resize1 = trans_totensor(img_resize1)
writer.add_image("img_resize1", img_resize1)
# 使用Resize 对图片进行等比例的缩放 735 550
print(img.size)
trans_resize = transforms.Resize(550)
trans_compose = transforms.Compose([trans_resize, trans_totensor])
# img_resize2 = trans_resize(img)
# img_resize2 = trans_totensor(img_resize2)
# PIL To TOTENSOR
img_resize2 = trans_compose1(img)
print(img_resize2.size)
writer.add_image("img_resize2", img_resize2)
transforms.RandomCrop
# 使用RandomCrop 随机裁剪指定大小的图片
trans_randomCrop = transforms.RandomCrop([30,30])
trans_compose2 = transforms.Compose([trans_randomCrop, trans_totensor])
for i in range(10):
img_randomCrop = trans_compose2(img)
print(img_randomCrop.size)
# PIL To TOTENSOR
writer.add_image("img_randomCrop", img_randomCrop,i+1)
使用方法总结
关注方法描述的 输入和输出
关注方法的参数
torchvision中的数据集使用
CIFAR10
https://pytorch.org/vision/stable/generated/torchvision.datasets.CIFAR10.html#torchvision.datasets.CIFAR10
Parameters
- root (string)
– Root directory of dataset where directory cifar-10-batches-py exists or will be saved to if download is set to True.
- train (bool, optional)
– If True, creates dataset from training set, otherwise creates from test set.
- transform (callable, optional)
– A function/transform that takes in an PIL image and returns a transformed version. E.g, transforms.RandomCrop
- target_transform (callable, optional)
– A function/transform that takes in the target and transforms it.
- download (bool, optional)
– If true, downloads the dataset from the internet and puts it in root directory. If dataset is already downloaded, it is not downloaded again.
code
import torchvision
from torch.utils.tensorboard import SummaryWriter
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
dataset_transform = torchvision.transforms.Compose([torchvision.transforms.ToTensor()])
train_set = torchvision.datasets.CIFAR10(root="./dataset3", transform=dataset_transform, train=True, download=True)
test_set = torchvision.datasets.CIFAR10(root="./dataset3", transform=dataset_transform, train=False, download=True)
# 写事务中
writer = SummaryWriter("log_CIFAR10")
for i in range(10):
img, target = test_set[i]
writer.add_image("test_set", img, i)
writer.close()
DataLoader中的数据集使用
https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader
torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=False,
sampler=None, batch_sampler=None, num_workers=0,
collate_fn=None, pin_memory=False, drop_last=False,
timeout=0, worker_init_fn=None, multiprocessing_context=None,
generator=None, *, prefetch_factor=2, persistent_workers=False)
Parameters
dataset (Dataset) – dataset from which to load the data.
batch_size (int, optional) 随机从dataset中抓取batch_size数量的 对应数据
– how many samples per batch to load (default: 1).
shuffle (bool, optional) 决定每次DataLoader的结果是否一样
– set to True to have the data reshuffled at every epoch (default: False).
sampler (Sampler or Iterable, optional) – defines the strategy to draw samples from the dataset. Can be any Iterable with __len__ implemented. If specified, shuffle must not be specified.
batch_sampler (Sampler or Iterable, optional) – like sampler, but returns a batch of indices at a time. Mutually exclusive with batch_size, shuffle, sampler, and drop_last.
num_workers (int, optional) 控制进程数进程
– how many subprocesses to use for data loading. 0 means that the data will be loaded in the main process. (default: 0)
collate_fn (callable, optional) – merges a list of samples to form a mini-batch of Tensor(s). Used when using batched loading from a map-style dataset.
pin_memory (bool, optional) – If True, the data loader will copy Tensors into CUDA pinned memory before returning them. If your data elements are a custom type, or your collate_fn returns a batch that is a custom type, see the example below.
drop_last (bool, optional) 处理多出来的数据
– set to True to drop the last incomplete batch, if the dataset size is not divisible by the batch size. If False and the size of dataset is not divisible by the batch size, then the last batch will be smaller. (default: False)
timeout (numeric, optional) – if positive, the timeout value for collecting a batch from workers. Should always be non-negative. (default: 0)
worker_init_fn (callable, optional) – If not None, this will be called on each worker subprocess with the worker id (an int in [0, num_workers - 1]) as input, after seeding and before data loading. (default: None)
torch.nn
nn.Module的使用
Python API https://pytorch.org/docs/stable/nn.html
NN -Neural Networks
Containers
容器、骨架
模块
Module
Base class for all neural network modules.
提供模板
import torch.nn as nn
import torch.nn.functional as F
class Model(nn.Module):
def __init__(self):
super().__init__()
self.conv1 = nn.Conv2d(1, 20, 5)
self.conv2 = nn.Conv2d(20, 20, 5)
# 处理输入的函数
# 定义每次调用时执行的计算。
# 应该被所有子类覆盖。
def forward(self, x):
# self.conv1(x) 对输入 x, 进行卷积
# F.relu() 非线性处理
x = F.relu(self.conv1(x))
# 二次卷积self.conv2(x)
return F.relu(self.conv2(x))
Sequential
A sequential container.
ModuleList
Holds submodules in a list.
ModuleDict
Holds submodules in a dictionary.
ParameterList
Holds parameters in a list.
ParameterDict
Holds parameters in a dictionary.
例子
import torch
from torch import nn
class Test_Neural_Networks(nn.Module):
# 参数的设置
def __init__(self):
super().__init__()
# 使用该类
def forward(self, input):
output = input + 1
return output
TNN = Test_Neural_Networks()
tensor_x = torch.tensor(1.0)
output = TNN(tensor_x)
print(output)
Sequential()
code
self.sequential1 = Sequential(
# c=3 32*32
Conv2d(in_channels=3, out_channels=32, kernel_size=(5, 5), stride=(1, 1), padding=2),
# c=32 32*32
MaxPool2d(kernel_size=2),
# c=32 16*16
Conv2d(in_channels=32, out_channels=32, kernel_size=(5, 5), stride=(1, 1), padding=2),
# c=32 16*16
MaxPool2d(kernel_size=2),
# c=32 8*8
Conv2d(in_channels=32, out_channels=64, kernel_size=(5, 5), stride=(1, 1), padding=2),
# c=64 8*8
MaxPool2d(kernel_size=2),
# c=64 4*4
Flatten(),
# c=1 1*1024 高度 = 1
Linear(1024, 64),
# c=1 1*64 高度 = 1
Linear(64, 10)
# c=1 1*10 高度 = 1
)
def forward(self, input):
input = self.conv1(input)
input = self.maxpool1(input)
input = self.conv2(input)
input = self.maxpool2(input)
input = self.conv3(input)
input = self.maxpool3(input)
input = self.flatten1(input)
hidden = self.linear1(input)
output = self.linear2(hidden)
return output
等效于
# c=3 32*32
self.conv1 = Conv2d(in_channels=3, out_channels=32, kernel_size=(5, 5), stride=(1, 1), padding=2)
# c=32 32*32
self.maxpool1 = MaxPool2d(kernel_size=2)
# c=32 16*16
self.conv2 = Conv2d(in_channels=32, out_channels=32, kernel_size=(5, 5), stride=(1, 1), padding=2)
# c=32 16*16
self.maxpool2 = MaxPool2d(kernel_size=2)
# c=32 8*8
self.conv3 = Conv2d(in_channels=32, out_channels=64, kernel_size=(5, 5), stride=(1, 1), padding=2)
# c=64 8*8
self.maxpool3 = MaxPool2d(kernel_size=2)
# c=64 4*4
self.flatten1 = Flatten()
# c=1 1*1024 高度 = 1
self.linear1 = Linear(1024, 64)
# c=1 1*64 高度 = 1
self.linear2 = Linear(64, 10)
# c=1 1*10 高度 = 1
def forward(self, input):
output = self.sequential1(input)
return output
torch.nn 是 torch.nn.functional封装
torch.nn.functional
conv1d
Applies a 1D convolution over an input signal composed of several input planes.
conv2d
对2D卷积进行操作
Applies a 2D convolution over an input image composed of several input planes.
Pameters
input – input tensor of shape (\text{minibatch} , \text{in\_channels} , iH , iW)(minibatch,in_channels,iH,iW)
weight 权重、卷积核
groups 通常设定为1
– filters of shape (\text{out\_channels} , \frac{\text{in\_channels}}{\text{groups}} , kH , kW)(out_channels,groupsin_channels ,kH,kW)
bias 编制
– optional bias tensor of shape (\text{out\_channels})(out_channels). Default: None
stride 卷积核 的步幅和方向(右，右下)
– the stride of the convolving kernel. Can be a single number or a tuple (sH, sW). Default: 1
卷积核的移动为 在输入矩阵上
向右移动sW直到尽头
向下移动SH的左边
向右移动sW直到尽头
向下移动SH的左边
padding 对输入的矩阵两边进行填充，宽度为1 例如 2*2 -> 3*3
介绍
–implicit paddings on both sides of the input. Can be a string {‘valid’, ‘same’}, single number or a tuple (padH, padW). Default: 0 padding='valid' is the same as no padding. padding='same' pads the input so the output has the same shape as the input. However, this mode doesn’t support any stride values other than 1
例子
[[1,2],
[3,4]] -> padding = 1
[[0,0,0,0],
[0,1,2,0],
[0,3,4,0],
[0,0,0,0]]
conv3d
Applies a 3D convolution over an input image composed of several input planes.
conv_transpose1d
Applies a 1D transposed convolution operator over an input signal composed of several input planes, sometimes also called “deconvolution”.
conv_transpose2d
Applies a 2D transposed convolution operator over an input image composed of several input planes, sometimes also called “deconvolution”.
conv_transpose3d
Applies a 3D transposed convolution operator over an input image composed of several input planes, sometimes also called “deconvolution”
unfold
Extracts sliding local blocks from a batched input tensor.
fold
Combines an array of sliding local blocks into a large containing tensor.
code
import torch
from torch import nn
import torch.nn.functional as F
input_matrix_2D = [[1, 2, 0, 3, 1],
[0, 1, 2, 3, 1],
[1, 2, 1, 0, 0],
[5, 2, 3, 1, 1],
[2, 1, 0, 1, 1], ]
input_kernel_2D = [[1, 2, 1],
[0, 1, 0],
[2, 1, 0], ]
matrix_torch = torch.tensor(input_matrix_2D)
kernel_torch = torch.tensor(input_kernel_2D)
# input tensor of shape (\text{minibatch} , \text{in\_channels} , iH , iW)(minibatch,in_channels,iH,iW)
# 2.根据要求做输入矩阵的 变换
# input, (channel,batch_size, matrisx_h, matrisx_w)
matrix_torch = torch.reshape(matrix_torch, (1, 1, 5, 5))
kernel_torch = torch.reshape(kernel_torch, (1, 1, 3, 3))
# out_{i}{j} = input1*w1 + input2*w2+ ... + inputn*wn
output = F.conv2d(input=matrix_torch, weight=kernel_torch, bias=None, stride=1)
print(output)
output2 = F.conv2d(input=matrix_torch, weight=kernel_torch, bias=None, stride=2)
print(output2)
output2 = F.conv2d(input=matrix_torch, weight=kernel_torch, bias=None, stride=1, padding=1)
print(output2)
torch.nn.CONV2D
https://pytorch.org/docs/stable/generated/torch.nn.Conv2d.html#torch.nn.Conv2d
以Layers为结尾应该是 往骨架中填充的程序
Convolution Layers
卷积层
torch.nn.Conv2d(in_channels, out_channels, kernel_size,
stride=1, padding=0, dilation=1, groups=1,
bias=True, padding_mode='zeros', device=None, dtype=None)
Parameters
in_channels (int) – Number of channels in the input image
out_channels (int) – Number of channels produced by the convolution
kernel_size (int or tuple) – Size of the convolving kernel
stride (int or tuple, optional) – Stride of the convolution. Default: 1
padding (int, tuple or str, optional) – Padding added to all four sides of the input. Default: 0
padding_mode (string, optional) – 'zeros', 'reflect', 'replicate' or 'circular'. Default: 'zeros'
dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1
groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – If True, adds a learnable bias to the output. Default: True
code
import torch
import torchvision
from torch import nn
from torch.nn import Conv2d
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
dataset_transform = torchvision.transforms.Compose([torchvision.transforms.ToTensor()])
train_set = torchvision.datasets.CIFAR10(root="./dataset3", transform=dataset_transform, train=True, download=True)
test_set = torchvision.datasets.CIFAR10(root="./dataset3", transform=dataset_transform, train=False, download=True)
dataloader = DataLoader(test_set, batch_size=64, shuffle=True)
# 将32*32像素的图片，经过3*3卷积核，转换未30*30的图片
class simple_neural_network(nn.Module):
def __init__(self):
# 完成simple_neural_network的父类 nn.Module 的初始化
super(simple_neural_network, self).__init__()
self.conv1 = Conv2d(in_channels=3, out_channels=6,
kernel_size=3, stride=1, padding=0)
def forward(self, x):
x = self.conv1(x)
return x
# 使用simple_neural_network
neural_network1 = simple_neural_network()
print(neural_network1)
# 写入tensorboard
writer = SummaryWriter("logs_conv2D")
tage = 0
for data_loder_item in dataloader:
imgs, targets = data_loder_item
# 进行卷积，实施上就是对图片每个 组成像素点的编码的RGB段，这3给channel进行变换，对图片进行放大或缩小（变换）
output = neural_network1(imgs)
# 64 images, 3 is RGB, 32 is height, 32 is width
print(imgs.shape)
print(output.shape)
print("----------------------------------")
writer.add_images("imgs", imgs, tage)
# -1 为无视 batch_size=64,根据reshape来进行计算
output = torch.reshape(output, (-1, 3, 30, 30))
writer.add_images("output", output, tage)
tage = tage + 1
Pooling layers的nn.MaxPool2d
https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html#torch.nn.MaxPool2d
作用
根据n*n的池化核，从输入的矩阵中，选出最大的值
取数据集中更加重要的数据，以达到减少数据量的目的
Parameters
kernel_size – the size of the window to take a max over
stride 默认值为池化核的大小kernel_size，不重复比较值
– the stride of the window. Default value is kernel_size
padding – implicit zero padding to be added on both sides
dilation – a parameter that controls the stride of elements in the window
return_indices – if True, will return the max indices along with the outputs. Useful for torch.nn.MaxUnpool2d later
ceil_mode 向下取整floor，向上取整ceiling
– when True, will use ceil instead of floor to compute the output shape
Padding layer
作用
对输入的tensor进行填充
用的少
Non-linear Activations
作用
给神经网咯引入非线性的特特征
code
class simple_nerual_network(nn.Module):
def __init__(self):
super(simple_nerual_network, self).__init__()
self.relu1 = ReLU()
self.sigmoid1 = Sigmoid()
def forward(self, input):
output = self.relu1(input)
return output
Normalization Layers
作用
加快神经网络的训练速度
code
class simple_nerual_network(nn.Module):
def __init__(self):
super(simple_nerual_network, self).__init__()
self.batchNorm2d1 = BatchNorm2d(3)
def forward(self, input):
output = self.batchNorm2d1(input)
return output
Recurrent Layers
作用
适用于文字识别中，适用于特定的网络结构，适用于特定网络
Transformer Layers
作用
适用于特定的网络结构，适用于特定网络
Linear Layers
https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear
作用
使用较多，适用于特定网络
方法
torch.nn.Linear(in_features, out_features, bias=True, device=None, dtype=None)
nn.Identity
不区分参数的占位符标识运算符。
g1 = k1*x1 + k2*x2 + ... + kn*xn + b
k_i \in (1,n) 为权重 由模型根据输入进行数学运算获得初始化的值
b is bias，由模型根据输入进行数学运算获得初始化的值
g1 为hidden layer的input
insight
神经网络的k_i和b进行调整，让神经网络的预测更加合理
Dropout Layers
作用
防止过拟合
方法
nn.Dropout
在训练过程中，使用来自伯努利分布的样本，以概率p随机地将输入张量的一些元素归零。
Sparse Layers
作用
使用于NLP，适用于特定网络
Distance Functions
作用
计算两个值之间的误差
Loss Functions
作用
衡量module的output和实际的target、label(目标)的差距loss
然后我们可以根据loss，指导module让它的输出更加接近 target(反向传播)
loss越小越好
反向传播
尝试如何调整神经网络的参数，使loss变小
因为是从loss的方向计算，和网络的输出的顺序相反
输出
给每个需要调优的参数一个梯度（grab）
优化的过程中根据梯度选择优化器(optimizer)对参数进行调优
nn.L1Loss
https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html#torch.nn.L1Loss
Creates a criterion that measures the mean absolute error (MAE) between each element in the input x and target y.
Vision Layers
Shuffle Layers
DataParallel Layers (multi-GPU, distributed)
Utilities
Quantized Functions
Lazy Modules Initialization
TORCH.OPTIM
https://pytorch.org/docs/stable/optim.html
code
# lr 是 学习速率
# 构造优化器
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
optimizer = optim.Adam([var1, var2], lr=0.0001)
for input, target in dataset:
# 将grad清零
optimizer.zero_grad()
output = model(input)
loss = loss_fn(output, target)
loss.backward()
optimizer.step()
使用和修改现有的网络模型
https://pytorch.org/vision/stable/index.html
下载数据集
安装下载数据集的工具
pip install scipy
Parameters
pretrained (bool) – If True, returns a model pre-trained on ImageNet
progress (bool) – If True, displays a progress bar of the download to stderr
VGG模型
定义
VGG 16-layer model (configuration “D”) “Very Deep Convolutional Networks For Large-Scale Image Recognition”. The required minimum input size of the model is 32x32.
VGG 16 层模型（配置“D”）“用于大规模图像识别的非常深的卷积网络”。模型所需的最小输入大小为 32x32。
# 在module中添加layer
vgg16_true.classifier.add_module('add_linear',nn.Linear(1000,10))
print(vgg16_true)
# 在module中修改已有的ayer: 修改名称为vgg16_false的module中classifier的索引为6的layer
vgg16_false.classifier[6] = nn.Linear(4096,10)
网络模型的保存和读取
保存
vgg16_false = torchvision.models.vgg16(pretrained='false')
# 保存方法 1 保存模型的结构 和 模型的参数 适用小模型
# torch.save(vgg16_false,"./models1_vgg16.pth")
# 保存方法 2 将vgg16_false的状态中的参数保存成字典，不保存结构 (官方推荐) 适用大模型
torch.save(vgg16_false.state_dict(),"./models2_vgg16.pth")
# 保存方式1的缺点
class simple_nerual_network(nn.Module):
def __init__(self):
。。。
)
def forward(self, input):
output = self.sequential1(input)
return output
nerual_network = simple_nerual_network()
torch.save(nerual_network,"./models3_nerual_network.pth")
保存方法1的缺点
AttributeError: Can't get attribute 'simple_nerual_network'
加载
# 加载方法 1 加载模型的结构 和 模型的参数 适用小模型
vgg16_false1 = torch.load("./models1_vgg16.pth")
# print(vgg16_false1)
# 加载方法 2 加载模型的参数
vgg16_false2 = torchvision.models.vgg16(pretrained='false')
vgg16_false2.load_state_dict(torch.load("./models2_vgg16.pth"))
# print(vgg16_false2)
# 保存方式 1 的缺点
# 需要确保正确的模型
nerual_network = torch.load("./models3_nerual_network.pth")
print(nerual_network)
模型训练的流程
import torchvision
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
from L22_model import *
# 1. 准备数据集 10个标签 每个图片只有一种确定的标签
train_dataset = torchvision.datasets.CIFAR10("./dataset3", train=True,
transform=torchvision.transforms.ToTensor(), download=True)
test_dataset = torchvision.datasets.CIFAR10("./dataset3", train=False,
transform=torchvision.transforms.ToTensor(), download=True)
# 2. 输出数据集的长度
print("训练数据的 长度： {}".format(len(train_dataset)))
print("测试数据的 长度： {}".format(len(test_dataset)))
# 3. 使用DataLoader加载数据集
dataloder_train_set = DataLoader(train_dataset, batch_size=64)
dataloder_test_set = DataLoader(test_dataset, batch_size=64)
# 4. tensorboard记录数据
writer = SummaryWriter("./log_train")
# 5. 引入 实例化的、需要训练的模型
nerual_network = simple_nerual_network()
# 6. 引入损失函数
loss_fn = nn.CrossEntropyLoss()
# 7. 引入优化器 optim
learning_rate = 1e-2
optimizer = torch.optim.SGD(nerual_network.parameters(), lr=learning_rate)
# 8. 设置训练网络的一些参数
# 记录训练的次数
total_train_step = 0
# 记录测试的次数
total_test_step = 0
# 设置训练的轮数
epoch = 20
for i in range(epoch):
print("------------NO {} train is START------------".format(i))
loss_epoch_train = 0
# 9 开始训练模型
nerual_network.train()
for data_item in dataloder_train_set:
img, target = data_item
outputs = nerual_network(img)
loss = loss_fn(outputs, target)
loss_epoch_train = loss_epoch_train + loss.item()
# 使用优化器
optimizer.zero_grad()
# 使用loss进行反向传播, 获得每个参数节点的梯度grab
loss.backward()
# 优化器根据 梯度grab 对模型进行优化
optimizer.step()
print("NO {}, total_train_step: {}, loss: {}: ".format(i, total_train_step, loss.item()))
# writer记录数据
writer.add_scalar("train_loss", loss.item(), global_step=total_train_step)
total_train_step = total_train_step + 1
print("NO {} 整体训练数据集 loss_epoch_train is : {}".format(i, loss_epoch_train))
writer.add_scalar("loss_epoch_train", loss.item(), global_step=total_train_step)
# 测试 过程中不需要调优 torch.no_grab()
loss_epoch_test = 0
accuracy_epoch_test = 0
with torch.no_grad():
# 10 开始测试模型，在训练后的模型中 使用测试数据集 计算loss
nerual_network.eval()
for data_item in dataloder_test_set:
img, target = data_item
outputs = nerual_network(img)
loss = loss_fn(outputs, target)
loss_epoch_test = loss_epoch_test + loss.item()
# 计算正确率
accuracy = (outputs.argmax(1)==target).sum()
accuracy_epoch_test = accuracy_epoch_test + accuracy
print("NO {} 整体测试数据集 loss_epoch_test is : {}".format(epoch, loss_epoch_test))
print("NO {} 整体测试数据集 正确率 is : {}".format(epoch, accuracy_epoch_test / len(test_dataset)))
# writer记录数据
writer.add_scalar("loss_epoch_test", loss_epoch_test, total_test_step)
writer.add_scalar("accuracy_epoch_test", accuracy_epoch_test / len(test_dataset), total_test_step)
total_test_step = total_test_step + 1
# 11 保存模型 每一轮的模型
torch.save(nerual_network, "model22_train_{}.pth".format(i))
print("NO.{} 模型已经保存".format(i))
writer.close()
关于模型训练中的细节
module应该是model中的模块
model.eval()
Sets the module in evaluation mode.
model.train()
定义
Sets the module in training mode.
介绍
This has any effect only on certain modules. See documentations of
particular modules for details of their behaviors in training/evaluation
mode, if they are affected, e.g. :class:`Dropout`, :class:`BatchNorm`,
etc.
##利用GPU训练模式
方式一
class simple_nerual_network(nn.Module):
def __init__(self):
super(simple_nerual_network, self).__init__()
# 使用Sequential
self.sequential1 = Sequential(
# c=3 32*32
Conv2d(in_channels=3, out_channels=32, kernel_size=(5, 5), stride=(1, 1), padding=2),
# c=32 32*32
MaxPool2d(kernel_size=2),
# c=32 16*16
Conv2d(in_channels=32, out_channels=32, kernel_size=(5, 5), stride=(1, 1), padding=2),
# c=32 16*16
MaxPool2d(kernel_size=2),
# c=32 8*8
Conv2d(in_channels=32, out_channels=64, kernel_size=(5, 5), stride=(1, 1), padding=2),
# c=64 8*8
MaxPool2d(kernel_size=2),
# c=64 4*4
Flatten(),
# c=1 1*1024 高度 = 1
Linear(1024, 64),
# c=1 1*64 高度 = 1
Linear(64, 10)
# c=1 1*10 高度 = 1
)
def forward(self, input):
output = self.sequential1(input)
return output
# 利用GPU训练模型 G1
# 1. 准备数据集 10个标签 每个图片只有一种确定的标签
train_dataset = torchvision.datasets.CIFAR10("./dataset3", train=True,
transform=torchvision.transforms.ToTensor(), download=True)
test_dataset = torchvision.datasets.CIFAR10("./dataset3", train=False,
transform=torchvision.transforms.ToTensor(), download=True)
# 2. 输出数据集的长度
print("训练数据的 长度： {}".format(len(train_dataset)))
print("测试数据的 长度： {}".format(len(test_dataset)))
# 3. 使用DataLoader加载数据集
dataloder_train_set = DataLoader(train_dataset, batch_size=64)
dataloder_test_set = DataLoader(test_dataset, batch_size=64)
# 4. tensorboard记录数据
writer = SummaryWriter("./log_GPU1_train")
# 5. 引入 实例化的、需要训练的模型
nerual_network = simple_nerual_network()
# G1.1 将网络模型转移到cuda上
if torch.cuda.is_available():
nerual_network = nerual_network.cuda()
# 6. 引入损失函数
loss_fn = nn.CrossEntropyLoss()
# G1.2 将损失函数转移到cuda上
if torch.cuda.is_available():
loss_fn = loss_fn.cuda()
# 7. 引入优化器 optim
learning_rate = 1e-2
optimizer = torch.optim.SGD(nerual_network.parameters(), lr=learning_rate)
# 8. 设置训练网络的一些参数
# 记录训练的次数
total_train_step = 0
# 记录测试的次数
total_test_step = 0
# 设置训练的轮数
epoch = 20
for i in range(epoch):
print("------------NO {} train is START------------".format(i))
loss_epoch_train = 0
# 9 开始训练模型
nerual_network.train()
for data_item in dataloder_train_set:
imgs, targets = data_item
# G1.3 将input、label 转移到cuda上
if torch.cuda.is_available():
imgs = imgs.cuda()
targets = targets.cuda()
# 使用模型
outputs = nerual_network(imgs)
loss = loss_fn(outputs, targets)
loss_epoch_train = loss_epoch_train + loss.item()
# 使用优化器
optimizer.zero_grad()
# 使用loss进行反向传播, 获得每个参数节点的梯度grab
loss.backward()
# 优化器根据 梯度grab 对模型进行优化
optimizer.step()
print("NO {}, total_train_step: {}, loss: {}: ".format(i, total_train_step, loss.item()))
# writer记录数据
writer.add_scalar("train_loss", loss.item(), global_step=total_train_step)
total_train_step = total_train_step + 1
print("NO {} 整体训练数据集 loss_epoch_train is : {}".format(i, loss_epoch_train))
writer.add_scalar("loss_epoch_train", loss.item(), global_step=total_train_step)
# 测试 过程中不需要调优 torch.no_grab()
loss_epoch_test = 0
accuracy_epoch_test = 0
with torch.no_grad():
# 10 开始测试模型，在训练后的模型中 使用测试数据集 计算loss
nerual_network.eval()
for data_item in dataloder_test_set:
imgs, targets = data_item
# G1.3 将input、label 转移到cuda上
if torch.cuda.is_available():
imgs = imgs.cuda()
targets = targets.cuda()
# 使用模型
outputs = nerual_network(imgs)
loss = loss_fn(outputs, targets)
loss_epoch_test = loss_epoch_test + loss.item()
# 计算正确率
accuracy = (outputs.argmax(1)==targets).sum()
accuracy_epoch_test = accuracy_epoch_test + accuracy
print("NO {} 整体测试数据集 loss_epoch_test is : {}".format(epoch, loss_epoch_test))
print("NO {} 整体测试数据集 正确率 is : {}".format(epoch, accuracy_epoch_test / len(test_dataset)))
# writer记录数据
writer.add_scalar("loss_epoch_test", loss_epoch_test, total_test_step)
writer.add_scalar("accuracy_epoch_test", accuracy_epoch_test / len(test_dataset), total_test_step)
total_test_step = total_test_step + 1
# 11 保存模型 每一轮的模型
torch.save(nerual_network, "model22_train_{}.pth".format(i))
print("NO.{} 模型已经保存".format(i))
writer.close()
方式二
class simple_nerual_network(nn.Module):
def __init__(self):
super(simple_nerual_network, self).__init__()
# 使用Sequential
self.sequential1 = Sequential(
# c=3 32*32
Conv2d(in_channels=3, out_channels=32, kernel_size=(5, 5), stride=(1, 1), padding=2),
# c=32 32*32
MaxPool2d(kernel_size=2),
# c=32 16*16
Conv2d(in_channels=32, out_channels=32, kernel_size=(5, 5), stride=(1, 1), padding=2),
# c=32 16*16
MaxPool2d(kernel_size=2),
# c=32 8*8
Conv2d(in_channels=32, out_channels=64, kernel_size=(5, 5), stride=(1, 1), padding=2),
# c=64 8*8
MaxPool2d(kernel_size=2),
# c=64 4*4
Flatten(),
# c=1 1*1024 高度 = 1
Linear(1024, 64),
# c=1 1*64 高度 = 1
Linear(64, 10)
# c=1 1*10 高度 = 1
)
def forward(self, input):
output = self.sequential1(input)
return output
# 利用GPU训练模型 G2
# G2.1 定义训练device cpu\ cuda:0
device = torch.device("cuda:0" if torch.cuda.is_available() else "cuda")
# 1. 准备数据集 10个标签 每个图片只有一种确定的标签
train_dataset = torchvision.datasets.CIFAR10("./dataset3", train=True,
transform=torchvision.transforms.ToTensor(), download=True)
test_dataset = torchvision.datasets.CIFAR10("./dataset3", train=False,
transform=torchvision.transforms.ToTensor(), download=True)
# 2. 输出数据集的长度
print("训练数据的 长度： {}".format(len(train_dataset)))
print("测试数据的 长度： {}".format(len(test_dataset)))
# 3. 使用DataLoader加载数据集
dataloder_train_set = DataLoader(train_dataset, batch_size=64)
dataloder_test_set = DataLoader(test_dataset, batch_size=64)
# 4. tensorboard记录数据
writer = SummaryWriter("./log_GPU1_train")
# 5. 引入 实例化的、需要训练的模型
nerual_network = simple_nerual_network()
# G2.1 将网络模型转移到cuda上
nerual_network = nerual_network.to(device)
# 6. 引入损失函数
loss_fn = nn.CrossEntropyLoss()
# G2.2 将损失函数转移到cuda上
loss_fn = loss_fn.to(device)
# 7. 引入优化器 optim
learning_rate = 1e-2
optimizer = torch.optim.SGD(nerual_network.parameters(), lr=learning_rate)
# 8. 设置训练网络的一些参数
# 记录训练的次数
total_train_step = 0
# 记录测试的次数
total_test_step = 0
# 设置训练的轮数
epoch = 20
for i in range(epoch):
print("------------NO {} train is START------------".format(i))
loss_epoch_train = 0
# 9 开始训练模型
nerual_network.train()
for data_item in dataloder_train_set:
imgs, targets = data_item
# G2.3 将input、label 转移到cuda上
imgs = imgs.to(device)
targets = targets.to(device)
# 使用模型
outputs = nerual_network(imgs)
loss = loss_fn(outputs, targets)
loss_epoch_train = loss_epoch_train + loss.item()
# 使用优化器
optimizer.zero_grad()
# 使用loss进行反向传播, 获得每个参数节点的梯度grab
loss.backward()
# 优化器根据 梯度grab 对模型进行优化
optimizer.step()
print("NO {}, total_train_step: {}, loss: {}: ".format(i, total_train_step, loss.item()))
# writer记录数据
writer.add_scalar("train_loss", loss.item(), global_step=total_train_step)
total_train_step = total_train_step + 1
print("NO {} 整体训练数据集 loss_epoch_train is : {}".format(i, loss_epoch_train))
writer.add_scalar("loss_epoch_train", loss.item(), global_step=total_train_step)
# 测试 过程中不需要调优 torch.no_grab()
loss_epoch_test = 0
accuracy_epoch_test = 0
with torch.no_grad():
# 10 开始测试模型，在训练后的模型中 使用测试数据集 计算loss
nerual_network.eval()
for data_item in dataloder_test_set:
imgs, targets = data_item
# G1.3 将input、label 转移到cuda上
if torch.cuda.is_available():
imgs = imgs.cuda()
targets = targets.cuda()
# 使用模型
outputs = nerual_network(imgs)
loss = loss_fn(outputs, targets)
loss_epoch_test = loss_epoch_test + loss.item()
# 计算正确率
accuracy = (outputs.argmax(1) == targets).sum()
accuracy_epoch_test = accuracy_epoch_test + accuracy
print("NO {} 整体测试数据集 loss_epoch_test is : {}".format(epoch, loss_epoch_test))
print("NO {} 整体测试数据集 正确率 is : {}".format(epoch, accuracy_epoch_test / len(test_dataset)))
# writer记录数据
writer.add_scalar("loss_epoch_test", loss_epoch_test, total_test_step)
writer.add_scalar("accuracy_epoch_test", accuracy_epoch_test / len(test_dataset), total_test_step)
total_test_step = total_test_step + 1
# 11 保存模型 每一轮的模型
torch.save(nerual_network, "model22_train_{}.pth".format(i))
print("NO.{} 模型已经保存".format(i))
writer.close()
关于没有GPU的解决办法
https://colab.research.google.com/notebooks/welcome.ipynb
Question & Answer
Python中with的使用
https://blog.csdn.net/Ego_Bai/article/details/80873242
file = open("/tmp/foo.txt")
try:
data = file.read()
finally:
file.close()
两者等效
with open("/tmp/foo.txt") as file:
data = file.read()
RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor
原因
RuntimeError: 输入类型 (torch.FloatTensor) 和权重类型 (torch.cuda.FloatTensor) 应该相同，或者输入应该是 MKLDNN 张量并且权重是密集张量
保存的模型是使用GPU训练的，输入和标注的数据也应该使用GPU
解决方法
原理
让网络模型、数据标注、损失函数在同样的环境中GPU或CPU
一 让在GPU中的模型对应到CPU
nerual_network = torch.load("./model22_train_6.pth",map_location=torch.device("cpu"))
二 把数据转到cuda(GPU)中
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
img = img.to(device)
targets = targets.to(device)
###NotImplementedError
解决方法
def foward(self, input):
改为
def forward(self, input):
使用writer.add_images发生AssertionError
原因
add_images只能处理channel为3
解决方法
使用reshape把channel改为3
output = torch.reshape(output, (-1, 3, 30, 30))
AssertionError: size of input tensor and input format are different. tensor shape: (64, 3, 32, 32), input_format: CHW
解决方法
writer.add_image("log_DataLoader1", imgs, step)
改为
writer.add_images("log_DataLoader1", imgs, step)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)>
解决方法
添加以下代码至程序中
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
原因
可能是目标网站的安全证书失效
Python中下划线在变量名和方法名中的含义
原文链接：https://blog.csdn.net/cnds123/article/details/118707525
python中的下划线主要分为如下几类：
前置单下划线，如_var，
主要是用于将变量或方法定义为私有属性。它对于程序员而言是一种提示，这里的私有属性并非如java中的私有，而是一种约定俗成，想强行访问仍旧可以正常访问。
后置单下划线，如var_,
主要用于将变量区别，如变量名称和关键字冲突，后面加个下划线区别开来。
前置双下划线，如__var,
类中带前置双下划线的变量或方法，都会触发python解释器的名称修饰，是不能直接通过该名称去访问到的，实现了一定程度上的隔离。
前后都有双下划线，如__var__,
常见的有__init__()对象构造函数，这类方法是python中的魔法方法（特殊方法），用于特殊用途。
单下划线本身，如_ 在Python REPLs如IDLE Shell中是一个特殊变量（可以表示一个临时值），它表示解释器计算的最后一个表达式的结果。
AttributeError: type object ‘Image’ has no attribute ‘open’
原因
from PIL.Image import Image
解决方法
from PIL import Image
关于Python Console中设置相对路径出现乱码问题
Python Console
定义
Python程序的Script
问题描述
如设置 img = "dataset\train\ants"
原因
'\' 导致字符转义
解决方法
img = "dataset\train\ants"
改为
1. img = "dataset/train/ants"
2. img = "dataset\\train\\ants"
add_image方法出现TypeError: Cannot handle this data type: (1, 1, 512), |u1
原因
Shape:
img_tensor: Default is :math:`(3, H, W)`. You can use ``torchvision.utils.make_grid()`` to
convert a batch of tensor into 3xHxW format or call ``add_images`` and let us do the job.
Tensor with :math:`(1, H, W)`, :math:`(H, W)`, :math:`(H, W, 3)` is also suitable as long as
corresponding ``dataformats`` argument is passed, e.g. ``CHW``, ``HWC``, ``HW``.
解决方法
使用type(img)查看dataformats
使用add_image时设置对应的dataformats
例如
writer.add_image("img_array1",img_array,1,dataformats='HWC')
1878
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
