PyTorch学习笔记（30）模型微调(Finetune)

最新推荐文章于 2022-07-08 11:22:41 发布

TongYixuan_LUT

最新推荐文章于 2022-07-08 11:22:41 发布

阅读量553

点赞数

分类专栏： PyTorch学习笔记文章标签： pytorch

本文链接：https://blog.csdn.net/qq_33357094/article/details/104655461

版权

模型微调(Finetune)

Transfer Learning ：机器学习分值，研究源域(source domain) 的知识如何应用到目标域(target domain)

Model Finetune: 模型的迁移学习

模型微调步骤:

1.获取预训练模型参数
2.加载模型(load_state_dict)
3.修改输出层

模型微调训练方法

1.固定预训练的参数(requires_grad = False;lr = 0)
2.Features Extractor 较小学习率(params_group)

训练数据

预训练模型



# -*- coding: utf-8 -*-


import os
import numpy as np
import torch
import torch.nn as nn
from torch.utils.data import DataLoader
import torchvision.transforms as transforms
import torch.optim as optim
from matplotlib import pyplot as plt
from tools.my_dataset import AntsDataset
from tools.common_tools import set_seed
import torchvision.models as models
import torchvision
BASEDIR = os.path.dirname(os.path.abspath(__file__))
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("use device :{}".format(device))

set_seed(1)  # 设置随机种子
label_name = {
   "ants": 0, "bees": 1}

# 参数设置
MAX_EPOCH = 25
BATCH_SIZE = 16
LR = 0.001
log_interval = 10
val_interval = 1
classes = 2
start_epoch = -1
lr_decay_step = 7


# ============================ step 1/5 数据 ============================
data_dir = os.path.join(BASEDIR, "data/hymenoptera_data")
train_dir = os.path.join(data_dir, "train")
valid_dir = os.path.join(data_dir, "val")

norm_mean = [0.485, 0.456, 0.406]
norm_std = [0.229, 0.224, 0.225]

train_transform = transforms.Compose([
    transforms.RandomResizedCrop(224),
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
    transforms.Normalize(norm_mean, norm_std),
])

valid_transform = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(