A1.计算机视觉入门

单帧图像超分辨率

1. 环境搭建

远程环境
服务器, Ubuntu 16.04, GPU 1080 Ti

本地应用

  • MobaXterm(与远程建立ssh连接,方便SFTP方式管理远程文件)
  • Sublime(作为本地文本阅读器使用, Markdown,Code)
  • Pycharm(可选,方便工程管理,调试)
  • Vscode(可选,方便工程管理,调试)

2. 框架学习

  • 此处向Xinntao前辈学习,使用其BasicSR框架,此处为Github地址
  • 该框架可实现三类深度超分网络, PSNR-oriented SR(如 SCRCNN, SRResNet等),Enhanced SRGAN(同时可训练SRGAN),以及SFTGAN。
  • 代码分为五大部分

3. SRResnet的具体实现

本框架所选用默认训练集为DIV2K(分为训练部分和测试部分),本次实验所选用验证(测试)集为Set5

  1. DIV2K数据集处理:
  • a.先下载得到 DIV2K_train_HR与DIV2K_valid_HR两部分数据;
  • b.然后用codes/scripts中的 extract_subimags_single.py处理(需要修改其中保存路径),分别得到DIV2K_train_HR_sub,和DIV2K_valid_HR_sub;
  • c.接着用codes/scripts中的generate_mod_LR_bic.m进行处理(需要修改其中保存路径),此步为生成低分辨率的图像(设定mod=4, scale=4),得到DIV2K_train_HR_sub_bicLRx4,以及DIV2K_valid_HR_sub_bicLRx4;
  • d.接下来用codes/scripts中的create_lmdb.py处理,生成相应的DIV2K_train_HR_sub.imdb和DIV2K_valid_HR_sub_bicLRx4.imdb,至此图片处理暂时完成。
  • (对于Set5及Set14类似,但不需进行b的处理)
  1. 修改train_sr.json
    需要添加一些路径信息,如下所示,根据自己的具体设置更改。
, "path": {
    "root": "/media/sdc/cyli/BasicSR"
	, "experiments_root":"/media/sdc/cyli/BasicSR/experiments/SRresnet_PSNR_x4_DIV2K/"
	, "models":"/media/sdc/cyli/BasicSR/experiments/SRresnet_PSNR_x4_DIV2K/models"
	, "log":"/media/sdc/cyli/BasicSR/experiments/SRresnet_PSNR_x4_DIV2K"
	, "val_images": "/media/sdc/cyli/BasicSR/experiments/SRresnet_PSNR_x4_DIV2K/val_images"
    ,  "pretrain_model_G": null
  1. 训练开始
    待修改train_sr.json完成后可以,进入train.py所在目录,输入 python test.py -opt options/test/test_sr.json 即可开始训练。如果顺利在终端界面会出现如下图所示的输出信息。
    训练时终端输出结果

  2. 训练过程可视化
    可用Tensorboard 实现训练过程的可视化,在开始训练时会出现“BasicSR/tb_logger/XX”文件夹,在另一终端窗口输入 tensorboard --logdir ./BasicSR/tb_logger/XX/ ,会有输出结果类似于 “TensorBoard 1.12.0 at http://ubuntu:6006 (Press CTRL+C to quit)”,此时将ubuntu替换为你所用的远程服务器的地址,并在浏览器窗口打开相应地址即可得到类似如下图的结果。
    TensorBoard显示训练效果图

  3. (未完待续)

[1] : https://github.com/xinntao/public_figures/blob/master/BasicSR/code_framework.png

Network Building

###1. First meet
train.py

from models import create_model
# Create model
model = create_model(opt) #返回何种类型的model, sr, srgan, srrgan, stfgan

Here the opt is a dictionary

{'name': 'DnCNN_S_PSNR_sigma25_001', 'use_tb_logger': True, 'model': 'sr', 'scale': 1, 'gpu_ids': [0, 1, 2, 3], 'datasets': {'train': {'name': 'DIV2K', 'mode': 'NHR', 'dataroot_HR': '/media/sdc/cyli/datasets/DIV2K800/DIV2K800_train_sub/', 'dataroot_LR': None, 'subset_file': None, 'use_shuffle': True, 'n_workers': 8, 'batch_size': 128, 'HR_size': 40, 'use_flip': True, 'use_rot': True, 'data_type': 'img', 'phase': 'train', 'scale': 1}, 'val': {'name': 'val_set5', 'mode': 'NHR', 'dataroot_HR': '/media/sdc/cyli/datasets/val_set5/Set5/', 'dataroot_LR': None, 'phase': 'val', 'HR_size': 40, 'scale': 1, 'data_type': 'img'}}, 'path': {'root': '/media/sdc/cyli/BasicSR', 'experiments_root': '/media/sdc/cyli/BasicSR/experiments/DnCNN_S_PSNR_sigma25_001', 'models': '/media/sdc/cyli/BasicSR/experiments/DnCNN_S_PSNR_sigma25_001/models', 'log': '/media/sdc/cyli/BasicSR/experiments/DnCNN_S_PSNR_sigma25_001', 'val_images': '/media/sdc/cyli/BasicSR/experiments/DnCNN_S_PSNR_sigma25_001/val_images', 'pretrain_model_G': None}, 'network_G': {'which_model_G': 'dncnn', 'norm_type': None, 'mode': 'CNA', 'nf': 64, 'nb': None, 'in_nc': 3, 'out_nc': 3, 'gc': None, 'group': 1, 'scale': 1}, 'train': {'lr_G': 0.0002, 'lr_scheme': 'MultiStepLR', 'lr_steps': [200000, 400000, 600000, 800000], 'lr_gamma': 0.5, 'pixel_criterion': 'l1', 'pixel_weight': 1.0, 'val_freq': 5000.0, 'manual_seed': 0, 'niter': 1000000.0}, 'logger': {'print_freq': 200, 'save_checkpoint_freq': 5000.0}, 'timestamp': '190105-154603', 'is_train': True}

2. Second Meet: Model Type

Go to models folder, meet ‘int.py’, select the model type
The whole codes like this

def create_model(opt):
    model = opt['model']

    if model == 'sr':
        from .SR_model import SRModel as M
    elif model == 'srgan':
        from .SRGAN_model import SRGANModel as M
    elif model == 'srragan':
        from .SRRaGAN_model import SRRaGANModel as M
    elif model == 'sftgan':
        from .SFTGAN_ACD_model import SFTGAN_ACD_Model as M
    else:
        raise NotImplementedError('Model [{:s}] not recognized.'.format(model))
    m = M(opt)
    print('Model [{:s}] is created.'.format(m.__class__.__name__))
    return m

Here we know that 'model': 'sr', therefore 'sr'-> opt['model'] -> model, then model == 'sr' is True,
finally, run from .SR_modle import SRModel as M.

3. Third Meet: Parameters

  • Now we will go to the file /media/sdc/cyli/BasicSR/codes/models/SR_model.py
    Then we meet the body of the code
import os
from collections import OrderedDict

import torch
import torch.nn as nn
from torch.optim import lr_scheduler

import models.networks as networks
from .base_model import BaseModel

class SRModel(BaseModel): # first 继承了BaseModel的功能,然后定义了一些基本的函数
    def __init__(self, opt):
        pass
        #... 
    def feed_data(self, data, need_HR=True):
        #...
    def optimize_parameters(self, step):
        #...
    def test(self):
    def test_x8(self):
    def get_current_log(self):
    def get_current_visuals(self, need_HR=True):
    def print_network(self):
    def load(self):
    def save(self, iter_label):    

4 Forth Meet: Network Structure

Use import models.networks as networks
/media/sdc/cyli/BasicSR/codes/models/networks.py

  • Import some packages
import functools
import torch
import torch.nn as nn
from torch.nn import init

import models.modules.architecture as arch
import models.modules.sft_arch as sft_arch
  • Define Initialization methods
####################
# initialize
####################
def weights_init_normal(m, std=0.02):

def weights_init_kaiming(m, scale=1):

def weights_init_orthogonal(m):

def init_weights(net, init_type='kaiming', scale=1, std=0.02):
    # scale for 'kaiming', std for 'normal'.
    print('initialization method [{:s}]'.format(init_type))
    if init_type == 'normal':
        weights_init_normal_ = functools.partial(weights_init_normal, std=std)
        net.apply(weights_init_normal_)
    elif init_type == 'kaiming':
        weights_init_kaiming_ = functools.partial(weights_init_kaiming, scale=scale)
        net.apply(weights_init_kaiming_)
    elif init_type == 'orthogonal':
        net.apply(weights_init_orthogonal)
    else:
        raise NotImplementedError('initialization method [{:s}] not implemented'.format(init_type))
  • Define Generator
####################
# define network
####################
# Generator
def define_G(opt):
    gpu_ids = opt['gpu_ids']
    opt_net = opt['network_G']
    which_model = opt_net['which_model_G']

    if which_model == 'sr_resnet':  # SRResNet
        netG = arch.SRResNet(in_nc=opt_net['in_nc'], out_nc=opt_net['out_nc'], nf=opt_net['nf'], \
            nb=opt_net['nb'], upscale=opt_net['scale'], norm_type=opt_net['norm_type'], \
            act_type='relu', mode=opt_net['mode'], upsample_mode='pixelshuffle')
    ################################################################################################
    elif which_model == 'dncnn':
        netG = arch.DnCNN(in_nc=opt_net['in_nc'], out_nc=opt_net['out_nc'],nf=opt_net['nf'],\
            nb=opt_net['nb'],upscale=opt_net['scale'], norm_type=opt_net['norm_type'], \
            act_type='relu', mode=opt_net['mode'], upsample_mode='pixelshuffle')

    ################################################################################################   
    
    elif which_model == 'srcnn':
        netG = arch.SRCNN(in_nc=opt_net['in_nc'], out_nc=opt_net['out_nc'],nf=opt_net['nf'],\
            nb=opt_net['nb'],upscale=opt_net['scale'], norm_type=opt_net['norm_type'], \
            act_type='relu', mode=opt_net['mode'], upsample_mode='pixelshuffle')

    ################################################################################################
    elif which_model == 'fsrcnn': # FSRCNN
        netG = arch.FSRCNN(in_nc=opt_net['in_nc'], out_nc=opt_net['out_nc'], nf=opt_net['nf'],\
                           nb=opt_net['nb'],upscale=opt_net['scale'], norm_type=opt_net['norm_type'], \
            act_type='relu', mode=opt_net['mode'], upsample_mode='pixelshuffle')
    ################################################################################################

    elif which_model == 'sft_arch':  # SFT-GAN
        netG = sft_arch.SFT_Net()

    elif which_model == 'RRDB_net':  # RRDB
        netG = arch.RRDBNet(in_nc=opt_net['in_nc'], out_nc=opt_net['out_nc'], nf=opt_net['nf'],
            nb=opt_net['nb'], gc=opt_net['gc'], upscale=opt_net['scale'], norm_type=opt_net['norm_type'],
            act_type='leakyrelu', mode=opt_net['mode'], upsample_mode='upconv')
    else:
        raise NotImplementedError('Generator model [{:s}] not recognized'.format(which_model))

    if opt['is_train']:
        init_weights(netG, init_type='kaiming', scale=0.1)
    if gpu_ids:
        assert torch.cuda.is_available()
        netG = nn.DataParallel(netG)
    return netG
  • Define Discriminator
# Discriminator
def define_D(opt):
    gpu_ids = opt['gpu_ids']
    opt_net = opt['network_D']
    which_model = opt_net['which_model_D']

    if which_model == 'discriminator_vgg_128':
        netD = arch.Discriminator_VGG_128(in_nc=opt_net['in_nc'], base_nf=opt_net['nf'], \
            norm_type=opt_net['norm_type'], mode=opt_net['mode'], act_type=opt_net['act_type'])

    elif which_model == 'dis_acd':  # sft-gan, Auxiliary Classifier Discriminator
        netD = sft_arch.ACD_VGG_BN_96()

    elif which_model == 'discriminator_vgg_96':
        netD = arch.Discriminator_VGG_96(in_nc=opt_net['in_nc'], base_nf=opt_net['nf'], \
            norm_type=opt_net['norm_type'], mode=opt_net['mode'], act_type=opt_net['act_type'])
    elif which_model == 'discriminator_vgg_192':
        netD = arch.Discriminator_VGG_192(in_nc=opt_net['in_nc'], base_nf=opt_net['nf'], \
            norm_type=opt_net['norm_type'], mode=opt_net['mode'], act_type=opt_net['act_type'])
    elif which_model == 'discriminator_vgg_128_SN':
        netD = arch.Discriminator_VGG_128_SN()
    else:
        raise NotImplementedError('Discriminator model [{:s}] not recognized'.format(which_model))

    init_weights(netD, init_type='kaiming', scale=1)
    if gpu_ids:
        netD = nn.DataParallel(netD)
    return netD


def define_F(opt, use_bn=False):
    gpu_ids = opt['gpu_ids']
    device = torch.device('cuda' if gpu_ids else 'cpu')
    # pytorch pretrained VGG19-54, before ReLU.
    if use_bn:
        feature_layer = 49
    else:
        feature_layer = 34
    netF = arch.VGGFeatureExtractor(feature_layer=feature_layer, use_bn=use_bn, \
        use_input_norm=True, device=device)
    # netF = arch.ResNet101FeatureExtractor(use_input_norm=True, device=device)
    if gpu_ids:
        netF = nn.DataParallel(netF)
    netF.eval()  # No need to train
    return netF


5 The Base Model

  • The base model in /media/sdc/cyli/BasicSR/codes/models/base_model.py
    Base model only have some fuctions that is not relating to the model structure
  • Import packages
import os
import torch
import torch.nn as nn
  • Define BaseModel() Class
class BaseModel():
    def __init__(self, opt):
        self.opt = opt
        self.save_dir = opt['path']['models']  # save models
        self.device = torch.device('cuda' if opt['gpu_ids'] is not None else 'cpu')
        self.is_train = opt['is_train']
        self.schedulers = []
        self.optimizers = []

    def feed_data(self, data):
        pass

    def optimize_parameters(self):
        pass

    def get_current_visuals(self):
        pass

    def get_current_losses(self):
        pass

    def print_network(self):
        pass

    def save(self, label):
        pass

    def load(self):
        pass

    def update_learning_rate(self):
        for scheduler in self.schedulers:
            scheduler.step()

    def get_current_learning_rate(self):
        return self.optimizers[0].param_groups[0]['lr']

    # helper printing function that can be used by subclasses
    def get_network_description(self, network):
        if isinstance(network, nn.DataParallel):
            network = network.module
        s = str(network)
        n = sum(map(lambda x: x.numel(), network.parameters()))
        return s, n

    # helper saving function that can be used by subclasses
    def save_network(self, save_dir, network, network_label, iter_label):
        save_filename = '{}_{}.pth'.format(iter_label, network_label)
        save_path = os.path.join(save_dir, save_filename)
        if isinstance(network, nn.DataParallel):
            network = network.module
        state_dict = network.state_dict()
        for key, param in state_dict.items():
            state_dict[key] = param.cpu()
        torch.save(state_dict, save_path)

    # helper loading function that can be used by subclasses
    def load_network(self, load_path, network, strict=True):
        if isinstance(network, nn.DataParallel):
            network = network.module
        network.load_state_dict(torch.load(load_path), strict=strict)

©️2020 CSDN 皮肤主题: 大白 设计师: CSDN官方博客 返回首页
实付0元
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值