SETI Breakthrough Listen - E.T. Signal Search 第三课作业

最新推荐文章于 2022-03-30 15:33:14 发布

程序员狐小李

最新推荐文章于 2022-03-30 15:33:14 发布

阅读量318

点赞数

文章标签：深度学习

原文链接：https://blog.csdn.net/charles5101/article/details/107881638

版权

自定义模型结构

介绍

PyTorch Image Models (timm)是一个图像模型（models）、层（layers）、实用程序（utilities）、优化器（optimizers）、调度器（schedulers）、数据加载/增强（data-loaders / augmentations）和参考训练/验证脚本（reference training / validation scripts）的集合，目的是将各种SOTA模型组合在一起，从而能够重现ImageNet的训练结果。
timm库链接：https://github.com/rwightman/pytorch-image-models#introduction.

我们最主要的是使用其中的模型，大部分模型都包含了预训练权重。

实战

使用现有模型

import timm
model = timm.create_model('gluon_resnext101_32x4d', pretrained=True)
model.eval()

改造模型

model = timm.create_model('gluon_resnext101_32x4d', pretrained=True, num_classes=NUM_FINETUNE_CLASSES)

使用VIT模型

定义Vit模型

class VITModel(nn.Module):
    """
    Model Class for VIT Model
    """
    def __init__(self, model_name=Config.model_name, pretrained=True):
        super(VITModel, self).__init__()
        self.model = timm.create_model(model_name, pretrained, in_chans=1)
        self.model.head = nn.Linear(self.model.head.in_features, 1)
    
    def forward(self, x):
        x = self.model(x)
        return x

训练代码

# Training Code
if __name__ == '__main__':
    if torch.cuda.is_available():
        print("[INFO] Using GPU: {}\n".format(torch.cuda.get_device_name()))
        DEVICE = torch.device('cuda:0')
    else:
        print("\n[INFO] GPU not found. Using CPU: {}\n".format(platform.processor()))
        DEVICE = torch.device('cpu')
    
    kfold = StratifiedKFold(n_splits=Config.N_SPLITS, shuffle=True, random_state=2021)
    
    fold_scores = {}
    
    for fold_, (trn_idx, val_idx) in enumerate(kfold.split(train_labels, train_labels['target'])):
        print(f"{'='*40} Fold: {fold_} {'='*40}")
        
        train_data = train_labels.loc[trn_idx]
        valid_data = train_labels.loc[val_idx]
        
        print(f"[INFO] Training on {trn_idx.shape[0]} samples and validating on {valid_data.shape[0]} samples")

        # Make Training and Validation Datasets
        training_set = SETIData(
            images=train_data['path'].values,
            targets=train_data['target'].values
        )

        validation_set = SETIData(
            images=valid_data['path'].values,
            targets=valid_data['target'].values
        )

        train = DataLoader(
            training_set,
            batch_size=Config.TRAIN_BS,
            shuffle=True,
            num_workers=8,
            pin_memory=True
        )

        valid = DataLoader(
            validation_set,
            batch_size=Config.VALID_BS,
            shuffle=False,
            num_workers=8
        )

        model = VITModel().to(DEVICE)
        print(f"[INFO] Training Model: {Config.model_name}")
        nb_train_steps = int(len(train_data) / Config.TRAIN_BS * Config.NB_EPOCHS)
        optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)

        trainer = Trainer(model, optimizer, None, train, valid, DEVICE)
        
        per_fold_score = []
        for epoch in range(1, Config.NB_EPOCHS+1):
            print(f"\n{'--'*5} EPOCH: {epoch} {'--'*5}\n")

            # Train for 1 epoch
            tr_lss = trainer.train_one_epoch()
            
            # Validate for 1 epoch
            current_roc, vl_lss = trainer.valid_one_epoch()
            print(f"Validation ROC-AUC: {current_roc:.4f}")
            
            per_fold_score.append(current_roc)
            torch.save(trainer.get_model().state_dict(), f"{Config.model_name}_fold_{fold_}.pt")
        
        fold_scores[fold_] = per_fold_score
        del training_set, validation_set, train, valid, model, optimizer, trainer, current_roc
        gc.collect()
        torch.cuda.empty_cache()

掌握模型速度和FLOPs的计算

FLOPs：注意s小写，是floating point operations的缩写（s表复数），意指浮点运算数，理解为计算量。可以用来衡量算法/模型的复杂度，或者说衡量模型速度

import torchvision.models as models

import torch
from ptflops  import get_model_complexity_info


with torch.cuda.device(0):
  net = models.resnet18()
  flops, params = get_model_complexity_info(net, (3, 224, 224), as_strings=True, print_per_layer_stat=True) #不用写batch_size大小，默认batch_size=1 		            
  print('Flops:  ' + flops)
  print('Params: ' + params)

ResNet(
11.69 M, 100.000% Params, 1.822 GMac, 100.000% MACs,
(conv1): Conv2d(0.009 M, 0.080% Params, 0.118 GMac, 6.477% MACs, 3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(bn1): BatchNorm2d(0.0 M, 0.001% Params, 0.002 GMac, 0.088% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(0.0 M, 0.000% Params, 0.001 GMac, 0.044% MACs, inplace)
(maxpool): MaxPool2d(0.0 M, 0.000% Params, 0.001 GMac, 0.044% MACs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(layer1): Sequential(
0.148 M, 1.266% Params, 0.465 GMac, 25.510% MACs,
(0): BasicBlock(
0.074 M, 0.633% Params, 0.232 GMac, 12.755% MACs,
(conv1): Conv2d(0.037 M, 0.315% Params, 0.116 GMac, 6.344% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(0.0 M, 0.001% Params, 0.0 GMac, 0.022% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.022% MACs, inplace)
(conv2): Conv2d(0.037 M, 0.315% Params, 0.116 GMac, 6.344% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(0.0 M, 0.001% Params, 0.0 GMac, 0.022% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
0.074 M, 0.633% Params, 0.232 GMac, 12.755% MACs,
(conv1): Conv2d(0.037 M, 0.315% Params, 0.116 GMac, 6.344% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(0.0 M, 0.001% Params, 0.0 GMac, 0.022% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.022% MACs, inplace)
(conv2): Conv2d(0.037 M, 0.315% Params, 0.116 GMac, 6.344% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(0.0 M, 0.001% Params, 0.0 GMac, 0.022% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(layer2): Sequential(
0.526 M, 4.496% Params, 0.412 GMac, 22.635% MACs,
(0): BasicBlock(
0.23 M, 1.969% Params, 0.181 GMac, 9.913% MACs,
(conv1): Conv2d(0.074 M, 0.631% Params, 0.058 GMac, 3.172% MACs, 64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(0.0 M, 0.002% Params, 0.0 GMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.011% MACs, inplace)
(conv2): Conv2d(0.147 M, 1.261% Params, 0.116 GMac, 6.344% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(0.0 M, 0.002% Params, 0.0 GMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
0.008 M, 0.072% Params, 0.007 GMac, 0.363% MACs,
(0): Conv2d(0.008 M, 0.070% Params, 0.006 GMac, 0.352% MACs, 64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(0.0 M, 0.002% Params, 0.0 GMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
0.295 M, 2.527% Params, 0.232 GMac, 12.722% MACs,
(conv1): Conv2d(0.147 M, 1.261% Params, 0.116 GMac, 6.344% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(0.0 M, 0.002% Params, 0.0 GMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.011% MACs, inplace)
(conv2): Conv2d(0.147 M, 1.261% Params, 0.116 GMac, 6.344% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(0.0 M, 0.002% Params, 0.0 GMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(layer3): Sequential(
2.1 M, 17.962% Params, 0.412 GMac, 22.596% MACs,
(0): BasicBlock(
0.919 M, 7.862% Params, 0.18 GMac, 9.891% MACs,
(conv1): Conv2d(0.295 M, 2.523% Params, 0.058 GMac, 3.172% MACs, 128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(0.001 M, 0.004% Params, 0.0 GMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.006% MACs, inplace)
(conv2): Conv2d(0.59 M, 5.046% Params, 0.116 GMac, 6.344% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(0.001 M, 0.004% Params, 0.0 GMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
0.033 M, 0.285% Params, 0.007 GMac, 0.358% MACs,
(0): Conv2d(0.033 M, 0.280% Params, 0.006 GMac, 0.352% MACs, 128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(0.001 M, 0.004% Params, 0.0 GMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
1.181 M, 10.100% Params, 0.232 GMac, 12.705% MACs,
(conv1): Conv2d(0.59 M, 5.046% Params, 0.116 GMac, 6.344% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(0.001 M, 0.004% Params, 0.0 GMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.006% MACs, inplace)
(conv2): Conv2d(0.59 M, 5.046% Params, 0.116 GMac, 6.344% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(0.001 M, 0.004% Params, 0.0 GMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(layer4): Sequential(
8.394 M, 71.806% Params, 0.411 GMac, 22.577% MACs,
(0): BasicBlock(
3.673 M, 31.422% Params, 0.18 GMac, 9.880% MACs,
(conv1): Conv2d(1.18 M, 10.092% Params, 0.058 GMac, 3.172% MACs, 256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(0.001 M, 0.009% Params, 0.0 GMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.003% MACs, inplace)
(conv2): Conv2d(2.359 M, 20.183% Params, 0.116 GMac, 6.344% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(0.001 M, 0.009% Params, 0.0 GMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
0.132 M, 1.130% Params, 0.006 GMac, 0.355% MACs,
(0): Conv2d(0.131 M, 1.121% Params, 0.006 GMac, 0.352% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(0.001 M, 0.009% Params, 0.0 GMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
4.721 M, 40.384% Params, 0.231 GMac, 12.697% MACs,
(conv1): Conv2d(2.359 M, 20.183% Params, 0.116 GMac, 6.344% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(0.001 M, 0.009% Params, 0.0 GMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.003% MACs, inplace)
(conv2): Conv2d(2.359 M, 20.183% Params, 0.116 GMac, 6.344% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(0.001 M, 0.009% Params, 0.0 GMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(avgpool): AdaptiveAvgPool2d(0.0 M, 0.000% Params, 0.0 GMac, 0.001% MACs, output_size=(1, 1))
(fc): Linear(0.513 M, 4.389% Params, 0.001 GMac, 0.028% MACs, in_features=512, out_features=1000, bias=True)
)
Flops: 1.82 GMac
Params: 11.69 M

再试试vgg16，可以得出：

VGG(
138.358 M, 100.000% Params, 15.504 GMac, 100.000% MACs,
(features): Sequential(
14.715 M, 10.635% Params, 15.38 GMac, 99.202% MACs,
(0): Conv2d(0.002 M, 0.001% Params, 0.09 GMac, 0.580% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(1): ReLU(0.0 M, 0.000% Params, 0.003 GMac, 0.021% MACs, inplace)
(2): Conv2d(0.037 M, 0.027% Params, 1.853 GMac, 11.951% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(3): ReLU(0.0 M, 0.000% Params, 0.003 GMac, 0.021% MACs, inplace)
(4): MaxPool2d(0.0 M, 0.000% Params, 0.003 GMac, 0.021% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(5): Conv2d(0.074 M, 0.053% Params, 0.926 GMac, 5.976% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(6): ReLU(0.0 M, 0.000% Params, 0.002 GMac, 0.010% MACs, inplace)
(7): Conv2d(0.148 M, 0.107% Params, 1.851 GMac, 11.941% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(8): ReLU(0.0 M, 0.000% Params, 0.002 GMac, 0.010% MACs, inplace)
(9): MaxPool2d(0.0 M, 0.000% Params, 0.002 GMac, 0.010% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(10): Conv2d(0.295 M, 0.213% Params, 0.926 GMac, 5.971% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(11): ReLU(0.0 M, 0.000% Params, 0.001 GMac, 0.005% MACs, inplace)
(12): Conv2d(0.59 M, 0.426% Params, 1.85 GMac, 11.936% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(13): ReLU(0.0 M, 0.000% Params, 0.001 GMac, 0.005% MACs, inplace)
(14): Conv2d(0.59 M, 0.426% Params, 1.85 GMac, 11.936% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(15): ReLU(0.0 M, 0.000% Params, 0.001 GMac, 0.005% MACs, inplace)
(16): MaxPool2d(0.0 M, 0.000% Params, 0.001 GMac, 0.005% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(17): Conv2d(1.18 M, 0.853% Params, 0.925 GMac, 5.968% MACs, 256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(18): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.003% MACs, inplace)
(19): Conv2d(2.36 M, 1.706% Params, 1.85 GMac, 11.933% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(20): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.003% MACs, inplace)
(21): Conv2d(2.36 M, 1.706% Params, 1.85 GMac, 11.933% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(22): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.003% MACs, inplace)
(23): MaxPool2d(0.0 M, 0.000% Params, 0.0 GMac, 0.003% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(24): Conv2d(2.36 M, 1.706% Params, 0.463 GMac, 2.983% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(25): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.001% MACs, inplace)
(26): Conv2d(2.36 M, 1.706% Params, 0.463 GMac, 2.983% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(27): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.001% MACs, inplace)
(28): Conv2d(2.36 M, 1.706% Params, 0.463 GMac, 2.983% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(29): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.001% MACs, inplace)
(30): MaxPool2d(0.0 M, 0.000% Params, 0.0 GMac, 0.001% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(avgpool): AdaptiveAvgPool2d(0.0 M, 0.000% Params, 0.0 GMac, 0.000% MACs, output_size=(7, 7))
(classifier): Sequential(
123.643 M, 89.365% Params, 0.124 GMac, 0.798% MACs,
(0): Linear(102.765 M, 74.275% Params, 0.103 GMac, 0.663% MACs, in_features=25088, out_features=4096, bias=True)
(1): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.000% MACs, inplace)
(2): Dropout(0.0 M, 0.000% Params, 0.0 GMac, 0.000% MACs, p=0.5)
(3): Linear(16.781 M, 12.129% Params, 0.017 GMac, 0.108% MACs, in_features=4096, out_features=4096, bias=True)
(4): ReLU(0.0 M, 0.000% Params, 0.0 GMac, 0.000% MACs, inplace)
(5): Dropout(0.0 M, 0.000% Params, 0.0 GMac, 0.000% MACs, p=0.5)
(6): Linear(4.097 M, 2.961% Params, 0.004 GMac, 0.026% MACs, in_features=4096, out_features=1000, bias=True)
)
)
Flops: 15.5 GMac
Params: 138.36 M

程序员狐小李

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
SETI Breakthrough Listen - E.T. Signal Search 第三课作业

自定义模型结构使用timm库使用VIT模型定义Vit模型class VITModel(nn.Module): """ Model Class for VIT Model """ def __init__(self, model_name=Config.model_name, pretrained=True): super(VITModel, self).__init__() self.model = timm.create_model(
复制链接

扫一扫