1DCNN添加Residual Block, python代码实现及讲解

:以下代码基于jupyter notebook编辑器,pytorch
以下为Residual block示意图 接下来将实现左边的示意图
在这里插入图片描述

构建一个 ResNet_basic_block类

import torch
import torch.nn as nn
import torch.nn.functional as F
class ResNet_basic_block(nn.Module):
    def __init__(self, in_channels, out_channels):
        super().__init__()
        self.conv1 = nn.Conv2d(in_channels = in_channels,
                              out_channels = out_channels,
                              kernel_size = 3,  #3*3卷积
                              padding = 1,  #通过padding操作使x维度不变
                              bias = False)
        self.bn1 = nn.BatchNorm2d(num_features = out_channels)
        self.conv2 = nn.Conv2d(in_channels = out_channels,
                              out_channels = out_channels,
                              kernel_size = 3,   #3*3卷积
                              padding = 1,   #通过padding操作使x维度不变
                              bias = False)
        self.bn2 = nn.BatchNorm2d(num_features = out_channels)
    def forward(self, x):
        out = self.conv1(x)
        out = self.bn1(out)
        out = F.relu(self.bn1(out), inplace = True)
        out = self.conv2(out)
        out = self.bn2(out)
        return out+x
Res = ResNet_basic_block(in_channels=32, out_channels=64)
x = torch.randn(16,32,1024) #产生一个batch=16, channel=32, length=1024
print(x.shape)
y = Res(x)
print(y.shape)
>>>
torch.Size([16, 32, 1024])
torch.Size([16, 64, 1024])

接下来在1DCNN中添加Residual Block模块

构建一个1DCNN,其名称为Net(),用作4分类

在这里插入图片描述
**注:**1DCNN模型参考学位论文《基于多尺度卷积神经网络的电机故障诊断方法研究-王威》

# 输入x = torch.rand(64, 1, 1024)的模型
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv1d(in_channels = 1,out_channels= 64,kernel_size = 32, stride = 8, padding = 12)
        self.pool1 = nn.MaxPool1d(kernel_size=2, stride=2)
        self.BN = nn.BatchNorm1d(num_features=64)
        
        self.conv7_1 = nn.Conv1d(in_channels=64, out_channels=64, kernel_size=7, stride=1, padding=3)
        self.pool7_1 = nn.MaxPool1d(kernel_size=2, stride=2)
        self.conv7_2 = nn.Conv1d(in_channels=64, out_channels=128, kernel_size=7, stride=1, padding=3)
        self.pool7_2 = nn.MaxPool1d(kernel_size=2, stride=2)
        self.conv7_3 = nn.Conv1d(in_channels=128, out_channels=256, kernel_size=7, stride=1, padding=3)
        self.pool7_3 = nn.MaxPool1d(kernel_size=2, stride=2)
        
        self.pool2 = nn.MaxPool1d(kernel_size=8, stride=1)
        self.fc = nn.Linear(in_features=256, out_features=4)  ##这里的4096是计算出来的
        self.softmax = nn.Softmax()
        
    def forward(self, x):
        x = self.conv1(x)  ## x:Batch, 1, 1024
        x = self.pool1(x)
        
        x3 = self.conv7_1(x)
        x3 = self.pool7_1(x3)
        x3  = self.conv7_2(x3)
        x3 = self.pool7_2(x3)
        x3 = self.conv7_3(x3)
        x3 = self.pool7_3(x3)
        
        x3 = self.pool2(x3)
        
        Batch, Channel, Length = x3.size()
        x3 = x3.view(Batch, -1)
        
        x3 = self.fc(x3)
        return x3

测试一下这个构建的1DCNN

model = Net()
x = torch.randn(64, 1, 1024) # batchsize = 64, channel=1, length=1024
y = model(x)
print(y.shape)
>>>结果输出
torch.Size([64, 4])

在Net()里添加Residual Block模块

# x = torch.rand(64, 1, 1024)的模型
class Net_with_Res(nn.Module):
    def __init__(self):
        super(Net_with_Res, self).__init__()
        self.conv1 = nn.Conv1d(in_channels = 1,out_channels= 64,kernel_size = 32, stride = 8, padding = 12)
        self.pool1 = nn.MaxPool1d(kernel_size=2, stride=2)
        self.BN = nn.BatchNorm1d(num_features=64)
        
        self.conv7_1 = nn.Conv1d(in_channels=64, out_channels=64, kernel_size=7, stride=1, padding=3)
        self.pool7_1 = nn.MaxPool1d(kernel_size=2, stride=2)
        self.Res7_1 = ResNet_basic_block(in_channels=64, out_channels=64) # 第1个Residual block
        self.conv7_2 = nn.Conv1d(in_channels=64, out_channels=128, kernel_size=7, stride=1, padding=3)
        self.pool7_2 = nn.MaxPool1d(kernel_size=2, stride=2)
        self.Res7_2 = ResNet_basic_block(in_channels=128, out_channels=128) # 第2个Residual block
        self.conv7_3 = nn.Conv1d(in_channels=128, out_channels=256, kernel_size=7, stride=1, padding=3)
        self.pool7_3 = nn.MaxPool1d(kernel_size=2, stride=2)
        
        self.pool2 = nn.MaxPool1d(kernel_size=8, stride=1)
        self.fc = nn.Linear(in_features=256, out_features=4)  ##这里的4096是计算出来的
        
    def forward(self, x):
        x = self.conv1(x)  ## x:Batch, 1, 1024
        x = self.pool1(x)
        
        x3 = self.conv7_1(x)
        x3 = self.pool7_1(x3)
        # 添加第1个Residual Block
        x3 = self.Res7_1(x3)
        x3  = self.conv7_2(x3)
        x3 = self.pool7_2(x3)
        # 添加第2个Residual Block
        x3 = self.Res7_2(x3)
        x3 = self.conv7_3(x3)
        x3 = self.pool7_3(x3)
        
        x3 = self.pool2(x3)
        
        Batch, Channel, Length = x3.size()
        x3 = x3.view(Batch, -1)
        
        x3 = self.fc(x3)
        return x3

测试一下这个加了Residual Block的1DCNN

model = Net_with_Res()
x = torch.randn(64, 1, 1024)
y = model(x)
print(y.shape)
>>>结果输出
torch.Size([64, 4])

参考资料:基于多尺度卷积神经网络的电机故障诊断方法研究-王威-中国矿业大学

欢迎关注公众号:故障诊断与python学习

  • 6
    点赞
  • 53
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 2
    评论
自注意力(Self-Attention)机制是Transformer架构的核心组成部分,最初在机器翻译任务中由 Vaswani 等人在2017年的论文《Attention is All You Need》中引入。在Python中,特别是在使用像PyTorch或TensorFlow这样的深度学习库时,我们可以用以下步骤来实现自注意力机制: 1. **Query、Key和Value的计算**: - 对输入序列中的每个词(或单词的向量表示),我们创建三个版本:一个是查询(Query)向量,用于找出与其他向量的相关度;一个是键(Key)向量,用于存储相似度信息;还有一个值(Value)向量,用来存储原始信息。 ```python def compute_attention(query, key, value): attention_scores = torch.matmul(query, key.transpose(-2, -1)) / math.sqrt(key.size(-1)) attention_weights = softmax(attention_scores, dim=-1) output = torch.matmul(attention_weights, value) return output, attention_weights ``` 2. **加性注意力**: 这是最常见的自注意力形式,通过softmax函数将得分转换为概率分布。 3. **多头注意力(Multi-Head Attention)**: 分离为多个子注意力机制,每个子机制处理输入的不同线性变换,然后合并结果,以捕捉不同尺度的信息。 ```python def multi_head_attention(query, key, value, num_heads): query_heads = torch.split(query, query.size(-1) // num_heads, dim=-1) key_heads = torch.split(key, key.size(-1) // num_heads, dim=-1) value_heads = torch.split(value, value.size(-1) // num_heads, dim=-1) attended_heads = [compute_attention(head_q, head_k, head_v) for head_q, head_k, head_v in zip(query_heads, key_heads, value_heads)] attended_heads = [h for h in attended_heads] concatenated = torch.cat(attended_heads, dim=-1) return concatenated ``` 4. **残差连接和层归一化**: 这些是Transformer层的标准组成部分,用于优化模型性能。 ```python def self_attention_layer(inputs, num_heads, dropout): # 前向传播过程 att_output = multi_head_attention(inputs, inputs, inputs, num_heads) att_output = dropout(att_output) att_output = residual_connection(inputs, att_output) normalized_output = layer_norm(inputs + att_output) return normalized_output ```
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

故障诊断与python学习

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值