循环神经网络(Recurrent Neural Network,RNN)是一类用于处理序列数据的神经网络。它通过引入循环结构来处理序列中的依赖关系,使得网络能够保持对先前信息的记忆。
视频推荐:【循环神经网络】5分钟搞懂RNN,3D动画深入浅出
基本原理:
RNN的基本原理是引入循环结构,使得网络可以在处理当前输入的同时保留先前的信息。每个时间步,RNN都会接收当前输入和上一个时间步的隐藏状态,并产生当前时间步的输出和新的隐藏状态。这种结构允许网络对序列中的信息进行建模,适用于自然语言处理、时间序列分析等任务。
RNN的隐藏状态更新公式:
h
t
=
tanh
(
W
i
h
⋅
x
t
+
b
i
h
+
W
h
h
⋅
h
t
−
1
+
b
h
h
)
h_t = \text{tanh}(W_{ih} \cdot x_t + b_{ih} + W_{hh} \cdot h_{t-1} + b_{hh})
ht=tanh(Wih⋅xt+bih+Whh⋅ht−1+bhh)
其中:
- h t h_t ht 是时间步 t t t 的隐藏状态。
- x t x_t xt 是时间步 t t t 的输入。
- W i h W_{ih} Wih 和 W h h W_{hh} Whh是权重矩阵。
- b i h b_{ih} bih 和 b h h b_{hh} bhh 是偏置向量。
- tanh \text{tanh} tanh 是双曲正切激活函数。
Python代码示例:
以下是一个简单的用PyTorch实现的RNN示例。
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
from torch.utils.data import DataLoader, TensorDataset
# 构建数据集
sentences = [
"This is a positive example.",
"This is another positive example.",
"Negative sentiment detected here.",
"A neutral statement.",
"This is a negative example."
]
# 标签,1表示正序,0表示逆序
labels = [1, 1, 0, 1 , 0]
# 构建词汇表
word_to_idx = {word: idx + 1 for idx, word in enumerate(set(" ".join(sentences).split()))}
word_to_idx['<pad>'] = 0 # 用于padding
idx_to_word = {idx: word for word, idx in word_to_idx.items()}
vocab_size = len(word_to_idx)
# 将句子转换为索引序列
def sentence_to_indices(sentence):
return [word_to_idx[word] for word in sentence.split()]
data = [sentence_to_indices(sentence) for sentence in sentences]
# 找到最长的句子
max_len = max(map(len, data))
# 将所有句子填充到相同的长度
data_padded = [sentence + [0] * (max_len - len(sentence)) for sentence in data]
# 转换为PyTorch张量
X = torch.tensor(data_padded, dtype=torch.long)
y = torch.tensor(labels, dtype=torch.float).view(-1, 1)
# 构建RNN模型
class SimpleRNN(nn.Module):
def __init__(self, input_size, hidden_size, output_size):
super(SimpleRNN, self).__init__()
self.embedding = nn.Embedding(input_size, hidden_size)
self.rnn = nn.RNN(hidden_size, hidden_size, batch_first=True)
self.fc = nn.Linear(hidden_size, output_size)
def forward(self, x):
x = self.embedding(x)
out, _ = self.rnn(x)
out = self.fc(out[:, -1, :])
return out
# 初始化模型、损失函数和优化器
input_size = vocab_size
hidden_size = 32
output_size = 1
model = SimpleRNN(input_size, hidden_size, output_size)
criterion = nn.BCEWithLogitsLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
# 将数据转换为DataLoader
dataset = TensorDataset(X, y)
data_loader = DataLoader(dataset, batch_size=2, shuffle=True)
# 训练模型
num_epochs = 100
for epoch in range(num_epochs):
for batch_X, batch_y in data_loader:
optimizer.zero_grad()
output = model(batch_X)
loss = criterion(output, batch_y)
loss.backward()
optimizer.step()
if (epoch + 1) % 10 == 0:
print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {loss.item():.4f}')
# 测试模型
test_sentence = "Negative detected here."
test_data = torch.tensor(sentence_to_indices(test_sentence), dtype=torch.long).view(1, -1)
with torch.no_grad():
model.eval()
prediction = torch.sigmoid(model(test_data))
print(f'Test Sentence: {test_sentence}')
print(f'Predicted Probability: {prediction.item():.4f}')
print(f'Predicted Class: {"Positive" if prediction.item() > 0.5 else "Negative"}')