MINI-RNN代码学习

代码来源：https://gist.github.com/karpathy/d4dee566867f8291f086

这是一个简易的RNN代码，用于学习RNN的基本原理，前向传播和反向传播的基本式子

【文本输入】

import numpy as np

# data I/O
data = open('input.txt', 'r').read()  # should be simple plain text file
chars = list(set(data))#set用于掉重复元素！！！得到文本里一共有的各个字符
data_size, vocab_size = len(data), len(chars)
print ('data has %d characters, %d unique.' % (data_size, vocab_size))
char_to_ix = {ch: i for i, ch in enumerate(chars)}#获得字符与其index的对应关系字典
ix_to_char = {i: ch for i, ch in enumerate(chars)}

# hyperparameters
hidden_size = 100  # size of hidden layer of neurons
seq_length = 25  # number of steps to unroll the RNN for
learning_rate = 1e-1

# model parameters
Wxh = np.random.randn(hidden_size, vocab_size) * 0.01  # input to hidden
Whh = np.random.randn(hidden_size, hidden_size) * 0.01  # hidden to hidden
Why = np.random.randn(vocab_size, hidden_size) * 0.01  # hidden to output
bh = np.zeros((hidden_size, 1))  # hidden bias
by = np.zeros((vocab_size, 1))  # output bias

【训练】

n, p = 0, 0 # p是指针，指向送入文档的现有位置，n是iteration数
mWxh, mWhh, mWhy = np.zeros_like(Wxh), np.zeros_like(Whh), np.zeros_like(Why)
mbh, mby = np