自然语言处理（六）： Deep Learning for NLP: Feedforward Networks

目录

1. Deep Learning

1.2 Feed-forward NN

1.3 Neuron

1.4 Matrix Vector Notation 矩阵向量表示法

1.5 Output Layer

1.6 Learning from Data

1.7 Regularisation 正则化

1.8 Dropout

2. Applications in NLP

2.1 Topic Classification

2.2 Topic Classification - Training

2.3 Topic Classification - Prediction

2.4 Topic Classification - Improvements

2.5 Language Model Revisited

2.6 Language Models as Classifiers

2.6 Feed-forward NN Language Model

2.7 Word Embeddings

2.8 Topic Classification 

2.9 Training a FFNN LM

2.10 Input and Output Word Embeddings

2.11 Language Model: Architecture

2.12 Advantages of FFNN LM 

3. Convolutional Networks

3.1 Convolutional Networks for NLP

4. Final Words

---

1. Deep Learning

• A branch of machine learning 机器学习的一个分支
• Re-branded name for neural networks 神经网络的改名
• Why deep? Many layers are chained together in modern deep learning models 为什么是深度？在现代深度学习模型中，许多层都是连在一起的。
• Neural networks: historically inspired by the way computation works in the brain 神经网络：历史上受到大脑中计算方式的启发
  • Consists of computation units called neurons 由称为神经元的计算单元组成

1.2 Feed-forward NN

• Aka multilayer perceptrons 又名多层感知器
• Each arrow carries a weight, reflecting its importance 每个箭头都有一个权重，反映其重要性
• Certain layers have nonlinear activation functions 某些层有非线性激活函数

1.3 Neuron

Each neuron is a function 每个神经元都是一个函数

- given input x, computes real-value (scalar) h 给定输入x，计算实值（标量）

- scales input (with weights, w) and adds offset (bias, b)

- applies non-linear function:

• logistic sigmoid
• hyperbolic sigmoid (tanh)
• rectified linear unit

- w and b are parameters of the model

1.4 Matrix Vector Notation 矩阵向量表示法

- Typically have several hidden units, i.e.

- Each with its own weights () and bias term ()

- Can be expressed using matrix and vector operators 可以用矩阵和向量运算符表示

- Where a matrix comprisine the weight vectors, and is a vector of all bias terms

- Non-linear function applied element-wise 非线性函数的单元应用

1.5 Output Layer

- Binary