Capsule Networks胶囊网络（二）

Dynamic Routing

Conventional net: 层之间完全连接

• 任何两个神经元之间的加权都是可以训练的
• 训练之后，这些权重是固定的，与输入无关
  CapsNet 也有这些权重
• 但是additional routing coefficients 用于训练权重（coefficient value 由输入决定，因此是动态的）
• 没有动态路由的CapsNet和常规网络一样

CNN中类似动态路由的成分是max pooling
Max pooling是从前一层到后一层选择路由信息的粗糙方法。

Coefficients operate on capsule level

• The contribution of a capsule to another capsule in the next layer is determined by the weight matrix and a single scalar coefficient
• The first capsule u∈R3 contributes to the capsule s∈R2 in the next layer by 0.1⋅W1,2⋅u where
  0.1 is the routing coefficient and W1,2 is a 2x3 matrix whose subscript denotes the indices of capsules in the two layers.
• The total contribution of a capsule is exactly 1.Capsule u route to capsule v with routing coefficient 0.9 and to capsule s with 0.1.
• This is a voting scheme where each capsule in layer before vote for each capsule in layer after.
  (前面对后面投票)

Dynamic Routing: Routing by Agreement

Updating the coefficients.
Idea: a capsule votes more if its transformation agrees with the majority of other capsules
Shows routing coefficients to the first capsule in second layer denoted by ci1 for capsule ui
In the beginning all ci1 initialized to 1⁄2 since each capsule routes to all two capsules in the next layer The activation of the first capsule s1 in the next layer is the sum of all transformed and scaled vectors ci1Wi1ui from the left layer
The amount of change in coefficient (i.e. vote) is determined by the dot product between s_i and each summand ci1Wi1ui

Comparison to fully connected neural network

全连接网络

胶囊网络

front-up content

Computing input/output vectors of a capsule

Routing by Agreement

CapsNet Architecture

Performance of Capsule Networks

Matrix capsules with EM routing