Neural Architectures for Named Entity Recognition
2.3 CRF Tagging Models
-
Input sentence: X = ( X 1 , X 2 , . . . , X N ) \mathbf{X}=\mathbf{(X_1,X_2,...,X_N)} X=(X1,X2,...,XN)
-
Matrix of scores (output by BiLSTM) : P ∈ R n × k \mathbf{P}\in \mathbb{R}^{n\times k} P∈Rn×k
-Score of the j t h j^{th} jth of i t h i^{th} ithword in a sentence : P i , j P_{i,j} Pi,j
-
The numbers of distinct tags: k k k
-
Sequence of predictions: y = ( y 1 , y 2 , . . . , y n ) \mathbf{y}=(y_1,y_2,...,y_n) y=(y1,y2,...,yn)
y \mathbf{y} y's score: s ( X , y ) = ∑ i = 0 n A y i , y i + 1 + ∑ i = 0 n P i , y i s(\mathbf{X}, \mathbf{y})=\sum\limits_{i=0}^nA_{y_i,y_{i+1}}+\sum\limits_{i=0}^nP_{i,y_i} s(X,y)=i=0∑nAyi,yi+1+i=0∑nPi,yi
-
Matrix of transition scores: A ∈ R ( k + 2 ) × ( k + 2 ) A \in \mathbb{R}^{(k+2) \times(k+2)} A∈R(k+2)×(k+2)
- score of a transition from the tag i i i to tag j j j: A i , j A_{i,j} Ai,j
- start and end tag: y 0 , y n y_0, y_n y0,yn
Probability for the sequence y \mathbf{y} y:
p ( y ∣ X ) = e s ( X , y ) ∑ y ~ ∈ Y x e s ( X , y ~ ) p(\mathbf{y}|\mathbf{X})=\frac{e^{s(\mathbf{X}, \mathbf{y})}}{\sum_{\tilde{y}}\in\mathbf{Y_x}e^{s(\mathbf{X}, \mathbf{\tilde{y}})}} p(y∣X)=∑y~∈Yxes(X,y~)es(X,y)
Maximize the log_probability of the correct tag sequence:
l o g ( p ( y ∣ X ) ) = s ( X , y ) − l o g ( ∑ y ~ ∈ Y x e s ( X , y ~ ) = s ( X , y ) − l o g a d d y ~ ∈ Y x s ( X , y ~ ) log(p(\mathbf{y}|\mathbf{X}))={s(\mathbf{X}, \mathbf{y})}-log(\sum\limits_{{\tilde{y}}\in\mathbf{Y_x}}e^{s(\mathbf{X}, \mathbf{\tilde{y}}})=s(\mathbf{X}, \mathbf{y})-{logadd}_{\tilde{y}\in\mathbf{Y_x}} s(\mathbf{X}, \mathbf{\tilde{y}}) log(p(y∣X))=s(X,y)−log(y~∈Yx∑es(X,y~)=s(X,y)−logaddy~∈Yxs(X,y~)
- All possible tag sequences for a sentence X \mathbf{X} X: Y X \mathbf{Y_X} YX
Predict the output sequence by:
y ∗ = a r g m a x y ~ ∈ Y X s ( X , y ~ ) \mathbf{y}^*=argmax_{\tilde{y}\in \mathbf{Y_X}} s(\mathbf{X}, \mathbf{\tilde{y}}) y∗=argmaxy~∈YXs(X,y~)