【论文笔记】A Symmetric Local Search Network for Emotion-Cause Pair Extraction

A Symmetric Local Search Network for Emotion-Cause Pair Extraction

Abstract

要解决的问题：Emotion-cause pair extraction (ECPE)

已有的方案：a two-step method (Xia & Ding, 2019)

Xia & Ding (2019) 的缺点：未考虑情感从句与原因从句之间的相关性

To tackle this task, a two-step method was proposed by previous study which first extracted emotion clauses and cause clauses individually, then paired the emotion and cause clauses, and filtered out the pairs without causality.

作者的工作核心：local search

Symmetric Local Search Network (SLSN): perform the detection and matching simultaneously by local search

SLSN 的两个对称的子网络：

• the emotion subnetwork
• the cause subnetwork

两个子网络的组成部分：

• a clause representation learner - a specially-designed cross-subnetwork component
• a local pair searcher (LPS)

SLSN consists of two symmetric subnetworks, namely the emotion subnetwork and the cause subnetwork.

Each subnetwork is composed of a clause representation learner and a local pair searcher. The local pair searcher is a specially-designed cross-subnetwork component which can extract the local emotion-cause pairs.

Introduction

作者认为：模拟人类的行为，应该同时考虑情感/原因的检测与匹配

However, when humans deal with the ECPE task, they usually consider the detection and matching problems at the same time.

local search 的优点：可以忽略距离很远的 emotion-cause pair

The advantage of local search is that the wrong pairs (e.g., (c4, c12)) beyond the local context scope can be avoided. Additionally, when local searching the cause clause corresponding to the target emotion clause, humans not only judge whether the clause is a cause clause, but also consider whether it matches the target emotion clause.

LPS 引入 local context window 以限制 local search 的 context scope

Specifically, the LPS introduces a local context window to limit the scope of context for local search.

Symmetric Local Search Network

Task Definition

每个数据集 $D$ 中包含了多个文档 $d$
$$d=[c_1,c_2,\cdots ,c_n]$$

• 令 $c^e$​ 表示情感从句（emotion clause）
• 令 $c^c$ 表示原因从句（cause clause）

目标：提取所有的 emotion-cause pairs
P = { ⋯   , ( c e , c c ) , ⋯   } P = \{\cdots, (c^e, c^c), \cdots\} P={⋯,(ce,cc),⋯}

An Overview of SLSN

• input - a sequence of clauses from a document
• output - the local pair labels for these clauses

对于每个 $c_i$，SLSN 预测两种标签（label）：

• E-LC label - ${\hat{y}}_i^{elc}$​
  • the emotion label (E-label) ${\hat{y}}_i^e$​​ of the i-th clause
  • the local cause labels (LC-label) $({\hat{y}}_{i-1}^c,{\hat{y}}_i^c,{\hat{y}}_{i+1}^c)$​ of the clauses near the i-th clause
• C-LE label - ${\hat{y}}_i^{cle}$​
  • the cause label (C-label) ${\hat{y}}_i^c$ of the i-th clause
  • the local emotion labels (LE-label) $({\hat{y}}_{i-1}^e,{\hat{y}}_i^e,{\hat{y}}_{i+1}^e)$​ of the clauses near the i-th clause

令 ${\hat{y}}_i^{elc}$​ 所对应的 E-C pair set 为 $P_{elc}$​，${\hat{y}}_i^{cle}$​ 所对应的 E-C pair set 为 $P_{cle}$​，则最终的 E-C pair set 为 $P_{elc}\cup P_{cle}$​。

Components of SLSN

SLSN 的两个 subnetworks：

• the emotion subnetwork (E-net)
  • for the E-LC label prediction
• the cause subnetwork (C-net)
  • for the C-LE label prediction

E-net 与 C-net 相似的结构：词嵌入（word embedding）、clause encoder、隐层状态学习（hidden state learning）

E-net and C-net have similar structures in terms of word embedding, clause encoder, and hidden state learning.

Word Embedding

输入：$d=[c_1,c_2,\cdots ,c_n]$ with $c_i=[w_i^1,w_i^2,\cdots ,w_i^{li}]$（$c_i$ 包含了 $li$ 个词）

输出：$v_i=[v_1^i,v_i^2,\cdots ,v_i^{li}]$​

Clause Encoder

结构：word-level Bi-LSTM & Attention

目的：学习从句的表示（learn the representation of clauses）

【以 E-net 为例】

Bi-LSTM 输入：$v_i=[v_1^i,v_i^2,\cdots ,v_i^{li}]$

Bi-LSTM 输出 & Attention 输入：$r_i=[r_1^i,r_i^2,\cdots ,r_i^{li}]$​

Attention 将 $r_i$ 映射至 $s_i^e$ 并聚集它们：
$$\begin{gathered} u_i^j=tanh(W_w{r}_i^j+b_w) \\ {a}_i^j=\frac{exp((u_i^j{)}^T{u}_s)}{{\sum }_{t}exp((u_i^t{)}^T{u}_s)} \\ {s}_i^e=\sum _j{a}_i^j{r}_i^j \end{gathered}$$

Hidden State Learning

使用一个 clause-level Bi-LSTM

【以 E-net 为例】

输入：$[s_1^e,s_2^e,\cdots ,s_n^e]$​

输出：$[h_1^e,h_2^e,\cdots ,h_n^e]$​

同理，C-net 输出 $[h_1^c,h_2^c,\cdots ,h_n^c]$​​

Local Pair Searcher

在 E-net 与 C-net 中构造对称的结构，以预测每个从句的 local pair labels

【以 E-net 为例】

在 E-net 预测 E-label 的时候，只使用从句的 emotion hidden state，并使用 softmax 进行预测：
$${\hat{y}}_i^e=softmax(W_e{h}_i^e+b_e)$$
在 E-net 预测 LC-label 时：若当前从句非情感从句（即预测当前句子不是 emotion clause），则对应的 LC-label 是个零向量（zero vector）；否则，对所有 local context window 中的所有从句预测 LC-label。

LPS 首先计算每个 clause 的 emotion attention ratio ${\lambda }_j$：
$$\begin{gathered} \gamma (h_i^e,h_j^c)=h_i^e{h}_j^c \\ {\lambda }_j=\frac{exp(\gamma (h_i^e,h_j^c))}{{\sum }_{j=i-k}^{i+k}exp(\gamma (h_i^e,h_j^c))} \end{gathered}$$
其中，$\gamma (h_i^e,h_j^c)$ 是度量 local cause 与 target emotion 之间相关性的 emotion attention function，通过将 emotion hidden state 与 cause hidden state 相乘获得；使用 softmax 计算窗口内的 emotion attention ratio ${\lambda }_j$，${\lambda }_j$​ 被用于 scale (不知道 scale 怎么翻译) the origin hidden states：
$$q_j^{lc}={\lambda }_j\cdot h_j^c$$
其中，$q_j^{lc}$ 就是第 j 个 local context clause 的 cause hidden state。

作者还使用了一个 local Bi-LSTM 层去学习每个 local context clause 的 contextualized reqresentation：
$$\begin{gathered} \overrightarrow{o_j}=\overrightarrow{LST{M}_{lc}}(q_j^{lc}),j\in [i-k,i+k] \\ \overleftarrow{o_j}=\overleftarrow{LST{M}_{lc}}(q_j^{lc}),j\in [i-k,i+k] \end{gathered}$$

最终，将 $\overrightarrow{o_j}$​ 与 $\overleftarrow{o_j}$​ 进行拼接​得到 $o_j$，通过 $o_j$​​ 预测第 j 个位置的 local context clause 的 LC-label ${\hat{y}}_j^{lc}$：
$${\hat{y}}_j^{lc}=softmax(W_{lc}{o}_j+b_{lc})$$

Model Training

E-net 用于预测 E-LC label，C-net 用于预测 C-LE label，因此 SLSN 的 loss function 是两者的加权平均：
$$\begin{gathered} L=\alpha L^{elc}+(1-\alpha )L^{cle} \\ {L}^{elc}=\beta L^e+(1-\beta )L^{lc} \\ {L}^{cle}=\beta L^c+(1-\beta )L^{le} \end{gathered}$$
$L^e$​​、$L^{lc}$​​、$L^c$​​、$L^{le}$​​ 是预测 E-label ${\hat{y}}_i^e$​​、LC-label ${\hat{y}}_i^{lc}$​​、C-label ${\hat{y}}_i^c$​​、LE-label ${\hat{y}}_i^{le}$​​ 的交叉熵损失：
$$\begin{gathered} L^e=-\frac{1}{n}\sum _{i=1}^n\eta y_i^{e}log({\hat{y}}_i^e) \\ {L}^{lc}=-\frac{1}{p^e(2k+1)}\sum _{i=1}^{n}I({\hat{y}}_i^e=1)\sum _{j=i-k}^{i+k}{y}_j^{lc}log({\hat{y}}_j^{lc}) \\ {L}^c=-\frac{1}{n}\sum _{i=1}^n\eta y_i^{c}log({\hat{y}}_i^c) \\ {L}^{le}=-\frac{1}{p^c(2k+1)}\sum _{i=1}^{n}I({\hat{y}}_i^c=1)\sum _{j=i-k}^{i+k}{y}_j^{le}log({\hat{y}}_j^{le}) \end{gathered}$$
其中，$I(\cdot )$ 是个 indicator function。

参考文献

Rui Xia and Zixiang Ding. 2019. Emotion-cause pair extraction: A new task to emotion analysis in texts. In Proceedings ofthe 57th Conference ofthe Association for Computational Linguistics, pages 1003–1012.