exbert_AI explanation

在写之前，先要说一下explanation ，interpretable的区别，前者主要是面向黑盒子而言，就是未知的事物，或者是白箱操作，但一般在使用时，也并多未做多加区别。
注意视图提供了模型自我注意的交互视图。在这里，用户可以改变层次，选择头部，并查看聚集的注意力。标记可以被屏蔽
搜索结果显示在语料库视图中，首先显示最高相似性匹配。
Summary视图显示匹配元数据的柱状图摘要，这有助于获取嵌入在搜索语料库中编码的元数据的快照

An overview of the different components of the tool. The token “escape” is selected and
masked at 0-[all]. The results from a corpus search by token embedding are shown and summarized
in (d-g). Users can enter a sentence in (a) and modify the attention view through selections in (b). Self
attention is displayed in ©. The blue matrices show the attention of a head (column) to a token (row).
Tokens and heads that are selected in © can be searched over the annotated corpus (shown: Wizard
of Oz) with results presented in (d). Every token in (d) displays its linguistic metadata on hover. A
colored summary of the matched token (black highlight) and its context is shown in (e), which can
be expanded or collapsed with the buttons above it. The histograms in (f) and (g) summarize the
metadata of the results in (d) for the matched token and the token of max attention, respectively

sentence中mask的词是escape.

在bert模型的layer1时，mask的词指向自身，并没有学到什么有用的信息。

在bert模型的第5层中，所有注意头学到的信息，此时，相比于第一层，POS（词性标注）信息bert学到了，认为是动词的几率笔记比较大

这是在bert的第7层时，选择第二个注意力头的结果，可以看出的是，这个头已经确定了mask的词是动词

图中的token指向后续的token或者前向的token，
图3a，c确认某些负责人学会关注后续或之前的代币。我们称这些头为定位头，因为它们检测到当前标记的偏移量[Clark等人，2019]。虽然很简单，但位置头可以编码有关单词的重要信息。通过头部搜索可以揭示该头部可以看到多少来自令牌嵌入的信息。图3d底部的DEP摘要还显示，头部不仅与种子标记的位置匹配，而且还学会了寻找介词后面的单词是所有格的情况 confirm that certain heads learn to attend to succeeding or preceding tokens. We call
these heads positional heads in that they detect an offset from the current token [Clark et al., 2019].
Though simple, the positional head can encode important information about the attended-to word.
Searching by head can reveal how much information from the token embeddings is visible to that
head. A brief exploration of the attention in positional head 2-0 shows that the head is truly positional,
matching the following word 50/50 times as seen in the lower histogram in Figure 3c. It also seems to
match the POS belonging to the seed token (in this case, “of” is an ADP).
![](https://img-blog.csdnimg.cn/e38ecb08f56e4240a9bb7cb3e477f1a2.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBASmluZ19oYXBwaW5lc3M=,size_13,color_FFFFFF,t_70,g_se,x_16

of 这一个token，指向了her，根据offset可以看出，与下面的单词48/50（看着像是48）匹配，如图3c中较低的直方图所示。它似乎也与属于种子令牌的POS相匹配（在本例中，“of”是一个ADP）。

最后，可以看到，以下3个注意力头，一个是关注token的前一个词，一个是关注token的本身，一个是关注token的下一个词