LAA ： random fourier features的又一应用

        在论文NFormer: Robust Person Re-identification with Neighbor Transformer中的Landmark Agent Attention是对random fourier features 随机傅里叶特征 的又一在attention 降维上的应用：

        与测量高维表示向量之间的相似性不同，我们提出了一种更有效的方法来获得近似的亲和矩阵A。其关键思想是将高维表示向量z映射到低维编码空间，从而使等式（3）中的亲和性计算更加高效，这是受随机傅里叶特征启发的:

        方形的水平方向为矩阵的第一维度，数值方向为第二维度。(The horizontal side of the rectangles indicates the first dimension of the according matrices, while the vertical side indicates the second dimension. )

        输入由 $z\in R^{N\times d}$表示, the query, key and value 矩阵$q,k,v\in R^{N\times d}$ 由线性映射生成. “原有代表”（landmark agents） $z_l\in R^{N\times l}$ are sampled from z to map the q, k of d-dimension to qe, ke of l-dimension.

        Then the approximate affinity matrix $\tilde{A}$ is obtained by the multiplication of qe and ke. In this way, the time complexity of obtaining the affinity matrix reduces from O(N 2d) to O(N 2l), since the l is much smaller than d in practice. Then, the RNS is applied to $\tilde{A}$ and turns the affinities into sparse attention weights. The final output u is obtained by weighted aggregation of value matrix v.

注：以上提到的RNS(reciprocal  neighbour softmax)等更多信息详见原论文。