【论文笔记】Embedding of Embedding (EOE) : Joint Embedding for Coupled Heterogeneous Networks

一种network embedding 的思路
网络结点向量化

这种向量化可以表示出不同网络之间的连边（通过引入一个矩阵（harmonious embedding matrix））

Future work：
1. 多个网络的损失函数更复杂的结合方式（目前是简单的相加）
2. 从两个网络扩展到多个网络

Applications

• visualization
• link prediction
• multi-class classification and multi-label classification.

Particularly, multi-class classification and multi-label classification in networks are similar to community detection.

abstract & introduction

The authors introduced the basic idea of network embedding. And they claim that features in the latent space is very important.

Previous methods focus on the intra-network, and most of them are designed for dimension reduction of existing features. The authors propose the embedding of inter-network edges of two different type networks(like author-word networks, as a heterogeneous networks), and try to mine the latent features from the inter-network edges.

They also raised a concept of harmonious embedding matrix to further embed the embeddings that only encode intra-network edges.

They propose an alternating optimization algorithm to solve the learning objective of the EOE in which the learning objective is optimized with respect to one type of variable at a time until convergence.

Related work

A couple of graph or network embedding methods have been proposed previously, but they are originally designed for dimension reduction of existing features . Specifically, their objectives are to learn low-dimensional latent representations of existing features so that learning complexity brought by feature dimension would be significantly reduced.

LINE preserves both interaction information and non-interaction information, which is similar to the proposed EOE. But the proposed EOE model differs from LINE in the formulation of cost function. The proposed EOE is designed for embedding of couple heterogeneous networks.

Prelimimaries

To reconcile the heterogeneities of the two latent spaces, they introduce a harmonious embedding matrix to further embed the embeddings from one latent space to another latent space.
As:

And the harmonious embedding matrix is:

Model

The difination of loss function:

To cast both these two regulations to an optimization problem, small probabilities of pairs of vertices with edges and large probabilities of pairs of vertices without edges should be penalized.

*And now I am wondering:
If the application of the algorithm is to predict links between nodes, how does it work if we penalize large probabilities of pairs of vertices without edges?
1. If the algo there has both train and predict process, the method is understandable.
2. If we just use the algo straight forward to vectorize nodes, it is not good.*

The truth comes out with the first assumption. And there is no problem.