word2vec

最新推荐文章于 2024-06-15 19:55:22 发布

sohero

最新推荐文章于 2024-06-15 19:55:22 发布

阅读量543

点赞数

文章标签： foudation

Instead of capturing cooccurrence counts directly
Predict surrounding words of every word
Faster and can easily incorporate a new sentence/document or add a word to the vocabulary

Predict surrounding words in a window of length $c$ of every word.
Objective function: Maximize the $log$ probability of any context word given the current center word:
$J (θ) = 1 T \sum t = 1 T \sum - c \leq j \leq c, j \neq 0 l o g p (w t + j | w t)$ $J(\theta)=\frac{1}{T} \sum_{t=1}^T \sum_{-c\leq j \leq c,j \neq 0} log p(w_{t+j}|w_t)$
$p (w t + j | w t) = p (w o | w i) = e x p ( v ' T w o v w i ) \sum W w = 1 e x p ( v ' T w v w i )$ $p(w_{t+j}|w_t)=p(w_o|w_i)=\frac{exp(v_{w_o}'^T v_{w_i})}{\sum_{w=1}^W exp(v_w'^T v_{w_i})}$
where $v$ and $v'$ are input and output vector representations of $w$ (so every word has two vectors)