词向量与词向量拼接_05.序列模型 W2.自然语言处理与词嵌入（作业：词向量+Emoji表情生成）...

05.序列模型 W2.自然语言处理与词嵌入(作业：词向量+Emoji表情生成)

  

日期：2020-10-04

浏览：15    评论：0

核心提示：文章目录作业1：1. 余弦相似度2. 单词类比3. 词向量纠偏3.1 消除对非性别词语的偏见3.2 性别词的均衡算法作业2：Emojify表情生成1. Baseline model: Emojifier-V11.1 数据集1.2 模型预览1.3 实现 Emojifier-V11.4 在训练集上测试2. Emojifier-V2: Using LSTMs in Keras2.1 模型预览2.2 Keras and mini-batching2.3 Embedding 层2.3 建立 Emojifier-V2

文章目录

作业1：

1. 余弦相似度

2. 单词类比

3. 词向量纠偏

3.1 消除对非性别词语的偏见

3.2 性别词的均衡算法

作业2：Emojify表情生成

1. Baseline model: Emojifier-V1

1.1 数据集

1.2 模型预览

1.3 实现 Emojifier-V1

1.4 在训练集上测试

2. Emojifier-V2: Using LSTMs in Keras

2.1 模型预览

2.2 Keras and mini-batching

2.3 Embedding 层

2.3 建立 Emojifier-V2

测试题：参考博文

笔记：W2.自然语言处理与词嵌入

作业1：

加载预训练的 单词向量，用 c o s ( θ ) cos(\theta)cos(θ) 余弦夹角 测量相似度

使用词嵌入解决类比问题

修改词嵌入降低性比歧视

import numpy as np

from w2v_utils import *

这个作业使用 50-维的 GloVe vectors 表示单词

words, word_to_vec_map = read_glove_vecs('data/glove.6B.50d.txt')

1. 余弦相似度

CosineSimilarity(u, v) = u . v ∣ ∣ u ∣ ∣ 2 ∣ ∣ v ∣ ∣ 2 = c o s ( θ ) \text{CosineSimilarity(u, v)} = \frac {u . v} {||u||_2 ||v||_2} = cos(\theta)CosineSimilarity(u, v)=∣∣u∣∣2​∣∣v∣∣2​u.v​=cos(θ)

其中 ∣ ∣ u ∣ ∣ 2 = ∑ i = 1 n u i 2 ||u||_2 = \sqrt{\sum_{i=1}^{n} u_i^2}∣∣u∣∣2​=∑i=1n​ui2​​

# GRADED FUNCTION: cosine_similarity

def cosine_similarity(u, v):

""" Cosine similarity reflects the degree of similariy between u and v Arguments: u -- a word vector of shape (n,) v -- a word vector of shape (n,) Returns: cosine_similarity -- the cosine similarity between u and v defined by the formula above. """

distance = 0.0

### START CODE HERE ###

# Compute the dot product between u and v (≈1 line)

dot = np.dot(u, v)

# Compute the L2 norm of u (≈1 line)

norm_u = np.linalg.norm(u)

# Compute the L2 norm of v (≈1 line)

norm_v = np.linalg.norm(v)

# Compute the cosine similarity defined by formula (1) (≈1 line)

cosine_similarity = dot/(norm_u*norm_v)

### END CODE HERE ###

return cosine_similarity

2. 单词类比

例如：男人：女人 --> 国王：王后

# GRADED FUNCTION: complete_analogy

def complete_analogy(word_a, word_b, word_c, word_to_vec_map):

""" Performs the word analogy task as explained above: a is to b as c is to ____. Arguments: word_a -- a word, string word_b -- a word, string word_c -- a word, string word_to_vec_map -- dictionary that maps words to their corresponding vectors. Returns: best_word -- the word such that v_b - v_a is close to v_best_word - v_c, as measured by cosine similarity """

# convert words to lower case

word_a, word_b, word_c = word_a.lower(), word_b.lower(), word_c.lower()

### START CODE HERE ###

# Get the word embeddings v_a, v_b and v_c (≈1-3 lines)

e_a, e_b, e_c = word_to_vec_map[word_a],word_to_vec_map[word_b],word_to_vec_map[word_c]

### END CODE HERE ###

words = word_to_vec_map.keys()

max_cosine_sim = -100 # Initialize max_cosine_sim to a large negative number

best_word = None # Initialize best_word with None, it will help keep track of the word to output

# loop over the whole word vector set

for w in words:

# to avoid best_word being one of the input words, pass on them.

if w in [word_a, word_b, word_c] :

continue

### START CODE HERE ###

# Compute cosine similarity between the vector (e_b - e_a) and the vector ((w's vector representation) - e_c) (≈1 line)

cosine_sim = cosine_similarity(e_b-e_a, word_to_vec_map[w]-e_c)

# If the cosine_sim is more than the max_cosine_sim seen so far,

# then: set the new max_cosine_sim to the current cosine_sim and the best_word to the current word (≈3 lines)

if cosine_sim > max_cosine_sim:

max_cosine_sim = cosine_sim

best_word = w

### END CODE HERE ###

return best_word

测试：

triads_to_try = [('italy', 'italian', 'spain'), ('india', 'delhi', 'japan'), ('man', 'woman', 'boy'), ('small', 'smaller', 'large')]

for triad in triads_to_try:

print ('{} -> {} :: {} -> {}'.format( *triad, complete_analogy(*triad,word_to_vec_map)))

输出：

italy -> italian :: spain -> spanish

india -> delhi :: japan -> tokyo

man -> woman :: boy -> girl

small -> smaller :: large -> larger

额外测试：

good -> ok :: bad -> oops(糟糕)

father -> dad :: mother -> mom

3. 词向量纠偏

研究反映在单词嵌入中的性别偏见，并探索减少这种偏见的算法

g = word_to_vec_map['woman'] - word_to_vec_map['man']

print(g)

输出：向量(50维)

[-0.087144 0.2182 -0.40986 -0.03922 -0.1032 0.94165

-0.06042 0.32988 0.46144 -0.35962 0.31102 -0.86824

0.96006 0.01073 0.24337 0.08193 -1.02722 -0.21122

0.695044 -0.00222 0.29106 0.5053 -0.099454 0.40445

0.30181 0.1355 -0.0606 -0.07131 -0.19245 -0.06115

-0.3204 0.07165 -0.13337 -0.25068714 -0.14293 -0.224957

-0.149 0.048882 0.12191 -0.27362 -0.165476 -0.20426

0.54376 -0.271425 -0.10245 -0.32108 0.2516 -0.33455

-0.04371 0.01258 ]

print ('List of names and their similarities with constructed vector:')

# girls and boys name

name_list = ['john', 'marie', 'sophie', 'ronaldo', 'priya', 'rahul', 'danielle', 'reza', 'katy', 'yasmin']

for w in name_list:

print (w, cosine_similarity(word_to_vec_map[w], g))

输出：

List of names and their similarities with constructed vector:

john -0.23163356145973724

marie 0.315597935396073

sophie 0.31868789859418784

ronaldo -0.31244796850329437

priya 0.17632041839009402

rahul -0.16915471039231716

danielle 0.24393299216283895

reza -0.07930429672199553

katy 0.2831068659572615

yasmin 0.2331385776792876

可以看出，

女性的名字往往与向量

打赏

本文转载自：网络

所有权利归属于原作者，如文章来源标示错误或侵犯了您的权利请联系微信13520258486

最近资讯中心

最新资讯中心

相关评论