Coursera 吴恩达DeepLearning.AI 第五课 sequence model 序列模型 第二周 Operations on word vectors - v2

只做了计分部分

Operations on word vectors

Welcome to your first assignment of this week!

Because word embeddings are very computionally expensive to train, most ML practitioners will load a pre-trained set of embeddings.

After this assignment you will be able to:

• Load pre-trained word vectors, and measure similarity using cosine similarity
• Use word embeddings to solve word analogy problems such as Man is to Woman as King is to __.
• Modify word embeddings to reduce their gender bias

Let's get started! Run the following cell to load the packages you will need.

In [ ]:

import numpy as np

from w2v_utils import *

Next, lets load the word vectors. For this assignment, we will use 50-dimensional GloVe vectors to represent words. Run the following cell to load the word_to_vec_map.

In [ ]:

words, word_to_vec_map = read_glove_vecs('data/glove.6B.50d.txt')

You've loaded:

• words: set of words in the vocabulary.
• word_to_vec_map: dictionary mapping words to their GloVe vector representation.

You've seen that one-hot vectors do not do a good job cpaturing what words are similar. GloVe vectors provide much more useful information about the meaning of individual words. Lets now see how you can use GloVe vectors to decide how similar two words are.

1 - Cosine similarity

To measure how similar two words are, we need a way to measure the degree of similarity between two embedding vectors for the two words. Given two vectors uu and vv, cosine similarity is defined as follows:

CosineSimilarity(u, v)=u.v||u||2||v||2=cos(θ)(1)(1)CosineSimilarity(u, v)=u.v||u||2||v||2=cos(θ)

where u.vu.v is the dot product (or inner product) of two vectors, ||u||2||u||2 is the norm (or length) of the vector uu, and θθ is the angle between uu and vv. This similarity depends on the angle between uu and vv. If uu and vv are very similar, their cosine similarity will be close to 1; if they are dissimilar, the cosine similarity will take a smaller value.

Figure 1: The cosine of the angle between two vectors is a measure of how similar they are

Exercise: Implement the function cosine_similarity() to evaluate similarity between word vectors.

Reminder: The norm of uu is defined as ||u||2=∑ni=1u2i⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯√||u||2=∑i=1nui2

In [ ]:

# GRADED FUNCTION: cosine_similarity

​

def cosine_similarity(u, v):

    """

    Cosine similarity reflects the degree of similariy between u and v

    Arguments:

        u -- a word vector of shape (n,)          

        v -- a word vector of shape (n,)

​

    Returns:

        cosine_similarity -- the cosine similarity between u and v defined by the formula above.

    """

    distance = 0.0

    ### START CODE HERE ###

    # Compute the dot product between u and v (≈1 line)

    dot = np.dot(u,v)

    # Compute the L2 norm of u (≈1 line)

    norm_u = np.sqrt(np.sum(np.square(u)))

    # Compute the L2 norm of v (≈1 line)

    norm_v =np.sqrt(np.sum(np.square(v)))

    # Compute the cosine similarity defined by formula (1) (≈1 line)

    cosine_similarity = dot/(norm_u*norm_v)

    ### END CODE HERE ###

    return cosine_similarity

In [ ]:

father = word_to_vec_map["father"]

mother = word_to_vec_map["mother"]

ball = word_to_vec_map["ball"]

crocodile = word_to_vec_map["crocodile"]

france = word_to_vec_map["france"]

italy = word_to_vec_map["italy"]

paris = word_to_vec_map["paris"]

rome = word_to_vec_map["rome"]

​

print("cosine_similarity(father, mother) = ", cosine_similarity(father, mother))

print("cosine_similarity(ball, crocodile) = ",cosine_similarity(ball, crocodile))

print("cosine_similarity(france - paris, rome - italy) = ",cosine_similarity(france - paris, rome - italy))

​

​

​

Expected Output:

cosine_similarity(father, mother) =	0.890903844289
cosine_similarity(ball, crocodile) =	0.274392462614
cosine_similarity(france - paris, rome - italy) =	-0.675147930817

After you get the correct expected output, please feel free to modify the inputs and measure the cosine similarity between other pairs of words! Playing around the cosine similarity of other inputs will give you a better sense of how word vectors behave.

2 - Word analogy task

In the word analogy task, we complete the sentence "a is to b as c is to __". An example is 'man is to woman as king is to queen' . In detail, we are trying to find a word d, such that the associated word vectors ea,eb,ec,edea,eb,ec,ed are related in the following manner: eb−ea≈ed−eceb−ea≈ed−ec. We will measure the similarity between eb−eaeb−ea and ed−eced−ec using cosine similarity.

Exercise: Complete the code below to be able to perform word analogies!

In [ ]:

# GRADED FUNCTION: complete_analogy

​

def complete_analogy(word_a, word_b, word_c, word_to_vec_map):

    """

    Performs the word analogy task as explained above: a is to b as c is to ____. 

    Arguments:

    word_a -- a word, string

    word_b -- a word, string

    word_c -- a word, string

    word_to_vec_map -- dictionary that maps words to their corresponding vectors. 

    Returns:

    best_word --  the word such that v_b - v_a is close to v_best_word - v_c, as measured by cosine similarity

    """

    # convert words to lower case

    word_a, word_b, word_c = word_a.lower(), word_b.lower(), word_c.lower()

    ### START CODE HERE ###

    # Get the word embeddings v_a, v_b and v_c (≈1-3 lines)

    e_a, e_b, e_c = [word_to_vec_map[word_a],word_to_vec_map[word_b],word_to_vec_map[word_c]]

    ### END CODE HERE ###

    words = word_to_vec_map.keys()

    max_cosine_sim = -100              # Initialize max_cosine_sim to a large negative number

    best_word = None                   # Initialize best_word with None, it will help keep track of the word to output

​

    # loop over the whole word vector set

    for w in words:        

        # to avoid best_word being one of the input words, pass on them.

        if w in [word_a, word_b, word_c] :

            continue

        ### START CODE HERE ###

        # Compute cosine similarity between the vector (e_b - e_a) and the vector ((w's vector representation) - e_c)  (≈1 line)

        cosine_sim = cosine_similarity(e_b-e_a,word_to_vec_map[w]-e_c)

        # If the cosine_sim is more than the max_cosine_sim seen so far,

            # then: set the new max_cosine_sim to the current cosine_sim and the best_word to the current word (≈3 lines)

        if cosine_sim > max_cosine_sim:

            max_cosine_sim = cosine_sim

            best_word = w

        ### END CODE HERE ###

    return best_word

Run the cell below to test your code, this may take 1-2 minutes.

In [ ]:

triads_to_try = [('italy', 'italian', 'spain'), ('india', 'delhi', 'japan'), ('man', 'woman', 'boy'), ('small', 'smaller', 'large')]

for triad in triads_to_try:

    print ('{} -> {} :: {} -> {}'.format( *triad, complete_analogy(*triad,word_to_vec_map)))

Expected Output:

italy -> italian ::	spain -> spanish
india -> delhi ::	japan -> tokyo
man -> woman ::	boy -> girl
small -> smaller ::	large -> larger

Once you get the correct expected output, please feel free to modify the input cells above to test your own analogies. Try to find some other analogy pairs that do work, but also find some where the algorithm doesn't give the right answer: For example, you can try small->smaller as big->?.

Congratulations!

You've come to the end of this assignment. Here are the main points you should remember:

• Cosine similarity a good way to compare similarity between pairs of word vectors. (Though L2 distance works too.)
• For NLP applications, using a pre-trained set of word vectors from the internet is often a good way to get started.

Even though you have finished the graded portions, we recommend you take a look too at the rest of this notebook.

Congratulations on finishing the graded portions of this notebook!