深度学习Course5第三周Sequence Models & Attention Mechanism习题整理

1. Consider using this encoder-decoder model for machine translation.
   
   True/False: This model is a “conditional language model” in the sense that the decoder portion (shown in green) is modeling the probability of the input sentence $x$.

• True
• False

解析：The encoder-decoder model for machine translation models the probability of the output sentence y conditioned on the input sentence x. The encoder portion is shown in green, while the decoder portion is shown in purple.

2. In beam search, if you increase the beam width B, which of the following would you expect to be true?

• Beam search will converge after fewer steps.
• Beam search will use up less memory.
• Beam search will generally find better solutions (i.e. do a better job maximizing $P(y\mid x))$.
• Beam search will run more quickly.

解析：As the beam width increases, beam search runs more slowly, uses up more memory, and converges after more steps, but generally finds better solutions.

3. In machine translation, if we carry out beam search without using sentence normalization, the algorithm will tend to output overly short translations.

• True
• False

4. Suppose you are building a speech recognition system, which uses an RNN model to map from audio clip $x$ to a text transcript $y$. Your algorithm uses beam search to try to find the value of $y$ that maximizes $P(y\mid x)$.On a dev set example, given an input audio clip, your algorithm outputs the transcript $\hat{y}$​= “I’m building an A Eye system in Silly con Valley.”, whereas a human gives a much superior transcript $y^{\ast }$ “I’m building an AI system in Silicon Valley.”According to your model,
   $P(\hat{y}\mid x)$ = $1.09\ast 10^{-}7$
   $P(y^{\ast }\mid x)$ = $7.21\ast 10^{-}8$
   Would you expect increasing the beam width B to help correct this example?

• Yes, because $P(y\ast \mid x)\le P({\hat{y}}^{\mid }x)$ indicates the error should be attributed to the RNN rather than to the search algorithm.
• Yes, because $P(y\ast \mid x)\le P({\hat{y}}^{\mid }x)$ indicates the error should be attributed to the search algorithm rather than to the RNN.
• No, because $P(y\ast \mid x)\le P({\hat{y}}^{\mid }x)$ indicates the error should be attributed to the search algorithm rather than to the RNN.
• No, because $P(y\ast \mid x)\le P({\hat{y}}^{\mid }x)$ indicates the error should be attributed to the RNN rather than to the search algorithm.

5. Continuing the example from Q4, suppose you work on your algorithm for a few more weeks, and now find that for the vast majority of examples on which your algorithm makes a mistake, $P(y\ast \mid x)>P({\hat{y}}^{\mid }x)$. This suggests you should focus your attention on improving the search algorithm.

• True
• False

6. Consider the attention model for machine translation.
   
   Which of the following statements about ${\alpha }^{<t,t’>}$ are true? Check all that apply.

• ${\sum }_{t^{{}^{\prime }}}{\alpha }^{<t,t^{{}^{\prime }}>}=1$
• We expect ${\alpha }^{<t,t^{{}^{\prime }}>}$ to be generally larger for values of ${\alpha }^{<t^{{}^{\prime }}>}$ that are highly relevant to the value the network should output for $y^{<t^{{}^{\prime }}>}$. (Note the indices in the superscripts.)
• ${\alpha }^{<t,t^{{}^{\prime }}>}$ is equal to the amount of attention$y^{<t>}$ should pay to ${\alpha }^{<t^{{}^{\prime }}>}$This should not be
• ${\sum }_{t^{{}^{\prime }}}{\alpha }^{<t,t^{{}^{\prime }}>}=0$

7. The network learns where to “pay attention” by learning the values $e^{<t,t’>}$, which are computed using a small neural network:
   We can replace $s^{<t-1>}$ with $s^{<t>}$ as an input to this neural network because $s^{<t>}$ is independent of ${\alpha }^{<t,t’>}$ and $e^{<t,t’>}$.

• True
• False

8. Compared to the encoder-decoder model shown in Question 1 of this quiz (which does not use an attention mechanism), we expect the attention model to have the least advantage when:

• The input sequence length $T_x$ is small.
• The input sequence length $T_x$ is large.

解析：The encoder-decoder model works quite well with short sentences. The true advantage for the attention model occurs when the input sentence is large.

9. Under the CTC model, identical repeated characters not separated by the “blank” character (_) are collapsed. Under the CTC model, what does the following string collapse to?

__c_oo_o_kk___b_ooooo__oo__kkk

• cokbok
• cookbook
• coookkboooooookkk
• cook book

10. In trigger word detection, $x^{<t>}$ is:

• Whether someone has just finished saying the trigger word at time $t$.
• Features of the audio (such as spectrogram features) at time $t$.
• Whether the trigger word is being said at time $t$.
• The $t$-th input word, represented as either a one-hot vector or a word embedding.