translation attention

---

前言

将keras数据集下载到本地 datasets dir

    # Download the file
    path_to_zip = tf.keras.utils.get_file(
        'spa-eng.zip', origin='http://storage.googleapis.com/download.tensorflow.org/data/spa-eng.zip',
        extract=True, cache_dir='.')

You know I’d rather die than betray a comrade.
你知道我是宁死都不出卖战友的。
punctuation

out

path_to_file:.\datasets/spa-eng/spa.txt
preprocess_sentence(en_sentence),
may i borrow this book ?
preprocess_sentence(sp_sentence).encode(‘utf-8’),
b’ \xc2\xbf puedo tomar prestado este libro ? ’
str.decode,
¿ puedo tomar prestado este libro ?
word_pairs[:2]:
[[’ go . ', ’ ve . ‘], [’ go . ', ’ vete . ‘]]
*word_pairs[:2]:
[’ go . ', ’ ve . ‘] [’ go . ', ’ vete . ‘]
list(zip(*word_pairs[:2])):
[(’ go . ', ’ go . ‘), (’ ve . ', ’ vete . ‘)]
en[-1]:
if you want to sound like a native speaker , you must be willing to practice saying the same sentence over and over in the same way that banjo players practice the same phrase over and over until they can play it correctly and at the desired tempo .
sp[-1]:
si quieres sonar como un hablante nativo , debes estar dispuesto a practicar diciendo la misma frase una y otra vez de la misma manera en que un musico de banjo practica el mismo fraseo una y otra vez hasta que lo puedan tocar correctamente y en el tiempo esperado .
len(en):
118964
word_pairs[:2]:
[[’ go . ', ’ ve . ‘], [’ go . ', ’ vete . ‘]]
*word_pairs[:2]:
[’ go . ', ’ ve . ‘] [’ go . ', ’ vete . ‘]
list(zip(*word_pairs[:2])):
[(’ go . ', ’ go . ‘), (’ ve . ', ’ vete . ')]
max_length_targ：,11
max_length_inp：,16
len(input_tensor_train), len(target_tensor_train), len(input_tensor_val), len(target_tensor_val):
24000 24000 6000 6000
input_tensor_train.shape
(24000, 16)
input_tensor_val.shape
(6000, 16)
target_tensor_train.shape
(24000, 11)
target_tensor_val.shape
(6000, 11)

Input Language; index to word mapping
1 ---->
2878 ----> seguid
2312 ----> remando
3 ----> .
2 ---->

Target Language; index to word mapping
1 ---->
111 ----> keep
1739 ----> paddling
3 ----> .
2 ---->

二、使用步骤

random_state

    # Creating training and validation sets using an 80-20 split
    input_tensor_train, input_tensor_val, target_tensor_train, target_tensor_val = train_test_split(input_tensor,
                                                                                                    target_tensor,
                                                                                                    random_state=10,
                                                                                                    test_size=0.2)

attention weights

attention weights = softmax(score, axis = 1). Softmax by default is applied on the last axis but here we want to apply it on the 1st axis, since the shape of score is (batch_size, max_length, hidden_size). Max_length is the length of our input. Since we are trying to assign a weight to each input, softmax should be applied on that axis.

总结

    checkpoint_path = "./checkpoints/train"
    ckpt = tf.train.Checkpoint(encoder=encoder,
                               decoder=decoder,
                               optimizer=optimizer)
    ckpt_manager = tf.train.CheckpointManager(ckpt, checkpoint_path, max_to_keep=5)

    start_epoch = 0
    if ckpt_manager.latest_checkpoint:
        start_epoch = int(ckpt_manager.latest_checkpoint.split('-')[-1])
        # restoring the latest checkpoint in checkpoint_path
        ckpt.restore(ckpt_manager.latest_checkpoint)

ckpt_manager.save()