Pytorch-seq2seq机器翻译模型（不含attention和含attention两个版本）

由于语料短，训练时间也短，模型性能不好，以下演示过程。

语料链接：https://pan.baidu.com/s/1wpP4t_GSyPAD6HTsIoGPZg
提取码：jqq8

数据格式如图（先英文，再空格，再繁体中文）：

以下代码运行在Google Colab上。 

导包：

import os
import sys
import math
from collections import Counter
import numpy as np
import random

import torch
import torch.nn as nn
import torch.nn.functional as F

import nltk
nltk.download('punkt')

1.数据预处理

1.1读入中英文数据

• 英文使用nltk的word tokenizer来分词，并且使用小写字母
• 中文直接使用单个汉字作为基本单元

def load_data(in_file):
    cn = []
    en = []
    num_examples = 0
    with open(in_file, 'r') as f:
        for line in f:
            line = line.strip().split("	")
            
            en.append(["BOS"] + nltk.word_tokenize(line[0].lower()) + ["EOS"])
            cn.append(["BOS"] + [c for c in line[1]] + ["EOS"])
    return en, cn

train_file = "nmt/en-cn/train.txt"
dev_file = "nmt/en-cn/dev.txt"
train_en, train_cn = load_data(train_file)
dev_en, dev_cn = load_data(dev_file)

查看返回的数据内容：

print(dev_en[:2])
print(dev_cn[:2])

[['BOS', 'she', 'put', 'the', 'magazine', 'on', 'the', 'table', '.', 'EOS'], ['BOS', 'hey', ',', 'what', 'are', 'you', 'doing', 'here', '?', 'EOS']]

[['BOS', '她', '把', '雜', '誌', '放', '在', '桌', '上', '。', 'EOS'], ['BOS', '嘿', '，', '你', '在', '這', '做', '什', '麼', '？', 'EOS']]

1.2构建单词表

UNK_IDX = 0
PAD_IDX = 1
def build_dict(sentences, max_words=50000):
    word_count = Counter()
    for sentence in sentences:
        for s in sentence:
            word_count[s] += 1
    ls = word_count.most_common(max_words)
    total_words = len(ls) + 2
    word_dict = {w[0]: index+2 for index, w in enumerate(ls)}
    word_dict["UNK"] = UNK_IDX
    word_dict["PAD"] = PAD_IDX
    return word_dict, total_words      #total_words所有单词数，最大50002

en_dict, en_total_words = build_dict(train_en)
cn_dict, cn_total_words = build_dict(train_cn)
inv_en_dict = {v: k for k, v in en_dict.items()}    #英文：索引到单词
inv_cn_dict = {v: k for k, v in cn_dict.items()}    #中文：索引到字

1.3把单词全部转变成数字

sort_by_len=True的目的是为了使得一个batch中的句子长度差不多，所以按长度排序。

def encode(en_sentences, cn_sentences, en_dict, cn_dict, sort_by_len=True):        

    length = len(en_sentences)
    out_en_sentences = [[en_dict.get(w, 0) for w in sent] for sent in en_sentences]
    out_cn_sentences = [[cn_dict.get(w, 0) for w in sent] for sent in cn_sentences]

    # sort sentences by word lengths
    def len_argsort(seq):
        return sorted(range(len(seq)), key=lambda x: len(seq[x]))
       
    # 把中文和英文按照同样的顺序排序
    if sort_by_len:
        sorted_index = len_argsort(out_en_sentences)
        out_en_sentences = [out_en_sentences[i] for i in sorted_index]
        out_cn_sentences = [out_cn_sentences[i] for i in sorted_index]
        
    return out_en_sentences, out_cn_sentences

train_en, train_cn = encode(train_en, train_cn, en_dict, cn_dict)
dev_en, dev_cn = encode(dev_en, dev_cn, en_dict, cn_dict)

查看返回的数据内容：

print(train_cn[2])
print([inv_cn_dict[i] for i in train_cn[2]])
print([inv_en_dict[i] for i in train_en[2]])

[2, 982, 2028, 8, 4, 3]

['BOS', '祝', '贺', '你', '。', 'EOS']

['BOS', 'congratulations', '!', 'EOS']

1.4把全部句子分成batch

def get_minibatches(n, minibatch_size, shuffle=True):  #n是传进来的句子数
    idx_list = np.arange(0, n, minibatch_size)   #[0, 1, ..., n-1]按minibatch_size大小分割
    if shuffle:
        np.random.shuffle(idx_list)
    minibatches = []
    for idx in idx_list:
        minibatches.append(np.arange(idx, min(idx + minibatch_size, n)))
    return minibatches

查看上面函数的功能：

get_minibatches(100, 15)
[array([60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74]),
 array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14]),
 array([75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89]),
 array([45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59]),
 array([30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44]),
 array([90, 91, 92, 93, 94, 95, 96, 97, 98, 99]),
 array([15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29])]

def prepare_data(seqs):   #seqs传入的是minibatches中的一个minibatch对应的batch_size个句子索引（嵌套列表），此处batch_size=64

    lengths = [len(seq) for seq in seqs]   
    n_samples = len(seqs)            
    max_len = np.max(lengths)  #batch_size个句子中最长句子长度

    x = np.zeros((n_samples, max_len)).astype('int32')
    x_lengths = np.array(lengths).astype("int32")
    for idx, seq in enumerate(seqs):
        x[idx, :lengths[idx]] = seq
    return x, x_lengths             

def gen_examples(en_sentences, cn_sentences, batch_size):
    minibatches = get_minibatches(len(en_sentences), batch_size)
    all_ex = []
    for minibatch in minibatches:
        mb_en_sentences = [en_sentences[t] for t in minibatch]
        mb_cn_sentences = [cn_sentences[t] for t in minibatch]
        mb_x, mb_x_len = prepare_data(mb_en_sentences)
        mb_y, mb_y_len = prepare_data(mb_cn_sentences)
        all_ex.append((mb_x, mb_x_len, mb_y, mb_y_len))
    return all_ex     #返回内容依次是batch_size个英文句子索引，英文句子长度，中文句子索引，中文句子长度

batch_size = 64
train_data = gen_examples(train_en, train_cn, batch_size)
dev_data = gen_examples(dev_en, dev_cn, batch_size)

2.Encoder Decoder模型（没有Attention版本）

2.1定义计算损失的函数

# masked cross entropy loss
class LanguageModelCriterion(nn.Module):
    def __init__(self):
        super(LanguageModelCriterion, self).__init__()

    def forward(self, input, target, mask):   #把mask的部分忽略掉
        # input: (batch_size * seq_len) * vocab_size
        input = input.contiguous().view(-1, input.size(2))
        # target: batch_size * 1
        target = target.contiguous().view(-1, 1)
        mask = mask.contiguous().view(-1, 1)
        output = -input.gather(1, target) * mask
        output = torch.sum(output) / torch.sum(mask)

        return output

2.2Encoder部分

Encoder模型的任务是把输入文字传入embedding层和GRU层，转换成一些hidden states作为后续的context vectors；

对nn.utils.rnn.pack_padded_sequence和nn.utils.rnn.pad_packed_sequence的理解：http://www.mamicode.com/info-detail-2493083.html

class PlainEncoder(nn.Module):
    def __init__(self, vocab_size, hidden_size, dropout=0.2):       #假设embedding_size=hidden_size
        super(PlainEncoder, self).__init__()
        self.embed = nn.Embedding(vocab_size, hidden_size)
        self.rnn = nn.GRU(hidden_size, hidden_size, batch_first=True)
        self.dropout = nn.Dropout(dropout)

    def forward(self, x, lengths):   #最后一个hidden_state要取出来作为context vector，所以需要lengths
        sorted_len, sorted_idx = lengths.sort(0, descending=True)   #把batch里面的seq按照长度降序排列
        x_sorted = x[sorted_idx.long()]
        embedded = self.dropout(self.embed(x_sorted))
        
        #句子padding到一样长度的（真实句长会比padding的短），为了rnn时能取到真实长度的最后状态，先pack_padded_sequence进行处理
        packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, sorted_len.long().cpu().data.numpy(), batch_first=True)
        packed_out, hid = self.rnn(packed_embedded)
        out, _ = nn.utils.rnn.pad_packed_sequence(packed_out, batch_first=True)    #回到padding长度
        
        _, original_idx = sorted_idx.sort(0, descending=False)                     #排序回原来的样子
        out = out[original_idx.long()].contiguous()
        hid = hid[:, original_idx.long()].contiguous()
        
        return out, hid[[-1]]   #hid[[-1]]相当于out[:, -1]

2.3Decoder部分

Decoder会根据已经翻译的句子内容和context vectors，来决定下一个输出的单词；

class PlainDecoder(nn.Module):
    def __init__(self, vocab_size, hidden_size, dropout=0.2):
        super(PlainDecoder, self).__init__()
        self.embed = nn.Embedding(vocab_size, hidden_size)
        self.rnn = nn.GRU(hidden_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, vocab_size)
        self.dropout = nn.Dropout(dropout)
        
    def forward(self, y, y_lengths, hid):    #和PlainEncoder的forward过程大致差不多，区别在于hidden_state不是0而是传入的
        sorted_len, sorted_idx = y_lengths.sort(0, descending=True)
        y_sorted = y[sorted_idx.long()]
        hid = hid[:, sorted_idx.long()]

        y_sorted = self.dropout(self.embed(y_sorted))             #[batch_size, y_lengths, embed_size=hidden_size]
        
        packed_seq = nn.utils.rnn.pack_padded_sequence(y_sorted, sorted_len.long().cpu().data.numpy(), batch_first=True)
        out, hid = self.rnn(packed_seq, hid)
        unpacked, _ = nn.utils.rnn.pad_packed_sequence(out, batch_first=True)

        _, original_idx = sorted_idx.sort(0, descending=False)
        output_seq = unpacked[original_idx.long()].contiguous()   #[batch_size, y_lengths, hidden_size]
        hid = hid[:, original_idx.long()].contiguous()            #[1, batch_size, hidden_size]

        output = F.log_softmax(self.fc(output_seq), -1)           #[batch_size, y_lengths, vocab_size]
        
        return output, hid

2.4构建Seq2Seq模型

构建Seq2Seq模型把encoder, attention, decoder串到一起；

class PlainSeq2Seq(nn.Module):
    def __init__(self, encoder, decoder):
        super(PlainSeq2Seq, self).__init__()
        self.encoder = encoder
        self.decoder = decoder
        
    def forward(self, x, x_lengths, y, y_lengths):
        encoder_out, hid = self.encoder(x, x_lengths)
        output, hid = self.decoder(y, y_lengths, hid)
        return output, None
    
    def translate(self, x, x_lengths, y, max_length=10):
        encoder_out, hid = self.encoder(x, x_lengths)
        preds = []
        batch_size = x.shape[0]
        attns = []
        for i in range(max_length):
            output, hid = self.decoder(y=y, y_lengths=torch.ones(batch_size).long().to(y.device), hid=hid)
            y = output.max(2)[1].view(batch_size, 1)
            preds.append(y)
            
        return torch.cat(preds, 1), None

2.5定义损失函数

# masked cross entropy loss
class LanguageModelCriterion(nn.Module):
    def __init__(self):
        super(LanguageModelCriterion, self).__init__()

    def forward(self, input, target, mask):
        # input: (batch_size * seq_len) * vocab_size
        input = input.contiguous().view(-1, input.size(2))
        # target: batch_size * 1
        target = target.contiguous().view(-1, 1)
        mask = mask.contiguous().view(-1, 1)
        output = -input.gather(1, target) * mask
        output = torch.sum(output) / torch.sum(mask)

        return output

3.创建模型

定义模型、损失、优化器。

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dropout = 0.2
hidden_size = 100
encoder = PlainEncoder(vocab_size=en_total_words, hidden_size=hidden_size, dropout=dropout)
decoder = PlainDecoder(vocab_size=cn_total_words, hidden_size=hidden_size, dropout=dropout)
model = PlainSeq2Seq(encoder, decoder)
model = model.to(device)
loss_fn = LanguageModelCriterion().to(device)
optimizer = torch.optim.Adam(model.parameters())

4.评估模型

def evaluate(model, data):
    model.eval()
    total_num_words = total_loss = 0.
    with torch.no_grad():
        for it, (mb_x, mb_x_len, mb_y, mb_y_len) in enumerate(data):
            mb_x = torch.from_numpy(mb_x).to(device).long()
            mb_x_len = torch.from_numpy(mb_x_len).to(device).long()
            mb_input = torch.from_numpy(mb_y[:, :-1]).to(device).long()
            mb_output = torch.from_numpy(mb_y[:, 1:]).to(device).long()
            mb_y_len = torch.from_numpy(mb_y_len-1).to(device).long()
            mb_y_len[mb_y_len<=0] = 1

            mb_pred, attn = model(mb_x, mb_x_len, mb_input, mb_y_len)

            mb_out_mask = torch.arange(mb_y_len.max().item(), device=device)[None, :] < mb_y_len[:, None]
            mb_out_mask = mb_out_mask.float()

            loss = loss_fn(mb_pred, mb_output, mb_out_mask)

            num_words = torch.sum(mb_y_len).item()
            total_loss += loss.item() * num_words
            total_num_words += num_words
    print("Evaluation loss", total_loss/total_num_words)

5.训练模型

def train(model, data, num_epochs=20):
    for epoch in range(num_epochs):
        model.train()
        total_num_words = total_loss = 0.
        for it, (mb_x, mb_x_len, mb_y, mb_y_len) in enumerate(data):
            mb_x = torch.from_numpy(mb_x).to(device).long()
            mb_x_len = torch.from_numpy(mb_x_len).to(device).long()
            mb_input = torch.from_numpy(mb_y[:, :-1]).to(device).long()
            mb_output = torch.from_numpy(mb_y[:, 1:]).to(device).long()
            mb_y_len = torch.from_numpy(mb_y_len-1).to(device).long()
            mb_y_len[mb_y_len<=0] = 1
            
            mb_pred, attn = model(mb_x, mb_x_len, mb_input, mb_y_len)
            
            mb_out_mask = torch.arange(mb_y_len.max().item(), device=device)[None, :] < mb_y_len[:, None]
            mb_out_mask = mb_out_mask.float()
            
            loss = loss_fn(mb_pred, mb_output, mb_out_mask)
            
            num_words = torch.sum(mb_y_len).item()
            total_loss += loss.item() * num_words
            total_num_words += num_words
            
            # 更新模型
            optimizer.zero_grad()
            loss.backward()
            torch.nn.utils.clip_grad_norm_(model.parameters(), 5.)
            optimizer.step()
            
            if it % 100 == 0:
                print("Epoch", epoch, "iteration", it, "loss", loss.item())

                
        print("Epoch", epoch, "Training loss", total_loss/total_num_words)
        if epoch % 5 == 0:
            evaluate(model, dev_data)

训练100次：

train(model, train_data, num_epochs=100)

训练结果（training loss在不断下降）：

Epoch 0 iteration 0 loss 8.084440231323242
Epoch 0 iteration 100 loss 4.8448944091796875
Epoch 0 iteration 200 loss 4.879772663116455
Epoch 0 Training loss 5.477221919210141
Evaluation loss 4.821030395389826
Epoch 1 iteration 0 loss 4.69868278503418
Epoch 1 iteration 100 loss 4.085171699523926
Epoch 1 iteration 200 loss 4.312857151031494
Epoch 1 Training loss 4.579521701350524
Epoch 2 iteration 0 loss 4.193971633911133
Epoch 2 iteration 100 loss 3.678673267364502
Epoch 2 iteration 200 loss 4.019515514373779
Epoch 2 Training loss 4.186071368925457
Epoch 3 iteration 0 loss 3.8352835178375244
Epoch 3 iteration 100 loss 3.3954527378082275
Epoch 3 iteration 200 loss 3.774580240249634
Epoch 3 Training loss 3.9222166424267986
Epoch 4 iteration 0 loss 3.585063934326172
Epoch 4 iteration 100 loss 3.215750217437744
Epoch 4 iteration 200 loss 3.626997232437134
Epoch 4 Training loss 3.722608096150466
Epoch 5 iteration 0 loss 3.411375045776367
Epoch 5 iteration 100 loss 3.0424859523773193
Epoch 5 iteration 200 loss 3.492255926132202
Epoch 5 Training loss 3.5699179079587195
Evaluation loss 3.655821240952787
Epoch 6 iteration 0 loss 3.273927927017212
Epoch 6 iteration 100 loss 2.897022247314453
Epoch 6 iteration 200 loss 3.355715036392212
Epoch 6 Training loss 3.4411540739967426
Epoch 7 iteration 0 loss 3.16508412361145
Epoch 7 iteration 100 loss 2.7818763256073
Epoch 7 iteration 200 loss 3.241000175476074
Epoch 7 Training loss 3.330995073153501
Epoch 8 iteration 0 loss 3.081458806991577
Epoch 8 iteration 100 loss 2.692844867706299
Epoch 8 iteration 200 loss 3.159105062484741
Epoch 8 Training loss 3.237538761219645
Epoch 9 iteration 0 loss 2.983361005783081
Epoch 9 iteration 100 loss 2.5852301120758057
Epoch 9 iteration 200 loss 3.076793670654297
Epoch 9 Training loss 3.1542968146839754
Epoch 10 iteration 0 loss 2.88155198097229
Epoch 10 iteration 100 loss 2.504387617111206
Epoch 10 iteration 200 loss 2.9708898067474365
Epoch 10 Training loss 3.0766581801071924
Evaluation loss 3.3804360915245204
Epoch 11 iteration 0 loss 2.805739164352417
Epoch 11 iteration 100 loss 2.417832612991333
Epoch 11 iteration 200 loss 2.9001076221466064
Epoch 11 Training loss 3.0072335865815747
Epoch 12 iteration 0 loss 2.7389864921569824
Epoch 12 iteration 100 loss 2.352132558822632
Epoch 12 iteration 200 loss 2.864527702331543
Epoch 12 Training loss 2.945309993148362
Epoch 13 iteration 0 loss 2.6841001510620117
Epoch 13 iteration 100 loss 2.2722346782684326
Epoch 13 iteration 200 loss 2.8002915382385254
Epoch 13 Training loss 2.8879525671218156
Epoch 14 iteration 0 loss 2.641491651535034
Epoch 14 iteration 100 loss 2.237807273864746
Epoch 14 iteration 200 loss 2.7538034915924072
Epoch 14 Training loss 2.833802188663957
Epoch 15 iteration 0 loss 2.5613601207733154
Epoch 15 iteration 100 loss 2.149299144744873
Epoch 15 iteration 200 loss 2.671037435531616
Epoch 15 Training loss 2.7850014679518598
Evaluation loss 3.2569677577366516
Epoch 16 iteration 0 loss 2.5330140590667725
Epoch 16 iteration 100 loss 2.0988974571228027
Epoch 16 iteration 200 loss 2.611022472381592
Epoch 16 Training loss 2.7354116963192716
Epoch 17 iteration 0 loss 2.485084295272827
Epoch 17 iteration 100 loss 2.0532665252685547
Epoch 17 iteration 200 loss 2.604226589202881
Epoch 17 Training loss 2.6934350694497957
Epoch 18 iteration 0 loss 2.4521820545196533
Epoch 18 iteration 100 loss 2.0395381450653076
Epoch 18 iteration 200 loss 2.5578808784484863
Epoch 18 Training loss 2.651303096776386
Epoch 19 iteration 0 loss 2.390338182449341
Epoch 19 iteration 100 loss 1.9780246019363403
Epoch 19 iteration 200 loss 2.5150232315063477
Epoch 19 Training loss 2.611681331448251
Epoch 20 iteration 0 loss 2.352649211883545
Epoch 20 iteration 100 loss 1.9426053762435913
Epoch 20 iteration 200 loss 2.4782586097717285
Epoch 20 Training loss 2.5747013451744616
Evaluation loss 3.194680030596711
Epoch 21 iteration 0 loss 2.3205008506774902
Epoch 21 iteration 100 loss 1.9143742322921753
Epoch 21 iteration 200 loss 2.4607479572296143
Epoch 21 Training loss 2.5404243457594116
Epoch 22 iteration 0 loss 2.3100969791412354
Epoch 22 iteration 100 loss 1.912932276725769
Epoch 22 iteration 200 loss 2.4103682041168213
Epoch 22 Training loss 2.507626390779296
Epoch 23 iteration 0 loss 2.228956699371338
Epoch 23 iteration 100 loss 1.8543353080749512
Epoch 23 iteration 200 loss 2.3663489818573
Epoch 23 Training loss 2.475231424650597
Epoch 24 iteration 0 loss 2.199277639389038
Epoch 24 iteration 100 loss 1.8272788524627686
Epoch 24 iteration 200 loss 2.3518714904785156
Epoch 24 Training loss 2.4439996520576863
Epoch 25 iteration 0 loss 2.198460817337036
Epoch 25 iteration 100 loss 1.7921738624572754
Epoch 25 iteration 200 loss 2.3299384117126465
Epoch 25 Training loss 2.416539151404694
Evaluation loss 3.1583419660450347
Epoch 26 iteration 0 loss 2.1647706031799316
Epoch 26 iteration 100 loss 1.725657343864441
Epoch 26 iteration 200 loss 2.268852710723877
Epoch 26 Training loss 2.3919890312051444
Epoch 27 iteration 0 loss 2.1400880813598633
Epoch 27 iteration 100 loss 1.7474910020828247
Epoch 27 iteration 200 loss 2.256742000579834
Epoch 27 Training loss 2.3595162004913086
Epoch 28 iteration 0 loss 2.0979115962982178
Epoch 28 iteration 100 loss 1.7000322341918945
Epoch 28 iteration 200 loss 2.2546005249023438
Epoch 28 Training loss 2.3335356415568618
Epoch 29 iteration 0 loss 2.1031572818756104
Epoch 29 iteration 100 loss 1.6599613428115845
Epoch 29 iteration 200 loss 2.2020833492279053
Epoch 29 Training loss 2.311978717884133
Epoch 30 iteration 0 loss 2.041980028152466
Epoch 30 iteration 100 loss 1.6663353443145752
Epoch 30 iteration 200 loss 2.1463098526000977
Epoch 30 Training loss 2.2902015222655807
Evaluation loss 3.133273747140961
Epoch 31 iteration 0 loss 2.0045719146728516
Epoch 31 iteration 100 loss 1.6515719890594482
Epoch 31 iteration 200 loss 2.1130664348602295
Epoch 31 Training loss 2.2633183437027657
Epoch 32 iteration 0 loss 1.9948643445968628
Epoch 32 iteration 100 loss 1.6262538433074951
Epoch 32 iteration 200 loss 2.1329450607299805
Epoch 32 Training loss 2.242057023454951
Epoch 33 iteration 0 loss 1.9623773097991943
Epoch 33 iteration 100 loss 1.6022558212280273
Epoch 33 iteration 200 loss 2.092766523361206
Epoch 33 Training loss 2.219300144243463
Epoch 34 iteration 0 loss 1.929176688194275
Epoch 34 iteration 100 loss 1.57985258102417
Epoch 34 iteration 200 loss 2.067972183227539
Epoch 34 Training loss 2.199957146669663
Epoch 35 iteration 0 loss 1.9449653625488281
Epoch 35 iteration 100 loss 1.5760831832885742
Epoch 35 iteration 200 loss 2.056731939315796
Epoch 35 Training loss 2.1790822226814464
Evaluation loss 3.13363336627263
Epoch 36 iteration 0 loss 1.8961074352264404
Epoch 36 iteration 100 loss 1.5195672512054443
Epoch 36 iteration 200 loss 2.0268213748931885
Epoch 36 Training loss 2.160204240618562
Epoch 37 iteration 0 loss 1.9172203540802002
Epoch 37 iteration 100 loss 1.495902180671692
Epoch 37 iteration 200 loss 1.9827772378921509
Epoch 37 Training loss 2.139063811380212
Epoch 38 iteration 0 loss 1.8988227844238281
Epoch 38 iteration 100 loss 1.5224453210830688
Epoch 38 iteration 200 loss 1.972291111946106
Epoch 38 Training loss 2.1211086652629887
Epoch 39 iteration 0 loss 1.8728121519088745
Epoch 39 iteration 100 loss 1.4476994276046753
Epoch 39 iteration 200 loss 1.9898269176483154
Epoch 39 Training loss 2.1024907934743258
Epoch 40 iteration 0 loss 1.8664008378982544
Epoch 40 iteration 100 loss 1.4997611045837402
Epoch 40 iteration 200 loss 1.9541966915130615
Epoch 40 Training loss 2.086313187411815
Evaluation loss 3.1282314096494708
Epoch 41 iteration 0 loss 1.865237832069397
Epoch 41 iteration 100 loss 1.4755399227142334
Epoch 41 iteration 200 loss 1.9337103366851807
Epoch 41 Training loss 2.068258631932244
Epoch 42 iteration 0 loss 1.790804147720337
Epoch 42 iteration 100 loss 1.4380069971084595
Epoch 42 iteration 200 loss 1.9523491859436035
Epoch 42 Training loss 2.0498001934027874
Epoch 43 iteration 0 loss 1.7979768514633179
Epoch 43 iteration 100 loss 1.436006784439087
Epoch 43 iteration 200 loss 1.9101322889328003
Epoch 43 Training loss 2.0354298580230195
Epoch 44 iteration 0 loss 1.7717180252075195
Epoch 44 iteration 100 loss 1.412601351737976
Epoch 44 iteration 200 loss 1.8883790969848633
Epoch 44 Training loss 2.0182710578663032
Epoch 45 iteration 0 loss 1.7614871263504028
Epoch 45 iteration 100 loss 1.3429900407791138
Epoch 45 iteration 200 loss 1.862486720085144
Epoch 45 Training loss 2.0034489605129595
Evaluation loss 3.13050353642062
Epoch 46 iteration 0 loss 1.753187656402588
Epoch 46 iteration 100 loss 1.3810824155807495
Epoch 46 iteration 200 loss 1.8526273965835571
Epoch 46 Training loss 1.9899710891643612
Epoch 47 iteration 0 loss 1.7567869424819946
Epoch 47 iteration 100 loss 1.3430988788604736
Epoch 47 iteration 200 loss 1.8135911226272583
Epoch 47 Training loss 1.9723690433387957
Epoch 48 iteration 0 loss 1.7263280153274536
Epoch 48 iteration 100 loss 1.3430798053741455
Epoch 48 iteration 200 loss 1.8229252099990845
Epoch 48 Training loss 1.9580909331705005
Epoch 49 iteration 0 loss 1.731834888458252
Epoch 49 iteration 100 loss 1.325390100479126
Epoch 49 iteration 200 loss 1.8075029850006104
Epoch 49 Training loss 1.9418853706725143
Epoch 50 iteration 0 loss 1.7218893766403198
Epoch 50 iteration 100 loss 1.2710607051849365
Epoch 50 iteration 200 loss 1.8196479082107544
Epoch 50 Training loss 1.9300463292027463
Evaluation loss 3.1402900424368902
Epoch 51 iteration 0 loss 1.701721429824829
Epoch 51 iteration 100 loss 1.2720820903778076
Epoch 51 iteration 200 loss 1.7759710550308228
Epoch 51 Training loss 1.9192517232508806
Epoch 52 iteration 0 loss 1.7286512851715088
Epoch 52 iteration 100 loss 1.2737478017807007
Epoch 52 iteration 200 loss 1.7545547485351562
Epoch 52 Training loss 1.906238278183267
Epoch 53 iteration 0 loss 1.6672327518463135
Epoch 53 iteration 100 loss 1.3138436079025269
Epoch 53 iteration 200 loss 1.8045201301574707
Epoch 53 Training loss 1.8922825534741075
Epoch 54 iteration 0 loss 1.617557168006897
Epoch 54 iteration 100 loss 1.22885262966156
Epoch 54 iteration 200 loss 1.7750707864761353
Epoch 54 Training loss 1.8807705430479014
Epoch 55 iteration 0 loss 1.66348135471344
Epoch 55 iteration 100 loss 1.2331219911575317
Epoch 55 iteration 200 loss 1.7303975820541382
Epoch 55 Training loss 1.867195544079556
Evaluation loss 3.145431456349013
Epoch 56 iteration 0 loss 1.6259342432022095
Epoch 56 iteration 100 loss 1.2141388654708862
Epoch 56 iteration 200 loss 1.6984847784042358
Epoch 56 Training loss 1.8548133653506713
Epoch 57 iteration 0 loss 1.605487585067749
Epoch 57 iteration 100 loss 1.1920335292816162
Epoch 57 iteration 200 loss 1.7253336906433105
Epoch 57 Training loss 1.8387836396466541
Epoch 58 iteration 0 loss 1.600136160850525
Epoch 58 iteration 100 loss 1.2192472219467163
Epoch 58 iteration 200 loss 1.6888371706008911
Epoch 58 Training loss 1.83046734055076
Epoch 59 iteration 0 loss 1.6042535305023193
Epoch 59 iteration 100 loss 1.2362377643585205
Epoch 59 iteration 200 loss 1.6654771566390991
Epoch 59 Training loss 1.8226244935892273
Epoch 60 iteration 0 loss 1.5602766275405884
Epoch 60 iteration 100 loss 1.201045036315918
Epoch 60 iteration 200 loss 1.6702684164047241
Epoch 60 Training loss 1.8102721190615219
Evaluation loss 3.154303393916162
Epoch 61 iteration 0 loss 1.5679781436920166
Epoch 61 iteration 100 loss 1.2105367183685303
Epoch 61 iteration 200 loss 1.6650742292404175
Epoch 61 Training loss 1.7970227477404426
Epoch 62 iteration 0 loss 1.5734565258026123
Epoch 62 iteration 100 loss 1.1602052450180054
Epoch 62 iteration 200 loss 1.583187222480774
Epoch 62 Training loss 1.787027303402099
Epoch 63 iteration 0 loss 1.563283920288086
Epoch 63 iteration 100 loss 1.1829460859298706
Epoch 63 iteration 200 loss 1.6458944082260132
Epoch 63 Training loss 1.7742324239103342
Epoch 64 iteration 0 loss 1.5429617166519165
Epoch 64 iteration 100 loss 1.1225509643554688
Epoch 64 iteration 200 loss 1.6353931427001953
Epoch 64 Training loss 1.7665018986396424
Epoch 65 iteration 0 loss 1.5284583568572998
Epoch 65 iteration 100 loss 1.1426113843917847
Epoch 65 iteration 200 loss 1.6138485670089722
Epoch 65 Training loss 1.7557591437816458
Evaluation loss 3.166533922994568
Epoch 66 iteration 0 loss 1.5184751749038696
Epoch 66 iteration 100 loss 1.127056360244751
Epoch 66 iteration 200 loss 1.611910343170166
Epoch 66 Training loss 1.7446940747065838
Epoch 67 iteration 0 loss 1.4880752563476562
Epoch 67 iteration 100 loss 1.1075133085250854
Epoch 67 iteration 200 loss 1.6138321161270142
Epoch 67 Training loss 1.7374662356132202
Epoch 68 iteration 0 loss 1.5260978937149048
Epoch 68 iteration 100 loss 1.12235689163208
Epoch 68 iteration 200 loss 1.6129950284957886
Epoch 68 Training loss 1.7253250324901928
Epoch 69 iteration 0 loss 1.5172449350357056
Epoch 69 iteration 100 loss 1.1174883842468262
Epoch 69 iteration 200 loss 1.551174283027649
Epoch 69 Training loss 1.7166664929363027
Epoch 70 iteration 0 loss 1.5006300210952759
Epoch 70 iteration 100 loss 1.0905342102050781
Epoch 70 iteration 200 loss 1.5446460247039795
Epoch 70 Training loss 1.70989819337649
Evaluation loss 3.1750113054724385
Epoch 71 iteration 0 loss 1.4726097583770752
Epoch 71 iteration 100 loss 1.086822509765625
Epoch 71 iteration 200 loss 1.5575647354125977
Epoch 71 Training loss 1.697000935158525
Epoch 72 iteration 0 loss 1.449334979057312
Epoch 72 iteration 100 loss 1.0667144060134888
Epoch 72 iteration 200 loss 1.530726671218872
Epoch 72 Training loss 1.6881878283419123
Epoch 73 iteration 0 loss 1.4603246450424194
Epoch 73 iteration 100 loss 1.0751914978027344
Epoch 73 iteration 200 loss 1.5088605880737305
Epoch 73 Training loss 1.6805761044806562
Epoch 74 iteration 0 loss 1.4748084545135498
Epoch 74 iteration 100 loss 1.0556395053863525
Epoch 74 iteration 200 loss 1.5206905603408813
Epoch 74 Training loss 1.6673887956853506
Epoch 75 iteration 0 loss 1.454646348953247
Epoch 75 iteration 100 loss 1.0396276712417603
Epoch 75 iteration 200 loss 1.518398404121399
Epoch 75 Training loss 1.6633919350661184
Evaluation loss 3.189181657332237
Epoch 76 iteration 0 loss 1.4616646766662598
Epoch 76 iteration 100 loss 0.9838554859161377
Epoch 76 iteration 200 loss 1.4613702297210693
Epoch 76 Training loss 1.6526747506920867
Epoch 77 iteration 0 loss 1.4646761417388916
Epoch 77 iteration 100 loss 1.0383753776550293
Epoch 77 iteration 200 loss 1.5081768035888672
Epoch 77 Training loss 1.6462943129725018
Epoch 78 iteration 0 loss 1.4008097648620605
Epoch 78 iteration 100 loss 1.0147686004638672
Epoch 78 iteration 200 loss 1.5017434358596802
Epoch 78 Training loss 1.6352284007247493
Epoch 79 iteration 0 loss 1.4189144372940063
Epoch 79 iteration 100 loss 1.0126101970672607
Epoch 79 iteration 200 loss 1.4195480346679688
Epoch 79 Training loss 1.628015456811747
Epoch 80 iteration 0 loss 1.4199804067611694
Epoch 80 iteration 100 loss 1.0256879329681396
Epoch 80 iteration 200 loss 1.4564563035964966
Epoch 80 Training loss 1.6227562783981957
Evaluation loss 3.2074876046135703
Epoch 81 iteration 0 loss 1.431972622871399
Epoch 81 iteration 100 loss 1.0110960006713867
Epoch 81 iteration 200 loss 1.4414775371551514
Epoch 81 Training loss 1.6157781071711008
Epoch 82 iteration 0 loss 1.4158073663711548
Epoch 82 iteration 100 loss 0.9702512621879578
Epoch 82 iteration 200 loss 1.4209394454956055
Epoch 82 Training loss 1.605166310639776
Epoch 83 iteration 0 loss 1.3871146440505981
Epoch 83 iteration 100 loss 1.0183656215667725
Epoch 83 iteration 200 loss 1.4292359352111816
Epoch 83 Training loss 1.5961119023327037
Epoch 84 iteration 0 loss 1.3919366598129272
Epoch 84 iteration 100 loss 0.9692129492759705
Epoch 84 iteration 200 loss 1.4092985391616821
Epoch 84 Training loss 1.5897755956223851
Epoch 85 iteration 0 loss 1.355398416519165
Epoch 85 iteration 100 loss 0.9916797280311584
Epoch 85 iteration 200 loss 1.423561453819275
Epoch 85 Training loss 1.5878568289810793
Evaluation loss 3.2138472480503295
Epoch 86 iteration 0 loss 1.351928472518921
Epoch 86 iteration 100 loss 0.9997824430465698
Epoch 86 iteration 200 loss 1.4049323797225952
Epoch 86 Training loss 1.5719682346027806
Epoch 87 iteration 0 loss 1.3508714437484741
Epoch 87 iteration 100 loss 0.9411044716835022
Epoch 87 iteration 200 loss 1.4019731283187866
Epoch 87 Training loss 1.5641802139809575
Epoch 88 iteration 0 loss 1.347946047782898
Epoch 88 iteration 100 loss 0.9493017792701721
Epoch 88 iteration 200 loss 1.3770906925201416
Epoch 88 Training loss 1.5587840858982533
Epoch 89 iteration 0 loss 1.320084571838379
Epoch 89 iteration 100 loss 0.9223963022232056
Epoch 89 iteration 200 loss 1.4065088033676147
Epoch 89 Training loss 1.5548267858027334
Epoch 90 iteration 0 loss 1.3534889221191406
Epoch 90 iteration 100 loss 0.9281108975410461
Epoch 90 iteration 200 loss 1.3821330070495605
Epoch 90 Training loss 1.5474867314671616
Evaluation loss 3.2276618163204667
Epoch 91 iteration 0 loss 1.3667511940002441
Epoch 91 iteration 100 loss 0.8797598481178284
Epoch 91 iteration 200 loss 1.3776274919509888
Epoch 91 Training loss 1.536482189982952
Epoch 92 iteration 0 loss 1.3355433940887451
Epoch 92 iteration 100 loss 0.9130176901817322
Epoch 92 iteration 200 loss 1.3042923212051392
Epoch 92 Training loss 1.5308507835779057
Epoch 93 iteration 0 loss 1.2953367233276367
Epoch 93 iteration 100 loss 0.9194003939628601
Epoch 93 iteration 200 loss 1.3469970226287842
Epoch 93 Training loss 1.519625581403501
Epoch 94 iteration 0 loss 1.322600245475769
Epoch 94 iteration 100 loss 0.9003701210021973
Epoch 94 iteration 200 loss 1.3512846231460571
Epoch 94 Training loss 1.5193673748787049
Epoch 95 iteration 0 loss 1.2789180278778076
Epoch 95 iteration 100 loss 0.9352515339851379
Epoch 95 iteration 200 loss 1.3609877824783325
Epoch 95 Training loss 1.5135782739054082
Evaluation loss 3.2474015759319284
Epoch 96 iteration 0 loss 1.3051612377166748
Epoch 96 iteration 100 loss 0.8885603547096252
Epoch 96 iteration 200 loss 1.3272497653961182
Epoch 96 Training loss 1.5079536183100883
Epoch 97 iteration 0 loss 1.2671339511871338
Epoch 97 iteration 100 loss 0.8706735968589783
Epoch 97 iteration 200 loss 1.305412769317627
Epoch 97 Training loss 1.4974833326540824
Epoch 98 iteration 0 loss 1.308292269706726
Epoch 98 iteration 100 loss 0.9079441428184509
Epoch 98 iteration 200 loss 1.2940715551376343
Epoch 98 Training loss 1.4928753682563118
Epoch 99 iteration 0 loss 1.276250958442688
Epoch 99 iteration 100 loss 0.890657901763916
Epoch 99 iteration 200 loss 1.3286609649658203
Epoch 99 Training loss 1.4852960116094391

6.翻译

def translate_dev(i):
    en_sent = " ".join([inv_en_dict[w] for w in dev_en[i]])  #原来的英文
    print(en_sent)
    cn_sent = " ".join([inv_cn_dict[w] for w in dev_cn[i]])  #原来的中文
    print("".join(cn_sent))

    mb_x = torch.from_numpy(np.array(dev_en[i]).reshape(1, -1)).long().to(device)
    mb_x_len = torch.from_numpy(np.array([len(dev_en[i])])).long().to(device)
    bos = torch.Tensor([[cn_dict["BOS"]]]).long().to(device)

    translation, attn = model.translate(mb_x, mb_x_len, bos)
    translation = [inv_cn_dict[i] for i in translation.data.cpu().numpy().reshape(-1)]
    trans = []
    for word in translation:
        if word != "EOS":
            trans.append(word)
        else:
            break
    print("".join(trans))           #翻译后的中文

for i in range(100, 120):
    translate_dev(i)
    print()

执行结果如下（样本少加上训练时间太短造成翻译效果不太好）：

BOS you have nice skin . EOS
BOS 你 的 皮 膚 真 好 。 EOS
你有很多好。

BOS you 're UNK correct . EOS
BOS 你 部 分 正 确 。 EOS
你是个好人。

BOS everyone admired his courage . EOS
BOS 每 個 人 都 佩 服 他 的 勇 氣 。 EOS
他們的電話讓他們一個

BOS what time is it ? EOS
BOS 几 点 了 ？ EOS
它还是什么？

BOS i 'm free tonight . EOS
BOS 我 今 晚 有 空 。 EOS
我今晚有空。

BOS here is your book . EOS
BOS 這 是 你 的 書 。 EOS
你的書桌是舊。

BOS they are at lunch . EOS
BOS 他 们 在 吃 午 饭 。 EOS
他们正在吃米饭。

BOS this chair is UNK . EOS
BOS 這 把 椅 子 很 UNK 。 EOS
這是真的最好的人。

BOS it 's pretty heavy . EOS
BOS 它 真 重 。 EOS
它是真的。

BOS many attended his funeral . EOS
BOS 很 多 人 都 参 加 了 他 的 葬 礼 。 EOS
AI他的襪子。

BOS training will be provided . EOS
BOS 会 有 训 练 。 EOS
人们的货品造成的。

BOS someone is watching you . EOS
BOS 有 人 在 看 著 你 。 EOS
有人叫醒汤姆。

BOS i slapped his face . EOS
BOS 我 摑 了 他 的 臉 。 EOS
我有他的兄弟。

BOS i like UNK music . EOS
BOS 我 喜 歡 流 行 音 樂 。 EOS
我喜欢狗在家。

BOS tom had no children . EOS
BOS T o m 沒 有 孩 子 。 EOS
汤姆不需要做什么。

BOS please lock the door . EOS
BOS 請 把 門 鎖 上 。 EOS
請把門打開。

BOS tom has calmed down . EOS
BOS 汤 姆 冷 静 下 来 了 。 EOS
汤姆睡着了。

BOS please speak more loudly . EOS
BOS 請 說 大 聲 一 點 兒 。 EOS
請講慢一點。

BOS keep next sunday free . EOS
BOS 把 下 周 日 空 出 来 。 EOS
下午可以轉下。

BOS i made a mistake . EOS
BOS 我 犯 了 一 個 錯 。 EOS
我有些意生。

7.Encoder Decoder模型（含Attention版本）

7.1Encoder

Encoder模型的任务是把输入文字传入embedding层和GRU层，转换成一些hidden states作为后续的context vectors；

class Encoder(nn.Module):
    def __init__(self, vocab_size, embed_size, enc_hidden_size, dec_hidden_size, dropout=0.2):
        super(Encoder, self).__init__()
        self.embed = nn.Embedding(vocab_size, embed_size)
        self.rnn = nn.GRU(embed_size, enc_hidden_size, batch_first=True, bidirectional=True)
        self.dropout = nn.Dropout(dropout)
        self.fc = nn.Linear(enc_hidden_size * 2, dec_hidden_size)

    def forward(self, x, lengths):
        sorted_len, sorted_idx = lengths.sort(0, descending=True)
        x_sorted = x[sorted_idx.long()]
        embedded = self.dropout(self.embed(x_sorted))
        
        packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, sorted_len.long().cpu().data.numpy(), batch_first=True)
        packed_out, hid = self.rnn(packed_embedded)
        out, _ = nn.utils.rnn.pad_packed_sequence(packed_out, batch_first=True)
        _, original_idx = sorted_idx.sort(0, descending=False)
        out = out[original_idx.long()].contiguous()
        hid = hid[:, original_idx.long()].contiguous()
        
        hid = torch.cat([hid[-2], hid[-1]], dim=1)          #双向，所以拼接
        hid = torch.tanh(self.fc(hid)).unsqueeze(0) 

        return out, hid

7.2Luong Attention

图中h_t是第t个step下GRU的输出，即output，$overline{^{h_{s}}}$是encoder后的context，用QKV模型来解释的话，query就是h_t，key和value都是$overline{^{h_{s}}}$；

根据context vectors和当前的输出hidden states，计算输出；

class Attention(nn.Module):
    def __init__(self, enc_hidden_size, dec_hidden_size):
        super(Attention, self).__init__()

        self.enc_hidden_size = enc_hidden_size
        self.dec_hidden_size = dec_hidden_size

        self.linear_in = nn.Linear(enc_hidden_size*2, dec_hidden_size, bias=False)        #线性变换
        self.linear_out = nn.Linear(enc_hidden_size*2 + dec_hidden_size, dec_hidden_size)

    def forward(self, output, context, mask):
        # output: [batch_size, output_len, dec_hidden_size]
        # context: [batch_size, context_len, 2*enc_hidden_size] *2的原因：GRU是双向的

        batch_size = output.size(0)
        output_len = output.size(1)
        context_len = context.size(1)

        #context_in: [batch_size, context_len, dec_hidden_size]
        context_in = self.linear_in(context.view(batch_size*context_len, -1)).view(batch_size, context_len, -1)

        # context_in.transpose(1,2): [batch_size, dec_hidden_size, context_len]
        attn = torch.bmm(output, context_in.transpose(1,2))       #attn: [batch_size, output_len, context_len]

        attn.data.masked_fill(mask, -1e6)

        attn = F.softmax(attn, dim=2)                             #attn: [batch_size, output_len, context_len]

        context = torch.bmm(attn, context)                        #context: [batch_size, output_len, enc_hidden_size]

        output = torch.cat((context, output), dim=2)              #output: [batch_size, output_len, hidden_size*2]

        output = output.view(batch_size*output_len, -1)
        output = torch.tanh(self.linear_out(output))
        output = output.view(batch_size, output_len, -1)

        return output, attn

7.3Decoder

 Decoder会根据已经翻译的句子内容和context vectors，来决定下一个输出的单词；

class Decoder(nn.Module):
    def __init__(self, vocab_size, embed_size, enc_hidden_size, dec_hidden_size, dropout=0.2):
        super(Decoder, self).__init__()
        self.embed = nn.Embedding(vocab_size, embed_size)
        self.attention = Attention(enc_hidden_size, dec_hidden_size)
        self.rnn = nn.GRU(embed_size, hidden_size, batch_first=True)
        self.out = nn.Linear(dec_hidden_size, vocab_size)
        self.dropout = nn.Dropout(dropout)

    def create_mask(self, x_len, y_len):  # a mask of shape x_len * y_len
        
        device = x_len.device
        max_x_len = x_len.max()
        max_y_len = y_len.max()
        x_mask = torch.arange(max_x_len, device=x_len.device)[None, :] < x_len[:, None]
        y_mask = torch.arange(max_y_len, device=x_len.device)[None, :] < y_len[:, None]
        mask = (1 - x_mask[:, :, None] * y_mask[:, None, :]).byte()
        return mask
    
    def forward(self, ctx, ctx_lengths, y, y_lengths, hid):
        sorted_len, sorted_idx = y_lengths.sort(0, descending=True)
        y_sorted = y[sorted_idx.long()]
        hid = hid[:, sorted_idx.long()]
        
        y_sorted = self.dropout(self.embed(y_sorted)) # batch_size, output_length, embed_size

        packed_seq = nn.utils.rnn.pack_padded_sequence(y_sorted, sorted_len.long().cpu().data.numpy(), batch_first=True)
        out, hid = self.rnn(packed_seq, hid)
        unpacked, _ = nn.utils.rnn.pad_packed_sequence(out, batch_first=True)
        _, original_idx = sorted_idx.sort(0, descending=False)
        output_seq = unpacked[original_idx.long()].contiguous()
        hid = hid[:, original_idx.long()].contiguous()

        mask = self.create_mask(y_lengths, ctx_lengths)

        output, attn = self.attention(output_seq, ctx, mask)  #根据原来的output_seq和context来计算
        output = F.log_softmax(self.out(output), -1)
        
        return output, hid, attn

7.4构建Seq2Seq模型把encoder, attention, decoder串到一起

class Seq2Seq(nn.Module):
    def __init__(self, encoder, decoder):
        super(Seq2Seq, self).__init__()
        self.encoder = encoder
        self.decoder = decoder
        
    def forward(self, x, x_lengths, y, y_lengths):
        encoder_out, hid = self.encoder(x, x_lengths)
        output, hid, attn = self.decoder(ctx=encoder_out, 
                    ctx_lengths=x_lengths,
                    y=y,
                    y_lengths=y_lengths,
                    hid=hid)
        return output, attn
    
    def translate(self, x, x_lengths, y, max_length=100):
        encoder_out, hid = self.encoder(x, x_lengths)
        preds = []
        batch_size = x.shape[0]
        attns = []
        for i in range(max_length):
            output, hid, attn = self.decoder(ctx=encoder_out, 
                    ctx_lengths=x_lengths,
                    y=y,
                    y_lengths=torch.ones(batch_size).long().to(y.device),
                    hid=hid)
            y = output.max(2)[1].view(batch_size, 1)
            preds.append(y)
            attns.append(attn)
        return torch.cat(preds, 1), torch.cat(attns, 1)

7.5训练函数并调用上面的train函数

dropout = 0.2
embed_size = hidden_size = 100
encoder = Encoder(vocab_size=en_total_words, embed_size=embed_size, enc_hidden_size=hidden_size, dec_hidden_size=hidden_size, dropout=dropout)
decoder = Decoder(vocab_size=cn_total_words, embed_size=embed_size, enc_hidden_size=hidden_size, dec_hidden_size=hidden_size, dropout=dropout)
model = Seq2Seq(encoder, decoder)
model = model.to(device)
loss_fn = LanguageModelCriterion().to(device)
optimizer = torch.optim.Adam(model.parameters())

train(model, train_data, num_epochs=100)

训练结果：

Epoch 0 iteration 0 loss 1.3026132583618164
Epoch 0 iteration 100 loss 0.8847191333770752
Epoch 0 iteration 200 loss 1.285671353340149
Epoch 0 Training loss 1.4803871257447
Evaluation loss 3.260634059314127
Epoch 1 iteration 0 loss 1.2434465885162354
Epoch 1 iteration 100 loss 0.8472312092781067
Epoch 1 iteration 200 loss 1.282746434211731
Epoch 1 Training loss 1.4731217075462495
Epoch 2 iteration 0 loss 1.2593930959701538
Epoch 2 iteration 100 loss 0.8484001159667969
Epoch 2 iteration 200 loss 1.2862968444824219
Epoch 2 Training loss 1.466728115795638
Epoch 3 iteration 0 loss 1.2554501295089722
Epoch 3 iteration 100 loss 0.9115875363349915
Epoch 3 iteration 200 loss 1.2563236951828003
Epoch 3 Training loss 1.4607854827943827
Epoch 4 iteration 0 loss 1.217956304550171
Epoch 4 iteration 100 loss 0.8641748428344727
Epoch 4 iteration 200 loss 1.2998305559158325
Epoch 4 Training loss 1.4587181747145395
Epoch 5 iteration 0 loss 1.258739709854126
Epoch 5 iteration 100 loss 0.8705984354019165
Epoch 5 iteration 200 loss 1.2102816104888916
Epoch 5 Training loss 1.4507371513452623
Evaluation loss 3.266629208261664
Epoch 6 iteration 0 loss 1.259811282157898
Epoch 6 iteration 100 loss 0.8492067456245422
Epoch 6 iteration 200 loss 1.3064922094345093
Epoch 6 Training loss 1.4432446458560053
Epoch 7 iteration 0 loss 1.2411160469055176
Epoch 7 iteration 100 loss 0.8373231291770935
Epoch 7 iteration 200 loss 1.2500189542770386
Epoch 7 Training loss 1.436364381060567
Epoch 8 iteration 0 loss 1.1868956089019775
Epoch 8 iteration 100 loss 0.814584493637085
Epoch 8 iteration 200 loss 1.2773609161376953
Epoch 8 Training loss 1.4354508132900903
Epoch 9 iteration 0 loss 1.2234464883804321
Epoch 9 iteration 100 loss 0.797888457775116
Epoch 9 iteration 200 loss 1.2435855865478516
Epoch 9 Training loss 1.424914875345232
Epoch 10 iteration 0 loss 1.2067270278930664
Epoch 10 iteration 100 loss 0.8425077795982361
Epoch 10 iteration 200 loss 1.2325958013534546
Epoch 10 Training loss 1.4212906077384722
Evaluation loss 3.2876189393276327
Epoch 11 iteration 0 loss 1.221406102180481
Epoch 11 iteration 100 loss 0.80806964635849
Epoch 11 iteration 200 loss 1.3028448820114136
Epoch 11 Training loss 1.4154276829998698
Epoch 12 iteration 0 loss 1.1890984773635864
Epoch 12 iteration 100 loss 0.827181875705719
Epoch 12 iteration 200 loss 1.1675362586975098
Epoch 12 Training loss 1.4132606964483012
Epoch 13 iteration 0 loss 1.2002121210098267
Epoch 13 iteration 100 loss 0.8232781291007996
Epoch 13 iteration 200 loss 1.2605061531066895
Epoch 13 Training loss 1.407515715564216
Epoch 14 iteration 0 loss 1.1855664253234863
Epoch 14 iteration 100 loss 0.8178666234016418
Epoch 14 iteration 200 loss 1.2378345727920532
Epoch 14 Training loss 1.3966619677770713
Epoch 15 iteration 0 loss 1.1885008811950684
Epoch 15 iteration 100 loss 0.7523401975631714
Epoch 15 iteration 200 loss 1.1757400035858154
Epoch 15 Training loss 1.3940533722612007
Evaluation loss 3.3011410674061716
Epoch 16 iteration 0 loss 1.185882806777954
Epoch 16 iteration 100 loss 0.8129084706306458
Epoch 16 iteration 200 loss 1.2022055387496948
Epoch 16 Training loss 1.3908352825348185
Epoch 17 iteration 0 loss 1.145820140838623
Epoch 17 iteration 100 loss 0.7933529615402222
Epoch 17 iteration 200 loss 1.1954973936080933
Epoch 17 Training loss 1.3862186002415022
Epoch 18 iteration 0 loss 1.1626101732254028
Epoch 18 iteration 100 loss 0.8041335940361023
Epoch 18 iteration 200 loss 1.1879560947418213
Epoch 18 Training loss 1.3828558699833502
Epoch 19 iteration 0 loss 1.1661605834960938
Epoch 19 iteration 100 loss 0.7746578454971313
Epoch 19 iteration 200 loss 1.167975902557373
Epoch 19 Training loss 1.3737090146397222
Epoch 20 iteration 0 loss 1.1992604732513428
Epoch 20 iteration 100 loss 0.7750277519226074
Epoch 20 iteration 200 loss 1.1533249616622925
Epoch 20 Training loss 1.3699699581049805
Evaluation loss 3.316624780553762
Epoch 21 iteration 0 loss 1.182730793952942
Epoch 21 iteration 100 loss 0.7664387822151184
Epoch 21 iteration 200 loss 1.1734970808029175
Epoch 21 Training loss 1.3634166858854262
Epoch 22 iteration 0 loss 1.1587318181991577
Epoch 22 iteration 100 loss 0.7660608291625977
Epoch 22 iteration 200 loss 1.1832681894302368
Epoch 22 Training loss 1.3601878647219552
Epoch 23 iteration 0 loss 1.123557209968567
Epoch 23 iteration 100 loss 0.7884796857833862
Epoch 23 iteration 200 loss 1.131569266319275
Epoch 23 Training loss 1.3543664767232568
Epoch 24 iteration 0 loss 1.1566004753112793
Epoch 24 iteration 100 loss 0.7894638180732727
Epoch 24 iteration 200 loss 1.1293442249298096
Epoch 24 Training loss 1.3513351050205646
Epoch 25 iteration 0 loss 1.1237646341323853
Epoch 25 iteration 100 loss 0.7442751526832581
Epoch 25 iteration 200 loss 1.1396199464797974
Epoch 25 Training loss 1.3436930389138495
Evaluation loss 3.331576243054354
Epoch 26 iteration 0 loss 1.1391510963439941
Epoch 26 iteration 100 loss 0.7658866047859192
Epoch 26 iteration 200 loss 1.130005121231079
Epoch 26 Training loss 1.3387227258896204
Epoch 27 iteration 0 loss 1.086417555809021
Epoch 27 iteration 100 loss 0.7512990236282349
Epoch 27 iteration 200 loss 1.1055928468704224
Epoch 27 Training loss 1.3332018813254016
Epoch 28 iteration 0 loss 1.1308163404464722
Epoch 28 iteration 100 loss 0.7653459310531616
Epoch 28 iteration 200 loss 1.1437530517578125
Epoch 28 Training loss 1.3318316266073582
Epoch 29 iteration 0 loss 1.1284910440444946
Epoch 29 iteration 100 loss 0.7385256886482239
Epoch 29 iteration 200 loss 1.076254963874817
Epoch 29 Training loss 1.327704983812702
Epoch 30 iteration 0 loss 1.1279666423797607
Epoch 30 iteration 100 loss 0.7510428428649902
Epoch 30 iteration 200 loss 1.10474693775177
Epoch 30 Training loss 1.3247037412152105
Evaluation loss 3.345638094832775
Epoch 31 iteration 0 loss 1.1144018173217773
Epoch 31 iteration 100 loss 0.7183322906494141
Epoch 31 iteration 200 loss 1.1657849550247192
Epoch 31 Training loss 1.3181022928511037
Epoch 32 iteration 0 loss 1.1624877452850342
Epoch 32 iteration 100 loss 0.6971022486686707
Epoch 32 iteration 200 loss 1.1033793687820435
Epoch 32 Training loss 1.313637083400949
Epoch 33 iteration 0 loss 1.0961930751800537
Epoch 33 iteration 100 loss 0.7509954571723938
Epoch 33 iteration 200 loss 1.0901885032653809
Epoch 33 Training loss 1.3105013603183797
Epoch 34 iteration 0 loss 1.0936028957366943
Epoch 34 iteration 100 loss 0.7300226092338562
Epoch 34 iteration 200 loss 1.094140648841858
Epoch 34 Training loss 1.3085180236466905
Epoch 35 iteration 0 loss 1.1358038187026978
Epoch 35 iteration 100 loss 0.6928472518920898
Epoch 35 iteration 200 loss 1.1031907796859741
Epoch 35 Training loss 1.2983291715098229
Evaluation loss 3.3654267819449917
Epoch 36 iteration 0 loss 1.0817443132400513
Epoch 36 iteration 100 loss 0.7034777998924255
Epoch 36 iteration 200 loss 1.1244701147079468
Epoch 36 Training loss 1.294685624884497
Epoch 37 iteration 0 loss 1.0067986249923706
Epoch 37 iteration 100 loss 0.6711763739585876
Epoch 37 iteration 200 loss 1.0877138376235962
Epoch 37 Training loss 1.2908666166705178
Epoch 38 iteration 0 loss 1.0796058177947998
Epoch 38 iteration 100 loss 0.6984289288520813
Epoch 38 iteration 200 loss 1.0992212295532227
Epoch 38 Training loss 1.289693898836594
Epoch 39 iteration 0 loss 1.1193760633468628
Epoch 39 iteration 100 loss 0.7441080212593079
Epoch 39 iteration 200 loss 1.0557031631469727
Epoch 39 Training loss 1.287817969907393
Epoch 40 iteration 0 loss 1.0878312587738037
Epoch 40 iteration 100 loss 0.7390894889831543
Epoch 40 iteration 200 loss 1.0931909084320068
Epoch 40 Training loss 1.281862247889987
Evaluation loss 3.3775776429435034
Epoch 41 iteration 0 loss 1.1135987043380737
Epoch 41 iteration 100 loss 0.6786257028579712
Epoch 41 iteration 200 loss 1.056801676750183
Epoch 41 Training loss 1.2759448500226234
Epoch 42 iteration 0 loss 1.0649049282073975
Epoch 42 iteration 100 loss 0.6774815320968628
Epoch 42 iteration 200 loss 1.0807018280029297
Epoch 42 Training loss 1.2713608683004023
Epoch 43 iteration 0 loss 1.0711919069290161
Epoch 43 iteration 100 loss 0.6655244827270508
Epoch 43 iteration 200 loss 1.0616692304611206
Epoch 43 Training loss 1.2693709204800718
Epoch 44 iteration 0 loss 1.0423146486282349
Epoch 44 iteration 100 loss 0.7055337429046631
Epoch 44 iteration 200 loss 1.0746649503707886
Epoch 44 Training loss 1.2649514066760854
Epoch 45 iteration 0 loss 1.0937353372573853
Epoch 45 iteration 100 loss 0.6939021348953247
Epoch 45 iteration 200 loss 1.1060905456542969
Epoch 45 Training loss 1.2591645727085945
Evaluation loss 3.393269126438251
Epoch 46 iteration 0 loss 1.1005926132202148
Epoch 46 iteration 100 loss 0.6948174238204956
Epoch 46 iteration 200 loss 1.0675958395004272
Epoch 46 Training loss 1.2555609077507983
Epoch 47 iteration 0 loss 1.0566778182983398
Epoch 47 iteration 100 loss 0.6904436349868774
Epoch 47 iteration 200 loss 1.0723766088485718
Epoch 47 Training loss 1.2552127211907091
Epoch 48 iteration 0 loss 1.0497757196426392
Epoch 48 iteration 100 loss 0.6351101398468018
Epoch 48 iteration 200 loss 1.0661102533340454
Epoch 48 Training loss 1.2479233140629313
Epoch 49 iteration 0 loss 1.0470858812332153
Epoch 49 iteration 100 loss 0.6707669496536255
Epoch 49 iteration 200 loss 1.063056230545044
Epoch 49 Training loss 1.2453716254928995
Epoch 50 iteration 0 loss 1.0854698419570923
Epoch 50 iteration 100 loss 0.6165581345558167
Epoch 50 iteration 200 loss 1.0804699659347534
Epoch 50 Training loss 1.243395479327141
Evaluation loss 3.4102849531750494
Epoch 51 iteration 0 loss 1.0279858112335205
Epoch 51 iteration 100 loss 0.6448107957839966
Epoch 51 iteration 200 loss 1.0390673875808716
Epoch 51 Training loss 1.2358123475125082
Epoch 52 iteration 0 loss 1.0429105758666992
Epoch 52 iteration 100 loss 0.7124451994895935
Epoch 52 iteration 200 loss 1.061672329902649
Epoch 52 Training loss 1.233326693902576
Epoch 53 iteration 0 loss 1.0357102155685425
Epoch 53 iteration 100 loss 0.6381393074989319
Epoch 53 iteration 200 loss 1.0036036968231201
Epoch 53 Training loss 1.2297246983027847
Epoch 54 iteration 0 loss 1.0590764284133911
Epoch 54 iteration 100 loss 0.6603832840919495
Epoch 54 iteration 200 loss 1.0215944051742554
Epoch 54 Training loss 1.227340017322883
Epoch 55 iteration 0 loss 1.0460106134414673
Epoch 55 iteration 100 loss 0.67122882604599
Epoch 55 iteration 200 loss 1.0344772338867188
Epoch 55 Training loss 1.2244369935263697
Evaluation loss 3.4193579365732183
Epoch 56 iteration 0 loss 1.032409429550171
Epoch 56 iteration 100 loss 0.6183319091796875
Epoch 56 iteration 200 loss 0.9782896637916565
Epoch 56 Training loss 1.2178285635372972
Epoch 57 iteration 0 loss 1.0382548570632935
Epoch 57 iteration 100 loss 0.6902874708175659
Epoch 57 iteration 200 loss 1.016508936882019
Epoch 57 Training loss 1.214647328633978
Epoch 58 iteration 0 loss 1.0595533847808838
Epoch 58 iteration 100 loss 0.6885846853256226
Epoch 58 iteration 200 loss 1.0221766233444214
Epoch 58 Training loss 1.2100419675457097
Epoch 59 iteration 0 loss 1.014621615409851
Epoch 59 iteration 100 loss 0.602800190448761
Epoch 59 iteration 200 loss 1.037442684173584
Epoch 59 Training loss 1.2131489746903632
Epoch 60 iteration 0 loss 1.0217640399932861
Epoch 60 iteration 100 loss 0.6246439814567566
Epoch 60 iteration 200 loss 1.00297212600708
Epoch 60 Training loss 1.204652290841725
Evaluation loss 3.4292290158399075
Epoch 61 iteration 0 loss 0.9992070198059082
Epoch 61 iteration 100 loss 0.645142138004303
Epoch 61 iteration 200 loss 0.9961024522781372
Epoch 61 Training loss 1.2066241352823563
Epoch 62 iteration 0 loss 0.9980950951576233
Epoch 62 iteration 100 loss 0.6504135131835938
Epoch 62 iteration 200 loss 1.000308632850647
Epoch 62 Training loss 1.1984729866171178
Epoch 63 iteration 0 loss 0.9869410395622253
Epoch 63 iteration 100 loss 0.6618863344192505
Epoch 63 iteration 200 loss 0.981200635433197
Epoch 63 Training loss 1.1967191445048035
Epoch 64 iteration 0 loss 0.9695953130722046
Epoch 64 iteration 100 loss 0.6359274387359619
Epoch 64 iteration 200 loss 0.9904515743255615
Epoch 64 Training loss 1.194521029171779
Epoch 65 iteration 0 loss 0.9505796432495117
Epoch 65 iteration 100 loss 0.6068794131278992
Epoch 65 iteration 200 loss 0.980348527431488
Epoch 65 Training loss 1.189270519480765
Evaluation loss 3.454153442993674
Epoch 66 iteration 0 loss 1.0304545164108276
Epoch 66 iteration 100 loss 0.6792566776275635
Epoch 66 iteration 200 loss 0.9789241552352905
Epoch 66 Training loss 1.1869953296382767
Epoch 67 iteration 0 loss 0.957666277885437
Epoch 67 iteration 100 loss 0.584879994392395
Epoch 67 iteration 200 loss 1.0174148082733154
Epoch 67 Training loss 1.184179561090835
Epoch 68 iteration 0 loss 1.043166995048523
Epoch 68 iteration 100 loss 0.6168758869171143
Epoch 68 iteration 200 loss 1.0030053853988647
Epoch 68 Training loss 1.1824355462851552
Epoch 69 iteration 0 loss 1.0165300369262695
Epoch 69 iteration 100 loss 0.6542645692825317
Epoch 69 iteration 200 loss 1.0191236734390259
Epoch 69 Training loss 1.176021675397731
Epoch 70 iteration 0 loss 0.9590736031532288
Epoch 70 iteration 100 loss 0.6157773733139038
Epoch 70 iteration 200 loss 1.0451829433441162
Epoch 70 Training loss 1.1732503092442255
Evaluation loss 3.4715566423642277
Epoch 71 iteration 0 loss 0.971733570098877
Epoch 71 iteration 100 loss 0.5589802265167236
Epoch 71 iteration 200 loss 1.0018212795257568
Epoch 71 Training loss 1.1694891346833023
Epoch 72 iteration 0 loss 1.0042874813079834
Epoch 72 iteration 100 loss 0.6543828248977661
Epoch 72 iteration 200 loss 0.968835175037384
Epoch 72 Training loss 1.1667191714442264
Epoch 73 iteration 0 loss 0.9512341022491455
Epoch 73 iteration 100 loss 0.5809782147407532
Epoch 73 iteration 200 loss 0.9460022449493408
Epoch 73 Training loss 1.165780372824424
Epoch 74 iteration 0 loss 0.9838390946388245
Epoch 74 iteration 100 loss 0.6115572452545166
Epoch 74 iteration 200 loss 0.9821975827217102
Epoch 74 Training loss 1.1619031632185661
Epoch 75 iteration 0 loss 0.9615085124969482
Epoch 75 iteration 100 loss 0.5715279579162598
Epoch 75 iteration 200 loss 0.9673617482185364
Epoch 75 Training loss 1.1592393025041507
Evaluation loss 3.4810480487503015
Epoch 76 iteration 0 loss 0.9920525550842285
Epoch 76 iteration 100 loss 0.6243174076080322
Epoch 76 iteration 200 loss 0.9598985910415649
Epoch 76 Training loss 1.1506768550866349
Epoch 77 iteration 0 loss 0.9717826843261719
Epoch 77 iteration 100 loss 0.5903583765029907
Epoch 77 iteration 200 loss 0.9472079873085022
Epoch 77 Training loss 1.151601228059984
Epoch 78 iteration 0 loss 0.9331899881362915
Epoch 78 iteration 100 loss 0.6189018487930298
Epoch 78 iteration 200 loss 0.9951513409614563
Epoch 78 Training loss 1.1474281610158772
Epoch 79 iteration 0 loss 0.9012037515640259
Epoch 79 iteration 100 loss 0.5837778449058533
Epoch 79 iteration 200 loss 0.9066386818885803
Epoch 79 Training loss 1.142656700489289
Epoch 80 iteration 0 loss 0.9931736588478088
Epoch 80 iteration 100 loss 0.5927265882492065
Epoch 80 iteration 200 loss 0.938447892665863
Epoch 80 Training loss 1.1434640075302192
Evaluation loss 3.491394963812294
Epoch 81 iteration 0 loss 0.9227023720741272
Epoch 81 iteration 100 loss 0.5467157363891602
Epoch 81 iteration 200 loss 0.9126712083816528
Epoch 81 Training loss 1.1427882346320761
Epoch 82 iteration 0 loss 0.9733406901359558
Epoch 82 iteration 100 loss 0.564643144607544
Epoch 82 iteration 200 loss 0.9918593764305115
Epoch 82 Training loss 1.1362837826371996
Epoch 83 iteration 0 loss 0.9489978551864624
Epoch 83 iteration 100 loss 0.5791521668434143
Epoch 83 iteration 200 loss 0.9270768165588379
Epoch 83 Training loss 1.136011173156159
Epoch 84 iteration 0 loss 0.9410436749458313
Epoch 84 iteration 100 loss 0.5409624576568604
Epoch 84 iteration 200 loss 0.8918321132659912
Epoch 84 Training loss 1.128506034776273
Epoch 85 iteration 0 loss 0.9554007053375244
Epoch 85 iteration 100 loss 0.571331799030304
Epoch 85 iteration 200 loss 0.9672144055366516
Epoch 85 Training loss 1.1277133646535586
Evaluation loss 3.5075310684848158
Epoch 86 iteration 0 loss 0.9104467630386353
Epoch 86 iteration 100 loss 0.5656437277793884
Epoch 86 iteration 200 loss 0.9324206113815308
Epoch 86 Training loss 1.126375005188875
Epoch 87 iteration 0 loss 0.9339620471000671
Epoch 87 iteration 100 loss 0.5636867880821228
Epoch 87 iteration 200 loss 0.8825109601020813
Epoch 87 Training loss 1.1222316938253494
Epoch 88 iteration 0 loss 0.904504120349884
Epoch 88 iteration 100 loss 0.5706378221511841
Epoch 88 iteration 200 loss 0.9415532350540161
Epoch 88 Training loss 1.1215731092872845
Epoch 89 iteration 0 loss 0.9489354491233826
Epoch 89 iteration 100 loss 0.6389216184616089
Epoch 89 iteration 200 loss 0.8783397078514099
Epoch 89 Training loss 1.1199876689692458
Epoch 90 iteration 0 loss 0.909376323223114
Epoch 90 iteration 100 loss 0.6190019249916077
Epoch 90 iteration 200 loss 0.9191233515739441
Epoch 90 Training loss 1.1181392741798546
Evaluation loss 3.508678977926201
Epoch 91 iteration 0 loss 0.9080389738082886
Epoch 91 iteration 100 loss 0.5580074191093445
Epoch 91 iteration 200 loss 0.9494779706001282
Epoch 91 Training loss 1.1147855064570311
Epoch 92 iteration 0 loss 0.900802731513977
Epoch 92 iteration 100 loss 0.573580801486969
Epoch 92 iteration 200 loss 0.9199456572532654
Epoch 92 Training loss 1.1107969536786537
Epoch 93 iteration 0 loss 0.9345868229866028
Epoch 93 iteration 100 loss 0.5590959787368774
Epoch 93 iteration 200 loss 0.90354984998703
Epoch 93 Training loss 1.105984925602608
Epoch 94 iteration 0 loss 0.9008861780166626
Epoch 94 iteration 100 loss 0.5503742098808289
Epoch 94 iteration 200 loss 0.8791723251342773
Epoch 94 Training loss 1.1053885063813342
Epoch 95 iteration 0 loss 0.899246096611023
Epoch 95 iteration 100 loss 0.6236768364906311
Epoch 95 iteration 200 loss 0.8661567568778992
Epoch 95 Training loss 1.0993307278503
Evaluation loss 3.5332032706941585
Epoch 96 iteration 0 loss 0.8837733864784241
Epoch 96 iteration 100 loss 0.5473974943161011
Epoch 96 iteration 200 loss 0.9025910496711731
Epoch 96 Training loss 1.0998253373283113
Epoch 97 iteration 0 loss 0.922965407371521
Epoch 97 iteration 100 loss 0.5556969046592712
Epoch 97 iteration 200 loss 0.9027858972549438
Epoch 97 Training loss 1.096842199480861
Epoch 98 iteration 0 loss 0.8947715759277344
Epoch 98 iteration 100 loss 0.5312948822975159
Epoch 98 iteration 200 loss 0.9379984736442566
Epoch 98 Training loss 1.0949072217540066
Epoch 99 iteration 0 loss 0.8829227685928345
Epoch 99 iteration 100 loss 0.5451477766036987
Epoch 99 iteration 200 loss 0.8783729672431946
Epoch 99 Training loss 1.092216717956385

7.6调用上面的translate_dev函数

for i in range(100,120):
    translate_dev(i)
    print()

执行结果如下：

BOS you have nice skin . EOS
BOS 你 的 皮 膚 真 好 。 EOS
你有足球的食物都好了

BOS you 're UNK correct . EOS
BOS 你 部 分 正 确 。 EOS
你是个好厨师。

BOS everyone admired his courage . EOS
BOS 每 個 人 都 佩 服 他 的 勇 氣 。 EOS
他們每個人都很好奇。

BOS what time is it ? EOS
BOS 几 点 了 ？ EOS
它是什麼？

BOS i 'm free tonight . EOS
BOS 我 今 晚 有 空 。 EOS
我今晚沒有空。

BOS here is your book . EOS
BOS 這 是 你 的 書 。 EOS
你這附書是讀書。

BOS they are at lunch . EOS
BOS 他 们 在 吃 午 饭 。 EOS
他们在吃米饭。

BOS this chair is UNK . EOS
BOS 這 把 椅 子 很 UNK 。 EOS
這是真的最好的。

BOS it 's pretty heavy . EOS
BOS 它 真 重 。 EOS
它很有。

BOS many attended his funeral . EOS
BOS 很 多 人 都 参 加 了 他 的 葬 礼 。 EOS
仔细把他的罪恶着。

BOS training will be provided . EOS
BOS 会 有 训 练 。 EOS
克林變得餓了。

BOS someone is watching you . EOS
BOS 有 人 在 看 著 你 。 EOS
有人在信封信。

BOS i slapped his face . EOS
BOS 我 摑 了 他 的 臉 。 EOS
我是他的兄弟。

BOS i like UNK music . EOS
BOS 我 喜 歡 流 行 音 樂 。 EOS
我喜歡音樂。

BOS tom had no children . EOS
BOS T o m 沒 有 孩 子 。 EOS
Tom沒有太累。

BOS please lock the door . EOS
BOS 請 把 門 鎖 上 。 EOS
請關門。

BOS tom has calmed down . EOS
BOS 汤 姆 冷 静 下 来 了 。 EOS
汤姆向伤极了。

BOS please speak more loudly . EOS
BOS 請 說 大 聲 一 點 兒 。 EOS
請講話。

BOS keep next sunday free . EOS
BOS 把 下 周 日 空 出 来 。 EOS
下周举一直流出席。

BOS i made a mistake . EOS
BOS 我 犯 了 一 個 錯 。 EOS
我有一个梦意。

翻译结果依然一般。