Transformer 完整逐步手算
「我爱水课」→「I love easy courses」· d=4, h=2, d_k=d_v=2, N_enc=N_dec=2 · 全程编号不跳步 · 训练 / 推理双轨
🟢 Encoder（训练=推理）
🟠 Decoder-训练
🌸 Decoder-推理
🟡 对比
1 1.1 建立词表 Vocab（V=8） 输入准备
🎯 架构功能讲解Tokenization 是架构图的最底层输入：把人类文字映射为离散整数 ID，Transformer 才能处理。真实系统用 BPE 子词，本例按要求每字/每词一个 token。
ID 0 → 我
ID 1 → 爱
ID 2 → 水
ID 3 → 课
ID 4 → I
ID 5 → love
ID 6 → easy
ID 7 → courses
2 1.2 源序列 token 化（Encoder 输入） 输入准备
🎯 架构功能讲解输入流水线：把文字 token 变成模型可处理的向量（E+PE）。训练与推理在此阶段完全一致，无任何区别。
「我」 → ID 0
「爱」 → ID 1
「水」 → ID 2
「课」 → ID 3
3 1.3 目标序列 token 化（Decoder 输入） 输入准备
🎯 架构功能讲解输入流水线：把文字 token 变成模型可处理的向量（E+PE）。训练与推理在此阶段完全一致，无任何区别。
「I」 → ID 4
「love」 → ID 5
「easy」 → ID 6
「courses」 → ID 7
4 2.1 定义嵌入矩阵 E ∈ R^(8×4) 输入准备
🎯 架构功能讲解Embedding 层把离散 ID 变成可学习的连续向量。查表 = 取 E 的第 i 行。训练中这些向量会被调整到语义相近的词彼此靠近。
行 0 = 「我」: [0.5, -0.3, 0.8, 0.1]
行 1 = 「爱」: [-0.2, 0.6, -0.1, 0.4]
行 2 = 「水」: [0.3, 0.2, 0.5, -0.4]
行 3 = 「课」: [-0.6, 0.1, 0.3, 0.7]
行 4 = 「I」: [0.4, 0.5, -0.2, 0.3]
行 5 = 「love」: [0.7, -0.1, 0.6, -0.3]
行 6 = 「easy」: [-0.5, 0.4, 0.2, 0.6]
行 7 = 「courses」: [0.1, -0.6, 0.4, 0.2]
5 2.2 查表：源序列嵌入 X_src = E[[0,1,2,3]] 输入准备
🎯 架构功能讲解输入流水线：把文字 token 变成模型可处理的向量（E+PE）。训练与推理在此阶段完全一致，无任何区别。
「我」= ID 0 → 取 E 第 0 行 = [0.5, -0.3, 0.8, 0.1]
「爱」= ID 1 → 取 E 第 1 行 = [-0.2, 0.6, -0.1, 0.4]
「水」= ID 2 → 取 E 第 2 行 = [0.3, 0.2, 0.5, -0.4]
「课」= ID 3 → 取 E 第 3 行 = [-0.6, 0.1, 0.3, 0.7]
6 2.3 查表：目标序列嵌入 X_tgt = E[[4,5,6,7]] 输入准备
🎯 架构功能讲解输入流水线：把文字 token 变成模型可处理的向量（E+PE）。训练与推理在此阶段完全一致，无任何区别。
「I」= ID 4 → 取 E 第 4 行 = [0.4, 0.5, -0.2, 0.3]
「love」= ID 5 → 取 E 第 5 行 = [0.7, -0.1, 0.6, -0.3]
「easy」= ID 6 → 取 E 第 6 行 = [-0.5, 0.4, 0.2, 0.6]
「courses」= ID 7 → 取 E 第 7 行 = [0.1, -0.6, 0.4, 0.2]
7 3.1 位置编码 PE（sin/cos 公式） 输入准备
🎯 架构功能讲解自注意力对顺序不敏感（打乱 token 结果不变），必须显式注入位置。正弦/余弦编码给每个位置独一无二的向量，再与嵌入逐元素相加。
公式: PE(pos,2i)=sin(pos/10000^(2i/d)), PE(pos,2i+1)=cos(pos/10000^(2i/d))
d=4 → i=0: 分母 10000^0=1；i=1: 分母 10000^0.5=100
8 3.2 位置 0 的编码值 输入准备
🎯 架构功能讲解输入流水线：把文字 token 变成模型可处理的向量（E+PE）。训练与推理在此阶段完全一致，无任何区别。
PE[0][0] = sin(0/1) = 0
PE[0][1] = cos(0/1) = 1
PE[0][2] = sin(0/100) = 0
PE[0][3] = cos(0/100) = 1
9 3.2 位置 1 的编码值 输入准备
🎯 架构功能讲解输入流水线：把文字 token 变成模型可处理的向量（E+PE）。训练与推理在此阶段完全一致，无任何区别。
PE[1][0] = sin(1/1) = 0.8415
PE[1][1] = cos(1/1) = 0.5403
PE[1][2] = sin(1/100) = 0.01
PE[1][3] = cos(1/100) = 1
10 3.2 位置 2 的编码值 输入准备
🎯 架构功能讲解输入流水线：把文字 token 变成模型可处理的向量（E+PE）。训练与推理在此阶段完全一致，无任何区别。
PE[2][0] = sin(2/1) = 0.9093
PE[2][1] = cos(2/1) = -0.4161
PE[2][2] = sin(2/100) = 0.02
PE[2][3] = cos(2/100) = 0.9998
11 3.2 位置 3 的编码值 输入准备
🎯 架构功能讲解输入流水线：把文字 token 变成模型可处理的向量（E+PE）。训练与推理在此阶段完全一致，无任何区别。
PE[3][0] = sin(3/1) = 0.1411
PE[3][1] = cos(3/1) = -0.99
PE[3][2] = sin(3/100) = 0.03
PE[3][3] = cos(3/100) = 0.9996
12 4.1 Encoder 输入 X_enc_in = X_src + PE（逐元素相加） 输入准备
🎯 架构功能讲解嵌入 + 位置编码相加后，每个 token 的向量同时携带语义和位置信息，作为 Encoder 第一层的输入。
X_enc_in[0][0] = 0.5 + 0 = 0.5
X_enc_in[0][1] = -0.3 + 1 = 0.7
X_enc_in[0][2] = 0.8 + 0 = 0.8
X_enc_in[0][3] = 0.1 + 1 = 1.1
X_enc_in[1][0] = -0.2 + 0.8415 = 0.6415
X_enc_in[1][1] = 0.6 + 0.5403 = 1.1403
X_enc_in[1][2] = -0.1 + 0.01 = -0.09
X_enc_in[1][3] = 0.4 + 1 = 1.4
X_enc_in[2][0] = 0.3 + 0.9093 = 1.2093
X_enc_in[2][1] = 0.2 + -0.4161 = -0.2161
X_enc_in[2][2] = 0.5 + 0.02 = 0.52
X_enc_in[2][3] = -0.4 + 0.9998 = 0.5998
X_enc_in[3][0] = -0.6 + 0.1411 = -0.4589
X_enc_in[3][1] = 0.1 + -0.99 = -0.89
X_enc_in[3][2] = 0.3 + 0.03 = 0.33
X_enc_in[3][3] = 0.7 + 0.9996 = 1.6996
13 4.2 Decoder 输入 X_dec_in = X_tgt + PE（逐元素相加） 输入准备
🎯 架构功能讲解输入流水线：把文字 token 变成模型可处理的向量（E+PE）。训练与推理在此阶段完全一致，无任何区别。
X_dec_in[0][0] = 0.4 + 0 = 0.4
X_dec_in[0][1] = 0.5 + 1 = 1.5
X_dec_in[0][2] = -0.2 + 0 = -0.2
X_dec_in[0][3] = 0.3 + 1 = 1.3
X_dec_in[1][0] = 0.7 + 0.8415 = 1.5415
X_dec_in[1][1] = -0.1 + 0.5403 = 0.4403
X_dec_in[1][2] = 0.6 + 0.01 = 0.61
X_dec_in[1][3] = -0.3 + 1 = 0.7
X_dec_in[2][0] = -0.5 + 0.9093 = 0.4093
X_dec_in[2][1] = 0.4 + -0.4161 = -0.0161
X_dec_in[2][2] = 0.2 + 0.02 = 0.22
X_dec_in[2][3] = 0.6 + 0.9998 = 1.5998
X_dec_in[3][0] = 0.1 + 0.1411 = 0.2411
X_dec_in[3][1] = -0.6 + -0.99 = -1.59
X_dec_in[3][2] = 0.4 + 0.03 = 0.43
X_dec_in[3][3] = 0.2 + 0.9996 = 1.1996
14 5.1 定义 Head1 投影权重 WQ1a/WK1a/WV1a（2×4） Encoder L1
🎯 架构功能讲解注意力第一步：每个 token 用 3 个投影矩阵生成 Q(查询)/K(键)/V(值)，各 2 维。Q=我在找什么，K=我是什么，V=我携带什么。
WQ1a 行0: [-0.3, 0.9, 0.5, 0.2]
WQ1a 行1: [-0.7, -0.7, -0.9, 0.7]
WK1a 行0: [0.2, 0.4, -1, 0.9]
WK1a 行1: [0.7, -0.6, -0.6, -0.6]
WV1a 行0: [-0.4, 0, -0.1, -0.4]
WV1a 行1: [0.2, -0.7, -0.4, -0.3]
15 5.2 定义 Head2 投影权重 WQ1b/WK1b/WV1b（2×4） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
WQ1b 行0: [-0.1, 0.6, -0.6, 0]
WQ1b 行1: [0.2, -0.9, 0.2, -0.7]
WK1b 行0: [-0.9, 0.9, 0.9, 0.6]
WK1b 行1: [-0.4, -0.8, 0.4, -0.1]
WV1b 行0: [-0.8, 0, -0.9, 0.8]
WV1b 行1: [-0.5, 0.3, -0.4, 0]
16 5.3 定义输出投影 WO1（4×4，融合两头的拼接结果） Encoder L1
🎯 架构功能讲解多头注意力的收尾：两头的输出拼接成 4 维后，经 WO1 线性变换回 d_model 维度，得到本层注意力的最终输出。
WO1 行0: [0.1, -0.6, 0.9, 0.6]
WO1 行1: [0.9, 0.8, 0.2, 0.8]
WO1 行2: [-0.8, -0.6, -0.9, -0.3]
WO1 行3: [-0.2, -0.5, 0.7, -0.3]
17 5.4 计算 Head1 Q1 = X_enc_in·WQ1aᵀ（逐元素） Encoder L1
🎯 架构功能讲解每个 token 一行：X_enc_in 的第 i 行（4 维）与 WQ1a 的第 j 列（4 维）点积，得到 Q1[i][j]。
Q1[0][0] = 0.5×-0.3 + 0.7×0.9 + 0.8×0.5 + 1.1×0.2 = 1.1
Q1[0][1] = 0.5×-0.7 + 0.7×-0.7 + 0.8×-0.9 + 1.1×0.7 = -0.79
Q1[1][0] = 0.6415×-0.3 + 1.1403×0.9 + -0.09×0.5 + 1.4×0.2 = 1.0688
Q1[1][1] = 0.6415×-0.7 + 1.1403×-0.7 + -0.09×-0.9 + 1.4×0.7 = -0.1863
Q1[2][0] = 1.2093×-0.3 + -0.2161×0.9 + 0.52×0.5 + 0.5998×0.2 = -0.1774
Q1[2][1] = 1.2093×-0.7 + -0.2161×-0.7 + 0.52×-0.9 + 0.5998×0.7 = -0.7433
Q1[3][0] = -0.4589×-0.3 + -0.89×0.9 + 0.33×0.5 + 1.6996×0.2 = -0.1584
Q1[3][1] = -0.4589×-0.7 + -0.89×-0.7 + 0.33×-0.9 + 1.6996×0.7 = 1.8369
18 5.5 计算 Head1 K1 = X_enc_in·WK1aᵀ（逐元素） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
K1[0][0] = 0.5×0.2 + 0.7×0.4 + 0.8×-1 + 1.1×0.9 = 0.57
K1[0][1] = 0.5×0.7 + 0.7×-0.6 + 0.8×-0.6 + 1.1×-0.6 = -1.21
K1[1][0] = 0.6415×0.2 + 1.1403×0.4 + -0.09×-1 + 1.4×0.9 = 1.9344
K1[1][1] = 0.6415×0.7 + 1.1403×-0.6 + -0.09×-0.6 + 1.4×-0.6 = -1.0211
K1[2][0] = 1.2093×0.2 + -0.2161×0.4 + 0.52×-1 + 0.5998×0.9 = 0.1752
K1[2][1] = 1.2093×0.7 + -0.2161×-0.6 + 0.52×-0.6 + 0.5998×-0.6 = 0.3043
K1[3][0] = -0.4589×0.2 + -0.89×0.4 + 0.33×-1 + 1.6996×0.9 = 0.7518
K1[3][1] = -0.4589×0.7 + -0.89×-0.6 + 0.33×-0.6 + 1.6996×-0.6 = -1.0049
19 5.6 计算 Head1 V1 = X_enc_in·WV1aᵀ（逐元素） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
V1[0][0] = 0.5×-0.4 + 0.7×0 + 0.8×-0.1 + 1.1×-0.4 = -0.72
V1[0][1] = 0.5×0.2 + 0.7×-0.7 + 0.8×-0.4 + 1.1×-0.3 = -1.04
V1[1][0] = 0.6415×-0.4 + 1.1403×0 + -0.09×-0.1 + 1.4×-0.4 = -0.8076
V1[1][1] = 0.6415×0.2 + 1.1403×-0.7 + -0.09×-0.4 + 1.4×-0.3 = -1.0539
V1[2][0] = 1.2093×-0.4 + -0.2161×0 + 0.52×-0.1 + 0.5998×-0.4 = -0.7756
V1[2][1] = 1.2093×0.2 + -0.2161×-0.7 + 0.52×-0.4 + 0.5998×-0.3 = 0.0052
V1[3][0] = -0.4589×-0.4 + -0.89×0 + 0.33×-0.1 + 1.6996×-0.4 = -0.5293
V1[3][1] = -0.4589×0.2 + -0.89×-0.7 + 0.33×-0.4 + 1.6996×-0.3 = -0.1106
20 5.7 计算 Head2 Q2 = X_enc_in·WQ1bᵀ（逐元素） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
Q2[0][0] = 0.5×-0.1 + 0.7×0.6 + 0.8×-0.6 + 1.1×0 = -0.11
Q2[0][1] = 0.5×0.2 + 0.7×-0.9 + 0.8×0.2 + 1.1×-0.7 = -1.14
Q2[1][0] = 0.6415×-0.1 + 1.1403×0.6 + -0.09×-0.6 + 1.4×0 = 0.674
Q2[1][1] = 0.6415×0.2 + 1.1403×-0.9 + -0.09×0.2 + 1.4×-0.7 = -1.8959
Q2[2][0] = 1.2093×-0.1 + -0.2161×0.6 + 0.52×-0.6 + 0.5998×0 = -0.5626
Q2[2][1] = 1.2093×0.2 + -0.2161×-0.9 + 0.52×0.2 + 0.5998×-0.7 = 0.1205
Q2[3][0] = -0.4589×-0.1 + -0.89×0.6 + 0.33×-0.6 + 1.6996×0 = -0.6861
Q2[3][1] = -0.4589×0.2 + -0.89×-0.9 + 0.33×0.2 + 1.6996×-0.7 = -0.4145
21 5.8 计算 Head2 K2 = X_enc_in·WK1bᵀ（逐元素） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
K2[0][0] = 0.5×-0.9 + 0.7×0.9 + 0.8×0.9 + 1.1×0.6 = 1.56
K2[0][1] = 0.5×-0.4 + 0.7×-0.8 + 0.8×0.4 + 1.1×-0.1 = -0.55
K2[1][0] = 0.6415×-0.9 + 1.1403×0.9 + -0.09×0.9 + 1.4×0.6 = 1.2079
K2[1][1] = 0.6415×-0.4 + 1.1403×-0.8 + -0.09×0.4 + 1.4×-0.1 = -1.3448
K2[2][0] = 1.2093×-0.9 + -0.2161×0.9 + 0.52×0.9 + 0.5998×0.6 = -0.455
K2[2][1] = 1.2093×-0.4 + -0.2161×-0.8 + 0.52×0.4 + 0.5998×-0.1 = -0.1628
K2[3][0] = -0.4589×-0.9 + -0.89×0.9 + 0.33×0.9 + 1.6996×0.6 = 0.9287
K2[3][1] = -0.4589×-0.4 + -0.89×-0.8 + 0.33×0.4 + 1.6996×-0.1 = 0.8576
22 5.9 计算 Head2 V2 = X_enc_in·WV1bᵀ（逐元素） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
V2[0][0] = 0.5×-0.8 + 0.7×0 + 0.8×-0.9 + 1.1×0.8 = -0.24
V2[0][1] = 0.5×-0.5 + 0.7×0.3 + 0.8×-0.4 + 1.1×0 = -0.36
V2[1][0] = 0.6415×-0.8 + 1.1403×0 + -0.09×-0.9 + 1.4×0.8 = 0.6878
V2[1][1] = 0.6415×-0.5 + 1.1403×0.3 + -0.09×-0.4 + 1.4×0 = 0.0574
V2[2][0] = 1.2093×-0.8 + -0.2161×0 + 0.52×-0.9 + 0.5998×0.8 = -0.9556
V2[2][1] = 1.2093×-0.5 + -0.2161×0.3 + 0.52×-0.4 + 0.5998×0 = -0.8775
V2[3][0] = -0.4589×-0.8 + -0.89×0 + 0.33×-0.9 + 1.6996×0.8 = 1.4297
V2[3][1] = -0.4589×-0.5 + -0.89×0.3 + 0.33×-0.4 + 1.6996×0 = -0.1696
23 5.10 计算 Head1 注意力分数 score1 = Q1·K1ᵀ/√2（逐元素） Encoder L1
🎯 架构功能讲解score[i][j] = Q_i 与 K_j 的点积 ÷ √2。点积衡量「token i 对 token j 的关注强度」；÷√2 防止维度大时数值过大导致 softmax 饱和。
score1[0][0] = (1.5829)/1.4142 = 1.1193
score1[0][1] = (2.9345)/1.4142 = 2.075
score1[0][2] = (-0.0477)/1.4142 = -0.0337
score1[0][3] = (1.6209)/1.4142 = 1.1462
score1[1][0] = (0.8346)/1.4142 = 0.5902
score1[1][1] = (2.2577)/1.4142 = 1.5964
score1[1][2] = (0.1306)/1.4142 = 0.0923
score1[1][3] = (0.9908)/1.4142 = 0.7006
score1[2][0] = (0.7984)/1.4142 = 0.5645
score1[2][1] = (0.416)/1.4142 = 0.2941
score1[2][2] = (-0.2573)/1.4142 = -0.1819
score1[2][3] = (0.6137)/1.4142 = 0.4339
score1[3][0] = (-2.3129)/1.4142 = -1.6355
score1[3][1] = (-2.1821)/1.4142 = -1.543
score1[3][2] = (0.5312)/1.4142 = 0.3756
score1[3][3] = (-1.9651)/1.4142 = -1.3895
24 5.11 计算 Head2 注意力分数 score2 = Q2·K2ᵀ/√2（逐元素） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
score2[0][0] = (0.4554)/1.4142 = 0.322
score2[0][1] = (1.4002)/1.4142 = 0.9901
score2[0][2] = (0.2356)/1.4142 = 0.1666
score2[0][3] = (-1.0798)/1.4142 = -0.7635
score2[1][0] = (2.0943)/1.4142 = 1.4809
score2[1][1] = (3.3639)/1.4142 = 2.3786
score2[1][2] = (0.0019)/1.4142 = 0.0014
score2[1][3] = (-0.9999)/1.4142 = -0.7071
score2[2][0] = (-0.944)/1.4142 = -0.6675
score2[2][1] = (-0.8417)/1.4142 = -0.5952
score2[2][2] = (0.2364)/1.4142 = 0.1671
score2[2][3] = (-0.4191)/1.4142 = -0.2964
score2[3][0] = (-0.8424)/1.4142 = -0.5956
score2[3][1] = (-0.2714)/1.4142 = -0.1919
score2[3][2] = (0.3797)/1.4142 = 0.2685
score2[3][3] = (-0.9926)/1.4142 = -0.7019
25 5.12 Head1 softmax 注意力权重 A1 Encoder L1
🎯 架构功能讲解softmax 把一行分数变成概率分布：exp 放大差异，除以总和保证每行和为 1。这就是「注意力权重」——模型决定每个 token 分多少注意力给别人。
Head1 softmax：A1[i] = softmax(score1[i])（每行 exp 归一化，和为 1）
行0（我）:
e^(1.1193, 2.075, -0.0337, 1.1462)
= 3.0626 + 7.9645 + 0.9669 + 3.1461 = 15.1401
A1[0] = 3.0626/15.1401 = 0.2023
A1[1] = 7.9645/15.1401 = 0.5261
A1[2] = 0.9669/15.1401 = 0.0639
A1[3] = 3.1461/15.1401 = 0.2078
行1（爱）:
e^(0.5902, 1.5964, 0.0923, 0.7006)
= 1.8043 + 4.9354 + 1.0967 + 2.0149 = 9.8514
A1[0] = 1.8043/9.8514 = 0.1832
A1[1] = 4.9354/9.8514 = 0.501
A1[2] = 1.0967/9.8514 = 0.1113
A1[3] = 2.0149/9.8514 = 0.2045
行2（水）:
e^(0.5645, 0.2941, -0.1819, 0.4339)
= 1.7586 + 1.342 + 0.8337 + 1.5433 = 5.4775
A1[0] = 1.7586/5.4775 = 0.3211
A1[1] = 1.342/5.4775 = 0.245
A1[2] = 0.8337/5.4775 = 0.1522
A1[3] = 1.5433/5.4775 = 0.2818
行3（课）:
e^(-1.6355, -1.543, 0.3756, -1.3895)
= 0.1949 + 0.2137 + 1.4559 + 0.2492 = 2.1137
A1[0] = 0.1949/2.1137 = 0.0922
A1[1] = 0.2137/2.1137 = 0.1011
A1[2] = 1.4559/2.1137 = 0.6888
A1[3] = 0.2492/2.1137 = 0.1179
26 5.13 Head2 softmax 注意力权重 A2 Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
Head2 softmax：A2[i] = softmax(score2[i])
行0（我）:
e^(0.322, 0.9901, 0.1666, -0.7635)
= 1.3799 + 2.6915 + 1.1813 + 0.466 = 5.7188
A2[0] = 1.3799/5.7188 = 0.2413
A2[1] = 2.6915/5.7188 = 0.4707
A2[2] = 1.1813/5.7188 = 0.2066
A2[3] = 0.466/5.7188 = 0.0815
行1（爱）:
e^(1.4809, 2.3786, 0.0014, -0.7071)
= 4.3968 + 10.7901 + 1.0014 + 0.4931 = 16.6813
A2[0] = 4.3968/16.6813 = 0.2636
A2[1] = 10.7901/16.6813 = 0.6468
A2[2] = 1.0014/16.6813 = 0.06
A2[3] = 0.4931/16.6813 = 0.0296
行2（水）:
e^(-0.6675, -0.5952, 0.1671, -0.2964)
= 0.513 + 0.5515 + 1.1819 + 0.7435 = 2.9899
A2[0] = 0.513/2.9899 = 0.1716
A2[1] = 0.5515/2.9899 = 0.1844
A2[2] = 1.1819/2.9899 = 0.3953
A2[3] = 0.7435/2.9899 = 0.2487
行3（课）:
e^(-0.5956, -0.1919, 0.2685, -0.7019)
= 0.5512 + 0.8254 + 1.3079 + 0.4956 = 3.1802
A2[0] = 0.5512/3.1802 = 0.1733
A2[1] = 0.8254/3.1802 = 0.2595
A2[2] = 1.3079/3.1802 = 0.4113
A2[3] = 0.4956/3.1802 = 0.1559
27 5.14 Head1 输出 O1 = A1·V1 Encoder L1
🎯 架构功能讲解加权求和：用注意力权重 A 对 V 加权混合。输出 = 全句信息按关注度的加权和——这就是「上下文感知表示」的来源。
Head1 输出 O1[i] = Σ_j A1[i][j]·V1[j]（逐元素）
行0（我）:
O1[0] = 0.2023×-0.72 + 0.5261×-0.8076 + 0.0639×-0.7756 + 0.2078×-0.5293 = -0.73
O1[1] = 0.2023×-1.04 + 0.5261×-1.0539 + 0.0639×0.0052 + 0.2078×-0.1106 = -0.7874
行1（爱）:
O1[0] = 0.1832×-0.72 + 0.501×-0.8076 + 0.1113×-0.7756 + 0.2045×-0.5293 = -0.7311
O1[1] = 0.1832×-1.04 + 0.501×-1.0539 + 0.1113×0.0052 + 0.2045×-0.1106 = -0.7405
行2（水）:
O1[0] = 0.3211×-0.72 + 0.245×-0.8076 + 0.1522×-0.7756 + 0.2818×-0.5293 = -0.6962
O1[1] = 0.3211×-1.04 + 0.245×-1.0539 + 0.1522×0.0052 + 0.2818×-0.1106 = -0.6225
行3（课）:
O1[0] = 0.0922×-0.72 + 0.1011×-0.8076 + 0.6888×-0.7756 + 0.1179×-0.5293 = -0.7447
O1[1] = 0.0922×-1.04 + 0.1011×-1.0539 + 0.6888×0.0052 + 0.1179×-0.1106 = -0.2119
28 5.15 Head2 输出 O2 = A2·V2 Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
Head2 输出 O2[i] = Σ_j A2[i][j]·V2[j]（逐元素）
行0（我）:
O2[0] = 0.2413×-0.24 + 0.4707×0.6878 + 0.2066×-0.9556 + 0.0815×1.4297 = 0.1849
O2[1] = 0.2413×-0.36 + 0.4707×0.0574 + 0.2066×-0.8775 + 0.0815×-0.1696 = -0.2549
行1（爱）:
O2[0] = 0.2636×-0.24 + 0.6468×0.6878 + 0.06×-0.9556 + 0.0296×1.4297 = 0.3665
O2[1] = 0.2636×-0.36 + 0.6468×0.0574 + 0.06×-0.8775 + 0.0296×-0.1696 = -0.1155
行2（水）:
O2[0] = 0.1716×-0.24 + 0.1844×0.6878 + 0.3953×-0.9556 + 0.2487×1.4297 = 0.0635
O2[1] = 0.1716×-0.36 + 0.1844×0.0574 + 0.3953×-0.8775 + 0.2487×-0.1696 = -0.4402
行3（课）:
O2[0] = 0.1733×-0.24 + 0.2595×0.6878 + 0.4113×-0.9556 + 0.1559×1.4297 = -0.0333
O2[1] = 0.1733×-0.36 + 0.2595×0.0574 + 0.4113×-0.8775 + 0.1559×-0.1696 = -0.4348
29 5.16 拼接：Concat[i] = [O1[i] | O2[i]]（两头的 2 维拼成 4 维） Encoder L1
🎯 架构功能讲解两个头从不同视角提取信息，拼接保留全部信息，交由 WO1 融合。
行0: [-0.73, -0.7874, 0.1849, -0.2549]
行1: [-0.7311, -0.7405, 0.3665, -0.1155]
行2: [-0.6962, -0.6225, 0.0635, -0.4402]
行3: [-0.7447, -0.2119, -0.0333, -0.4348]
30 5.17 多头输出 AttnOut1 = Concat·WO1ᵀ（逐元素） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
AttnOut1[0][0] = -0.73×0.1 + -0.7874×-0.6 + 0.1849×0.9 + -0.2549×0.6 = 0.4129
AttnOut1[0][1] = -0.73×0.9 + -0.7874×0.8 + 0.1849×0.2 + -0.2549×0.8 = -1.4539
AttnOut1[0][2] = -0.73×-0.8 + -0.7874×-0.6 + 0.1849×-0.9 + -0.2549×-0.3 = 0.9665
AttnOut1[0][3] = -0.73×-0.2 + -0.7874×-0.5 + 0.1849×0.7 + -0.2549×-0.3 = 0.7456
AttnOut1[1][0] = -0.7311×0.1 + -0.7405×-0.6 + 0.3665×0.9 + -0.1155×0.6 = 0.6318
AttnOut1[1][1] = -0.7311×0.9 + -0.7405×0.8 + 0.3665×0.2 + -0.1155×0.8 = -1.2694
AttnOut1[1][2] = -0.7311×-0.8 + -0.7405×-0.6 + 0.3665×-0.9 + -0.1155×-0.3 = 0.7339
AttnOut1[1][3] = -0.7311×-0.2 + -0.7405×-0.5 + 0.3665×0.7 + -0.1155×-0.3 = 0.8077
AttnOut1[2][0] = -0.6962×0.1 + -0.6225×-0.6 + 0.0635×0.9 + -0.4402×0.6 = 0.0968
AttnOut1[2][1] = -0.6962×0.9 + -0.6225×0.8 + 0.0635×0.2 + -0.4402×0.8 = -1.464
AttnOut1[2][2] = -0.6962×-0.8 + -0.6225×-0.6 + 0.0635×-0.9 + -0.4402×-0.3 = 1.0054
AttnOut1[2][3] = -0.6962×-0.2 + -0.6225×-0.5 + 0.0635×0.7 + -0.4402×-0.3 = 0.627
AttnOut1[3][0] = -0.7447×0.1 + -0.2119×-0.6 + -0.0333×0.9 + -0.4348×0.6 = -0.2382
AttnOut1[3][1] = -0.7447×0.9 + -0.2119×0.8 + -0.0333×0.2 + -0.4348×0.8 = -1.1943
AttnOut1[3][2] = -0.7447×-0.8 + -0.2119×-0.6 + -0.0333×-0.9 + -0.4348×-0.3 = 0.8833
AttnOut1[3][3] = -0.7447×-0.2 + -0.2119×-0.5 + -0.0333×0.7 + -0.4348×-0.3 = 0.362
31 6.1 残差连接 X1 = X_enc_in + AttnOut1 Encoder L1
🎯 架构功能讲解残差（Add）：把原始输入直接加到注意力输出上。梯度可以「抄近道」流回输入层，深层网络才不退化——这是 Transformer 能堆很多层的关键。
残差连接：X1 = X_enc_in + AttnOut1（逐元素）
X1[0][0] = 0.5 + 0.4129 = 0.9129
X1[0][1] = 0.7 + -1.4539 = -0.7539
X1[0][2] = 0.8 + 0.9665 = 1.7665
X1[0][3] = 1.1 + 0.7456 = 1.8456
X1[1][0] = 0.6415 + 0.6318 = 1.2733
X1[1][1] = 1.1403 + -1.2694 = -0.1291
X1[1][2] = -0.09 + 0.7339 = 0.6439
X1[1][3] = 1.4 + 0.8077 = 2.2076
X1[2][0] = 1.2093 + 0.0968 = 1.3061
X1[2][1] = -0.2161 + -1.464 = -1.6802
X1[2][2] = 0.52 + 1.0054 = 1.5254
X1[2][3] = 0.5998 + 0.627 = 1.2268
X1[3][0] = -0.4589 + -0.2382 = -0.6971
X1[3][1] = -0.89 + -1.1943 = -2.0842
X1[3][2] = 0.33 + 0.8833 = 1.2133
X1[3][3] = 1.6996 + 0.362 = 2.0616
32 6.2 LayerNorm 归一化 LN1a = LayerNorm(X1) Encoder L1
🎯 架构功能讲解LayerNorm（Norm）：对每个 token 的向量独立做标准化——减均值、除标准差，输出均值≈0、方差≈1。稳定每层数值分布，训练更快更稳。
LayerNorm：每行 (x-μ)/√(σ²+ε)，ε=1e-5
行0（我）:
均值 μ = (0.9129 + -0.7539 + 1.7665 + 1.8456)/4 = 0.9428
方差 σ² = ((-0.0299)² + (-1.6967)² + (0.8237)² + (0.9029)²)/4 = 1.0933
LN1a[0] = (0.9129 - 0.9428)/√(1.0933+ε) = -0.0286
LN1a[1] = (-0.7539 - 0.9428)/√(1.0933+ε) = -1.6227
LN1a[2] = (1.7665 - 0.9428)/√(1.0933+ε) = 0.7878
LN1a[3] = (1.8456 - 0.9428)/√(1.0933+ε) = 0.8634
行1（爱）:
均值 μ = (1.2733 + -0.1291 + 0.6439 + 2.2076)/4 = 0.9989
方差 σ² = ((0.2743)² + (-1.1281)² + (-0.355)² + (1.2087)²)/4 = 0.7337
LN1a[0] = (1.2733 - 0.9989)/√(0.7337+ε) = 0.3203
LN1a[1] = (-0.1291 - 0.9989)/√(0.7337+ε) = -1.317
LN1a[2] = (0.6439 - 0.9989)/√(0.7337+ε) = -0.4144
LN1a[3] = (2.2076 - 0.9989)/√(0.7337+ε) = 1.4111
行2（水）:
均值 μ = (1.3061 + -1.6802 + 1.5254 + 1.2268)/4 = 0.5945
方差 σ² = ((0.7116)² + (-2.2747)² + (0.9309)² + (0.6322)²)/4 = 1.7367
LN1a[0] = (1.3061 - 0.5945)/√(1.7367+ε) = 0.54
LN1a[1] = (-1.6802 - 0.5945)/√(1.7367+ε) = -1.7261
LN1a[2] = (1.5254 - 0.5945)/√(1.7367+ε) = 0.7063
LN1a[3] = (1.2268 - 0.5945)/√(1.7367+ε) = 0.4797
行3（课）:
均值 μ = (-0.6971 + -2.0842 + 1.2133 + 2.0616)/4 = 0.1234
方差 σ² = ((-0.8205)² + (-2.2076)² + (1.0899)² + (1.9382)²)/4 = 2.6228
LN1a[0] = (-0.6971 - 0.1234)/√(2.6228+ε) = -0.5066
LN1a[1] = (-2.0842 - 0.1234)/√(2.6228+ε) = -1.3631
LN1a[2] = (1.2133 - 0.1234)/√(2.6228+ε) = 0.673
LN1a[3] = (2.0616 - 0.1234)/√(2.6228+ε) = 1.1968
33 6.3 定义 FFN 权重 W1_1(8×4)/W1_2(4×8) Encoder L1
🎯 架构功能讲解FFN（Feed-Forward）：对每个 token 独立做两次线性变换 + ReLU。注意力负责跨 token「搬运信息」，FFN 负责逐 token「加工信息」。
W1_1 行0: [-0.4, 0.1, -0.7, 0.6]
W1_1 行1: [-0.9, 1, 0.5, -0.6]
W1_1 行2: [-1, 0.6, 0.4, 0.5]
W1_1 行3: [0.5, -0.9, -0.3, -0.8]
W1_1 行4: [0.7, 0.2, -0.3, -0.9]
W1_1 行5: [-0.4, -0.3, 0.5, 0.3]
W1_1 行6: [0.8, -0.1, -0.8, 0.4]
W1_1 行7: [0.5, 0.1, 0.5, 0]
W1_2 行0: [0, -0.1, -0.9, -0.8, -0.9, 0.3, -0.4, 0]
W1_2 行1: [0.8, -0.5, -0.2, 0.5, -0.5, -0.8, -0.4, -0.7]
W1_2 行2: [0.9, 0.6, 0.3, 0.7, 0.6, -0.6, 0.8, 0.1]
W1_2 行3: [0.6, 0.8, -0.4, -0.8, -0.5, -0.1, 0.6, 0.7]
34 6.4 FFN 第一层 H1 = LN1a·W1_1ᵀ（4→8 维，逐元素） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
H1[0][0] = -0.0286×-0.4 + -1.6227×0.1 + 0.7878×-0.7 + 0.8634×0.6 = -0.1842
H1[0][1] = -0.0286×-0.9 + -1.6227×1 + 0.7878×0.5 + 0.8634×-0.6 = -1.7211
H1[0][2] = -0.0286×-1 + -1.6227×0.6 + 0.7878×0.4 + 0.8634×0.5 = -0.1982
H1[0][3] = -0.0286×0.5 + -1.6227×-0.9 + 0.7878×-0.3 + 0.8634×-0.8 = 0.519
H1[0][4] = -0.0286×0.7 + -1.6227×0.2 + 0.7878×-0.3 + 0.8634×-0.9 = -1.358
H1[0][5] = -0.0286×-0.4 + -1.6227×-0.3 + 0.7878×0.5 + 0.8634×0.3 = 1.1511
H1[0][6] = -0.0286×0.8 + -1.6227×-0.1 + 0.7878×-0.8 + 0.8634×0.4 = -0.1454
H1[0][7] = -0.0286×0.5 + -1.6227×0.1 + 0.7878×0.5 + 0.8634×0 = 0.2173
H1[1][0] = 0.3203×-0.4 + -1.317×0.1 + -0.4144×-0.7 + 1.4111×0.6 = 0.877
H1[1][1] = 0.3203×-0.9 + -1.317×1 + -0.4144×0.5 + 1.4111×-0.6 = -2.6591
H1[1][2] = 0.3203×-1 + -1.317×0.6 + -0.4144×0.4 + 1.4111×0.5 = -0.5707
H1[1][3] = 0.3203×0.5 + -1.317×-0.9 + -0.4144×-0.3 + 1.4111×-0.8 = 0.3408
H1[1][4] = 0.3203×0.7 + -1.317×0.2 + -0.4144×-0.3 + 1.4111×-0.9 = -1.1849
H1[1][5] = 0.3203×-0.4 + -1.317×-0.3 + -0.4144×0.5 + 1.4111×0.3 = 0.4831
H1[1][6] = 0.3203×0.8 + -1.317×-0.1 + -0.4144×-0.8 + 1.4111×0.4 = 1.2839
H1[1][7] = 0.3203×0.5 + -1.317×0.1 + -0.4144×0.5 + 1.4111×0 = -0.1788
H1[2][0] = 0.54×-0.4 + -1.7261×0.1 + 0.7063×-0.7 + 0.4797×0.6 = -0.5952
H1[2][1] = 0.54×-0.9 + -1.7261×1 + 0.7063×0.5 + 0.4797×-0.6 = -2.1467
H1[2][2] = 0.54×-1 + -1.7261×0.6 + 0.7063×0.4 + 0.4797×0.5 = -1.0532
H1[2][3] = 0.54×0.5 + -1.7261×-0.9 + 0.7063×-0.3 + 0.4797×-0.8 = 1.2278
H1[2][4] = 0.54×0.7 + -1.7261×0.2 + 0.7063×-0.3 + 0.4797×-0.9 = -0.6109
H1[2][5] = 0.54×-0.4 + -1.7261×-0.3 + 0.7063×0.5 + 0.4797×0.3 = 0.7989
H1[2][6] = 0.54×0.8 + -1.7261×-0.1 + 0.7063×-0.8 + 0.4797×0.4 = 0.2314
H1[2][7] = 0.54×0.5 + -1.7261×0.1 + 0.7063×0.5 + 0.4797×0 = 0.4506
H1[3][0] = -0.5066×-0.4 + -1.3631×0.1 + 0.673×-0.7 + 1.1968×0.6 = 0.3133
H1[3][1] = -0.5066×-0.9 + -1.3631×1 + 0.673×0.5 + 1.1968×-0.6 = -1.2888
H1[3][2] = -0.5066×-1 + -1.3631×0.6 + 0.673×0.4 + 1.1968×0.5 = 0.5563
H1[3][3] = -0.5066×0.5 + -1.3631×-0.9 + 0.673×-0.3 + 1.1968×-0.8 = -0.1858
H1[3][4] = -0.5066×0.7 + -1.3631×0.2 + 0.673×-0.3 + 1.1968×-0.9 = -1.9062
H1[3][5] = -0.5066×-0.4 + -1.3631×-0.3 + 0.673×0.5 + 1.1968×0.3 = 1.3071
H1[3][6] = -0.5066×0.8 + -1.3631×-0.1 + 0.673×-0.8 + 1.1968×0.4 = -0.3286
H1[3][7] = -0.5066×0.5 + -1.3631×0.1 + 0.673×0.5 + 1.1968×0 = -0.0531
35 6.5 ReLU 激活 R1 = max(0, H1)（负数清零，逐元素） Encoder L1
🎯 架构功能讲解ReLU 引入非线性：负数全部归零。没有它，两层线性变换叠加还是线性，模型学不了复杂函数。
R1[0][0] = max(0, -0.1842) = 0
R1[0][1] = max(0, -1.7211) = 0
R1[0][2] = max(0, -0.1982) = 0
R1[0][3] = max(0, 0.519) = 0.519
R1[0][4] = max(0, -1.358) = 0
R1[0][5] = max(0, 1.1511) = 1.1511
R1[0][6] = max(0, -0.1454) = 0
R1[0][7] = max(0, 0.2173) = 0.2173
R1[1][0] = max(0, 0.877) = 0.877
R1[1][1] = max(0, -2.6591) = 0
R1[1][2] = max(0, -0.5707) = 0
R1[1][3] = max(0, 0.3408) = 0.3408
R1[1][4] = max(0, -1.1849) = 0
R1[1][5] = max(0, 0.4831) = 0.4831
R1[1][6] = max(0, 1.2839) = 1.2839
R1[1][7] = max(0, -0.1788) = 0
R1[2][0] = max(0, -0.5952) = 0
R1[2][1] = max(0, -2.1467) = 0
R1[2][2] = max(0, -1.0532) = 0
R1[2][3] = max(0, 1.2278) = 1.2278
R1[2][4] = max(0, -0.6109) = 0
R1[2][5] = max(0, 0.7989) = 0.7989
R1[2][6] = max(0, 0.2314) = 0.2314
R1[2][7] = max(0, 0.4506) = 0.4506
R1[3][0] = max(0, 0.3133) = 0.3133
R1[3][1] = max(0, -1.2888) = 0
R1[3][2] = max(0, 0.5563) = 0.5563
R1[3][3] = max(0, -0.1858) = 0
R1[3][4] = max(0, -1.9062) = 0
R1[3][5] = max(0, 1.3071) = 1.3071
R1[3][6] = max(0, -0.3286) = 0
R1[3][7] = max(0, -0.0531) = 0
36 6.6 FFN 第二层 F1 = R1·W1_2ᵀ（8→4 维，逐元素） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
F1[0][0] = 0×0 + 0×-0.1 + 0×-0.9 + 0.519×-0.8 + 0×-0.9 + 1.1511×0.3 + 0×-0.4 + 0.2173×0 = -0.0699
F1[0][1] = 0×0.8 + 0×-0.5 + 0×-0.2 + 0.519×0.5 + 0×-0.5 + 1.1511×-0.8 + 0×-0.4 + 0.2173×-0.7 = -0.8135
F1[0][2] = 0×0.9 + 0×0.6 + 0×0.3 + 0.519×0.7 + 0×0.6 + 1.1511×-0.6 + 0×0.8 + 0.2173×0.1 = -0.3056
F1[0][3] = 0×0.6 + 0×0.8 + 0×-0.4 + 0.519×-0.8 + 0×-0.5 + 1.1511×-0.1 + 0×0.6 + 0.2173×0.7 = -0.3782
F1[1][0] = 0.877×0 + 0×-0.1 + 0×-0.9 + 0.3408×-0.8 + 0×-0.9 + 0.4831×0.3 + 1.2839×-0.4 + 0×0 = -0.6413
F1[1][1] = 0.877×0.8 + 0×-0.5 + 0×-0.2 + 0.3408×0.5 + 0×-0.5 + 0.4831×-0.8 + 1.2839×-0.4 + 0×-0.7 = -0.028
F1[1][2] = 0.877×0.9 + 0×0.6 + 0×0.3 + 0.3408×0.7 + 0×0.6 + 0.4831×-0.6 + 1.2839×0.8 + 0×0.1 = 1.7651
F1[1][3] = 0.877×0.6 + 0×0.8 + 0×-0.4 + 0.3408×-0.8 + 0×-0.5 + 0.4831×-0.1 + 1.2839×0.6 + 0×0.7 = 0.9755
F1[2][0] = 0×0 + 0×-0.1 + 0×-0.9 + 1.2278×-0.8 + 0×-0.9 + 0.7989×0.3 + 0.2314×-0.4 + 0.4506×0 = -0.8351
F1[2][1] = 0×0.8 + 0×-0.5 + 0×-0.2 + 1.2278×0.5 + 0×-0.5 + 0.7989×-0.8 + 0.2314×-0.4 + 0.4506×-0.7 = -0.4332
F1[2][2] = 0×0.9 + 0×0.6 + 0×0.3 + 1.2278×0.7 + 0×0.6 + 0.7989×-0.6 + 0.2314×0.8 + 0.4506×0.1 = 0.6103
F1[2][3] = 0×0.6 + 0×0.8 + 0×-0.4 + 1.2278×-0.8 + 0×-0.5 + 0.7989×-0.1 + 0.2314×0.6 + 0.4506×0.7 = -0.6079
F1[3][0] = 0.3133×0 + 0×-0.1 + 0.5563×-0.9 + 0×-0.8 + 0×-0.9 + 1.3071×0.3 + 0×-0.4 + 0×0 = -0.1085
F1[3][1] = 0.3133×0.8 + 0×-0.5 + 0.5563×-0.2 + 0×0.5 + 0×-0.5 + 1.3071×-0.8 + 0×-0.4 + 0×-0.7 = -0.9063
F1[3][2] = 0.3133×0.9 + 0×0.6 + 0.5563×0.3 + 0×0.7 + 0×0.6 + 1.3071×-0.6 + 0×0.8 + 0×0.1 = -0.3354
F1[3][3] = 0.3133×0.6 + 0×0.8 + 0.5563×-0.4 + 0×-0.8 + 0×-0.5 + 1.3071×-0.1 + 0×0.6 + 0×0.7 = -0.1652
37 6.7 残差连接 X2 = LN1a + F1 Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
残差连接：X2 = LN1a + F1（逐元素）
X2[0][0] = -0.0286 + -0.0699 = -0.0984
X2[0][1] = -1.6227 + -0.8135 = -2.4362
X2[0][2] = 0.7878 + -0.3056 = 0.4821
X2[0][3] = 0.8634 + -0.3782 = 0.4853
X2[1][0] = 0.3203 + -0.6413 = -0.321
X2[1][1] = -1.317 + -0.028 = -1.345
X2[1][2] = -0.4144 + 1.7651 = 1.3507
X2[1][3] = 1.4111 + 0.9755 = 2.3867
X2[2][0] = 0.54 + -0.8351 = -0.2951
X2[2][1] = -1.7261 + -0.4332 = -2.1593
X2[2][2] = 0.7063 + 0.6103 = 1.3166
X2[2][3] = 0.4797 + -0.6079 = -0.1281
X2[3][0] = -0.5066 + -0.1085 = -0.6151
X2[3][1] = -1.3631 + -0.9063 = -2.2694
X2[3][2] = 0.673 + -0.3354 = 0.3376
X2[3][3] = 1.1968 + -0.1652 = 1.0315
38 6.8 LayerNorm 归一化 LN1b（Encoder L1 最终输出） Encoder L1
🎯 架构功能讲解Encoder 第 1 层：自注意力让源 token 互相交换信息（局部依赖），FFN 逐 token 加工。训练与推理共用同一份权重与计算。
LayerNorm：LN1b = LayerNorm(X2)
行0（我）:
均值 μ = (-0.0984 + -2.4362 + 0.4821 + 0.4853)/4 = -0.3918
方差 σ² = ((0.2934)² + (-2.0444)² + (0.8739)² + (0.8771)²)/4 = 1.4497
LN1b[0] = (-0.0984 - -0.3918)/√(1.4497+ε) = 0.2437
LN1b[1] = (-2.4362 - -0.3918)/√(1.4497+ε) = -1.698
LN1b[2] = (0.4821 - -0.3918)/√(1.4497+ε) = 0.7259
LN1b[3] = (0.4853 - -0.3918)/√(1.4497+ε) = 0.7284
行1（爱）:
均值 μ = (-0.321 + -1.345 + 1.3507 + 2.3867)/4 = 0.5178
方差 σ² = ((-0.8389)² + (-1.8628)² + (0.8329)² + (1.8688)²)/4 = 2.09
LN1b[0] = (-0.321 - 0.5178)/√(2.09+ε) = -0.5803
LN1b[1] = (-1.345 - 0.5178)/√(2.09+ε) = -1.2885
LN1b[2] = (1.3507 - 0.5178)/√(2.09+ε) = 0.5761
LN1b[3] = (2.3867 - 0.5178)/√(2.09+ε) = 1.2927
行2（水）:
均值 μ = (-0.2951 + -2.1593 + 1.3166 + -0.1281)/4 = -0.3165
方差 σ² = ((0.0214)² + (-1.8428)² + (1.6331)² + (0.1884)²)/4 = 1.5247
LN1b[0] = (-0.2951 - -0.3165)/√(1.5247+ε) = 0.0173
LN1b[1] = (-2.1593 - -0.3165)/√(1.5247+ε) = -1.4924
LN1b[2] = (1.3166 - -0.3165)/√(1.5247+ε) = 1.3225
LN1b[3] = (-0.1281 - -0.3165)/√(1.5247+ε) = 0.1526
行3（课）:
均值 μ = (-0.6151 + -2.2694 + 0.3376 + 1.0315)/4 = -0.3789
方差 σ² = ((-0.2363)² + (-1.8906)² + (0.7164)² + (1.4104)²)/4 = 1.5332
LN1b[0] = (-0.6151 - -0.3789)/√(1.5332+ε) = -0.1908
LN1b[1] = (-2.2694 - -0.3789)/√(1.5332+ε) = -1.5269
LN1b[2] = (0.3376 - -0.3789)/√(1.5332+ε) = 0.5786
LN1b[3] = (1.0315 - -0.3789)/√(1.5332+ε) = 1.1391
39 7.1 定义 Layer2 权重（WQ2a/WK2a/WV2a/WQ2b/WK2b/WV2b/WO2/W2_1/W2_2） Encoder L2
🎯 架构功能讲解Encoder 堆叠 2 层，每层结构相同但权重不同。Layer 2 在 Layer 1 的输出上继续做自注意力+FFN，学到更全局、更抽象的语义。
WQ2a 行0: [-1, 0, -0.2, -0.6]
WQ2a 行1: [-0.8, -0.3, 0.9, -0.4]
WK2a 行0: [0, 0.4, -0.3, 0.9]
WK2a 行1: [0.9, -0.5, 0, -0.4]
WV2a 行0: [-0.4, -0.9, 0.2, 0]
WV2a 行1: [-0.9, -0.4, 0.8, -0.5]
WQ2b 行0: [-0.7, 0, 1, -0.5]
WQ2b 行1: [0.3, 0.5, -0.5, 0.5]
WK2b 行0: [-0.3, 0.3, 0.3, 0.1]
WK2b 行1: [-0.8, 0.7, -0.4, -0.6]
WV2b 行0: [-0.9, 0.2, 0.4, -1]
WV2b 行1: [0, -0.5, 0.3, -0.7]
WO2 行0: [0.4, -0.2, 0.9, -0.7]
WO2 行1: [-0.3, -0.8, 0.8, 0.8]
WO2 行2: [-0.5, 0.3, 0.6, 0.1]
WO2 行3: [0.1, -0.5, -0.8, 0.8]
W2_1 行0: [0.8, 0.3, -0.3, -0.3]
W2_1 行1: [0.5, 0.8, 0.8, 0.6]
W2_1 行2: [0.3, -0.8, -0.7, 0.8]
W2_1 行3: [0.2, -1, -0.8, 0.3]
W2_1 行4: [-1, -0.7, 0.1, 0.4]
W2_1 行5: [0.3, -0.6, 0.4, -0.5]
W2_1 行6: [-0.3, 0.5, 0.3, 0.7]
W2_1 行7: [0.3, 0.1, -0.8, -0.3]
W2_2 行0: [-0.5, -0.5, 0.9, -0.2, 0.8, 0.3, 0.6, 0]
W2_2 行1: [0.2, 0, -0.6, 0.4, -0.4, -1, 0.3, -0.6]
W2_2 行2: [0.9, 0.9, 0.8, -0.3, -1, 0.9, -0.1, 0.9]
W2_2 行3: [0.9, 0.7, -0.4, -0.2, 0.7, -0.4, -0.7, 0.1]
40 7.2 计算 Q/K/V（6 个矩阵，输入=LN1b） Encoder L2
🎯 架构功能讲解计算方式与 5.4~5.9 完全相同（逐元素点积），只是输入换成 LN1b。
Q1[0] = [-0.8259, 0.6764]
Q1[1] = [-0.3106, 0.8522]
Q1[2] = [-0.3733, 1.5632]
Q1[3] = [-0.6083, 0.6758]
K1[0] = [-0.2413, 0.7769]
K1[1] = [0.4752, -0.395]
K1[2] = [-0.8564, 0.7008]
K1[3] = [0.2408, 0.1361]
V1[0] = [1.5759, 0.6764]
V1[1] = [1.507, 0.8522]
V1[2] = [1.6008, 1.5632]
V1[3] = [1.5662, 0.6758]
Q2[0] = [0.1911, -0.7746]
Q2[1] = [0.3359, -0.4601]
Q2[2] = [1.2342, -1.326]
Q2[3] = [0.1427, -0.5404]
K2[0] = [-0.2919, -2.1109]
K2[1] = [0.0896, -1.4438]
K2[2] = [-0.0409, -1.6791]
K2[3] = [-0.1133, -1.831]
V2[0] = [-0.997, 0.5568]
V2[1] = [-0.7977, -0.0878]
V2[2] = [0.0624, 1.0362]
V2[3] = [-1.0413, 0.1397]
41 7.3 注意力分数 score = Q·Kᵀ/√2（2 头） Encoder L2
🎯 架构功能讲解Encoder 第 2 层：在 L1 输出上继续自注意力+FFN，学到更全局的语义；最终输出即源句记忆，供 Decoder 交叉注意力查阅。
score1_L2[0][0] = (0.7248)/1.4142 = 0.5125
score1_L2[0][1] = (-0.6596)/1.4142 = -0.4664
score1_L2[0][2] = (1.1813)/1.4142 = 0.8353
score1_L2[0][3] = (-0.1069)/1.4142 = -0.0756
score1_L2[1][0] = (0.737)/1.4142 = 0.5212
score1_L2[1][1] = (-0.4842)/1.4142 = -0.3424
score1_L2[1][2] = (0.8632)/1.4142 = 0.6103
score1_L2[1][3] = (0.0412)/1.4142 = 0.0291
score1_L2[2][0] = (1.3045)/1.4142 = 0.9224
score1_L2[2][1] = (-0.7949)/1.4142 = -0.5621
score1_L2[2][2] = (1.4151)/1.4142 = 1.0006
score1_L2[2][3] = (0.1228)/1.4142 = 0.0868
score1_L2[3][0] = (0.6719)/1.4142 = 0.4751
score1_L2[3][1] = (-0.556)/1.4142 = -0.3932
score1_L2[3][2] = (0.9946)/1.4142 = 0.7033
score1_L2[3][3] = (-0.0545)/1.4142 = -0.0386
score2_L2[0][0] = (1.5793)/1.4142 = 1.1168
score2_L2[0][1] = (1.1355)/1.4142 = 0.8029
score2_L2[0][2] = (1.2928)/1.4142 = 0.9141
score2_L2[0][3] = (1.3966)/1.4142 = 0.9876
score2_L2[1][0] = (0.8731)/1.4142 = 0.6174
score2_L2[1][1] = (0.6943)/1.4142 = 0.491
score2_L2[1][2] = (0.7587)/1.4142 = 0.5365
score2_L2[1][3] = (0.8043)/1.4142 = 0.5687
score2_L2[2][0] = (2.4389)/1.4142 = 1.7245
score2_L2[2][1] = (2.0251)/1.4142 = 1.432
score2_L2[2][2] = (2.176)/1.4142 = 1.5387
score2_L2[2][3] = (2.2881)/1.4142 = 1.6179
score2_L2[3][0] = (1.0992)/1.4142 = 0.7773
score2_L2[3][1] = (0.7931)/1.4142 = 0.5608
score2_L2[3][2] = (0.9016)/1.4142 = 0.6375
score2_L2[3][3] = (0.9734)/1.4142 = 0.6883
42 7.4 softmax 注意力权重 A（Head1/Head2） Encoder L2
🎯 架构功能讲解观察：Layer 2 的注意力比 Layer 1 更均匀（没有 0.53 那样的尖峰）——高层学到更分散的全局关系，这正是「低层抓局部、高层抓全局」。
softmax 注意力权重 A（2 头，每行和为 1）
A1_L2 行0: [0.3019, 0.1134, 0.417, 0.1677]
A1_L2 行1: [0.3199, 0.1349, 0.3497, 0.1956]
A1_L2 行2: [0.3648, 0.0827, 0.3944, 0.1582]
A1_L2 行3: [0.3054, 0.1282, 0.3837, 0.1827]
A2_L2 行0: [0.2919, 0.2133, 0.2383, 0.2565]
A2_L2 行1: [0.2662, 0.2346, 0.2456, 0.2536]
A2_L2 行2: [0.2877, 0.2147, 0.2389, 0.2586]
A2_L2 行3: [0.2786, 0.2243, 0.2422, 0.2549]
43 7.5 加权求和 O = A·V（2 头） Encoder L2
🎯 架构功能讲解Encoder 第 2 层：在 L1 输出上继续自注意力+FFN，学到更全局的语义；最终输出即源句记忆，供 Decoder 交叉注意力查阅。
O1_L2[0] = [1.5768, 1.066]
O1_L2[1] = [1.5734, 1.0101]
O1_L2[2] = [1.5785, 1.0406]
O1_L2[3] = [1.5748, 1.0391]
O2_L2[0] = [-0.7133, 0.4266]
O2_L2[1] = [-0.7013, 0.4175]
O2_L2[2] = [-0.7125, 0.425]
O2_L2[3] = [-0.7069, 0.422]
44 7.6 拼接 + 输出投影 AttnOut2 = Concat·WO2ᵀ Encoder L2
🎯 架构功能讲解Encoder 第 2 层：在 L1 输出上继续自注意力+FFN，学到更全局的语义；最终输出即源句记忆，供 Decoder 交叉注意力查阅。
Concat[0] = [1.5768, 1.066, -0.7133, 0.4266]
Concat[1] = [1.5734, 1.0101, -0.7013, 0.4175]
Concat[2] = [1.5785, 1.0406, -0.7125, 0.425]
Concat[3] = [1.5748, 1.0391, -0.7069, 0.422]
AttnOut2[0][0] = 1.5768×0.4 + 1.066×-0.2 + -0.7133×0.9 + 0.4266×-0.7 = -0.5231
AttnOut2[0][1] = 1.5768×-0.3 + 1.066×-0.8 + -0.7133×0.8 + 0.4266×0.8 = -1.5552
AttnOut2[0][2] = 1.5768×-0.5 + 1.066×0.3 + -0.7133×0.6 + 0.4266×0.1 = -0.854
AttnOut2[0][3] = 1.5768×0.1 + 1.066×-0.5 + -0.7133×-0.8 + 0.4266×0.8 = 0.5367
AttnOut2[1][0] = 1.5734×0.4 + 1.0101×-0.2 + -0.7013×0.9 + 0.4175×-0.7 = -0.4961
AttnOut2[1][1] = 1.5734×-0.3 + 1.0101×-0.8 + -0.7013×0.8 + 0.4175×0.8 = -1.5071
AttnOut2[1][2] = 1.5734×-0.5 + 1.0101×0.3 + -0.7013×0.6 + 0.4175×0.1 = -0.8627
AttnOut2[1][3] = 1.5734×0.1 + 1.0101×-0.5 + -0.7013×-0.8 + 0.4175×0.8 = 0.5474
AttnOut2[2][0] = 1.5785×0.4 + 1.0406×-0.2 + -0.7125×0.9 + 0.425×-0.7 = -0.5155
AttnOut2[2][1] = 1.5785×-0.3 + 1.0406×-0.8 + -0.7125×0.8 + 0.425×0.8 = -1.536
AttnOut2[2][2] = 1.5785×-0.5 + 1.0406×0.3 + -0.7125×0.6 + 0.425×0.1 = -0.8621
AttnOut2[2][3] = 1.5785×0.1 + 1.0406×-0.5 + -0.7125×-0.8 + 0.425×0.8 = 0.5476
AttnOut2[3][0] = 1.5748×0.4 + 1.0391×-0.2 + -0.7069×0.9 + 0.422×-0.7 = -0.5095
AttnOut2[3][1] = 1.5748×-0.3 + 1.0391×-0.8 + -0.7069×0.8 + 0.422×0.8 = -1.5316
AttnOut2[3][2] = 1.5748×-0.5 + 1.0391×0.3 + -0.7069×0.6 + 0.422×0.1 = -0.8577
AttnOut2[3][3] = 1.5748×0.1 + 1.0391×-0.5 + -0.7069×-0.8 + 0.422×0.8 = 0.5411
45 7.7 残差连接 X3 = LN1b + AttnOut2 Encoder L2
🎯 架构功能讲解Encoder 第 2 层：在 L1 输出上继续自注意力+FFN，学到更全局的语义；最终输出即源句记忆，供 Decoder 交叉注意力查阅。
残差：X3 = LN1b + AttnOut2（逐元素）
X3[0][0] = 0.2437 + -0.5231 = -0.2794
X3[0][1] = -1.698 + -1.5552 = -3.2532
X3[0][2] = 0.7259 + -0.854 = -0.1281
X3[0][3] = 0.7284 + 0.5367 = 1.2651
X3[1][0] = -0.5803 + -0.4961 = -1.0764
X3[1][1] = -1.2885 + -1.5071 = -2.7957
X3[1][2] = 0.5761 + -0.8627 = -0.2866
X3[1][3] = 1.2927 + 0.5474 = 1.8401
X3[2][0] = 0.0173 + -0.5155 = -0.4982
X3[2][1] = -1.4924 + -1.536 = -3.0284
X3[2][2] = 1.3225 + -0.8621 = 0.4605
X3[2][3] = 0.1526 + 0.5476 = 0.7002
X3[3][0] = -0.1908 + -0.5095 = -0.7004
X3[3][1] = -1.5269 + -1.5316 = -3.0585
X3[3][2] = 0.5786 + -0.8577 = -0.279
X3[3][3] = 1.1391 + 0.5411 = 1.6802
46 7.8 LayerNorm 归一化 LN2a Encoder L2
🎯 架构功能讲解Encoder 第 2 层：在 L1 输出上继续自注意力+FFN，学到更全局的语义；最终输出即源句记忆，供 Decoder 交叉注意力查阅。
LayerNorm：LN2a = LayerNorm(X3)
行0:
均值 μ = (-0.2794 + -3.2532 + -0.1281 + 1.2651)/4 = -0.5989
方差 σ² = ((0.3195)² + (-2.6543)² + (0.4708)² + (1.864)²)/4 = 2.7109
LN2a[0] = (-0.2794 - -0.5989)/√(2.7109+ε) = 0.194
LN2a[1] = (-3.2532 - -0.5989)/√(2.7109+ε) = -1.6121
LN2a[2] = (-0.1281 - -0.5989)/√(2.7109+ε) = 0.2859
LN2a[3] = (1.2651 - -0.5989)/√(2.7109+ε) = 1.1321
行1:
均值 μ = (-1.0764 + -2.7957 + -0.2866 + 1.8401)/4 = -0.5797
方差 σ² = ((-0.4967)² + (-2.216)² + (0.293)² + (2.4197)²)/4 = 2.7746
LN2a[0] = (-1.0764 - -0.5797)/√(2.7746+ε) = -0.2982
LN2a[1] = (-2.7957 - -0.5797)/√(2.7746+ε) = -1.3304
LN2a[2] = (-0.2866 - -0.5797)/√(2.7746+ε) = 0.1759
LN2a[3] = (1.8401 - -0.5797)/√(2.7746+ε) = 1.4527
行2:
均值 μ = (-0.4982 + -3.0284 + 0.4605 + 0.7002)/4 = -0.5915
方差 σ² = ((0.0933)² + (-2.4369)² + (1.052)² + (1.2917)²)/4 = 2.1806
LN2a[0] = (-0.4982 - -0.5915)/√(2.1806+ε) = 0.0632
LN2a[1] = (-3.0284 - -0.5915)/√(2.1806+ε) = -1.6503
LN2a[2] = (0.4605 - -0.5915)/√(2.1806+ε) = 0.7124
LN2a[3] = (0.7002 - -0.5915)/√(2.1806+ε) = 0.8747
行3:
均值 μ = (-0.7004 + -3.0585 + -0.279 + 1.6802)/4 = -0.5894
方差 σ² = ((-0.1109)² + (-2.4691)² + (0.3104)² + (2.2696)²)/4 = 2.839
LN2a[0] = (-0.7004 - -0.5894)/√(2.839+ε) = -0.0658
LN2a[1] = (-3.0585 - -0.5894)/√(2.839+ε) = -1.4654
LN2a[2] = (-0.279 - -0.5894)/√(2.839+ε) = 0.1842
LN2a[3] = (1.6802 - -0.5894)/√(2.839+ε) = 1.347
47 7.9 FFN 第一层 H2 = LN2a·W2_1ᵀ（逐元素） Encoder L2
🎯 架构功能讲解Encoder 第 2 层：在 L1 输出上继续自注意力+FFN，学到更全局的语义；最终输出即源句记忆，供 Decoder 交叉注意力查阅。
H2[0][0] = 0.194×0.8 + -1.6121×0.3 + 0.2859×-0.3 + 1.1321×-0.3 = -0.7538
H2[0][1] = 0.194×0.5 + -1.6121×0.8 + 0.2859×0.8 + 1.1321×0.6 = -0.2846
H2[0][2] = 0.194×0.3 + -1.6121×-0.8 + 0.2859×-0.7 + 1.1321×0.8 = 2.0534
H2[0][3] = 0.194×0.2 + -1.6121×-1 + 0.2859×-0.8 + 1.1321×0.3 = 1.7618
H2[0][4] = 0.194×-1 + -1.6121×-0.7 + 0.2859×0.1 + 1.1321×0.4 = 1.4159
H2[0][5] = 0.194×0.3 + -1.6121×-0.6 + 0.2859×0.4 + 1.1321×-0.5 = 0.5738
H2[0][6] = 0.194×-0.3 + -1.6121×0.5 + 0.2859×0.3 + 1.1321×0.7 = 0.014
H2[0][7] = 0.194×0.3 + -1.6121×0.1 + 0.2859×-0.8 + 1.1321×-0.3 = -0.6714
H2[1][0] = -0.2982×0.8 + -1.3304×0.3 + 0.1759×-0.3 + 1.4527×-0.3 = -1.1262
H2[1][1] = -0.2982×0.5 + -1.3304×0.8 + 0.1759×0.8 + 1.4527×0.6 = -0.2011
H2[1][2] = -0.2982×0.3 + -1.3304×-0.8 + 0.1759×-0.7 + 1.4527×0.8 = 2.0138
H2[1][3] = -0.2982×0.2 + -1.3304×-1 + 0.1759×-0.8 + 1.4527×0.3 = 1.5658
H2[1][4] = -0.2982×-1 + -1.3304×-0.7 + 0.1759×0.1 + 1.4527×0.4 = 1.8281
H2[1][5] = -0.2982×0.3 + -1.3304×-0.6 + 0.1759×0.4 + 1.4527×-0.5 = 0.0528
H2[1][6] = -0.2982×-0.3 + -1.3304×0.5 + 0.1759×0.3 + 1.4527×0.7 = 0.4939
H2[1][7] = -0.2982×0.3 + -1.3304×0.1 + 0.1759×-0.8 + 1.4527×-0.3 = -0.799
H2[2][0] = 0.0632×0.8 + -1.6503×0.3 + 0.7124×-0.3 + 0.8747×-0.3 = -0.9207
H2[2][1] = 0.0632×0.5 + -1.6503×0.8 + 0.7124×0.8 + 0.8747×0.6 = -0.1939
H2[2][2] = 0.0632×0.3 + -1.6503×-0.8 + 0.7124×-0.7 + 0.8747×0.8 = 1.5403
H2[2][3] = 0.0632×0.2 + -1.6503×-1 + 0.7124×-0.8 + 0.8747×0.3 = 1.3554
H2[2][4] = 0.0632×-1 + -1.6503×-0.7 + 0.7124×0.1 + 0.8747×0.4 = 1.5132
H2[2][5] = 0.0632×0.3 + -1.6503×-0.6 + 0.7124×0.4 + 0.8747×-0.5 = 0.8567
H2[2][6] = 0.0632×-0.3 + -1.6503×0.5 + 0.7124×0.3 + 0.8747×0.7 = -0.0181
H2[2][7] = 0.0632×0.3 + -1.6503×0.1 + 0.7124×-0.8 + 0.8747×-0.3 = -0.9784
H2[3][0] = -0.0658×0.8 + -1.4654×0.3 + 0.1842×-0.3 + 1.347×-0.3 = -0.9516
H2[3][1] = -0.0658×0.5 + -1.4654×0.8 + 0.1842×0.8 + 1.347×0.6 = -0.2496
H2[3][2] = -0.0658×0.3 + -1.4654×-0.8 + 0.1842×-0.7 + 1.347×0.8 = 2.1012
H2[3][3] = -0.0658×0.2 + -1.4654×-1 + 0.1842×-0.8 + 1.347×0.3 = 1.7089
H2[3][4] = -0.0658×-1 + -1.4654×-0.7 + 0.1842×0.1 + 1.347×0.4 = 1.6488
H2[3][5] = -0.0658×0.3 + -1.4654×-0.6 + 0.1842×0.4 + 1.347×-0.5 = 0.2597
H2[3][6] = -0.0658×-0.3 + -1.4654×0.5 + 0.1842×0.3 + 1.347×0.7 = 0.2852
H2[3][7] = -0.0658×0.3 + -1.4654×0.1 + 0.1842×-0.8 + 1.347×-0.3 = -0.7178
48 7.10 ReLU 激活 R2 = max(0, H2) Encoder L2
🎯 架构功能讲解Encoder 第 2 层：在 L1 输出上继续自注意力+FFN，学到更全局的语义；最终输出即源句记忆，供 Decoder 交叉注意力查阅。
R2[0][0] = max(0, -0.7538) = 0
R2[0][1] = max(0, -0.2846) = 0
R2[0][2] = max(0, 2.0534) = 2.0534
R2[0][3] = max(0, 1.7618) = 1.7618
R2[0][4] = max(0, 1.4159) = 1.4159
R2[0][5] = max(0, 0.5738) = 0.5738
R2[0][6] = max(0, 0.014) = 0.014
R2[0][7] = max(0, -0.6714) = 0
R2[1][0] = max(0, -1.1262) = 0
R2[1][1] = max(0, -0.2011) = 0
R2[1][2] = max(0, 2.0138) = 2.0138
R2[1][3] = max(0, 1.5658) = 1.5658
R2[1][4] = max(0, 1.8281) = 1.8281
R2[1][5] = max(0, 0.0528) = 0.0528
R2[1][6] = max(0, 0.4939) = 0.4939
R2[1][7] = max(0, -0.799) = 0
R2[2][0] = max(0, -0.9207) = 0
R2[2][1] = max(0, -0.1939) = 0
R2[2][2] = max(0, 1.5403) = 1.5403
R2[2][3] = max(0, 1.3554) = 1.3554
R2[2][4] = max(0, 1.5132) = 1.5132
R2[2][5] = max(0, 0.8567) = 0.8567
R2[2][6] = max(0, -0.0181) = 0
R2[2][7] = max(0, -0.9784) = 0
R2[3][0] = max(0, -0.9516) = 0
R2[3][1] = max(0, -0.2496) = 0
R2[3][2] = max(0, 2.1012) = 2.1012
R2[3][3] = max(0, 1.7089) = 1.7089
R2[3][4] = max(0, 1.6488) = 1.6488
R2[3][5] = max(0, 0.2597) = 0.2597
R2[3][6] = max(0, 0.2852) = 0.2852
R2[3][7] = max(0, -0.7178) = 0
49 7.11 FFN 第二层 F2 = R2·W2_2ᵀ（逐元素） Encoder L2
🎯 架构功能讲解Encoder 第 2 层：在 L1 输出上继续自注意力+FFN，学到更全局的语义；最终输出即源句记忆，供 Decoder 交叉注意力查阅。
F2[0][0] = 0×-0.5 + 0×-0.5 + 2.0534×0.9 + 1.7618×-0.2 + 1.4159×0.8 + 0.5738×0.3 + 0.014×0.6 + 0×0 = 2.809
F2[0][1] = 0×0.2 + 0×0 + 2.0534×-0.6 + 1.7618×0.4 + 1.4159×-0.4 + 0.5738×-1 + 0.014×0.3 + 0×-0.6 = -1.6633
F2[0][2] = 0×0.9 + 0×0.9 + 2.0534×0.8 + 1.7618×-0.3 + 1.4159×-1 + 0.5738×0.9 + 0.014×-0.1 + 0×0.9 = 0.2133
F2[0][3] = 0×0.9 + 0×0.7 + 2.0534×-0.4 + 1.7618×-0.2 + 1.4159×0.7 + 0.5738×-0.4 + 0.014×-0.7 + 0×0.1 = -0.4219
F2[1][0] = 0×-0.5 + 0×-0.5 + 2.0138×0.9 + 1.5658×-0.2 + 1.8281×0.8 + 0.0528×0.3 + 0.4939×0.6 + 0×0 = 3.274
F2[1][1] = 0×0.2 + 0×0 + 2.0138×-0.6 + 1.5658×0.4 + 1.8281×-0.4 + 0.0528×-1 + 0.4939×0.3 + 0×-0.6 = -1.2179
F2[1][2] = 0×0.9 + 0×0.9 + 2.0138×0.8 + 1.5658×-0.3 + 1.8281×-1 + 0.0528×0.9 + 0.4939×-0.1 + 0×0.9 = -0.6887
F2[1][3] = 0×0.9 + 0×0.7 + 2.0138×-0.4 + 1.5658×-0.2 + 1.8281×0.7 + 0.0528×-0.4 + 0.4939×-0.7 + 0×0.1 = -0.2059
F2[2][0] = 0×-0.5 + 0×-0.5 + 1.5403×0.9 + 1.3554×-0.2 + 1.5132×0.8 + 0.8567×0.3 + 0×0.6 + 0×0 = 2.5827
F2[2][1] = 0×0.2 + 0×0 + 1.5403×-0.6 + 1.3554×0.4 + 1.5132×-0.4 + 0.8567×-1 + 0×0.3 + 0×-0.6 = -1.844
F2[2][2] = 0×0.9 + 0×0.9 + 1.5403×0.8 + 1.3554×-0.3 + 1.5132×-1 + 0.8567×0.9 + 0×-0.1 + 0×0.9 = 0.0835
F2[2][3] = 0×0.9 + 0×0.7 + 1.5403×-0.4 + 1.3554×-0.2 + 1.5132×0.7 + 0.8567×-0.4 + 0×-0.7 + 0×0.1 = -0.1707
F2[3][0] = 0×-0.5 + 0×-0.5 + 2.1012×0.9 + 1.7089×-0.2 + 1.6488×0.8 + 0.2597×0.3 + 0.2852×0.6 + 0×0 = 3.1174
F2[3][1] = 0×0.2 + 0×0 + 2.1012×-0.6 + 1.7089×0.4 + 1.6488×-0.4 + 0.2597×-1 + 0.2852×0.3 + 0×-0.6 = -1.4108
F2[3][2] = 0×0.9 + 0×0.9 + 2.1012×0.8 + 1.7089×-0.3 + 1.6488×-1 + 0.2597×0.9 + 0.2852×-0.1 + 0×0.9 = -0.2754
F2[3][3] = 0×0.9 + 0×0.7 + 2.1012×-0.4 + 1.7089×-0.2 + 1.6488×0.7 + 0.2597×-0.4 + 0.2852×-0.7 + 0×0.1 = -0.3316
50 7.12 残差连接 X4 = LN2a + F2 Encoder L2
🎯 架构功能讲解Encoder 第 2 层：在 L1 输出上继续自注意力+FFN，学到更全局的语义；最终输出即源句记忆，供 Decoder 交叉注意力查阅。
残差：X4 = LN2a + F2（逐元素）
X4[0][0] = 0.194 + 2.809 = 3.003
X4[0][1] = -1.6121 + -1.6633 = -3.2754
X4[0][2] = 0.2859 + 0.2133 = 0.4993
X4[0][3] = 1.1321 + -0.4219 = 0.7102
X4[1][0] = -0.2982 + 3.274 = 2.9758
X4[1][1] = -1.3304 + -1.2179 = -2.5482
X4[1][2] = 0.1759 + -0.6887 = -0.5127
X4[1][3] = 1.4527 + -0.2059 = 1.2468
X4[2][0] = 0.0632 + 2.5827 = 2.6458
X4[2][1] = -1.6503 + -1.844 = -3.4942
X4[2][2] = 0.7124 + 0.0835 = 0.7959
X4[2][3] = 0.8747 + -0.1707 = 0.7041
X4[3][0] = -0.0658 + 3.1174 = 3.0515
X4[3][1] = -1.4654 + -1.4108 = -2.8761
X4[3][2] = 0.1842 + -0.2754 = -0.0912
X4[3][3] = 1.347 + -0.3316 = 1.0154
51 7.13 ⭐ LayerNorm 归一化 → ENCODER 最终输出（= 给 Cross-Attn 的 K/V） Encoder L2
🎯 架构功能讲解Encoder 完成！这 4 个向量就是「我爱水课」的完整记忆。Decoder 的交叉注意力将反复查阅它们。
LayerNorm：ENC_OUT = LayerNorm(X4)
行0（我）:
均值 μ = (3.003 + -3.2754 + 0.4993 + 0.7102)/4 = 0.2343
方差 σ² = ((2.7687)² + (-3.5096)² + (0.265)² + (0.4759)²)/4 = 5.0701
ENC_OUT[0] = (3.003 - 0.2343)/√(5.0701+ε) = 1.2296
ENC_OUT[1] = (-3.2754 - 0.2343)/√(5.0701+ε) = -1.5587
ENC_OUT[2] = (0.4993 - 0.2343)/√(5.0701+ε) = 0.1177
ENC_OUT[3] = (0.7102 - 0.2343)/√(5.0701+ε) = 0.2114
行1（爱）:
均值 μ = (2.9758 + -2.5482 + -0.5127 + 1.2468)/4 = 0.2904
方差 σ² = ((2.6854)² + (-2.8386)² + (-0.8031)² + (0.9564)²)/4 = 4.2072
ENC_OUT[0] = (2.9758 - 0.2904)/√(4.2072+ε) = 1.3092
ENC_OUT[1] = (-2.5482 - 0.2904)/√(4.2072+ε) = -1.3839
ENC_OUT[2] = (-0.5127 - 0.2904)/√(4.2072+ε) = -0.3915
ENC_OUT[3] = (1.2468 - 0.2904)/√(4.2072+ε) = 0.4663
行2（水）:
均值 μ = (2.6458 + -3.4942 + 0.7959 + 0.7041)/4 = 0.1629
方差 σ² = ((2.483)² + (-3.6571)² + (0.633)² + (0.5412)²)/4 = 5.0583
ENC_OUT[0] = (2.6458 - 0.1629)/√(5.0583+ε) = 1.104
ENC_OUT[1] = (-3.4942 - 0.1629)/√(5.0583+ε) = -1.6261
ENC_OUT[2] = (0.7959 - 0.1629)/√(5.0583+ε) = 0.2814
ENC_OUT[3] = (0.7041 - 0.1629)/√(5.0583+ε) = 0.2406
行3（课）:
均值 μ = (3.0515 + -2.8761 + -0.0912 + 1.0154)/4 = 0.2749
方差 σ² = ((2.7766)² + (-3.151)² + (-0.3661)² + (0.7405)²)/4 = 4.5803
ENC_OUT[0] = (3.0515 - 0.2749)/√(4.5803+ε) = 1.2974
ENC_OUT[1] = (-2.8761 - 0.2749)/√(4.5803+ε) = -1.4723
ENC_OUT[2] = (-0.0912 - 0.2749)/√(4.5803+ε) = -0.171
ENC_OUT[3] = (1.0154 - 0.2749)/√(4.5803+ε) = 0.346
52 8.1 定义 Decoder 全部权重（Decoder 有自己的独立参数，不与 Encoder 共享） Decoder 权重
🎯 架构功能讲解Decoder 参数：与 Encoder 完全独立的一组权重，训练时一起更新，推理时冻结。
WQd1a 行0: [0.9, 0.4, 0.1, -0.8]
WQd1a 行1: [0.2, 1, -0.7, 0]
WKd1a 行0: [0.8, 0.5, 0.4, 0.4]
WKd1a 行1: [-0.3, -0.4, 0.6, 0.6]
WVd1a 行0: [0.7, 0.8, 0, 0]
WVd1a 行1: [0.6, 0.3, 0.4, 0.6]
WQd1b 行0: [0.8, -0.3, -0.2, -0.8]
WQd1b 行1: [0.2, -0.9, -0.1, 0.1]
WKd1b 行0: [-0.4, 0.2, -0.9, -0.9]
WKd1b 行1: [0.6, -0.3, -0.7, 0]
WVd1b 行0: [0.5, -0.6, 0.2, -0.8]
WVd1b 行1: [-0.9, 0.1, 0.1, 0.3]
WOd1 行0: [0.5, 1, 0, -0.4]
WOd1 行1: [0.6, -0.5, -0.1, -0.8]
WOd1 行2: [-0.9, 0.9, 0.7, 0.4]
WOd1 行3: [-0.2, -0.7, -0.7, -0.5]
53 8.2 Decoder Layer2 + Cross-Attn + 输出层权重 Decoder 权重
🎯 架构功能讲解Decoder 参数：与 Encoder 完全独立的一组权重，训练时一起更新，推理时冻结。
WQd2a 行0: [0.5, 0.6, -0.4, -0.6]
WQd2a 行1: [0.5, 0.6, 1, -0.2]
WKd2a 行0: [-0.3, 0.6, -0.3, 0.9]
WKd2a 行1: [0.7, -0.1, 0.5, 0.5]
WVd2a 行0: [-0.8, 0.8, 0, 0.7]
WVd2a 行1: [-0.4, 0.8, -0.2, -1]
WQc2a 行0: [0.9, 0.7, -0.9, -0.9]
WQc2a 行1: [-0.2, 0.6, 1, -0.7]
WKc2a 行0: [0.2, -0.2, 0.9, 0.7]
WKc2a 行1: [0.7, -0.1, -0.2, -0.5]
WVc2a 行0: [-0.9, 0.7, 0.6, 1]
WVc2a 行1: [1, 0.1, 0.5, 0.9]
WOc2 行0: [0.4, -0.6, -0.7, -1]
WOc2 行1: [-0.3, 0.2, -0.2, -0.1]
WOc2 行2: [0.8, -0.3, 0, 0.6]
WOc2 行3: [-0.2, 0.2, 0.7, 0.9]
54 8.3 掩码矩阵 M（Decoder 自注意力专用） Decoder 权重
🎯 架构功能讲解为什么掩码？生成是自回归的：写第 3 个词时绝不能看到第 4 个词，否则训练时「抄答案」、推理时无答案可抄。softmax 后 e^(-∞)=0，被遮位置的注意力强制归零。
M[i][j] = 0（j ≤ i，允许看自己及左边）
M[i][j] = -1e9 ≈ -∞（j > i，禁止看未来）
矩阵（上三角 = 红/禁止）：
[ 0, -∞, -∞, -∞]
[ 0, 0, -∞, -∞]
[ 0, 0, 0, -∞]
[ 0, 0, 0, 0]
55 9.1 计算 Q/K/V（输入 = X_dec_in = I love easy courses 的嵌入+位置） Decoder L1(训练)
🎯 架构功能讲解与 Encoder 自注意力唯一区别：这是目标语言（英文），且分数要加掩码。
Q1[0] = [-0.1, 1.72]
Q1[1] = [1.0645, 0.3216]
Q1[2] = [-0.8959, -0.0883]
Q1[3] = [-1.3356, -1.8428]
K1[0] = [1.51, -0.06]
K1[1] = [1.9773, 0.1474]
K1[2] = [1.0473, 0.9755]
K1[3] = [0.0497, 1.5414]
V1[0] = [1.48, 1.39]
V1[1] = [1.4313, 1.7209]
V1[2] = [0.2736, 1.2886]
V1[3] = [-1.1032, 0.5594]
Q2[0] = [-1.13, -1.12]
Q2[1] = [0.4191, -0.079]
Q2[2] = [-0.9916, 0.2344]
Q2[3] = [-0.3757, 1.5562]
K2[0] = [-0.85, -0.07]
K2[1] = [-1.7075, 0.3658]
K2[2] = [-1.8048, 0.0964]
K2[3] = [-1.881, 0.3207]
V2[0] = [-1.78, 0.16]
V2[1] = [0.0686, -1.0723]
V2[2] = [-1.0215, 0.132]
V2[3] = [0.2009, 0.0269]
56 9.2 掩码注意力分数 score = Q·Kᵀ/√2 + M（上三角被 -1e9 覆盖） Decoder L1(训练)
🎯 架构功能讲解对比 Encoder：这里上三角全部变成 ≈-∞，softmax 后这些位置权重必为 0。
score1[0][0] = (-0.2542)/1.4142 = -0.1797
score1[0][1] = (0.0558)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[0][2] = (1.5732)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[0][3] = (2.6462)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[1][0] = (1.5881)/1.4142 = 1.1229
score1[1][1] = (2.1522)/1.4142 = 1.5218
score1[1][2] = (1.4286)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[1][3] = (0.5486)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[2][0] = (-1.3476)/1.4142 = -0.9529
score1[2][1] = (-1.7845)/1.4142 = -1.2619
score1[2][2] = (-1.0244)/1.4142 = -0.7244
score1[2][3] = (-0.1806)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[3][0] = (-1.9062)/1.4142 = -1.3479
score1[3][1] = (-2.9126)/1.4142 = -2.0595
score1[3][2] = (-3.1965)/1.4142 = -2.2603
score1[3][3] = (-2.9068)/1.4142 = -2.0554
score2[0][0] = (1.0389)/1.4142 = 0.7346
score2[0][1] = (1.5198)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[0][2] = (1.9314)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[0][3] = (1.7664)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[1][0] = (-0.3507)/1.4142 = -0.248
score2[1][1] = (-0.7445)/1.4142 = -0.5265
score2[1][2] = (-0.764)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[1][3] = (-0.8137)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[2][0] = (0.8264)/1.4142 = 0.5844
score2[2][1] = (1.7788)/1.4142 = 1.2578
score2[2][2] = (1.8121)/1.4142 = 1.2814
score2[2][3] = (1.9403)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[3][0] = (0.2105)/1.4142 = 0.1488
score2[3][1] = (1.2108)/1.4142 = 0.8562
score2[3][2] = (0.8282)/1.4142 = 0.5856
score2[3][3] = (1.2058)/1.4142 = 0.8526
57 9.3 掩码 softmax 注意力权重 A Decoder L1(训练)
🎯 架构功能讲解关键观察：第 1 行（I）=[1,0,0,0] 只能看自己；第 2 行（love）=[0.40,0.60,0,0] 只能看 I 和 love；第 4 行（courses）无限制。每行权重和恒为 1。
掩码 softmax 权重 A（每行和为 1，被掩码处 = 0）
A1 行0（I）: [1, 0, 0, 0]
A1 行1（love）: [0.4016, 0.5984, 0, 0]
A1 行2（easy）: [0.3343, 0.2455, 0.4202, 0]
A1 行3（courses）: [0.4192, 0.2058, 0.1684, 0.2066]
A2 行0（I）: [1, 0, 0, 0]
A2 行1（love）: [0.5692, 0.4308, 0, 0]
A2 行2（easy）: [0.2013, 0.3947, 0.4041, 0]
A2 行3（courses）: [0.1516, 0.3075, 0.2346, 0.3064]
58 9.4 加权求和 O = A·V（2 头） Decoder L1(训练)
🎯 架构功能讲解Decoder 第 1 层（训练模式）：掩码自注意力约束「只能看左边」，保证自回归因果性；FFN 加工目标侧表示。
O1[0] = [1.48, 1.39]
O1[1] = [1.4508, 1.588]
O1[2] = [0.9611, 1.4286]
O1[3] = [0.7331, 1.2694]
O2[0] = [-1.78, 0.16]
O2[1] = [-0.9836, -0.3709]
O2[2] = [-0.7439, -0.3377]
O2[3] = [-0.4268, -0.2663]
59 9.5 拼接 + 输出投影 DAttnOut1 = Concat·WOd1ᵀ Decoder L1(训练)
🎯 架构功能讲解Decoder 第 1 层（训练模式）：掩码自注意力约束「只能看左边」，保证自回归因果性；FFN 加工目标侧表示。
Concat[0] = [1.48, 1.39, -1.78, 0.16]
Concat[1] = [1.4508, 1.588, -0.9836, -0.3709]
Concat[2] = [0.9611, 1.4286, -0.7439, -0.3377]
Concat[3] = [0.7331, 1.2694, -0.4268, -0.2663]
DAttnOut1[0][0] = 1.48×0.5 + 1.39×1 + -1.78×0 + 0.16×-0.4 = 2.066
DAttnOut1[0][1] = 1.48×0.6 + 1.39×-0.5 + -1.78×-0.1 + 0.16×-0.8 = 0.243
DAttnOut1[0][2] = 1.48×-0.9 + 1.39×0.9 + -1.78×0.7 + 0.16×0.4 = -1.263
DAttnOut1[0][3] = 1.48×-0.2 + 1.39×-0.7 + -1.78×-0.7 + 0.16×-0.5 = -0.103
DAttnOut1[1][0] = 1.4508×0.5 + 1.588×1 + -0.9836×0 + -0.3709×-0.4 = 2.4618
DAttnOut1[1][1] = 1.4508×0.6 + 1.588×-0.5 + -0.9836×-0.1 + -0.3709×-0.8 = 0.4716
DAttnOut1[1][2] = 1.4508×-0.9 + 1.588×0.9 + -0.9836×0.7 + -0.3709×0.4 = -0.7134
DAttnOut1[1][3] = 1.4508×-0.2 + 1.588×-0.7 + -0.9836×-0.7 + -0.3709×-0.5 = -0.5278
DAttnOut1[2][0] = 0.9611×0.5 + 1.4286×1 + -0.7439×0 + -0.3377×-0.4 = 2.0443
DAttnOut1[2][1] = 0.9611×0.6 + 1.4286×-0.5 + -0.7439×-0.1 + -0.3377×-0.8 = 0.2069
DAttnOut1[2][2] = 0.9611×-0.9 + 1.4286×0.9 + -0.7439×0.7 + -0.3377×0.4 = -0.2351
DAttnOut1[2][3] = 0.9611×-0.2 + 1.4286×-0.7 + -0.7439×-0.7 + -0.3377×-0.5 = -0.5027
DAttnOut1[3][0] = 0.7331×0.5 + 1.2694×1 + -0.4268×0 + -0.2663×-0.4 = 1.7425
DAttnOut1[3][1] = 0.7331×0.6 + 1.2694×-0.5 + -0.4268×-0.1 + -0.2663×-0.8 = 0.0609
DAttnOut1[3][2] = 0.7331×-0.9 + 1.2694×0.9 + -0.4268×0.7 + -0.2663×0.4 = 0.0774
DAttnOut1[3][3] = 0.7331×-0.2 + 1.2694×-0.7 + -0.4268×-0.7 + -0.2663×-0.5 = -0.6033
60 9.6 残差连接 DX1 = X_dec_in + DAttnOut1 Decoder L1(训练)
🎯 架构功能讲解Decoder 第 1 层（训练模式）：掩码自注意力约束「只能看左边」，保证自回归因果性；FFN 加工目标侧表示。
残差：DX1 = X_dec_in + DAttnOut1（逐元素）
DX1[0][0] = 0.4 + 2.066 = 2.466
DX1[0][1] = 1.5 + 0.243 = 1.743
DX1[0][2] = -0.2 + -1.263 = -1.463
DX1[0][3] = 1.3 + -0.103 = 1.197
DX1[1][0] = 1.5415 + 2.4618 = 4.0033
DX1[1][1] = 0.4403 + 0.4716 = 0.9119
DX1[1][2] = 0.61 + -0.7134 = -0.1034
DX1[1][3] = 0.7 + -0.5278 = 0.1721
DX1[2][0] = 0.4093 + 2.0443 = 2.4536
DX1[2][1] = -0.0161 + 0.2069 = 0.1908
DX1[2][2] = 0.22 + -0.2351 = -0.0151
DX1[2][3] = 1.5998 + -0.5027 = 1.0971
DX1[3][0] = 0.2411 + 1.7425 = 1.9836
DX1[3][1] = -1.59 + 0.0609 = -1.5291
DX1[3][2] = 0.43 + 0.0774 = 0.5074
DX1[3][3] = 1.1996 + -0.6033 = 0.5962
61 9.7 LayerNorm 归一化 DLN1a Decoder L1(训练)
🎯 架构功能讲解Decoder 第 1 层（训练模式）：掩码自注意力约束「只能看左边」，保证自回归因果性；FFN 加工目标侧表示。
LayerNorm：DLN1a = LayerNorm(DX1)
行0:
均值 μ = (2.466 + 1.743 + -1.463 + 1.197)/4 = 0.9857
方差 σ² = ((1.4802)² + (0.7573)² + (-2.4487)² + (0.2113)²)/4 = 2.2014
DLN1a[0] = (2.466 - 0.9857)/√(2.2014+ε) = 0.9977
DLN1a[1] = (1.743 - 0.9857)/√(2.2014+ε) = 0.5104
DLN1a[2] = (-1.463 - 0.9857)/√(2.2014+ε) = -1.6504
DLN1a[3] = (1.197 - 0.9857)/√(2.2014+ε) = 0.1424
行1:
均值 μ = (4.0033 + 0.9119 + -0.1034 + 0.1721)/4 = 1.246
方差 σ² = ((2.7573)² + (-0.3341)² + (-1.3494)² + (-1.0739)²)/4 = 2.6721
DLN1a[0] = (4.0033 - 1.246)/√(2.6721+ε) = 1.6868
DLN1a[1] = (0.9119 - 1.246)/√(2.6721+ε) = -0.2044
DLN1a[2] = (-0.1034 - 1.246)/√(2.6721+ε) = -0.8255
DLN1a[3] = (0.1721 - 1.246)/√(2.6721+ε) = -0.6569
行2:
均值 μ = (2.4536 + 0.1908 + -0.0151 + 1.0971)/4 = 0.9316
方差 σ² = ((1.522)² + (-0.7408)² + (-0.9467)² + (0.1655)²)/4 = 0.9472
DLN1a[0] = (2.4536 - 0.9316)/√(0.9472+ε) = 1.5638
DLN1a[1] = (0.1908 - 0.9316)/√(0.9472+ε) = -0.7612
DLN1a[2] = (-0.0151 - 0.9316)/√(0.9472+ε) = -0.9727
DLN1a[3] = (1.0971 - 0.9316)/√(0.9472+ε) = 0.1701
行3:
均值 μ = (1.9836 + -1.5291 + 0.5074 + 0.5962)/4 = 0.3895
方差 σ² = ((1.5941)² + (-1.9187)² + (0.1179)² + (0.2067)²)/4 = 1.5697
DLN1a[0] = (1.9836 - 0.3895)/√(1.5697+ε) = 1.2723
DLN1a[1] = (-1.5291 - 0.3895)/√(1.5697+ε) = -1.5314
DLN1a[2] = (0.5074 - 0.3895)/√(1.5697+ε) = 0.0941
DLN1a[3] = (0.5962 - 0.3895)/√(1.5697+ε) = 0.165
62 9.8 FFN 第一层 DH1 = DLN1a·Wd1_1ᵀ（逐元素） Decoder L1(训练)
🎯 架构功能讲解Decoder 第 1 层（训练模式）：掩码自注意力约束「只能看左边」，保证自回归因果性；FFN 加工目标侧表示。
DH1[0][0] = 0.9977×0.1 + 0.5104×0.4 + -1.6504×0.3 + 0.1424×-0.4 = -0.2482
DH1[0][1] = 0.9977×0.9 + 0.5104×0.5 + -1.6504×0.1 + 0.1424×0.2 = 1.0165
DH1[0][2] = 0.9977×-0.2 + 0.5104×-0.5 + -1.6504×-0.3 + 0.1424×0.5 = 0.1116
DH1[0][3] = 0.9977×-1 + 0.5104×-0.8 + -1.6504×-0.9 + 0.1424×-0.9 = -0.0487
DH1[0][4] = 0.9977×0.7 + 0.5104×0.4 + -1.6504×-0.1 + 0.1424×-0.8 = 0.9537
DH1[0][5] = 0.9977×0 + 0.5104×-0.1 + -1.6504×-0.7 + 0.1424×-0.1 = 1.09
DH1[0][6] = 0.9977×-0.2 + 0.5104×0.2 + -1.6504×0.3 + 0.1424×-0.9 = -0.7207
DH1[0][7] = 0.9977×-0.3 + 0.5104×0.3 + -1.6504×0 + 0.1424×0.7 = -0.0465
DH1[1][0] = 1.6868×0.1 + -0.2044×0.4 + -0.8255×0.3 + -0.6569×-0.4 = 0.1021
DH1[1][1] = 1.6868×0.9 + -0.2044×0.5 + -0.8255×0.1 + -0.6569×0.2 = 1.202
DH1[1][2] = 1.6868×-0.2 + -0.2044×-0.5 + -0.8255×-0.3 + -0.6569×0.5 = -0.316
DH1[1][3] = 1.6868×-1 + -0.2044×-0.8 + -0.8255×-0.9 + -0.6569×-0.9 = -0.1891
DH1[1][4] = 1.6868×0.7 + -0.2044×0.4 + -0.8255×-0.1 + -0.6569×-0.8 = 1.7071
DH1[1][5] = 1.6868×0 + -0.2044×-0.1 + -0.8255×-0.7 + -0.6569×-0.1 = 0.664
DH1[1][6] = 1.6868×-0.2 + -0.2044×0.2 + -0.8255×0.3 + -0.6569×-0.9 = -0.0346
DH1[1][7] = 1.6868×-0.3 + -0.2044×0.3 + -0.8255×0 + -0.6569×0.7 = -1.0272
DH1[2][0] = 1.5638×0.1 + -0.7612×0.4 + -0.9727×0.3 + 0.1701×-0.4 = -0.5079
DH1[2][1] = 1.5638×0.9 + -0.7612×0.5 + -0.9727×0.1 + 0.1701×0.2 = 0.9636
DH1[2][2] = 1.5638×-0.2 + -0.7612×-0.5 + -0.9727×-0.3 + 0.1701×0.5 = 0.4447
DH1[2][3] = 1.5638×-1 + -0.7612×-0.8 + -0.9727×-0.9 + 0.1701×-0.9 = -0.2325
DH1[2][4] = 1.5638×0.7 + -0.7612×0.4 + -0.9727×-0.1 + 0.1701×-0.8 = 0.7514
DH1[2][5] = 1.5638×0 + -0.7612×-0.1 + -0.9727×-0.7 + 0.1701×-0.1 = 0.74
DH1[2][6] = 1.5638×-0.2 + -0.7612×0.2 + -0.9727×0.3 + 0.1701×-0.9 = -0.9099
DH1[2][7] = 1.5638×-0.3 + -0.7612×0.3 + -0.9727×0 + 0.1701×0.7 = -0.5784
DH1[3][0] = 1.2723×0.1 + -1.5314×0.4 + 0.0941×0.3 + 0.165×-0.4 = -0.5231
DH1[3][1] = 1.2723×0.9 + -1.5314×0.5 + 0.0941×0.1 + 0.165×0.2 = 0.4218
DH1[3][2] = 1.2723×-0.2 + -1.5314×-0.5 + 0.0941×-0.3 + 0.165×0.5 = 0.5655
DH1[3][3] = 1.2723×-1 + -1.5314×-0.8 + 0.0941×-0.9 + 0.165×-0.9 = -0.2804
DH1[3][4] = 1.2723×0.7 + -1.5314×0.4 + 0.0941×-0.1 + 0.165×-0.8 = 0.1367
DH1[3][5] = 1.2723×0 + -1.5314×-0.1 + 0.0941×-0.7 + 0.165×-0.1 = 0.0708
DH1[3][6] = 1.2723×-0.2 + -1.5314×0.2 + 0.0941×0.3 + 0.165×-0.9 = -0.681
DH1[3][7] = 1.2723×-0.3 + -1.5314×0.3 + 0.0941×0 + 0.165×0.7 = -0.7256
63 9.9 ReLU 激活 DR1 = max(0, DH1) Decoder L1(训练)
🎯 架构功能讲解Decoder 第 1 层（训练模式）：掩码自注意力约束「只能看左边」，保证自回归因果性；FFN 加工目标侧表示。
DR1[0][0] = max(0, -0.2482) = 0
DR1[0][1] = max(0, 1.0165) = 1.0165
DR1[0][2] = max(0, 0.1116) = 0.1116
DR1[0][3] = max(0, -0.0487) = 0
DR1[0][4] = max(0, 0.9537) = 0.9537
DR1[0][5] = max(0, 1.09) = 1.09
DR1[0][6] = max(0, -0.7207) = 0
DR1[0][7] = max(0, -0.0465) = 0
DR1[1][0] = max(0, 0.1021) = 0.1021
DR1[1][1] = max(0, 1.202) = 1.202
DR1[1][2] = max(0, -0.316) = 0
DR1[1][3] = max(0, -0.1891) = 0
DR1[1][4] = max(0, 1.7071) = 1.7071
DR1[1][5] = max(0, 0.664) = 0.664
DR1[1][6] = max(0, -0.0346) = 0
DR1[1][7] = max(0, -1.0272) = 0
DR1[2][0] = max(0, -0.5079) = 0
DR1[2][1] = max(0, 0.9636) = 0.9636
DR1[2][2] = max(0, 0.4447) = 0.4447
DR1[2][3] = max(0, -0.2325) = 0
DR1[2][4] = max(0, 0.7514) = 0.7514
DR1[2][5] = max(0, 0.74) = 0.74
DR1[2][6] = max(0, -0.9099) = 0
DR1[2][7] = max(0, -0.5784) = 0
DR1[3][0] = max(0, -0.5231) = 0
DR1[3][1] = max(0, 0.4218) = 0.4218
DR1[3][2] = max(0, 0.5655) = 0.5655
DR1[3][3] = max(0, -0.2804) = 0
DR1[3][4] = max(0, 0.1367) = 0.1367
DR1[3][5] = max(0, 0.0708) = 0.0708
DR1[3][6] = max(0, -0.681) = 0
DR1[3][7] = max(0, -0.7256) = 0
64 9.10 FFN 第二层 DF1 = DR1·Wd1_2ᵀ（逐元素） Decoder L1(训练)
🎯 架构功能讲解Decoder 第 1 层（训练模式）：掩码自注意力约束「只能看左边」，保证自回归因果性；FFN 加工目标侧表示。
DF1[0][0] = 0×0.3 + 1.0165×-0.7 + 0.1116×-0.9 + 0×0.3 + 0.9537×-0.9 + 1.09×0.2 + 0×0.9 + 0×0.2 = -1.4523
DF1[0][1] = 0×-0.2 + 1.0165×0.3 + 0.1116×-0.1 + 0×0.1 + 0.9537×0.9 + 1.09×-0.2 + 0×0.9 + 0×0.8 = 0.9341
DF1[0][2] = 0×-0.6 + 1.0165×-0.9 + 0.1116×-0.8 + 0×-1 + 0.9537×-0.8 + 1.09×0.4 + 0×-0.9 + 0×-0.4 = -1.3311
DF1[0][3] = 0×0.7 + 1.0165×-1 + 0.1116×0.6 + 0×-0.4 + 0.9537×-0.8 + 1.09×0.4 + 0×0.3 + 0×0.8 = -1.2765
DF1[1][0] = 0.1021×0.3 + 1.202×-0.7 + 0×-0.9 + 0×0.3 + 1.7071×-0.9 + 0.664×0.2 + 0×0.9 + 0×0.2 = -2.2144
DF1[1][1] = 0.1021×-0.2 + 1.202×0.3 + 0×-0.1 + 0×0.1 + 1.7071×0.9 + 0.664×-0.2 + 0×0.9 + 0×0.8 = 1.7438
DF1[1][2] = 0.1021×-0.6 + 1.202×-0.9 + 0×-0.8 + 0×-1 + 1.7071×-0.8 + 0.664×0.4 + 0×-0.9 + 0×-0.4 = -2.2431
DF1[1][3] = 0.1021×0.7 + 1.202×-1 + 0×0.6 + 0×-0.4 + 1.7071×-0.8 + 0.664×0.4 + 0×0.3 + 0×0.8 = -2.2306
DF1[2][0] = 0×0.3 + 0.9636×-0.7 + 0.4447×-0.9 + 0×0.3 + 0.7514×-0.9 + 0.74×0.2 + 0×0.9 + 0×0.2 = -1.603
DF1[2][1] = 0×-0.2 + 0.9636×0.3 + 0.4447×-0.1 + 0×0.1 + 0.7514×0.9 + 0.74×-0.2 + 0×0.9 + 0×0.8 = 0.7729
DF1[2][2] = 0×-0.6 + 0.9636×-0.9 + 0.4447×-0.8 + 0×-1 + 0.7514×-0.8 + 0.74×0.4 + 0×-0.9 + 0×-0.4 = -1.5281
DF1[2][3] = 0×0.7 + 0.9636×-1 + 0.4447×0.6 + 0×-0.4 + 0.7514×-0.8 + 0.74×0.4 + 0×0.3 + 0×0.8 = -1.0019
DF1[3][0] = 0×0.3 + 0.4218×-0.7 + 0.5655×-0.9 + 0×0.3 + 0.1367×-0.9 + 0.0708×0.2 + 0×0.9 + 0×0.2 = -0.913
DF1[3][1] = 0×-0.2 + 0.4218×0.3 + 0.5655×-0.1 + 0×0.1 + 0.1367×0.9 + 0.0708×-0.2 + 0×0.9 + 0×0.8 = 0.1788
DF1[3][2] = 0×-0.6 + 0.4218×-0.9 + 0.5655×-0.8 + 0×-1 + 0.1367×-0.8 + 0.0708×0.4 + 0×-0.9 + 0×-0.4 = -0.913
DF1[3][3] = 0×0.7 + 0.4218×-1 + 0.5655×0.6 + 0×-0.4 + 0.1367×-0.8 + 0.0708×0.4 + 0×0.3 + 0×0.8 = -0.1635
65 9.11 残差连接 DX2 = DLN1a + DF1 Decoder L1(训练)
🎯 架构功能讲解Decoder 第 1 层（训练模式）：掩码自注意力约束「只能看左边」，保证自回归因果性；FFN 加工目标侧表示。
残差：DX2 = DLN1a + DF1（逐元素）
DX2[0][0] = 0.9977 + -1.4523 = -0.4546
DX2[0][1] = 0.5104 + 0.9341 = 1.4445
DX2[0][2] = -1.6504 + -1.3311 = -2.9815
DX2[0][3] = 0.1424 + -1.2765 = -1.1341
DX2[1][0] = 1.6868 + -2.2144 = -0.5276
DX2[1][1] = -0.2044 + 1.7438 = 1.5394
DX2[1][2] = -0.8255 + -2.2431 = -3.0686
DX2[1][3] = -0.6569 + -2.2306 = -2.8876
DX2[2][0] = 1.5638 + -1.603 = -0.0392
DX2[2][1] = -0.7612 + 0.7729 = 0.0117
DX2[2][2] = -0.9727 + -1.5281 = -2.5008
DX2[2][3] = 0.1701 + -1.0019 = -0.8318
DX2[3][0] = 1.2723 + -0.913 = 0.3593
DX2[3][1] = -1.5314 + 0.1788 = -1.3525
DX2[3][2] = 0.0941 + -0.913 = -0.8189
DX2[3][3] = 0.165 + -0.1635 = 0.0014
66 9.12 LayerNorm 归一化 DLN1b（Decoder L1 输出） Decoder L1(训练)
🎯 架构功能讲解Decoder 第 1 层（训练模式）：掩码自注意力约束「只能看左边」，保证自回归因果性；FFN 加工目标侧表示。
LayerNorm：DLN1b = LayerNorm(DX2)（Decoder L1 最终输出）
行0:
均值 μ = (-0.4546 + 1.4445 + -2.9815 + -1.1341)/4 = -0.7814
方差 σ² = ((0.3268)² + (2.2259)² + (-2.2)² + (-0.3527)²)/4 = 2.5065
DLN1b[0] = (-0.4546 - -0.7814)/√(2.5065+ε) = 0.2064
DLN1b[1] = (1.4445 - -0.7814)/√(2.5065+ε) = 1.406
DLN1b[2] = (-2.9815 - -0.7814)/√(2.5065+ε) = -1.3896
DLN1b[3] = (-1.1341 - -0.7814)/√(2.5065+ε) = -0.2228
行1:
均值 μ = (-0.5276 + 1.5394 + -3.0686 + -2.8876)/4 = -1.2361
方差 σ² = ((0.7085)² + (2.7755)² + (-1.8325)² + (-1.6515)²)/4 = 3.5727
DLN1b[0] = (-0.5276 - -1.2361)/√(3.5727+ε) = 0.3748
DLN1b[1] = (1.5394 - -1.2361)/√(3.5727+ε) = 1.4684
DLN1b[2] = (-3.0686 - -1.2361)/√(3.5727+ε) = -0.9695
DLN1b[3] = (-2.8876 - -1.2361)/√(3.5727+ε) = -0.8737
行2:
均值 μ = (-0.0392 + 0.0117 + -2.5008 + -0.8318)/4 = -0.84
方差 σ² = ((0.8009)² + (0.8517)² + (-1.6607)² + (0.0082)²)/4 = 1.0312
DLN1b[0] = (-0.0392 - -0.84)/√(1.0312+ε) = 0.7886
DLN1b[1] = (0.0117 - -0.84)/√(1.0312+ε) = 0.8387
DLN1b[2] = (-2.5008 - -0.84)/√(1.0312+ε) = -1.6354
DLN1b[3] = (-0.8318 - -0.84)/√(1.0312+ε) = 0.0081
行3:
均值 μ = (0.3593 + -1.3525 + -0.8189 + 0.0014)/4 = -0.4527
方差 σ² = ((0.812)² + (-0.8998)² + (-0.3662)² + (0.4541)²)/4 = 0.4523
DLN1b[0] = (0.3593 - -0.4527)/√(0.4523+ε) = 1.2072
DLN1b[1] = (-1.3525 - -0.4527)/√(0.4523+ε) = -1.3379
DLN1b[2] = (-0.8189 - -0.4527)/√(0.4523+ε) = -0.5445
DLN1b[3] = (0.0014 - -0.4527)/√(0.4523+ε) = 0.6752
67 10.1 计算 Q/K/V（输入 = DLN1b，Decoder L1 输出） Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
Q1[0] = [1.6363, -0.3983]
Q1[1] = [1.9805, 0.2737]
Q1[2] = [1.5468, -0.7395]
Q1[3] = [-0.3864, -0.8787]
K1[0] = [0.998, -0.8023]
K1[1] = [0.2731, -0.8061]
K1[2] = [0.7645, -0.3455]
K1[3] = [-0.3939, 1.0442]
V1[0] = [0.8037, 1.5429]
V1[1] = [0.2632, 2.0924]
V1[2] = [0.0457, 0.6745]
V1[3] = [-1.5635, -2.1195]
Q2[0] = [-0.6042, -0.0682]
Q2[1] = [-1.2734, 0.2807]
Q2[2] = [0.6214, 0.3022]
Q2[3] = [2.8616, 0.7219]
K2[0] = [-1.0326, 2.0791]
K2[1] = [-1.3182, 1.8815]
K2[2] = [-1.0536, 2.2847]
K2[3] = [0.0927, 0.3572]
V2[0] = [-0.0844, -0.3253]
V2[1] = [-0.7926, -1.0454]
V2[2] = [-0.4559, 0.2413]
V2[3] = [-0.6257, 1.4269]
68 10.2 掩码注意力分数 score = Q·Kᵀ/√2 + M Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
score1[0][0] = (1.9526)/1.4142 = 1.3807
score1[0][1] = (0.7679)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[0][2] = (1.3886)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[0][3] = (-1.0604)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[1][0] = (1.757)/1.4142 = 1.2424
score1[1][1] = (0.3202)/1.4142 = 0.2264
score1[1][2] = (1.4196)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[1][3] = (-0.4943)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[2][0] = (2.1371)/1.4142 = 1.5112
score1[2][1] = (1.0185)/1.4142 = 0.7202
score1[2][2] = (1.4381)/1.4142 = 1.0169
score1[2][3] = (-1.3815)/1.4142 + (-1e9) ≈ -∞ (掩码)
score1[3][0] = (0.3193)/1.4142 = 0.2258
score1[3][1] = (0.6028)/1.4142 = 0.4262
score1[3][2] = (0.0081)/1.4142 = 0.0058
score1[3][3] = (-0.7653)/1.4142 = -0.5412
score2[0][0] = (0.4821)/1.4142 = 0.3409
score2[0][1] = (0.6681)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[0][2] = (0.4808)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[0][3] = (-0.0804)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[1][0] = (1.8986)/1.4142 = 1.3425
score2[1][1] = (2.2067)/1.4142 = 1.5604
score2[1][2] = (1.9831)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[1][3] = (-0.0178)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[2][0] = (-0.0134)/1.4142 = -0.0094
score2[2][1] = (-0.2505)/1.4142 = -0.1771
score2[2][2] = (0.0357)/1.4142 = 0.0253
score2[2][3] = (0.1656)/1.4142 + (-1e9) ≈ -∞ (掩码)
score2[3][0] = (-1.4542)/1.4142 = -1.0283
score2[3][1] = (-2.4139)/1.4142 = -1.7069
score2[3][2] = (-1.3659)/1.4142 = -0.9658
score2[3][3] = (0.5232)/1.4142 = 0.37
69 10.3 掩码 softmax 权重 A Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
掩码 softmax 权重 A
A1 行0: [1, 0, 0, 0]
A1 行1: [0.7342, 0.2658, 0, 0]
A1 行2: [0.4846, 0.2197, 0.2956, 0]
A1 行3: [0.2866, 0.3502, 0.23, 0.1331]
A2 行0: [1, 0, 0, 0]
A2 行1: [0.4457, 0.5543, 0, 0]
A2 行2: [0.3471, 0.2935, 0.3594, 0]
A2 行3: [0.1511, 0.0766, 0.1608, 0.6115]
70 10.4 加权求和 O = A·V Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
O1[0] = [0.8037, 1.5429]
O1[1] = [0.66, 1.6889]
O1[2] = [0.4608, 1.4069]
O1[3] = [0.1249, 1.0481]
O2[0] = [-0.0844, -0.3253]
O2[1] = [-0.4769, -0.7244]
O2[2] = [-0.4258, -0.3331]
O2[3] = [-0.5294, 0.7821]
71 10.5 拼接 + 投影 EAttnOut1 = Concat·WOd2ᵀ Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
Concat[0] = [0.8037, 1.5429, -0.0844, -0.3253]
Concat[1] = [0.66, 1.6889, -0.4769, -0.7244]
Concat[2] = [0.4608, 1.4069, -0.4258, -0.3331]
Concat[3] = [0.1249, 1.0481, -0.5294, 0.7821]
EAttnOut1[0][0] = 0.8037×0.2 + 1.5429×-0.8 + -0.0844×-0.8 + -0.3253×0.4 = -1.1361
EAttnOut1[0][1] = 0.8037×-0.9 + 1.5429×0.6 + -0.0844×0.4 + -0.3253×-0.8 = 0.4289
EAttnOut1[0][2] = 0.8037×-0.8 + 1.5429×1 + -0.0844×-0.3 + -0.3253×-0.3 = 1.0228
EAttnOut1[0][3] = 0.8037×0.6 + 1.5429×0.9 + -0.0844×1 + -0.3253×0.5 = 1.6237
EAttnOut1[1][0] = 0.66×0.2 + 1.6889×-0.8 + -0.4769×-0.8 + -0.7244×0.4 = -1.1274
EAttnOut1[1][1] = 0.66×-0.9 + 1.6889×0.6 + -0.4769×0.4 + -0.7244×-0.8 = 0.8081
EAttnOut1[1][2] = 0.66×-0.8 + 1.6889×1 + -0.4769×-0.3 + -0.7244×-0.3 = 1.5213
EAttnOut1[1][3] = 0.66×0.6 + 1.6889×0.9 + -0.4769×1 + -0.7244×0.5 = 1.0769
EAttnOut1[2][0] = 0.4608×0.2 + 1.4069×-0.8 + -0.4258×-0.8 + -0.3331×0.4 = -0.826
EAttnOut1[2][1] = 0.4608×-0.9 + 1.4069×0.6 + -0.4258×0.4 + -0.3331×-0.8 = 0.5255
EAttnOut1[2][2] = 0.4608×-0.8 + 1.4069×1 + -0.4258×-0.3 + -0.3331×-0.3 = 1.2659
EAttnOut1[2][3] = 0.4608×0.6 + 1.4069×0.9 + -0.4258×1 + -0.3331×0.5 = 0.9504
EAttnOut1[3][0] = 0.1249×0.2 + 1.0481×-0.8 + -0.5294×-0.8 + 0.7821×0.4 = -0.0771
EAttnOut1[3][1] = 0.1249×-0.9 + 1.0481×0.6 + -0.5294×0.4 + 0.7821×-0.8 = -0.3211
EAttnOut1[3][2] = 0.1249×-0.8 + 1.0481×1 + -0.5294×-0.3 + 0.7821×-0.3 = 0.8723
EAttnOut1[3][3] = 0.1249×0.6 + 1.0481×0.9 + -0.5294×1 + 0.7821×0.5 = 0.8799
72 10.6 残差连接 EX1 Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
残差：EX1 = DLN1b + EAttnOut1（逐元素）
EX1[0][0] = 0.2064 + -1.1361 = -0.9297
EX1[0][1] = 1.406 + 0.4289 = 1.8348
EX1[0][2] = -1.3896 + 1.0228 = -0.3668
EX1[0][3] = -0.2228 + 1.6237 = 1.4009
EX1[1][0] = 0.3748 + -1.1274 = -0.7525
EX1[1][1] = 1.4684 + 0.8081 = 2.2765
EX1[1][2] = -0.9695 + 1.5213 = 0.5518
EX1[1][3] = -0.8737 + 1.0769 = 0.2032
EX1[2][0] = 0.7886 + -0.826 = -0.0373
EX1[2][1] = 0.8387 + 0.5255 = 1.3642
EX1[2][2] = -1.6354 + 1.2659 = -0.3695
EX1[2][3] = 0.0081 + 0.9504 = 0.9585
EX1[3][0] = 1.2072 + -0.0771 = 1.1302
EX1[3][1] = -1.3379 + -0.3211 = -1.659
EX1[3][2] = -0.5445 + 0.8723 = 0.3278
EX1[3][3] = 0.6752 + 0.8799 = 1.5551
73 10.7 LayerNorm 归一化 ELN1（= 送入 Cross-Attn 的 Q） Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
LayerNorm：ELN1 = LayerNorm(EX1)
行0:
均值 μ = (-0.9297 + 1.8348 + -0.3668 + 1.4009)/4 = 0.4848
方差 σ² = ((-1.4145)² + (1.35)² + (-0.8516)² + (0.9161)²)/4 = 1.347
ELN1[0] = (-0.9297 - 0.4848)/√(1.347+ε) = -1.2188
ELN1[1] = (1.8348 - 0.4848)/√(1.347+ε) = 1.1632
ELN1[2] = (-0.3668 - 0.4848)/√(1.347+ε) = -0.7338
ELN1[3] = (1.4009 - 0.4848)/√(1.347+ε) = 0.7893
行1:
均值 μ = (-0.7525 + 2.2765 + 0.5518 + 0.2032)/4 = 0.5697
方差 σ² = ((-1.3223)² + (1.7067)² + (-0.0179)² + (-0.3665)²)/4 = 1.199
ELN1[0] = (-0.7525 - 0.5697)/√(1.199+ε) = -1.2076
ELN1[1] = (2.2765 - 0.5697)/√(1.199+ε) = 1.5587
ELN1[2] = (0.5518 - 0.5697)/√(1.199+ε) = -0.0164
ELN1[3] = (0.2032 - 0.5697)/√(1.199+ε) = -0.3347
行2:
均值 μ = (-0.0373 + 1.3642 + -0.3695 + 0.9585)/4 = 0.479
方差 σ² = ((-0.5163)² + (0.8852)² + (-0.8485)² + (0.4795)²)/4 = 0.5
ELN1[0] = (-0.0373 - 0.479)/√(0.5+ε) = -0.7301
ELN1[1] = (1.3642 - 0.479)/√(0.5+ε) = 1.2519
ELN1[2] = (-0.3695 - 0.479)/√(0.5+ε) = -1.1999
ELN1[3] = (0.9585 - 0.479)/√(0.5+ε) = 0.6782
行3:
均值 μ = (1.1302 + -1.659 + 0.3278 + 1.5551)/4 = 0.3385
方差 σ² = ((0.7917)² + (-1.9975)² + (-0.0107)² + (1.2166)²)/4 = 1.5242
ELN1[0] = (1.1302 - 0.3385)/√(1.5242+ε) = 0.6412
ELN1[1] = (-1.659 - 0.3385)/√(1.5242+ε) = -1.6179
ELN1[2] = (0.3278 - 0.3385)/√(1.5242+ε) = -0.0087
ELN1[3] = (1.5551 - 0.3385)/√(1.5242+ε) = 0.9854
74 11.1 计算 Q（来自 Decoder ELN1）/ K、V（来自 ENCODER 最终输出） Cross-Attn(训练)
🎯 架构功能讲解Cross-Attention（交叉注意力）是翻译的灵魂：Q 来自目标语言（想表达什么），K/V 来自源语言记忆（有什么可用）。训练后生成「I」时权重会集中到「我」上 = 词对齐。
Q1[0] = [-0.3327, -0.3446] ← Decoder
Q1[1] = [0.3203, 1.3947] ← Decoder
Q1[2] = [0.6888, -0.7775] ← Decoder
Q1[3] = [-1.4345, -1.7975] ← Decoder
K1[0] = [0.8115, 0.8874] ← Encoder
K1[1] = [0.5126, 0.9] ← Encoder
K1[2] = [0.9677, 0.7588] ← Encoder
K1[3] = [0.6422, 0.9166] ← Encoder
V1[0] = [-1.9158, 1.3228] ← Encoder
V1[1] = [-1.9157, 1.3947] ← Encoder
V1[2] = [-1.7224, 1.2987] ← Encoder
V1[3] = [-1.9549, 1.376] ← Encoder
Q2[0] = [-1.9139, -0.2606] ← Decoder
Q2[1] = [-1.3887, -0.8673] ← Decoder
Q2[2] = [-1.4917, 0.3967] ← Decoder
Q2[3] = [0.5689, 0.5535] ← Decoder
K2[0] = [0.6828, -0.6314] ← Encoder
K2[1] = [1.1301, -0.3656] ← Encoder
K2[2] = [0.5557, -0.6446] ← Encoder
K2[3] = [0.9418, -0.494] ← Encoder
V2[0] = [0.019, 0.6432] ← Encoder
V2[1] = [-0.414, 0.2266] ← Encoder
V2[2] = [0.1248, 0.8795] ← Encoder
V2[3] = [-0.2235, 0.4006] ← Encoder
75 11.2 注意力分数 score = Q·Kᵀ/√2（⚠ 无掩码！可看全部源 token） Cross-Attn(训练)
🎯 架构功能讲解对比自注意力：这里没有掩码——Decoder 生成每个词时都可以自由查看 Encoder 的 4 个源 token，这正是翻译对齐的基础。
score1[0][0] = (-0.5758)/1.4142 = -0.4072
score1[0][1] = (-0.4807)/1.4142 = -0.3399
score1[0][2] = (-0.5835)/1.4142 = -0.4126
score1[0][3] = (-0.5295)/1.4142 = -0.3744
score1[1][0] = (1.4975)/1.4142 = 1.0589
score1[1][1] = (1.4194)/1.4142 = 1.0037
score1[1][2] = (1.3682)/1.4142 = 0.9675
score1[1][3] = (1.4841)/1.4142 = 1.0494
score1[2][0] = (-0.131)/1.4142 = -0.0926
score1[2][1] = (-0.3467)/1.4142 = -0.2451
score1[2][2] = (0.0766)/1.4142 = 0.0542
score1[2][3] = (-0.2703)/1.4142 = -0.1912
score1[3][0] = (-2.7592)/1.4142 = -1.951
score1[3][1] = (-2.3531)/1.4142 = -1.6639
score1[3][2] = (-2.7521)/1.4142 = -1.9461
score1[3][3] = (-2.5688)/1.4142 = -1.8164
score2[0][0] = (-1.1423)/1.4142 = -0.8077
score2[0][1] = (-2.0677)/1.4142 = -1.4621
score2[0][2] = (-0.8955)/1.4142 = -0.6332
score2[0][3] = (-1.6738)/1.4142 = -1.1836
score2[1][0] = (-0.4006)/1.4142 = -0.2832
score2[1][1] = (-1.2523)/1.4142 = -0.8855
score2[1][2] = (-0.2126)/1.4142 = -0.1503
score2[1][3] = (-0.8794)/1.4142 = -0.6219
score2[2][0] = (-1.269)/1.4142 = -0.8973
score2[2][1] = (-1.8309)/1.4142 = -1.2946
score2[2][2] = (-1.0846)/1.4142 = -0.7669
score2[2][3] = (-1.6009)/1.4142 = -1.132
score2[3][0] = (0.039)/1.4142 = 0.0276
score2[3][1] = (0.4406)/1.4142 = 0.3116
score2[3][2] = (-0.0407)/1.4142 = -0.0287
score2[3][3] = (0.2624)/1.4142 = 0.1855
76 11.3 softmax 权重 A（对齐矩阵） Cross-Attn(训练)
🎯 架构功能讲解每行是目标词对 4 个源词的注意力分配。随机权重下分布较均匀；训练后应锐化为「I↔我、love↔爱、easy↔水(课)、courses↔课」的对角对齐。
softmax 权重 A（行 = 目标 token，列 = 源 token）
A1 行0（I）→ 源[0.2441, 0.261, 0.2427, 0.2522]
A1 行1（love）→ 源[0.2598, 0.2458, 0.2371, 0.2573]
A1 行2（easy）→ 源[0.2549, 0.2189, 0.2952, 0.231]
A1 行3（courses）→ 源[0.2231, 0.2973, 0.2242, 0.2553]
A2 行0（I）→ 源[0.2944, 0.153, 0.3505, 0.2021]
A2 行1（love）→ 源[0.2939, 0.1609, 0.3357, 0.2095]
A2 行2（easy）→ 源[0.2776, 0.1866, 0.3163, 0.2195]
A2 行3（courses）→ 源[0.225, 0.2989, 0.2127, 0.2635]
77 11.4 加权求和 O = A·V Cross-Attn(训练)
🎯 架构功能讲解交叉注意力（训练模式）：Q 来自 Decoder、K/V 来自 Encoder 输出，实现目标词与源词的翻译对齐。无掩码。
O1[0] = [-1.8787, 1.3491]
O1[1] = [-1.88, 1.3485]
O1[2] = [-1.8677, 1.3437]
O1[3] = [-1.8824, 1.3524]
O2[0] = [-0.0592, 0.6132]
O2[1] = [-0.066, 0.6047]
O2[2] = [-0.0816, 0.5869]
O2[3] = [-0.1518, 0.505]
78 11.5 拼接 + 投影 CAttnOut = Concat·WOc2ᵀ Cross-Attn(训练)
🎯 架构功能讲解交叉注意力（训练模式）：Q 来自 Decoder、K/V 来自 Encoder 输出，实现目标词与源词的翻译对齐。无掩码。
Concat[0] = [-1.8787, 1.3491, -0.0592, 0.6132]
Concat[1] = [-1.88, 1.3485, -0.066, 0.6047]
Concat[2] = [-1.8677, 1.3437, -0.0816, 0.5869]
Concat[3] = [-1.8824, 1.3524, -0.1518, 0.505]
CAttnOut[0][0] = -1.8787×0.4 + 1.3491×-0.6 + -0.0592×-0.7 + 0.6132×-1 = -2.1328
CAttnOut[0][1] = -1.8787×-0.3 + 1.3491×0.2 + -0.0592×-0.2 + 0.6132×-0.1 = 0.7839
CAttnOut[0][2] = -1.8787×0.8 + 1.3491×-0.3 + -0.0592×0 + 0.6132×0.6 = -1.5397
CAttnOut[0][3] = -1.8787×-0.2 + 1.3491×0.2 + -0.0592×0.7 + 0.6132×0.9 = 1.156
CAttnOut[1][0] = -1.88×0.4 + 1.3485×-0.6 + -0.066×-0.7 + 0.6047×-1 = -2.1195
CAttnOut[1][1] = -1.88×-0.3 + 1.3485×0.2 + -0.066×-0.2 + 0.6047×-0.1 = 0.7864
CAttnOut[1][2] = -1.88×0.8 + 1.3485×-0.3 + -0.066×0 + 0.6047×0.6 = -1.5457
CAttnOut[1][3] = -1.88×-0.2 + 1.3485×0.2 + -0.066×0.7 + 0.6047×0.9 = 1.1437
CAttnOut[2][0] = -1.8677×0.4 + 1.3437×-0.6 + -0.0816×-0.7 + 0.5869×-1 = -2.0831
CAttnOut[2][1] = -1.8677×-0.3 + 1.3437×0.2 + -0.0816×-0.2 + 0.5869×-0.1 = 0.7867
CAttnOut[2][2] = -1.8677×0.8 + 1.3437×-0.3 + -0.0816×0 + 0.5869×0.6 = -1.5451
CAttnOut[2][3] = -1.8677×-0.2 + 1.3437×0.2 + -0.0816×0.7 + 0.5869×0.9 = 1.1134
CAttnOut[3][0] = -1.8824×0.4 + 1.3524×-0.6 + -0.1518×-0.7 + 0.505×-1 = -1.9631
CAttnOut[3][1] = -1.8824×-0.3 + 1.3524×0.2 + -0.1518×-0.2 + 0.505×-0.1 = 0.815
CAttnOut[3][2] = -1.8824×0.8 + 1.3524×-0.3 + -0.1518×0 + 0.505×0.6 = -1.6086
CAttnOut[3][3] = -1.8824×-0.2 + 1.3524×0.2 + -0.1518×0.7 + 0.505×0.9 = 0.9952
79 11.6 残差连接 CX1 Cross-Attn(训练)
🎯 架构功能讲解交叉注意力（训练模式）：Q 来自 Decoder、K/V 来自 Encoder 输出，实现目标词与源词的翻译对齐。无掩码。
残差：CX1 = ELN1 + CAttnOut（逐元素）
CX1[0][0] = -1.2188 + -2.1328 = -3.3515
CX1[0][1] = 1.1632 + 0.7839 = 1.9471
CX1[0][2] = -0.7338 + -1.5397 = -2.2735
CX1[0][3] = 0.7893 + 1.156 = 1.9454
CX1[1][0] = -1.2076 + -2.1195 = -3.3271
CX1[1][1] = 1.5587 + 0.7864 = 2.3451
CX1[1][2] = -0.0164 + -1.5457 = -1.5621
CX1[1][3] = -0.3347 + 1.1437 = 0.809
CX1[2][0] = -0.7301 + -2.0831 = -2.8132
CX1[2][1] = 1.2519 + 0.7867 = 2.0385
CX1[2][2] = -1.1999 + -1.5451 = -2.745
CX1[2][3] = 0.6782 + 1.1134 = 1.7916
CX1[3][0] = 0.6412 + -1.9631 = -1.3219
CX1[3][1] = -1.6179 + 0.815 = -0.8029
CX1[3][2] = -0.0087 + -1.6086 = -1.6173
CX1[3][3] = 0.9854 + 0.9952 = 1.9806
80 11.7 LayerNorm 归一化 CLN1（送入 FFN） Cross-Attn(训练)
🎯 架构功能讲解交叉注意力（训练模式）：Q 来自 Decoder、K/V 来自 Encoder 输出，实现目标词与源词的翻译对齐。无掩码。
LayerNorm：CLN1 = LayerNorm(CX1)
行0:
均值 μ = (-3.3515 + 1.9471 + -2.2735 + 1.9454)/4 = -0.4331
方差 σ² = ((-2.9184)² + (2.3803)² + (-1.8404)² + (2.3785)²)/4 = 5.8068
CLN1[0] = (-3.3515 - -0.4331)/√(5.8068+ε) = -1.2111
CLN1[1] = (1.9471 - -0.4331)/√(5.8068+ε) = 0.9878
CLN1[2] = (-2.2735 - -0.4331)/√(5.8068+ε) = -0.7637
CLN1[3] = (1.9454 - -0.4331)/√(5.8068+ε) = 0.987
行1:
均值 μ = (-3.3271 + 2.3451 + -1.5621 + 0.809)/4 = -0.4338
方差 σ² = ((-2.8933)² + (2.7789)² + (-1.1283)² + (1.2427)²)/4 = 4.7277
CLN1[0] = (-3.3271 - -0.4338)/√(4.7277+ε) = -1.3307
CLN1[1] = (2.3451 - -0.4338)/√(4.7277+ε) = 1.278
CLN1[2] = (-1.5621 - -0.4338)/√(4.7277+ε) = -0.5189
CLN1[3] = (0.809 - -0.4338)/√(4.7277+ε) = 0.5716
行2:
均值 μ = (-2.8132 + 2.0385 + -2.745 + 1.7916)/4 = -0.432
方差 σ² = ((-2.3812)² + (2.4706)² + (-2.313)² + (2.2236)²)/4 = 5.5171
CLN1[0] = (-2.8132 - -0.432)/√(5.5171+ε) = -1.0138
CLN1[1] = (2.0385 - -0.432)/√(5.5171+ε) = 1.0518
CLN1[2] = (-2.745 - -0.432)/√(5.5171+ε) = -0.9847
CLN1[3] = (1.7916 - -0.432)/√(5.5171+ε) = 0.9467
行3:
均值 μ = (-1.3219 + -0.8029 + -1.6173 + 1.9806)/4 = -0.4404
方差 σ² = ((-0.8815)² + (-0.3625)² + (-1.1769)² + (2.421)²)/4 = 2.0387
CLN1[0] = (-1.3219 - -0.4404)/√(2.0387+ε) = -0.6174
CLN1[1] = (-0.8029 - -0.4404)/√(2.0387+ε) = -0.2539
CLN1[2] = (-1.6173 - -0.4404)/√(2.0387+ε) = -0.8243
CLN1[3] = (1.9806 - -0.4404)/√(2.0387+ε) = 1.6956
81 12.1 FFN 第一层 CH1 = CLN1·Wd2_1ᵀ（逐元素） Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
CH1[0][0] = -1.2111×-0.7 + 0.9878×0.9 + -0.7637×0 + 0.987×-0.5 = 1.2432
CH1[0][1] = -1.2111×-0.1 + 0.9878×1 + -0.7637×0 + 0.987×-0.3 = 0.8128
CH1[0][2] = -1.2111×0.3 + 0.9878×-0.5 + -0.7637×-0.8 + 0.987×-0.7 = -0.9372
CH1[0][3] = -1.2111×-0.7 + 0.9878×-0.7 + -0.7637×-0.7 + 0.987×0.3 = 0.987
CH1[0][4] = -1.2111×-0.6 + 0.9878×-0.3 + -0.7637×0.8 + 0.987×-0.1 = -0.2794
CH1[0][5] = -1.2111×0.3 + 0.9878×-0.7 + -0.7637×-0.6 + 0.987×-0.9 = -1.4849
CH1[0][6] = -1.2111×-0.7 + 0.9878×-0.4 + -0.7637×-0.6 + 0.987×-0.8 = 0.1213
CH1[0][7] = -1.2111×-0.8 + 0.9878×-0.1 + -0.7637×-0.6 + 0.987×-0.3 = 1.0322
CH1[1][0] = -1.3307×-0.7 + 1.278×0.9 + -0.5189×0 + 0.5716×-0.5 = 1.7959
CH1[1][1] = -1.3307×-0.1 + 1.278×1 + -0.5189×0 + 0.5716×-0.3 = 1.2396
CH1[1][2] = -1.3307×0.3 + 1.278×-0.5 + -0.5189×-0.8 + 0.5716×-0.7 = -1.0232
CH1[1][3] = -1.3307×-0.7 + 1.278×-0.7 + -0.5189×-0.7 + 0.5716×0.3 = 0.5716
CH1[1][4] = -1.3307×-0.6 + 1.278×-0.3 + -0.5189×0.8 + 0.5716×-0.1 = -0.0573
CH1[1][5] = -1.3307×0.3 + 1.278×-0.7 + -0.5189×-0.6 + 0.5716×-0.9 = -1.4969
CH1[1][6] = -1.3307×-0.7 + 1.278×-0.4 + -0.5189×-0.6 + 0.5716×-0.8 = 0.2744
CH1[1][7] = -1.3307×-0.8 + 1.278×-0.1 + -0.5189×-0.6 + 0.5716×-0.3 = 1.0766
CH1[2][0] = -1.0138×-0.7 + 1.0518×0.9 + -0.9847×0 + 0.9467×-0.5 = 1.1829
CH1[2][1] = -1.0138×-0.1 + 1.0518×1 + -0.9847×0 + 0.9467×-0.3 = 0.8692
CH1[2][2] = -1.0138×0.3 + 1.0518×-0.5 + -0.9847×-0.8 + 0.9467×-0.7 = -0.7049
CH1[2][3] = -1.0138×-0.7 + 1.0518×-0.7 + -0.9847×-0.7 + 0.9467×0.3 = 0.9467
CH1[2][4] = -1.0138×-0.6 + 1.0518×-0.3 + -0.9847×0.8 + 0.9467×-0.1 = -0.5897
CH1[2][5] = -1.0138×0.3 + 1.0518×-0.7 + -0.9847×-0.6 + 0.9467×-0.9 = -1.3016
CH1[2][6] = -1.0138×-0.7 + 1.0518×-0.4 + -0.9847×-0.6 + 0.9467×-0.8 = 0.1224
CH1[2][7] = -1.0138×-0.8 + 1.0518×-0.1 + -0.9847×-0.6 + 0.9467×-0.3 = 1.0127
CH1[3][0] = -0.6174×-0.7 + -0.2539×0.9 + -0.8243×0 + 1.6956×-0.5 = -0.6441
CH1[3][1] = -0.6174×-0.1 + -0.2539×1 + -0.8243×0 + 1.6956×-0.3 = -0.7008
CH1[3][2] = -0.6174×0.3 + -0.2539×-0.5 + -0.8243×-0.8 + 1.6956×-0.7 = -0.5857
CH1[3][3] = -0.6174×-0.7 + -0.2539×-0.7 + -0.8243×-0.7 + 1.6956×0.3 = 1.6956
CH1[3][4] = -0.6174×-0.6 + -0.2539×-0.3 + -0.8243×0.8 + 1.6956×-0.1 = -0.3824
CH1[3][5] = -0.6174×0.3 + -0.2539×-0.7 + -0.8243×-0.6 + 1.6956×-0.9 = -1.0389
CH1[3][6] = -0.6174×-0.7 + -0.2539×-0.4 + -0.8243×-0.6 + 1.6956×-0.8 = -0.3282
CH1[3][7] = -0.6174×-0.8 + -0.2539×-0.1 + -0.8243×-0.6 + 1.6956×-0.3 = 0.5052
82 12.2 ReLU 激活 CR1 = max(0, CH1) Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
CR1[0][0] = max(0, 1.2432) = 1.2432
CR1[0][1] = max(0, 0.8128) = 0.8128
CR1[0][2] = max(0, -0.9372) = 0
CR1[0][3] = max(0, 0.987) = 0.987
CR1[0][4] = max(0, -0.2794) = 0
CR1[0][5] = max(0, -1.4849) = 0
CR1[0][6] = max(0, 0.1213) = 0.1213
CR1[0][7] = max(0, 1.0322) = 1.0322
CR1[1][0] = max(0, 1.7959) = 1.7959
CR1[1][1] = max(0, 1.2396) = 1.2396
CR1[1][2] = max(0, -1.0232) = 0
CR1[1][3] = max(0, 0.5716) = 0.5716
CR1[1][4] = max(0, -0.0573) = 0
CR1[1][5] = max(0, -1.4969) = 0
CR1[1][6] = max(0, 0.2744) = 0.2744
CR1[1][7] = max(0, 1.0766) = 1.0766
CR1[2][0] = max(0, 1.1829) = 1.1829
CR1[2][1] = max(0, 0.8692) = 0.8692
CR1[2][2] = max(0, -0.7049) = 0
CR1[2][3] = max(0, 0.9467) = 0.9467
CR1[2][4] = max(0, -0.5897) = 0
CR1[2][5] = max(0, -1.3016) = 0
CR1[2][6] = max(0, 0.1224) = 0.1224
CR1[2][7] = max(0, 1.0127) = 1.0127
CR1[3][0] = max(0, -0.6441) = 0
CR1[3][1] = max(0, -0.7008) = 0
CR1[3][2] = max(0, -0.5857) = 0
CR1[3][3] = max(0, 1.6956) = 1.6956
CR1[3][4] = max(0, -0.3824) = 0
CR1[3][5] = max(0, -1.0389) = 0
CR1[3][6] = max(0, -0.3282) = 0
CR1[3][7] = max(0, 0.5052) = 0.5052
83 12.3 FFN 第二层 CF1 = CR1·Wd2_2ᵀ（逐元素） Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
CF1[0][0] = 1.2432×0 + 0.8128×0.4 + 0×-0.9 + 0.987×0.6 + 0×0.3 + 0×-0.8 + 0.1213×0.7 + 1.0322×0.8 = 1.828
CF1[0][1] = 1.2432×-0.9 + 0.8128×-0.4 + 0×0.6 + 0.987×0.5 + 0×-0.6 + 0×-0.6 + 0.1213×-0.3 + 1.0322×0 = -0.9869
CF1[0][2] = 1.2432×0.2 + 0.8128×-0.3 + 0×-0.1 + 0.987×0.5 + 0×-0.9 + 0×-0.5 + 0.1213×0.4 + 1.0322×0.8 = 1.3726
CF1[0][3] = 1.2432×0 + 0.8128×0.1 + 0×-0.8 + 0.987×-0.1 + 0×0.1 + 0×-0.5 + 0.1213×-0.5 + 1.0322×-0.2 = -0.2845
CF1[1][0] = 1.7959×0 + 1.2396×0.4 + 0×-0.9 + 0.5716×0.6 + 0×0.3 + 0×-0.8 + 0.2744×0.7 + 1.0766×0.8 = 1.8921
CF1[1][1] = 1.7959×-0.9 + 1.2396×-0.4 + 0×0.6 + 0.5716×0.5 + 0×-0.6 + 0×-0.6 + 0.2744×-0.3 + 1.0766×0 = -1.9087
CF1[1][2] = 1.7959×0.2 + 1.2396×-0.3 + 0×-0.1 + 0.5716×0.5 + 0×-0.9 + 0×-0.5 + 0.2744×0.4 + 1.0766×0.8 = 1.2441
CF1[1][3] = 1.7959×0 + 1.2396×0.1 + 0×-0.8 + 0.5716×-0.1 + 0×0.1 + 0×-0.5 + 0.2744×-0.5 + 1.0766×-0.2 = -0.2857
CF1[2][0] = 1.1829×0 + 0.8692×0.4 + 0×-0.9 + 0.9467×0.6 + 0×0.3 + 0×-0.8 + 0.1224×0.7 + 1.0127×0.8 = 1.8115
CF1[2][1] = 1.1829×-0.9 + 0.8692×-0.4 + 0×0.6 + 0.9467×0.5 + 0×-0.6 + 0×-0.6 + 0.1224×-0.3 + 1.0127×0 = -0.9757
CF1[2][2] = 1.1829×0.2 + 0.8692×-0.3 + 0×-0.1 + 0.9467×0.5 + 0×-0.9 + 0×-0.5 + 0.1224×0.4 + 1.0127×0.8 = 1.3083
CF1[2][3] = 1.1829×0 + 0.8692×0.1 + 0×-0.8 + 0.9467×-0.1 + 0×0.1 + 0×-0.5 + 0.1224×-0.5 + 1.0127×-0.2 = -0.2715
CF1[3][0] = 0×0 + 0×0.4 + 0×-0.9 + 1.6956×0.6 + 0×0.3 + 0×-0.8 + 0×0.7 + 0.5052×0.8 = 1.4215
CF1[3][1] = 0×-0.9 + 0×-0.4 + 0×0.6 + 1.6956×0.5 + 0×-0.6 + 0×-0.6 + 0×-0.3 + 0.5052×0 = 0.8478
CF1[3][2] = 0×0.2 + 0×-0.3 + 0×-0.1 + 1.6956×0.5 + 0×-0.9 + 0×-0.5 + 0×0.4 + 0.5052×0.8 = 1.2519
CF1[3][3] = 0×0 + 0×0.1 + 0×-0.8 + 1.6956×-0.1 + 0×0.1 + 0×-0.5 + 0×-0.5 + 0.5052×-0.2 = -0.2706
84 12.4 残差连接 CX2 Decoder L2(训练)
🎯 架构功能讲解Decoder 第 2 层（训练模式）：掩码自注意力 + 交叉注意力（融合源句）+ FFN，产出目标位置最终表示。
残差：CX2 = CLN1 + CF1（逐元素）
CX2[0][0] = -1.2111 + 1.828 = 0.6169
CX2[0][1] = 0.9878 + -0.9869 = 0.0009
CX2[0][2] = -0.7637 + 1.3726 = 0.6089
CX2[0][3] = 0.987 + -0.2845 = 0.7025
CX2[1][0] = -1.3307 + 1.8921 = 0.5615
CX2[1][1] = 1.278 + -1.9087 = -0.6307
CX2[1][2] = -0.5189 + 1.2441 = 0.7252
CX2[1][3] = 0.5716 + -0.2857 = 0.2859
CX2[2][0] = -1.0138 + 1.8115 = 0.7977
CX2[2][1] = 1.0518 + -0.9757 = 0.0761
CX2[2][2] = -0.9847 + 1.3083 = 0.3235
CX2[2][3] = 0.9467 + -0.2715 = 0.6752
CX2[3][0] = -0.6174 + 1.4215 = 0.8041
CX2[3][1] = -0.2539 + 0.8478 = 0.5939
CX2[3][2] = -0.8243 + 1.2519 = 0.4277
CX2[3][3] = 1.6956 + -0.2706 = 1.425
85 12.5 ⭐ LayerNorm 归一化 DEC_OUT（Decoder 最终输出） Decoder L2(训练)
🎯 架构功能讲解Decoder 完成！DEC_OUT 的每一行 = 对应目标位置的最终表示，携带了源句信息（经 Cross-Attn）+ 目标句自身结构（经掩码自注意力）。
LayerNorm：DEC_OUT = LayerNorm(CX2)（⭐ Decoder 最终输出）
行0:
均值 μ = (0.6169 + 0.0009 + 0.6089 + 0.7025)/4 = 0.4823
方差 σ² = ((0.1346)² + (-0.4814)² + (0.1266)² + (0.2202)²)/4 = 0.0786
DEC_OUT[0] = (0.6169 - 0.4823)/√(0.0786+ε) = 0.48
DEC_OUT[1] = (0.0009 - 0.4823)/√(0.0786+ε) = -1.717
DEC_OUT[2] = (0.6089 - 0.4823)/√(0.0786+ε) = 0.4515
DEC_OUT[3] = (0.7025 - 0.4823)/√(0.0786+ε) = 0.7855
行1:
均值 μ = (0.5615 + -0.6307 + 0.7252 + 0.2859)/4 = 0.2355
方差 σ² = ((0.326)² + (-0.8661)² + (0.4897)² + (0.0504)²)/4 = 0.2747
DEC_OUT[0] = (0.5615 - 0.2355)/√(0.2747+ε) = 0.622
DEC_OUT[1] = (-0.6307 - 0.2355)/√(0.2747+ε) = -1.6525
DEC_OUT[2] = (0.7252 - 0.2355)/√(0.2747+ε) = 0.9344
DEC_OUT[3] = (0.2859 - 0.2355)/√(0.2747+ε) = 0.0962
行2:
均值 μ = (0.7977 + 0.0761 + 0.3235 + 0.6752)/4 = 0.4681
方差 σ² = ((0.3296)² + (-0.392)² + (-0.1446)² + (0.2071)²)/4 = 0.0815
DEC_OUT[0] = (0.7977 - 0.4681)/√(0.0815+ε) = 1.1543
DEC_OUT[1] = (0.0761 - 0.4681)/√(0.0815+ε) = -1.3729
DEC_OUT[2] = (0.3235 - 0.4681)/√(0.0815+ε) = -0.5065
DEC_OUT[3] = (0.6752 - 0.4681)/√(0.0815+ε) = 0.7251
行3:
均值 μ = (0.8041 + 0.5939 + 0.4277 + 1.425)/4 = 0.8127
方差 σ² = ((-0.0085)² + (-0.2188)² + (-0.385)² + (0.6123)²)/4 = 0.1428
DEC_OUT[0] = (0.8041 - 0.8127)/√(0.1428+ε) = -0.0226
DEC_OUT[1] = (0.5939 - 0.8127)/√(0.1428+ε) = -0.579
DEC_OUT[2] = (0.4277 - 0.8127)/√(0.1428+ε) = -1.0189
DEC_OUT[3] = (1.425 - 0.8127)/√(0.1428+ε) = 1.6205
86 13.1 定义输出权重 W_out（8×4，词表 8 个词） 输出层(训练)
🎯 架构功能讲解Linear 输出层：把 4 维表示映射回词表大小（8），得到每个词的 logits（未归一化分数）。
W_out 行0（我）: [-1, -0.4, -0.6, -0.3]
W_out 行1（爱）: [-0.8, 0.8, 0.2, 0.4]
W_out 行2（水）: [0.6, 0, -0.8, 0.1]
W_out 行3（课）: [0.2, 0.5, -0.1, -0.7]
W_out 行4（I）: [-0.4, -0.3, 0.3, 0.1]
W_out 行5（love）: [-0.3, 1, 0.2, -0.5]
W_out 行6（easy）: [-0.8, -0.7, -0.5, -0.7]
W_out 行7（courses）: [-0.6, -0.4, -0.7, 0.8]
87 13.2 计算 Logits = DEC_OUT·W_outᵀ（逐元素，4 位置 × 8 词） 输出层(训练)
🎯 架构功能讲解Logits[i][j] = 位置 i 预测第 j 个词的原始分数。分数越高 = 模型越倾向该词。
Logits[0][0] = 0.48×-1 + -1.717×-0.4 + 0.4515×-0.6 + 0.7855×-0.3 = -0.2998
Logits[0][1] = 0.48×-0.8 + -1.717×0.8 + 0.4515×0.2 + 0.7855×0.4 = -1.3531
Logits[0][2] = 0.48×0.6 + -1.717×0 + 0.4515×-0.8 + 0.7855×0.1 = 0.0054
Logits[0][3] = 0.48×0.2 + -1.717×0.5 + 0.4515×-0.1 + 0.7855×-0.7 = -1.3575
Logits[0][4] = 0.48×-0.4 + -1.717×-0.3 + 0.4515×0.3 + 0.7855×0.1 = 0.5371
Logits[0][5] = 0.48×-0.3 + -1.717×1 + 0.4515×0.2 + 0.7855×-0.5 = -2.1635
Logits[0][6] = 0.48×-0.8 + -1.717×-0.7 + 0.4515×-0.5 + 0.7855×-0.7 = 0.0423
Logits[0][7] = 0.48×-0.6 + -1.717×-0.4 + 0.4515×-0.7 + 0.7855×0.8 = 0.7112
Logits[1][0] = 0.622×-1 + -1.6525×-0.4 + 0.9344×-0.6 + 0.0962×-0.3 = -0.5504
Logits[1][1] = 0.622×-0.8 + -1.6525×0.8 + 0.9344×0.2 + 0.0962×0.4 = -1.5943
Logits[1][2] = 0.622×0.6 + -1.6525×0 + 0.9344×-0.8 + 0.0962×0.1 = -0.3647
Logits[1][3] = 0.622×0.2 + -1.6525×0.5 + 0.9344×-0.1 + 0.0962×-0.7 = -0.8626
Logits[1][4] = 0.622×-0.4 + -1.6525×-0.3 + 0.9344×0.3 + 0.0962×0.1 = 0.5369
Logits[1][5] = 0.622×-0.3 + -1.6525×1 + 0.9344×0.2 + 0.0962×-0.5 = -1.7003
Logits[1][6] = 0.622×-0.8 + -1.6525×-0.7 + 0.9344×-0.5 + 0.0962×-0.7 = 0.1247
Logits[1][7] = 0.622×-0.6 + -1.6525×-0.4 + 0.9344×-0.7 + 0.0962×0.8 = -0.2893
Logits[2][0] = 1.1543×-1 + -1.3729×-0.4 + -0.5065×-0.6 + 0.7251×-0.3 = -0.5187
Logits[2][1] = 1.1543×-0.8 + -1.3729×0.8 + -0.5065×0.2 + 0.7251×0.4 = -1.833
Logits[2][2] = 1.1543×0.6 + -1.3729×0 + -0.5065×-0.8 + 0.7251×0.1 = 1.1702
Logits[2][3] = 1.1543×0.2 + -1.3729×0.5 + -0.5065×-0.1 + 0.7251×-0.7 = -0.9125
Logits[2][4] = 1.1543×-0.4 + -1.3729×-0.3 + -0.5065×0.3 + 0.7251×0.1 = -0.1292
Logits[2][5] = 1.1543×-0.3 + -1.3729×1 + -0.5065×0.2 + 0.7251×-0.5 = -2.183
Logits[2][6] = 1.1543×-0.8 + -1.3729×-0.7 + -0.5065×-0.5 + 0.7251×-0.7 = -0.2167
Logits[2][7] = 1.1543×-0.6 + -1.3729×-0.4 + -0.5065×-0.7 + 0.7251×0.8 = 0.7912
Logits[3][0] = -0.0226×-1 + -0.579×-0.4 + -1.0189×-0.6 + 1.6205×-0.3 = 0.3794
Logits[3][1] = -0.0226×-0.8 + -0.579×0.8 + -1.0189×0.2 + 1.6205×0.4 = -0.0007
Logits[3][2] = -0.0226×0.6 + -0.579×0 + -1.0189×-0.8 + 1.6205×0.1 = 0.9636
Logits[3][3] = -0.0226×0.2 + -0.579×0.5 + -1.0189×-0.1 + 1.6205×-0.7 = -1.3264
Logits[3][4] = -0.0226×-0.4 + -0.579×-0.3 + -1.0189×0.3 + 1.6205×0.1 = 0.0391
Logits[3][5] = -0.0226×-0.3 + -0.579×1 + -1.0189×0.2 + 1.6205×-0.5 = -1.5862
Logits[3][6] = -0.0226×-0.8 + -0.579×-0.7 + -1.0189×-0.5 + 1.6205×-0.7 = -0.2015
Logits[3][7] = -0.0226×-0.6 + -0.579×-0.4 + -1.0189×-0.7 + 1.6205×0.8 = 2.2547
88 13.3 Softmax 概率分布 P = softmax(Logits) 输出层(训练)
🎯 架构功能讲解Softmax 把 logits 变成合法概率。训练时模型输出 4 个位置各自对 8 个词的概率（teacher forcing 并行预测）；推理时一次只预测 1 个词。
Softmax 概率（每行和为 1）
位置0（目标 I）:
e^(-0.2998, -1.3531, 0.0054, -1.3575, 0.5371, -2.1635, 0.0423, 0.7112)
= 0.741 + 0.2584 + 1.0054 + 0.2573 + 1.711 + 0.1149 + 1.0432 + 2.0364 = 7.1676
P[0] = 0.741/7.1676 = 0.1034
P[1] = 0.2584/7.1676 = 0.0361
P[2] = 1.0054/7.1676 = 0.1403
P[3] = 0.2573/7.1676 = 0.0359
P[4] = 1.711/7.1676 = 0.2387
P[5] = 0.1149/7.1676 = 0.016
P[6] = 1.0432/7.1676 = 0.1455
P[7] = 2.0364/7.1676 = 0.2841
位置1（目标 love）:
e^(-0.5504, -1.5943, -0.3647, -0.8626, 0.5369, -1.7003, 0.1247, -0.2893)
= 0.5767 + 0.2031 + 0.6944 + 0.4221 + 1.7107 + 0.1826 + 1.1328 + 0.7488 = 5.6711
P[0] = 0.5767/5.6711 = 0.1017
P[1] = 0.2031/5.6711 = 0.0358
P[2] = 0.6944/5.6711 = 0.1225
P[3] = 0.4221/5.6711 = 0.0744
P[4] = 1.7107/5.6711 = 0.3016
P[5] = 0.1826/5.6711 = 0.0322
P[6] = 1.1328/5.6711 = 0.1997
P[7] = 0.7488/5.6711 = 0.132
位置2（目标 easy）:
e^(-0.5187, -1.833, 1.1702, -0.9125, -0.1292, -2.183, -0.2167, 0.7912)
= 0.5953 + 0.1599 + 3.2227 + 0.4015 + 0.8788 + 0.1127 + 0.8052 + 2.2061 = 8.3821
P[0] = 0.5953/8.3821 = 0.071
P[1] = 0.1599/8.3821 = 0.0191
P[2] = 3.2227/8.3821 = 0.3845
P[3] = 0.4015/8.3821 = 0.0479
P[4] = 0.8788/8.3821 = 0.1048
P[5] = 0.1127/8.3821 = 0.0134
P[6] = 0.8052/8.3821 = 0.0961
P[7] = 2.2061/8.3821 = 0.2632
位置3（目标 courses）:
e^(0.3794, -0.0007, 0.9636, -1.3264, 0.0391, -1.5862, -0.2015, 2.2547)
= 1.4614 + 0.9993 + 2.6211 + 0.2654 + 1.0399 + 0.2047 + 0.8175 + 9.5328 = 16.9421
P[0] = 1.4614/16.9421 = 0.0863
P[1] = 0.9993/16.9421 = 0.059
P[2] = 2.6211/16.9421 = 0.1547
P[3] = 0.2654/16.9421 = 0.0157
P[4] = 1.0399/16.9421 = 0.0614
P[5] = 0.2047/16.9421 = 0.0121
P[6] = 0.8175/16.9421 = 0.0483
P[7] = 9.5328/16.9421 = 0.5627
89 13.4 预测 vs 目标 输出层(训练)
🎯 架构功能讲解随机权重下 4 个位置只猜对 1 个（位置 3 的 courses）。这就是训练前的样子。
argmax 预测结果 + 目标词对比
位置0: 目标=I(ID4), 预测=courses(p=0.2841), 目标概率 p(I)=0.2387
位置1: 目标=love(ID5), 预测=I(p=0.3016), 目标概率 p(love)=0.0322
位置2: 目标=easy(ID6), 预测=水(p=0.3845), 目标概率 p(easy)=0.0961
位置3: 目标=courses(ID7), 预测=courses(p=0.5627), 目标概率 p(courses)=0.5627
90 13.5 交叉熵损失（训练优化的目标） 输出层(训练)
🎯 架构功能讲解损失越小 = 模型越自信地给出目标词。反向传播从这里的梯度出发，逐层更新所有权重；下一轮前向时目标词概率会上升。这就是「训练」。
交叉熵损失 loss = -ln P(目标词)，全句平均
位置0: loss = -ln(0.2387) = 1.4325
位置1: loss = -ln(0.0322) = 3.4357
位置2: loss = -ln(0.0961) = 2.3428
位置3: loss = -ln(0.5627) = 0.5751
平均损失 = (1.4325 + 3.4357 + 2.3428 + 0.5751)/4 = 1.9465
91 14.1 推理设定：自回归生成 推理
🎯 架构功能讲解训练是一次性并行预测 4 个位置；推理是逐步串行生成——这是两者最本质的差别。
推理时模型一次只生成 1 个词，把新词拼进输入，再生成下一个（自回归）
Encoder 只需跑一次（源句不变），Decoder 每步输入长度 +1
本实例：第 1 步生成「love」（已生成 I），第 2 步生成「easy」（已生成 I love），第 3 步生成「courses」（已生成 I love easy）
（真实推理第 1 步由 生成 I，本例为复用同一套权重，从「已生成 I」演示后续生成）
92 14.2 推理第 1 步：输入 [I] → 预测第 2 词 推理
🎯 架构功能讲解注意：掩码自注意力在长度为 1 时没有实际遮蔽（只看自己），Cross-Attn 仍查阅完整源句记忆。这一步模型该输出「love」。
输入：只有 [I]（1 个 token，嵌入+位置 = X_dec_in 第 0 行）
X_inf1 = [0.4, 1.5, -0.2, 1.3]
Logits = [-0.2998, -1.3531, 0.0054, -1.3575, 0.5371, -2.1635, 0.0423, 0.7112]
Softmax = [0.1034, 0.0361, 0.1403, 0.0359, 0.2387, 0.016, 0.1455, 0.2841]
argmax → 预测「courses」(p=0.2841)；目标应为「love」(p=0.016)
93 14.3 推理第 2 步：输入 [I, love] → 预测第 3 词 推理
🎯 架构功能讲解掩码此时生效：位置 1（love）只能看 I 和 love。推理每步重复完整 Decoder 前向，只是输入序列更长。
输入：已生成 [I, love]（2 个 token）
X_inf2 行0 = [0.4, 1.5, -0.2, 1.3]
X_inf2 行1 = [1.5415, 0.4403, 0.61, 0.7]
最后位置 Logits = [-0.5504, -1.5943, -0.3647, -0.8626, 0.5369, -1.7003, 0.1247, -0.2893]
Softmax = [0.1017, 0.0358, 0.1225, 0.0744, 0.3016, 0.0322, 0.1997, 0.132]
argmax → 预测「I」(p=0.3016)；目标应为「easy」(p=0.1997)
94 14.4 推理第 3 步：输入 [I, love, easy] → 预测第 4 词 推理
🎯 架构功能讲解推理 3 步 + 起始 I 共 4 个词，输出完整句子「I love easy courses」。注意：推理用的是模型自己生成的词喂回输入（无 teacher forcing），所以一旦某步预测错，后续会连锁错。
输入：已生成 [I, love, easy]（3 个 token）
X_inf3 行0 = [0.4, 1.5, -0.2, 1.3]
X_inf3 行1 = [1.5415, 0.4403, 0.61, 0.7]
X_inf3 行2 = [0.4093, -0.0161, 0.22, 1.5998]
最后位置 Logits = [-0.5187, -1.833, 1.1702, -0.9125, -0.1292, -2.183, -0.2167, 0.7912]
Softmax = [0.071, 0.0191, 0.3845, 0.0479, 0.1048, 0.0134, 0.0961, 0.2632]
argmax → 预测「水」(p=0.3845)；目标应为「courses」(p=0.2632)
95 15.1 训练 vs 推理：逐步差异对比表 对比
🎯 架构功能讲解一句话总结：训练 = 并行 + teacher forcing + 有梯度；推理 = 串行 + 自回归 + 无梯度。Encoder 两边完全一样，差异全在 Decoder。
【阶段1 Tokenization】训练/推理完全相同：源句「我爱水课」→[0,1,2,3]；
【阶段2 嵌入+位置】完全相同：X = E + PE；
【阶段3 Encoder】完全相同：源句编码一次，训练推理都产出 ENC_OUT；
【阶段4 Decoder 输入】训练：一次性输入完整目标句 [I,love,easy,courses]（teacher forcing）；推理：只输入已生成部分，逐步 [I]→[I,love]→[I,love,easy]；
【阶段5 掩码自注意力】训练：4×4 掩码矩阵，一次算 4 个位置；推理：每步掩码尺寸 1×1→2×2→3×3，只算最后位置（前面位置已算过，可用 KV cache 缓存）；
【阶段6 Cross-Attention】训练：4 个目标位置同时对全部源词；推理：每步仅最后位置对全部源词（Q 只有一个向量）；
【阶段7 输出层】训练：一次输出 4×8 概率矩阵，算 4 个位置的平均损失；推理：一次输出 1×8 概率，取 argmax 一个词；
【阶段8 下一步输入】训练：不依赖预测，直接用真实目标词（teacher forcing）；推理：把 argmax 选中的词拼回输入（自回归，错一步连锁错）；
【阶段9 梯度】训练：有反向传播更新权重；推理：无梯度、无更新，权重冻结。