Llama3技术报告

Model architecture

standard decoder-only transformer
Compared to Llama 2, we made several key improvements:

1. Llama 3 uses a tokenizer with a vocabulary of 128K tokens that encodes language much more efficiently
2. To improve the inference efficiency of Llama 3 models, we’ve adopted grouped query attention (GQA) across both the 8B and 70B sizes
3. We trained the models on sequences of 8,192 tokens, using a mask to ensure self-attention does not cross document boundaries.

Instruction fine-tuning

1. supervised fine-tuning (SFT)
2. rejection sampling
3. proximal policy optimization (PPO)
4. direct preference optimization (DPO)：the preference rankings that are used in PPO and DPO has an outsized influence on the performance of aligned models