阿正的梦工坊

根据 transformers 文档，如果没有显式指定 label_pad_token_id，它通常会默认使用 -100，因为这是 CrossEntropyLoss 的 ignore_index 默认值。

In-Depth Analysis of Memory Requirements for Mixed Precision Training of Meta-Llama-3-8B-Instruct Model

混合精度训练通过 BF16 格式大幅减少显存需求，但关键的优化器参数（权重更新副本、一阶动量、二阶动量）仍然以 FP32 存储，保证数值稳定性和训练精度

In deep learning model training, a "step" refers to a single update of the model's parameters after processing a batch of training samples.

监督微调（Supervised Fine-Tuning, SFT）是指在一个已经经过预训练的大规模语言模型的基础上，使用标注数据进行进一步的训练，使其在某个特定任务上表现得更好。

在 SFT训练中，我们通常会使用有标签的训练数据进行微调。

协方差矩阵的值：决定了马氏距离的尺度和方向敏感性。分布形状的建模：通过调整协方差矩阵，可以控制分布的拉伸和旋转，以更精确地拟合数据。

主题模型是一种用于发现文档集合中潜在主题的概率生成模型。其中，LDA（Latent Dirichlet Allocation, 潜在狄利克雷分配）是最著名的主题模型之一。在 LDA 中，狄利克雷分布起到了核心作用，用于建模文档-主题分布和主题-单词分布。

BF16 offers a larger numerical range and is specifically optimized for deep learning tasks that require handling large gradients and weights.

Data Parallelism, Model Parallelism, and Tensor Parallelism: Parallel Computing Strategies in Deep Learning

By enabling and configuring the hybrid computation engine, DeepSpeed can intelligently manage memory and computation across multiple devices, improving efficiency and reducing training time.

This parameter allows users to manage and optimize the storage and version control of training states, facilitating efficient data storage and recovery during model training.

这些配置项涵盖了内存优化、自动调优、混合精度、分布式训练等多个方面，以及模型训练的其他细节方面，包括压缩、梯度处理、优化器配置、数据效率、流水线并行等

Gradient checkpointing in DeepSpeed is a technique designed to reduce memory usage when training large models by storing only a subset of intermediate activations during the forward pass.

By discarding intermediate activations and recomputing them when needed, gradient checkpointing reduces memory usage, making it feasible to train large models on memory-limited hardware.

为了避免显存不足，可以通过减少 微批次大小 和 梯度累积步数、使用 混合精度训练、或应用 梯度检查点 等技术来优化显存的使用。

Distributed gradient accumulation helps overcome memory limitations by allowing us to simulate larger batch sizes while using smaller mini-batches.

reduce_bucket_size is an essential parameter in DeepSpeed's ZeRO Stage 2 optimization, controlling the size of the buckets during gradient reduction.

By understanding the mechanics of gradient reduction and the impact of contiguous memory, we can optimize distributed training setups and improve model training efficiency across multiple devices.

dataset_info.json文件记得更改

Understanding Parquet Files and Arrow Format: A Guide with Hugging Face Datasets

DeepSpeed’s configuration is highly flexible, but tuning requires balancing memory efficiency and computational speed.

NCCL (NVIDIA Collective Communications Library) is a high-performance communication library developed by NVIDIA.

DeepSpeed introduces the ZeRO (Zero Redundancy Optimizer) optimization technique, a groundbreaking solution to reduce memory usage and improve efficiency during training.

KL散度（Kullback-Leibler Divergence）可衡量两个概率分布之间差异

训模型的经验

By using a held-out set, we ensure that the evaluation results are reliable and not influenced by overfitting to the training or development data.

upsampling and downsampling refer to adjusting the number of samples in a dataset

是成果的意思，可以指学术成果，产出。

PPO四个模型里面的Reference模型

map函数的默认行为是将这些新生成的字段添加到原始数据集的每个样本上，同时保留原有的字段，除非你告诉它删除这些字段。

通过最大化对数似然估计来学习参数\($\Phi$\)，从而使模型能够根据给定的输入序列\(x\)来生成输出序列\(y\)

We used the final layer hidden state of each sentence’s [CLS] token as its embedding

第四周的quiz和编程作业。第一门课结束

第三周的quiz和编程作业

第二周的quiz和编程作业

第一周的字幕，quiz

手动下载并上传

下载数据集到本地，手动上传到服务器

torch，cudatoolkit和cuda驱动版本要适配