值得一看的大模型资源

Stanford_alpha: https://github.com/tatsu-lab/stanford_alpaca
可以大致看一下； 

https://github.com/Duxiaoman-DI/XuanYuan
金融领域模型参考

https://github.com/tloen/alpaca-lora?tab=readme-ov-file
Alpaca_lora,可以找个GPU卡试一下

python finetune.py \
    --base_model 'baffo32/decapoda-research-llama-7B-hf' \
    --data_path 'alpaca_data.json' \
    --output_dir './lora-alpaca'

Transformer 参考博客
https://jalammar.github.io/illustrated-transformer/
https://towardsdatascience.com/transformers-explained-visually-part-1-overview-of-functionality-95a6dd460452

Awesome github for LLMs
https://github.com/Hannibal046/Awesome-LLM
https://github.com/HqWu-HITCS/Awesome-Chinese-LLM

阅读以下的源码

GitHub - karpathy/nanoGPT: The simplest, fastest repository for training/finetuning medium-sized GPTs.

读懂这篇论文

https://arxiv.org/abs/2309.10305

阅读这个源码

https://github.com/baichuan-inc/Baichuan-7B/blob/main/models/modeling_baichuan.py

推荐阅读

https://arxiv.org/pdf/2308.10792

https://arxiv.org/pdf/2402.06196