LLM大模型算法学习资源持续整理(2024-)

整理大模型数据、预训练、对齐、推理加速、微调、应用、工程部署等相关学习资源；

AI社区

huggingface

https://huggingface.co/
https://huggingface.co/learn 抱抱脸学习中心上有很多公开课程；

魔搭社区

https://www.modelscope.cn/home

waytoagi

飞书文档写的AGI知识库。
https://www.waytoagi.com/

datawhale

大模型开源课程
https://github.com/datawhalechina/so-large-lm
开源大模型部署指南：
https://github.com/datawhalechina/self-llm

---

斯坦福大规模语言模型winter 2022

https://stanford-cs324.github.io/winter2022/

李宏毅生成式AI课程 2024

https://speech.ee.ntu.edu.tw/~hylee/genai/2024-spring.php

AI agent

• AutoGPT: https://github.com/Significant-Gravitas/Auto-GPT
• AgentGPT: https://agentgpt.reworkd.ai/
• BabyAGI: https://github.com/yoheinakajima/babyagi
• Godmode: https://godmode.space/?ref=futuretools.io

transformer

attention

flash attention

《FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness》https://huggingface.co/papers/2205.14135

multi-query attention

group-query attention

https://cyrilzakka.github.io/llm-playbook/nested/gqa.html

位置编码

RoPE
《RoFormer: Enhanced Transformer with Rotary Position Embedding》中提出
https://paperswithcode.com/paper/roformer-enhanced-transformer-with-rotary

---

Kaggle model courses

https://www.kaggle.com/models

LLM101

karpathy更新中的大模型教程：
https://github.com/karpathy/LLM101n

llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

课程地址：https://github.com/mlabonne/llm-course

SJTU dive-into-llms

https://sjtullm.gitbook.io/dive-into-llms
https://github.com/Lordog/dive-into-llms

llm-cookbook

吴恩达LLM系列课程的笔记
https://github.com/datawhalechina/llm-cookbook?tab=readme-ov-file

刘志远团队大模型公开课

https://www.openbmb.cn/community/course

开源大模型

Meta Llama 3.1

https://ai.meta.com/blog/meta-llama-3/

https://llama.meta.com/

开源大模型微调

https://github.com/hiyouga/LLaMA-Factory

LLM benchmark测试

https://github.com/EleutherAI/lm-evaluation-harness

多模态大模型学习资料

ViT

https://paperswithcode.com/method/vision-transformer

CLIP

https://github.com/openai/CLIP

BLIP

SAM

https://segment-anything.com/

文生视频

cogvideo

https://github.com/THUDM/CogVideo

opensora

扩散模型

fastai diffusion course

From Deep Learning Foundations to Stable Diffusion,part 2 of Practical Deep Learning for Coders.
https://course.fast.ai/Lessons/part2.html

huggingface-diffusion-course

https://huggingface.co/learn/diffusion-course/unit0/1

模型微调

LoRA

https://www.codewithgpu.com/i/Akegarasu/lora-scripts/lora-train

模型训练

强化学习

openai的spinningup深度强化学习教程；
https://spinningup.openai.com/en/latest/user/introduction.html#what-this-is

deepspeed

https://www.deepspeed.ai/

streamingLLM

https://www.high-flyer.cn/blog/streamingllm/

本地大模型

Ollama

https://ollama.com/

训练数据采集

gpt-crawler

爬取网站构建自己的GPT本地数据库；
https://github.com/BuilderIO/gpt-crawler

firecrawl

根据网站链接，采集网页内容并转为markdown或json结构；
https://www.firecrawl.dev/

LLM全栈工程bootcamp

https://fullstackdeeplearning.com/llm-bootcamp/

推理加速

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://github.com/vllm-project/vllm

RAG

• Langchain-Chatchat : https://github.com/chatchat-space/Langchain-Chatchat
• ACL2023 tutorial: https://acl2023-retrieval-lm.github.io/
  
• llamaindex: https://docs.llamaindex.ai/en/stable/optimizing/production_rag/

graphRAG

https://siwei.io/graph-rag/

相关blog

https://lilianweng.github.io/
AIMO大模型数学竞赛解决方案：https://www.kaggle.com/c/ai-mathematical-olympiad-prize/code

AI native应用

asksia.ai

私人学习AI导师：https://www.asksia.ai/zh-TW?utm_source=homepage&utm_medium=organic&utm_campaign=homepage

LLM面试题

1. LLMs_interview_notes
2. NLP资料整理 fighting41love/funNLP