大语言模型生成式AI学习笔记——1. 1.1 大语言模型及生成式AI项目生命周期简介——​​​​​​​课程简介

Generative AI with Large Language Models

by DeepLearning.AI & Amazon Web Services

About this Course

In Generative AI with Large Language Models (LLMs), you’ll learn the fundamentals of how generative AI works, and how to deploy it in real-world applications. By taking this course, you'll learn to:

- Deeply understand generative AI, describing the key steps in a typical LLM-based generative AI lifecycle, from data gathering and model selection, to performance evaluation and deployment

- Describe in detail the transformer architecture that powers LLMs, how they’re trained, and how fine-tuning enables LLMs to be adapted to a variety of specific use cases

- Use empirical scaling laws to optimize the model's objective function across dataset size, compute budget, and inference requirements

- Apply state-of-the art training, tuning, inference, tools, and deployment methods to maximize the performance of models within the specific constraints of your project

- Discuss the challenges and opportunities that generative AI creates for businesses after hearing stories from industry researchers and practitioners Developers who have a good foundational understanding of how LLMs work, as well the best practices behind training and deploying them, will be able to make good decisions for their companies and more quickly build working prototypes.

This course will support learners in building practical intuition about how to best utilize this exciting new technology. This is an intermediate course, so you should have some experience coding in Python to get the most out of it. You should also be familiar with the basics of machine learning, such as supervised and unsupervised learning, loss functions, and splitting data into training, validation, and test sets. If you have taken the Machine Learning Specialization or Deep Learning Specialization from DeepLearning.AI, you’ll be ready to take this course and dive deeper into the fundamentals of generative AI.

大语言模型生成式人工智能

由DeepLearning.AI和亚马逊网络服务合作出品

关于这门课程

在《生成式人工智能与大型语言模型（LLMs）》这门课程中，你将学习生成式人工智能的基本原理，以及如何将其应用于现实世界的问题。通过学习这门课程，你将能够：

1. 深入理解生成式人工智能，描述典型的基于LLM的生成式人工智能生命周期中的关键步骤，从数据收集和模型选择，到性能评估和部署
2. 详细描述支撑LLMs的转换器架构，它们是如何训练的，以及如何通过微调使LLMs适应各种特定的用例
3. 使用经验性缩放定律来优化模型的目标函数，涵盖数据集大小、计算预算和推理需求
4. 应用最先进的训练、调整、推理、工具和部署方法，以在项目的具体约束条件下最大化模型的性能
5. 在听取行业研究人员和实践者的故事后，讨论生成式人工智能为企业创造的挑战和机遇

对于具有LLMs工作原理以及训练和部署它们的最佳实践的良好基础了解的开发者来说，他们将能够为他们的公司做出明智的决策，并更快地构建工作原型。这门课程将支持学习者建立关于如何最好地利用这一令人兴奋的新技术的实践直觉。这是一门中级课程，因此你应该有一些Python编程经验才能从中受益。你还应该熟悉机器学习的基础知识，如监督学习和非监督学习、损失函数以及将数据划分为训练、验证和测试集。如果你已经学习了DeepLearning.AI的机器学习专业化或深度学习专业化，那么你将准备好参加这门课程，并更深入地了解生成式人工智能的基础知识。

Introduction to LLMs and generative AI project lifecycle

Course Introduction

Welcome to this course on generative AI with large language models. Large language models or LLMs are a very exciting technology. But despite all the buzz and hype, one of the thing that is still underestimated by many people is their power as a developer too. Specifically, there are many machine learning and AI applications that used to take me many months to build that you can now build in days or maybe even small numbers of weeks.

This course will take a deep dive with you into how LLM technology actually works including going through many of the technical details, like model training, instruction tuning, fine-tuning, the generative AI project life cycle framework to help you plan and execute your projects and so on. Generative AI and LLMs specifically are a general purpose technology. That means that similar to other general purpose technologies like deep learning and electricity, is useful not just for a single application, but for a lot of different applications that span many corners of the economy. Similar to the rise of deep learning that started maybe 15 years ago or so, there's a lot of important where it lies ahead of us that needs to be done over many years by many people, I hope including you, to identify use cases and build specific applications. Because a lot of with this technology is so new and so few people really know how to use them, many companies are also right now scrambling to try to find and hire people that actually know how to build applications with LLMs.

I hope that this course will also help you, if you wish, better position yourself to get one of those jobs. I'm thrilled to bring you this course along with a group of fantastic instructors from the AWS team, Antje Barth, Mike Chambers, Shelbee Eigenbrode who are here with me today, as well as a fourth instructor Chris Fregly, who will be presenting the labs. Antje and Mike are both generative AI developer advocates. Shelby and Chris are both generative AI solutions architects. All of them have a lot of experience helping many different companies build many, many creative applications using LLMs. I look forward to all of them sharing this rich hands-on experience in this course. We've develop the content for this course with inputs from many industry experts and applied scientists at Amazon, AWS, Hugging Face and many top universities around the world.

Andrea, perhaps you can say a bit more about this course.

Sure. Thanks Andrew. It's a pleasure to work with you again on this course and the exciting area of generative AI. With this course on generative AI with large language models, we've created a series of lessons meant for AI enthusiasts, engineers, or data scientists. Looking to learn the technical foundations of how LLMs work, as well as the best practices behind training, tuning, and deploying them. In terms of prerequisites, we assume you are already familiar with Python programming and at least basic data science and machine learning concepts. If you have some experience with either Python or TensorFlow, that should be enough. In this course, you will explore in detail the steps that make up a typical generative AI project lifecycle, from scoping the problem and selecting a language model to optimizing a model for deployment and integrating into your applications. This course covers all of the topics, not just at a shallow level, but we'll take the time to make sure you come away with a deep technical understanding of all of these technologies and be well-positioned to really know what you're doing when you build your own generative AI projects.

Mike, why don't you tell us a little bit more details about what the learners will see in each week?

Absolutely, Andrea. Thank you. In Week 1, you will examine the transformer architecture that powers large language models, explore how these models are trained, and understand the compute resources required to develop these powerful LLMs. You'll also learn about a technique called in-context learning. How to guide the model to output at inference time with prompt engineering, and how to tune the most important generation parameters of LLMs for tuning your model output. In Week 2, you'll explore options for adapting pre-trained models to specific tasks and datasets via a process called instruction fine tuning. Then in Week 3, you'll see how to align the output of language models with human values in order to increase helpfulness and decrease potential harm and toxicity. Though we don't stop at the theory. Each week includes a hands-on lab where you'll be able to try out these techniques for yourself in an AWS environment that includes all the resources you need for working with large models at no cost to you.

Shelby, can you tell us a little bit more about the hands-on labs?

Sure thing, Mike. In the first hands-on lab, you'll construct a compare different prompts and inputs for a given generative task, in this case, dialogue summarization. You'll also explore different inference parameters and sampling strategies to gain intuition on how to further improve the generative model of responses. In the second hands-on lab, you'll find tune it existing large language model from Hugging Face, a popular open-source model hub. You'll play with both full fine-tuning and parameter efficient fine tuning or PEFT for short. You'll see how PEFT lets you make your workflow much more efficient. In the third lab, you get hands-on with reinforcement learning from human feedback or RLHF, you'll build a reward model classifier to label model responses as either toxic or non-toxic. Don't worry if you don't understand all these terms and concepts just yet. You'll dive into each of these topics in much more detail throughout this course. I'm thrilled to have Andrea, Mike, Shelby as well as Tris presenting this course to you that takes a deep technical dive into LLMs. You come away from this course having practice with many different concrete code examples for how to build or use LLMs. I'm sure that many of the code snippets will end up being directly useful in your own work. I hope you enjoy the course and use what you learn to build some really exciting applications. So that, let's go on to the next video where we start diving into how LLMs are being used to build applications.

大语言模型及生成式AI项目生命周期简介——课程简介

欢迎来到这门关于生成式AI和大型语言模型的课程。大型语言模型或LLMs是一项非常令人兴奋的技术。但尽管围绕它的热议和炒作不断，很多人仍然低估了它们作为开发者的强大能力。具体来说，许多过去需要我花费数月时间构建的机器学习和AI应用，现在你可以在短时间内构建出来，可能只需几天或者几周的时间。

这门课程将带你深入了解LLM技术的实际运作方式，包括许多技术细节，如模型训练、指令调整、微调，以及生成式AI项目生命周期框架，帮助你规划和执行你的项目等等。生成式AI和LLMs特别是一种通用技术。这意味着它类似于深度学习和电力等其他通用技术，不仅仅对单一应用有用，而且对经济中许多不同领域的各种应用都有用。与大约15年前兴起的深度学习类似，我们面前还有很多重要的工作需要在未来许多年里由许多人来完成，我希望包括你在内，来识别用例并构建特定应用。由于这项技术的新颖性以及真正知道如何使用它们的人还很少，许多公司现在也在争相寻找并雇佣真正知道如何利用LLMs构建应用的人员。

我希望这门课程也能帮助你，如果你愿意的话，更好地定位自己以获得这些工作之一。我很激动能与AWS团队的一群出色讲师一起为你带来这门课程，包括Antje Barth、Mike Chambers、Shelbee Eigenbrode，他们今天也在场，以及第四位讲师Chris Fregly，他将负责实验室部分。Antje和Mike都是生成式AI开发倡导者。Shelby和Chris都是生成式AI解决方案架构师。他们都在帮助许多不同的公司使用LLMs构建了许多有创意的应用方面拥有丰富的经验。我期待着他们在本课程中分享这些丰富的实践经验。我们根据亚马逊、AWS、Hugging Face以及世界各地许多顶尖大学的行业专家和应用科学家的意见开发了这门课程的内容。

Andrea，或许你可以多说一些关于这门课程的内容。

当然。谢谢Andrew。再次与你合作开设这门课程以及令人兴奋的生成式AI领域是我的荣幸。在这门关于生成式AI和大型语言模型的课程中，我们创建了一系列课程，适合AI爱好者、工程师或数据科学家。希望学习LLMs工作的 technical foundations（技术基础），以及训练、调整和部署它们的最佳实践。就先决条件而言，我们假设你已经熟悉Python编程，并且至少具备基本的数据科学和机器学习概念。如果你有一些Python或TensorFlow的经验，那应该就足够了。在这门课程中，你将详细探索构成典型生成式AI项目生命周期的各个步骤，从确定问题范围和选择语言模型，到优化模型部署并将其集成到你的应用中。这门课程涵盖了所有这些主题，不仅仅是浅层次的，而且我们会花时间确保你对所有这些技术有深入的技术理解，并且在构建自己的生成式AI项目时能够真正知道你在做什么。

Mike，不妨告诉我们每周学习者将会看到哪些更多细节？

绝对可以，Andrea。谢谢你。在第一周，你将检查驱动大型语言模型的转换器架构，探索这些模型是如何被训练的，并理解开发这些强大的LLMs所需的计算资源。你还将学习一种称为上下文内学习的技术。如何在推理时通过提示工程指导模型输出，以及如何调整LLMs最重要的生成参数来调整模型输出。在第二周，你将探索如何通过一种称为指令微调的过程，将预训练模型适应特定任务和数据集。然后在第三周，你将了解如何使语言模型的输出与人类价值观保持一致，以提高帮助性并减少潜在的危害和毒性。虽然我们不会停留在理论上。每周都包括一个动手实验室，你将能够在包含你需要的所有资源的AWS环境中亲自尝试这些技术，无需支付任何费用。

Shelby，你能告诉我们更多关于动手实验室的信息吗？

当然，Mike。在第一个动手实验室中，你将构建一个比较不同提示和输入的给定生成任务，在这种情况下是对话摘要。你还将探索不同的推理参数和采样策略，以获得如何进一步改善响应生成模型的直觉。在第二个动手实验室中，你将对来自Hugging Face的现有大型语言模型进行微调，这是一个流行的开源模型库。你将尝试完全微调和参数高效微调（简称PEFT）。你会看到PEFT如何让你的工作流更加高效。在第三个实验室中，你将亲身体验来自人类反馈的强化学习或RLHF，你将构建一个奖励模型分类器来标记模型响应为有毒或无毒。如果这些术语和概念你还不理解，不用担心。你将在本课程中更详细地深入了解这些主题。

我很激动能有Andrea、Mike、Shelby以及Tris为你呈现这门深入探讨LLMs的技术课程。你将从这门课程中获得许多不同具体代码示例的实践，用于构建或使用LLMs。我相信许多代码片段将直接对你自己的工作有用。我希望你喜欢这门课程，并使用你学到的知识构建一些真正令人兴奋的应用。那么，让我们进入下一个视频，开始深入了解LLMs是如何被用来构建应用的。