书生实战营（第三期）- 进阶关卡 - Lagent

自定义Agent智能体

---

---

Lagent介绍

• Lagent 是一个轻量级开源智能体框架

  • 可以让用户构建基于大语言模型的智能体
  • 提供一些典型工具以增强大语言模型的能力

• Lagent 目前支持的工具有

  • Arxiv 搜索
  • Bing 地图
  • Google学术搜索
  • Google搜索
  • 交互式 IPython 解释器
  • Python 解释器
  • PPT

环境配置

• 配置虚拟环境

# 创建环境
conda create -n agent_camp3 python=3.10 -y
# 激活环境
conda activate agent_camp3
# 安装 torch
conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia -y
# 安装其他依赖包
pip install termcolor==2.4.0
pip install lmdeploy==0.5.2

• 源码的方式安装lagent

# 创建目录以存放代码
mkdir -p /root/agent_camp3
cd /root/agent_camp3
git clone https://github.com/InternLM/lagent.git
cd lagent && git checkout 81e7ace && pip install -e . && cd ..

使用已有的智能体

1. 使用 LMDeploy部署InternLM2.5-7B-Chat,并启动一个API Server
   • lmdeploy serve api_server /share/new_models/Shanghai_AI_Laboratory/internlm2_5-7b-chat --model-name internlm2_5-7b-chat
2. 启动 Lagent 的 Web Demo
   • streamlit run examples/internlm2_agent_web_demo.py
   • 修改模型名称为：internlm2_5-7b-chat
   • 修改模型IP为：0.0.0.0:23333
   • 选择插件：ArxivSearch
3. 输入指令：请帮我在Arxiv上查询论文 MindSearch
   

自定义智能体

使用Lagent自定义工具分几步：

1. 继承BaseAction类
2. 实现工具方法
3. 工具方法要用@tool_api装饰

案例一：使用MagicMaker API以完成文生图

1. 在lagent/actions/目录下创建magicmaker.py
2. 工具调用实现：

# 引自 https://github.com/InternLM/Tutorial/blob/camp3/docs/L2/Lagent/readme.md
import json
import requests

from lagent.actions.base_action import BaseAction, tool_api
from lagent.actions.parser import BaseParser, JsonParser
from lagent.schema import ActionReturn, ActionStatusCode


class MagicMaker(BaseAction):
    styles_option = [
        'dongman',  # 动漫
        'guofeng',  # 国风
        'xieshi',   # 写实
        'youhua',   # 油画
        'manghe',   # 盲盒
    ]
    aspect_ratio_options = [
        '16:9', '4:3', '3:2', '1:1',
        '2:3', '3:4', '9:16'
    ]

    def __init__(self,
                 style='guofeng',
                 aspect_ratio='4:3'):
        super().__init__()
        if style in self.styles_option:
            self.style = style
        else:
            raise ValueError(f'The style must be one of {self.styles_option}')
        
        if aspect_ratio in self.aspect_ratio_options:
            self.aspect_ratio = aspect_ratio
        else:
            raise ValueError(f'The aspect ratio must be one of {aspect_ratio}')
    
    @tool_api
    def generate_image(self, keywords: str) -> dict:
        """Run magicmaker and get the generated image according to the keywords.

        Args:
            keywords (:class:`str`): the keywords to generate image

        Returns:
            :class:`dict`: the generated image
                * image (str): path to the generated image
        """
        try:
            response = requests.post(
                url='https://magicmaker.openxlab.org.cn/gw/edit-anything/api/v1/bff/sd/generate',
                data=json.dumps({
                    "official": True,
                    "prompt": keywords,
                    "style": self.style,
                    "poseT": False,
                    "aspectRatio": self.aspect_ratio
                }),
                headers={'content-type': 'application/json'}
            )
        except Exception as exc:
            return ActionReturn(
                errmsg=f'MagicMaker exception: {exc}',
                state=ActionStatusCode.HTTP_ERROR)
        image_url = response.json()['data']['imgUrl']
        return {'image': image_url}

3. 脚本``中新增插件

from lagent.actions import ActionExecutor, ArxivSearch, IPythonInterpreter
+ from lagent.actions.magicmaker import MagicMaker
from lagent.agents.internlm2_agent import INTERPRETER_CN, META_CN, PLUGIN_CN, Internlm2Agent, Internlm2Protocol

...
        action_list = [
            ArxivSearch(),
+             MagicMaker(),
        ]

4. 效果
   

• 生成的图像：
  

遇到的问题

三方库版本错误

1. 找不到模块 griffe.enumerations
   • pip install griffe==0.48.0

Package                   Version        Editable project location
------------------------- -------------- --------------------------------------
accelerate                0.33.0
addict                    2.4.0
altair                    5.4.0
annotated-types           0.7.0
anyio                     4.4.0
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
arxiv                     2.1.3
asttokens                 2.4.1
async-lru                 2.0.4
attrs                     24.2.0
babel                     2.16.0
backports.strenum         1.3.1
beautifulsoup4            4.12.3
bleach                    6.1.0
blinker                   1.8.2
Brotli                    1.0.9
cachetools                5.4.0
certifi                   2024.7.4
cffi                      1.17.0
charset-normalizer        3.3.2
click                     8.1.7
colorama                  0.4.6
comm                      0.2.2
debugpy                   1.8.5
decorator                 5.1.1
defusedxml                0.7.1
distro                    1.9.0
duckduckgo_search         5.3.1b1
einops                    0.8.0
exceptiongroup            1.2.2
executing                 2.0.1
fastapi                   0.112.1
fastjsonschema            2.20.0
feedparser                6.0.11
filelock                  3.13.1
fire                      0.6.0
fqdn                      1.5.1
fsspec                    2024.6.1
func_timeout              4.3.5
gitdb                     4.0.11
GitPython                 3.1.43
gmpy2                     2.1.2
griffe                    0.48.0
h11                       0.14.0
h2                        4.1.0
hpack                     4.0.0
httpcore                  1.0.5
httpx                     0.27.0
huggingface-hub           0.24.5
hyperframe                6.0.1
idna                      3.7
importlib_metadata        8.2.0
ipykernel                 6.29.5
ipython                   8.26.0
ipywidgets                8.1.3
isoduration               20.11.0
jedi                      0.19.1
Jinja2                    3.1.4
json5                     0.9.25
jsonpointer               3.0.0
jsonschema                4.23.0
jsonschema-specifications 2023.12.1
jupyter                   1.0.0
jupyter_client            8.6.2
jupyter-console           6.6.3
jupyter_core              5.7.2
jupyter-events            0.10.0
jupyter-lsp               2.2.5
jupyter_server            2.14.2
jupyter_server_terminals  0.5.3
jupyterlab                4.2.4
jupyterlab_pygments       0.3.0
jupyterlab_server         2.27.3
jupyterlab_widgets        3.0.11
lagent                    0.2.3          /root/tasks/task_L2/agent_camp3/lagent
lmdeploy                  0.5.2
markdown-it-py            3.0.0
MarkupSafe                2.1.3
matplotlib-inline         0.1.7
mdurl                     0.1.2
mistune                   3.0.2
mkl-fft                   1.3.8
mkl-random                1.2.4
mkl-service               2.4.0
mmengine-lite             0.10.4
mpmath                    1.3.0
narwhals                  1.4.2
nbclient                  0.10.0
nbconvert                 7.16.4
nbformat                  5.10.4
nest-asyncio              1.6.0
networkx                  3.3
notebook                  7.2.1
notebook_shim             0.2.4
numpy                     1.26.4
nvidia-cublas-cu12        12.6.0.22
nvidia-cuda-runtime-cu12  12.6.37
nvidia-curand-cu12        10.3.7.37
nvidia-nccl-cu12          2.22.3
overrides                 7.7.0
packaging                 24.1
pandas                    2.2.2
pandocfilters             1.5.1
parso                     0.8.4
peft                      0.11.1
pexpect                   4.9.0
phx-class-registry        4.1.0
pillow                    10.4.0
pip                       24.2
platformdirs              4.2.2
prometheus_client         0.20.0
prompt_toolkit            3.0.47
protobuf                  5.27.3
psutil                    6.0.0
ptyprocess                0.7.0
pure_eval                 0.2.3
pyarrow                   17.0.0
pycparser                 2.22
pydantic                  2.8.2
pydantic_core             2.20.1
pydeck                    0.9.1
Pygments                  2.18.0
pynvml                    11.5.3
PySocks                   1.7.1
python-dateutil           2.9.0.post0
python-json-logger        2.0.7
pytz                      2024.1
PyYAML                    6.0.1
pyzmq                     26.1.0
qtconsole                 5.5.2
QtPy                      2.4.1
referencing               0.35.1
regex                     2024.7.24
requests                  2.32.3
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rich                      13.7.1
rpds-py                   0.20.0
safetensors               0.4.4
Send2Trash                1.8.3
sentencepiece             0.2.0
setuptools                72.1.0
sgmllib3k                 1.0.0
shortuuid                 1.0.13
six                       1.16.0
smmap                     5.0.1
sniffio                   1.3.1
socksio                   1.0.0
soupsieve                 2.6
stack-data                0.6.3
starlette                 0.38.2
streamlit                 1.37.1
sympy                     1.12
tenacity                  8.5.0
termcolor                 2.4.0
terminado                 0.18.1
tiktoken                  0.7.0
timeout-decorator         0.5.0
tinycss2                  1.3.0
tokenizers                0.19.1
toml                      0.10.2
tomli                     2.0.1
torch                     2.1.2
torchaudio                2.1.2
torchvision               0.16.2
tornado                   6.4.1
tqdm                      4.66.5
traitlets                 5.14.3
transformers              4.44.0
triton                    2.1.0
types-python-dateutil     2.9.0.20240316
typing_extensions         4.11.0
tzdata                    2024.1
uri-template              1.3.0
urllib3                   2.2.2
uvicorn                   0.30.6
watchdog                  4.0.2
wcwidth                   0.2.13
webcolors                 24.8.0
webencodings              0.5.1
websocket-client          1.8.0
wheel                     0.43.0
widgetsnbextension        4.0.11
yapf                      0.40.2
zipp                      3.20.0

模型名称错误

• 在web demo上配置的模型名称错误，导致没有大语言模型的返回信息
  • KeyError: 'choices'
  • 需要跟启动的大语言模型Server的--model-name 保持一致