如何使用LangChain调用Google的大模型Gemini

当今，大型语言模型（LLM）已经成为人工智能领域的热门话题。近期，Gemini发布了一款新的大模型，而LangChain也发布了调用接口。在这篇文章中，我们将介绍如何使用LangChain调用Google的大模型Gemini。

通过 ChatGoogleGenerativeAI 和 langchain-google-genai 集成包中的类可以访问 Google gemini 和 gemini-vision 模型。

安装开发包

%pip install -U --quiet langchain-google-genai pillow

引入类和配置GOOGLE_API_KEY

import getpass
import os

if "GOOGLE_API_KEY" not in os.environ:
    os.environ["GOOGLE_API_KEY"] = getpass("Provide your Google API Key")

调用Gemini模型

from langchain_google_genai import ChatGoogleGenerativeAI

llm = ChatGoogleGenerativeAI(model="gemini-pro")
result = llm.invoke("Write a ballad about LangChain")
print(result.content)

In realms where data streams like fervent tides,
Where algorithms dance and knowledge abides,
A tale unfolds of LangChain, grand and bold,
A ballad sung in bits and bytes untold.

Amidst the codes and circuits' hum,
A spark ignited, a vision would come.
From minds of brilliance, a tapestry formed,
A model to learn, to comprehend, to transform.

In layers deep, its architecture wove,
A neural network, ever-growing, in love.
With language's essence, it sought to entwine,
To unlock the secrets, each word's design.

From texts vast and varied, it feasted and learned,
Its grasp on meaning, swiftly discerned.
Context and syntax, it embraced with grace,
Unraveling stories, at an astonishing pace.

Translations sprang forth, with seamless art,
Bridging tongues and weaving hearts apart.
From English to French, Chinese to Spanish, behold,
LangChain's prowess, like a language untold.

It summarized texts, with insights profound,
Extracting knowledge, without a sound.
Questions it answered, with eloquence rare,
A digital sage, beyond compare.

Yet, its journey faced trials and tribulations,
Obstacles that tested its dedication.
Biases and errors, it sought to transcend,
For fairness and accuracy, it would contend.

With every challenge, it emerged more strong,
Adaptive and resilient, all along.
For LangChain's purpose was etched in its core,
To empower humans, forevermore.

In classrooms and workplaces, it lent its hand,
A tireless assistant, at every demand.
It aided students, in their quests for knowledge,
And professionals thrived, with its guidance and homage.

As years unfurled, its fame grew wide,
A testament to its unwavering stride.
Researchers and scholars, they all took heed,
Of LangChain's brilliance, a groundbreaking deed.

And so, the ballad of LangChain resounds,
A tribute to progress, where innovation abounds.
In the annals of AI, its name shall be etched,
A pioneer, forever in our hearts sketched.

流式输出方式

ChatGoogleGenerativeAI 本机支持流式输出。

for chunk in llm.stream("Write a limerick about LLMs."):
    print(chunk.content)
    print("---")
#Note that each chunk may contain more than one "token"

There once was an AI named Bert,
Whose language skills were quite expert.
---

With a vast dataset,
It could chat, even bet,
And write limericks, for what it's worth.
---

results = llm.batch(
    [
        "What's 2+2?",
        "What's 3+5?",
    ]
)
for res in results:
    print(res.content)

4
8

支持上传图片

要提供图像，请传递内容为 类型的 List[dict] 人工消息，其中每个字典都包含图像值（type of ）或文本（type of image_url text ）值。的值 image_url 可以是以下任何值：

• 公开可访问的图片URL

• 可访问的 gcs 文件（例如，“gcs://path/to/file.png”）

• 本地文件路径

• base64 编码的图像（例如， data:image/png;base64,abcd124 ）

• PIL 图像

import requests
from IPython.display import Image

image_url = "https://picsum.photos/seed/picsum/300/300"
content = requests.get(image_url).content
Image(content)

from langchain_core.messages import HumanMessage
from langchain_google_genai import ChatGoogleGenerativeAI

llm = ChatGoogleGenerativeAI(model="gemini-pro-vision")
example
message = HumanMessage(
    content=[
        {
            "type": "text",
            "text": "What's in this image?",
        },  You can optionally provide text parts
        {"type": "image_url", "image_url": image_url},
    ]
)
llm.invoke([message])

AIMessage(content=' The image contains a snow-capped mountain peak.')

Gemini提示词常见问题

截至本文档撰写之时（2024 年 12 月 12 日），Gemini 对其接受的提示类型和结构有一些限制。具体说来：

1. 提供多模态（图像）输入时，您最多只能收到 1 条“人类”（用户）类型的消息。您不能传递多条消息（尽管单个人工消息可能有多个内容条目）

2. 不接受系统消息。

3. 对于常规聊天对话，消息必须遵循 human/ai/human/ai 交替模式。您不能按顺序提供 2 条 AI 或人工消息。

4. 如果消息违反了 LLM 的安全检查，则可能会被阻止。在这种情况下，模型将返回一个空响应。