LangChain向量数据库的使用

最新推荐文章于 2024-06-10 09:00:00 发布

hehui0921

最新推荐文章于 2024-06-10 09:00:00 发布

阅读量264

点赞数 1

分类专栏： LangChain 文章标签： langchain 数据库

本文链接：https://blog.csdn.net/oHeHui1/article/details/136042156

版权

LangChain 专栏收录该内容

59 篇文章 4 订阅

订阅专栏

本文介绍了如何使用Jina进行文本向量生成，通过Langchain的LLMChain和Chroma向量存储库，加载文本文件，分割并嵌入每个片段，最后实现基于上下文的问题回答功能。

摘要由CSDN通过智能技术生成

以下使用jina来做向量生成。存储使用Chroma 。

import ChatGLM
from langchain.chains import LLMChain
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

from langchain_community.tools.tavily_search import TavilySearchResults
from langchain.chains import LLMMathChain
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

from langchain_community.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain_community.vectorstores import Chroma


from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableParallel, RunnablePassthrough

# from langchain_openai.embeddings import OpenAIEmbeddings

from langchain_community.embeddings import JinaEmbeddings

# https://jina.ai/embeddings/
# https://python.langchain.com/docs/integrations/text_embedding/jina
# demo:  https://python.langchain.com/cookbook


embeddings = JinaEmbeddings(
    jina_api_key="jina_fa2c341a2f634f1381f7cfec767150caSconYmQA2XRAcVKfZ7-Zboaqeydu", model_name="jina-embeddings-v2-base-en"
)

# Load the document, split it into chunks, embed each chunk and load it into the vector store.
raw_documents = TextLoader('state_of_the_union.txt').load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
documents = text_splitter.split_documents(raw_documents)
vectorstore = Chroma.from_documents(documents, embeddings)
retriever = vectorstore.as_retriever()


template = """Answer the question based only on the following context:
{context}

Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)
llm = ChatGLM.ChatGLM_LLM()
output_parser = StrOutputParser()
setup_and_retrieval = RunnableParallel(
    {"context": retriever, "question": RunnablePassthrough()}
)
chain = setup_and_retrieval | prompt | llm | output_parser

print(chain.invoke("What did the president say about Ketanji Brown Jackson?"))