LangChain RAG 下册

路由Routing

完成第一步Query Translation之后就要进入第二步Routing，Routing的意义在于根据不同的问题类型走不同的策略，比如关系型数据库走nl2sql查询，向量查询走索引向量数据库查询。

逻辑和语义路由 Logical and Semantic routing

使用函数调用进行分类

1from typing import Literal
2
3from langchain_core.prompts import ChatPromptTemplate
4from langchain_core.pydantic_v1 import BaseModel, Field
5from langchain_openai import ChatOpenAI
6
7# Data model
8class RouteQuery(BaseModel):
9    """Route a user query to the most relevant datasource."""
10
11    datasource: Literal["python_docs", "js_docs", "golang_docs"] = Field(
12        ...,
13        description="Given a user question choose which datasource would be most relevant for answering their question",
14    )
15
16# LLM with function call 
17llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
18structured_llm = llm.with_structured_output(RouteQuery)
19
20# Prompt 
21system = """You are an expert at routing a user question to the appropriate data source.
22
23Based on the programming language the question is referring to, route it to the relevant data source."""
24
25prompt = ChatPromptTemplate.from_messages(
26    [
27        ("system", system),
28        ("human", "{question}"),
29    ]
30)
31
32# Define router 
33router = prompt | structured_llm

注意：我们使用函数调用来产生结构化输出。

1question = """Why doesn't the following code work:
2
3from langchain_core.prompts import ChatPromptTemplate
4
5prompt = ChatPromptTemplate.from_messages(["human", "speak in {language}"])
6prompt.invoke("french")
7"""
8
9result = router.invoke({"question": question})
10#result.datasource -> 'python_docs'
11def choose_route(result):
12    if "python_docs" in result.datasource.lower():
13        ### Logic here 
14        return "chain for python_docs"
15    elif "js_docs" in result.datasource.lower():
16        ### Logic here 
17        return "chain for js_docs"
18    else:
19        ### Logic here 
20        return "golang_docs"
21
22from langchain_core.runnables import RunnableLambda
23
24full_chain = router | RunnableLambda(choose_route)
25full_chain.invoke({"question": question})
26# 'chain for python_docs'
27

语义路由 Semantic routing

1from langchain.utils.math import cosine_similarity
2from langchain_core.output_parsers import StrOutputParser
3from langchain_core.prompts import PromptTemplate
4from langchain_core.runnables import RunnableLambda, RunnablePassthrough
5from langchain_openai import ChatOpenAI, OpenAIEmbeddings
6
7# Two prompts
8physics_template = """You are a very smart physics professor. \
9You are great at answering questions about physics in a concise and easy to understand manner. \
10When you don't know the answer to a question you admit that you don't know.
11
12Here is a question:
13{query}"""
14
15math_template = """You are a very good mathematician. You are great at answering math questions. \
16You are so good because you are able to break down hard problems into their component parts, \
17answer the component parts, and then put them together to answer the broader question.
18
19Here is a question:
20{query}"""
21
22# Embed prompts
23embeddings = OpenAIEmbeddings()
24prompt_templates = [physics_template, math_template]
25prompt_embeddings = embeddings.embed_documents(prompt_templates)
26
27# Route question to prompt 
28def prompt_router(input):
29    # Embed question
30    query_embedding = embeddings.embed_query(input["query"])
31    # Compute similarity
32    similarity = cosine_similarity([query_embedding], prompt_embeddings)[0]
33    most_similar = prompt_templates[similarity.argmax()]
34    # Chosen prompt 
35    print("Using MATH" if most_similar == math_template else "Using PHYSICS")
36    return PromptTemplate.from_template(most_similar)
37
38
39chain = (
40    {"query": RunnablePassthrough()}
41    | RunnableLambda(prompt_router)
42    | ChatOpenAI()
43    | StrOutputParser()
44)
45
46print(chain.invoke("What's a black hole"))

输出：

1Using PHYSICS
2A black hole is a region in space where the gravitational pull is so strong that nothing, not even light, can escape from it. It is formed when a massive star collapses in on itself. The boundary surrounding a black hole is called the event horizon. Beyond the event horizon, the gravitational pull is so intense that even time and space are distorted. Black holes are some of the most mysterious and fascinating objects in the universe.

结构化查询 Query Construction

元数据过滤器的查询结构 Query structuring for metadata filters

许多矢量存储包含元数据字段。这使得可以根据元数据过滤特定块成为可能。

1from langchain_community.document_loaders import YoutubeLoader
2
3docs = YoutubeLoader.from_youtube_url(
4    "https://www.youtube.com/watch?v=pbAd8O1Lvm4", add_video_info=True
5).load()
6
7docs[0].metadata

输出

1{'source': 'pbAd8O1Lvm4',
2 'title': 'Self-reflective RAG with LangGraph: Self-RAG and CRAG',
3 'description': 'Unknown',
4 'view_count': 11922,
5 'thumbnail_url': 'https://i.ytimg.com/vi/pbAd8O1Lvm4/hq720.jpg',
6 'publish_date': '2024-02-07 00:00:00',
7 'length': 1058,
8 'author': 'LangChain'}

假设我们已经建立了一个索引：

允许我们对每个文档的内容和标题进行非结构化搜索 并对查看次数、发布日期和长度使用范围过滤。

我们希望将自然语言转换为结构化搜索查询。我们可以为结构化搜索查询定义一个架构。

1import datetime
2from typing import Literal, Optional, Tuple
3from langchain_core.pydantic_v1 import BaseModel, Field
4
5class TutorialSearch(BaseModel):
6    """Search over a database of tutorial videos about a software library."""
7
8    content_search: str = Field(
9        ...,
10        description="Similarity search query applied to video transcripts.",
11    )
12    title_search: str = Field(
13        ...,
14        description=(
15            "Alternate version of the content search query to apply to video titles. "
16            "Should be succinct and only include key words that could be in a video "
17            "title."
18        ),
19    )
20    min_view_count: Optional[int] = Field(
21        None,
22        description="Minimum view count filter, inclusive. Only use if explicitly specified.",
23    )
24    max_view_count: Optional[int] = Field(
25        None,
26        description="Maximum view count filter, exclusive. Only use if explicitly specified.",
27    )
28    earliest_publish_date: Optional[datetime.date] = Field(
29        None,
30        description="Earliest publish date filter, inclusive. Only use if explicitly specified.",
31    )
32    latest_publish_date: Optional[datetime.date] = Field(
33        None,
34        description="Latest publish date filter, exclusive. Only use if explicitly specified.",
35    )
36    min_length_sec: Optional[int] = Field(
37        None,
38        description="Minimum video length in seconds, inclusive. Only use if explicitly specified.",
39    )
40    max_length_sec: Optional[int] = Field(
41        None,
42        description="Maximum video length in seconds, exclusive. Only use if explicitly specified.",
43    )
44
45    def pretty_print(self) -> None:
46        for field in self.__fields__:
47            if getattr(self, field) is not None and getattr(self, field) != getattr(
48                self.__fields__[field], "default", None
49            ):
50                print(f"{field}: {getattr(self, field)}")
51                
52from langchain_core.prompts import ChatPromptTemplate
53from langchain_openai import ChatOpenAI
54
55system = """You are an expert at converting user questions into database queries. \
56You have access to a database of tutorial videos about a software library for building LLM-powered applications. \
57Given a question, return a database query optimized to retrieve the most relevant results.
58
59If there are acronyms or words you are not familiar with, do not try to rephrase them."""
60prompt = ChatPromptTemplate.from_messages(
61    [
62        ("system", system),
63        ("human", "{question}"),
64    ]
65)
66llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
67structured_llm = llm.with_structured_output(TutorialSearch)
68query_analyzer = prompt | structured_llm
69query_analyzer.invoke({"question": "rag from scratch"}).pretty_print()                

输出：

1content_search: rag from scratch
2title_search: rag from scratch

1query_analyzer.invoke(
2    {"question": "videos on chat langchain published in 2023"}
3).pretty_print()

输出：

1content_search: chat langchain
2title_search: 2023
3earliest_publish_date: 2023-01-01
4latest_publish_date: 2024-01-01

1query_analyzer.invoke(
2    {"question": "videos that are focused on the topic of chat langchain that are published before 2024"}
3).pretty_print()

输出：

1content_search: chat langchain
2title_search: chat langchain
3earliest_publish_date: 2024-01-01

1query_analyzer.invoke(
2    {
3        "question": "how to use multi-modal models in an agent, only videos under 5 minutes"
4    }
5).pretty_print()

输出：

1content_search: multi-modal models agent
2title_search: multi-modal models agent
3max_length_sec: 300

索引Indexing

Multi-representation Indexing

1from langchain_community.document_loaders import WebBaseLoader
2from langchain_text_splitters import RecursiveCharacterTextSplitter
3
4loader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/")
5docs = loader.load()
6
7loader = WebBaseLoader("https://lilianweng.github.io/posts/2024-02-05-human-data-quality/")
8docs.extend(loader.load())
9
10import uuid
11
12from langchain_core.documents import Document
13from langchain_core.output_parsers import StrOutputParser
14from langchain_core.prompts import ChatPromptTemplate
15from langchain_openai import ChatOpenAI
16
17chain = (
18    {"doc": lambda x: x.page_content}
19    | ChatPromptTemplate.from_template("Summarize the following document:\n\n{doc}")
20    | ChatOpenAI(model="gpt-3.5-turbo",max_retries=0)
21    | StrOutputParser()
22)
23
24summaries = chain.batch(docs, {"max_concurrency": 5})
25
26from langchain.storage import InMemoryByteStore
27from langchain_openai import OpenAIEmbeddings
28from langchain_community.vectorstores import Chroma
29from langchain.retrievers.multi_vector import MultiVectorRetriever
30
31# The vectorstore to use to index the child chunks
32vectorstore = Chroma(collection_name="summaries",
33                     embedding_function=OpenAIEmbeddings())
34
35# The storage layer for the parent documents
36store = InMemoryByteStore()
37id_key = "doc_id"
38
39# The retriever
40retriever = MultiVectorRetriever(
41    vectorstore=vectorstore,
42    byte_store=store,
43    id_key=id_key,
44)
45doc_ids = [str(uuid.uuid4()) for _ in docs]
46
47# Docs linked to summaries
48summary_docs = [
49    Document(page_content=s, metadata={id_key: doc_ids[i]})
50    for i, s in enumerate(summaries)
51]
52
53# Add
54retriever.vectorstore.add_documents(summary_docs)
55retriever.docstore.mset(list(zip(doc_ids, docs)))
56
57query = "Memory in agents"
58sub_docs = vectorstore.similarity_search(query,k=1)
59sub_docs[0]

输出：

1Document(page_content='The document discusses the concept of building autonomous agents powered by Large Language Models (LLMs) as their core controllers. It covers components such as planning, memory, and tool use, along with case studies and proof-of-concept examples like AutoGPT and GPT-Engineer. Challenges like finite context length, planning difficulties, and reliability of natural language interfaces are also highlighted. The document provides references to related research papers and offers a comprehensive overview of LLM-powered autonomous agents.', metadata={'doc_id': 'cf31524b-fe6a-4b28-a980-f5687c9460ea'})

1retrieved_docs = retriever.get_relevant_documents(query,n_results=1)
2retrieved_docs[0].page_content[0:500]

输出：

1Number of requested results 4 is greater than number of elements in index 2, updating n_results = 2

1"\n\n\n\n\n\nLLM Powered Autonomous Agents | Lil'Log\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nLil'Log\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nPosts\n\n\n\n\nArchive\n\n\n\n\nSearch\n\n\n\n\nTags\n\n\n\n\nFAQ\n\n\n\n\nemojisearch.app\n\n\n\n\n\n\n\n\n\n      LLM Powered Autonomous Agents\n    \nDate: June 23, 2023  |  Estimated Reading Time: 31 min  |  Author: Lilian Weng\n\n\n \n\n\nTable of Contents\n\n\n\nAgent System Overview\n\nComponent One: Planning\n\nTask Decomposition\n\nSelf-Reflection\n\n\nComponent Two: Memory\n\nTypes of Memory\n\nMaximum Inner Product Search (MIPS)\n\n"

RAPTOR

ColBERT

RAGatouille 让 ColBERT 的使用变得非常简单。 ColBERT 为段落中的每个标记生成受上下文影响的向量。 ColBERT 类似地为查询中的每个标记生成向量。然后，每个文档的分数是每个查询嵌入与任何文档嵌入的最大相似度之和：

1from ragatouille import RAGPretrainedModel
2RAG = RAGPretrainedModel.from_pretrained("colbert-ir/colbertv2.0")
3
4import requests
5
6def get_wikipedia_page(title: str):
7    """
8    Retrieve the full text content of a Wikipedia page.
9
10    :param title: str - Title of the Wikipedia page.
11    :return: str - Full text content of the page as raw string.
12    """
13    # Wikipedia API endpoint
14    URL = "https://en.wikipedia.org/w/api.php"
15
16    # Parameters for the API request
17    params = {
18        "action": "query",
19        "format": "json",
20        "titles": title,
21        "prop": "extracts",
22        "explaintext": True,
23    }
24
25    # Custom User-Agent header to comply with Wikipedia's best practices
26    headers = {"User-Agent": "RAGatouille_tutorial/0.0.1 (ben@clavie.eu)"}
27
28    response = requests.get(URL, params=params, headers=headers)
29    data = response.json()
30
31    # Extracting page content
32    page = next(iter(data["query"]["pages"].values()))
33    return page["extract"] if "extract" in page else None
34
35full_document = get_wikipedia_page("Hayao_Miyazaki")
36
37RAG.index(
38    collection=[full_document],
39    index_name="Miyazaki-123",
40    max_document_length=180,
41    split_documents=True,
42)

1results = RAG.search(query="What animation studio did Miyazaki found?", k=3)
2results

1retriever = RAG.as_langchain_retriever(k=3)
2retriever.invoke("What animation studio did Miyazaki found?")

Retrieval

Re-ranking

1#### INDEXING ####
2
3# Load blog
4import bs4
5from langchain_community.document_loaders import WebBaseLoader
6loader = WebBaseLoader(
7    web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
8    bs_kwargs=dict(
9        parse_only=bs4.SoupStrainer(
10            class_=("post-content", "post-title", "post-header")
11        )
12    ),
13)
14blog_docs = loader.load()
15
16# Split
17from langchain.text_splitter import RecursiveCharacterTextSplitter
18text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
19    chunk_size=300, 
20    chunk_overlap=50)
21
22# Make splits
23splits = text_splitter.split_documents(blog_docs)
24
25# Index
26from langchain_openai import OpenAIEmbeddings
27# from langchain_cohere import CohereEmbeddings
28from langchain_community.vectorstores import Chroma
29vectorstore = Chroma.from_documents(documents=splits, 
30                                    # embedding=CohereEmbeddings()
31                                    embedding=OpenAIEmbeddings())
32
33
34retriever = vectorstore.as_retriever()
35
36from langchain.prompts import ChatPromptTemplate
37
38# RAG-Fusion
39template = """You are a helpful assistant that generates multiple search queries based on a single input query. \n
40Generate multiple search queries related to: {question} \n
41Output (4 queries):"""
42prompt_rag_fusion = ChatPromptTemplate.from_template(template)
43
44from langchain_core.output_parsers import StrOutputParser
45from langchain_openai import ChatOpenAI
46
47generate_queries = (
48    prompt_rag_fusion 
49    | ChatOpenAI(temperature=0)
50    | StrOutputParser() 
51    | (lambda x: x.split("\n"))
52)
53
54from langchain.load import dumps, loads
55
56def reciprocal_rank_fusion(results: list[list], k=60):
57    """ Reciprocal_rank_fusion that takes multiple lists of ranked documents 
58        and an optional parameter k used in the RRF formula """
59    
60    # Initialize a dictionary to hold fused scores for each unique document
61    fused_scores = {}
62
63    # Iterate through each list of ranked documents
64    for docs in results:
65        # Iterate through each document in the list, with its rank (position in the list)
66        for rank, doc in enumerate(docs):
67            # Convert the document to a string format to use as a key (assumes documents can be serialized to JSON)
68            doc_str = dumps(doc)
69            # If the document is not yet in the fused_scores dictionary, add it with an initial score of 0
70            if doc_str not in fused_scores:
71                fused_scores[doc_str] = 0
72            # Retrieve the current score of the document, if any
73            previous_score = fused_scores[doc_str]
74            # Update the score of the document using the RRF formula: 1 / (rank + k)
75            fused_scores[doc_str] += 1 / (rank + k)
76
77    # Sort the documents based on their fused scores in descending order to get the final reranked results
78    reranked_results = [
79        (loads(doc), score)
80        for doc, score in sorted(fused_scores.items(), key=lambda x: x[1], reverse=True)
81    ]
82
83    # Return the reranked results as a list of tuples, each containing the document and its fused score
84    return reranked_results
85
86question = "What is task decomposition for LLM agents?"
87retrieval_chain_rag_fusion = generate_queries | retriever.map() | reciprocal_rank_fusion
88docs = retrieval_chain_rag_fusion.invoke({"question": question})
89len(docs)

1from operator import itemgetter
2from langchain_core.runnables import RunnablePassthrough
3
4# RAG
5template = """Answer the following question based on this context:
6
7{context}
8
9Question: {question}
10"""
11
12prompt = ChatPromptTemplate.from_template(template)
13
14llm = ChatOpenAI(temperature=0)
15
16final_rag_chain = (
17    {"context": retrieval_chain_rag_fusion, 
18     "question": itemgetter("question")} 
19    | prompt
20    | llm
21    | StrOutputParser()
22)
23
24final_rag_chain.invoke({"question":question})

输出：

1'Task decomposition for LLM agents involves breaking down large tasks into smaller, manageable subgoals. This enables the agent to efficiently handle complex tasks by dividing them into smaller and simpler steps. Task decomposition can be achieved through techniques like Chain of Thought (CoT) and Tree of Thoughts, which prompt the model to think step by step and explore multiple reasoning possibilities at each step. Additionally, task decomposition can be done using simple prompting, task-specific instructions, or with human inputs.'

1from langchain_community.llms import Cohere
2from langchain.retrievers import  ContextualCompressionRetriever
3from langchain.retrievers.document_compressors import CohereRerank
4
5from langchain.retrievers.document_compressors import CohereRerank
6
7retriever = vectorstore.as_retriever(search_kwargs={"k": 10})
8
9# Re-rank
10compressor = CohereRerank()
11compression_retriever = ContextualCompressionRetriever(
12    base_compressor=compressor, base_retriever=retriever
13)
14
15compressed_docs = compression_retriever.get_relevant_documents(question)

如何学习AI大模型？

我在一线互联网企业工作十余年里，指导过不少同行后辈。帮助很多人得到了学习和成长。

我意识到有很多经验和知识值得分享给大家，也可以通过我们的能力和经验解答大家在人工智能学习中的很多困惑，所以在工作繁忙的情况下还是坚持各种整理和分享。但苦于知识传播途径有限，很多互联网行业朋友无法获得正确的资料得到学习提升，故此将并将重要的AI大模型资料包括AI大模型入门学习思维导图、精品AI大模型学习书籍手册、视频教程、实战学习等录播视频免费分享出来。

第一阶段： 从大模型系统设计入手，讲解大模型的主要方法；

第二阶段： 在通过大模型提示词工程从Prompts角度入手更好发挥模型的作用；

第三阶段： 大模型平台应用开发借助阿里云PAI平台构建电商领域虚拟试衣系统；

第四阶段： 大模型知识库应用开发以LangChain框架为例，构建物流行业咨询智能问答系统；

第五阶段： 大模型微调开发借助以大健康、新零售、新媒体领域构建适合当前领域大模型；

第六阶段： 以SD多模态大模型为主，搭建了文生图小程序案例；

第七阶段： 以大模型平台应用与开发为主，通过星火大模型，文心大模型等成熟大模型构建大模型行业应用。

👉学会后的收获：👈
• 基于大模型全栈工程实现（前端、后端、产品经理、设计、数据分析等），通过这门课可获得不同能力；

• 能够利用大模型解决相关实际项目需求： 大数据时代，越来越多的企业和机构需要处理海量数据，利用大模型技术可以更好地处理这些数据，提高数据分析和决策的准确性。因此，掌握大模型应用开发技能，可以让程序员更好地应对实际项目需求；

• 基于大模型和企业数据AI应用开发，实现大模型理论、掌握GPU算力、硬件、LangChain开发框架和项目实战技能， 学会Fine-tuning垂直训练大模型（数据准备、数据蒸馏、大模型部署）一站式掌握；

• 能够完成时下热门大模型垂直领域模型训练能力，提高程序员的编码能力： 大模型应用开发需要掌握机器学习算法、深度学习框架等技术，这些技术的掌握可以提高程序员的编码能力和分析能力，让程序员更加熟练地编写高质量的代码。

1.AI大模型学习路线图
2.100套AI大模型商业化落地方案
3.100集大模型视频教程
4.200本大模型PDF书籍
5.LLM面试题合集
6.AI产品经理资源合集

👉获取方式：
😝有需要的小伙伴，可以保存图片到wx扫描二v码免费领取【保证100%免费】🆓