使用MongoDB Atlas实现高效向量存储查询：一个实用指南

llzwxh888

于 2024-10-08 18:18:47 发布

阅读量172

点赞数 2

文章标签： mongodb 线性代数数据库 python

本文链接：https://blog.csdn.net/ppoojjj/article/details/142766150

版权

引言

MongoDB Atlas是一个强大的文档数据库，不仅可以作为传统数据库使用，还可以用作向量数据库。这篇文章将演示如何通过MongoDB Atlas向量存储与SelfQueryRetriever结合，来进行高效查询。我们将使用一个简化的电影摘要数据集进行说明。

主要内容

创建MongoDB Atlas VectorStore

在开始之前，确保安装必要的库：

%pip install --upgrade --quiet lark pymongo

接下来，我们需要获取OpenAI API密钥：

import os

OPENAI_API_KEY = "Use your OpenAI key"
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY

然后，使用以下设置连接到MongoDB Atlas并创建向量存储：

from langchain_community.vectorstores import MongoDBAtlasVectorSearch
from langchain_core.documents import Document
from langchain_openai import OpenAIEmbeddings
from pymongo import MongoClient

# 替换你的MongoDB Atlas连接字符串
CONNECTION_STRING = "Use your MongoDB Atlas connection string"
DB_NAME = "Name of your MongoDB Atlas database"
COLLECTION_NAME = "Name of your collection in the database"
INDEX_NAME = "Name of a search index defined on the collection"

# 使用API代理服务提高访问稳定性
MongoClient = MongoClient(CONNECTION_STRING)
collection = MongoClient[DB_NAME][COLLECTION_NAME]

embeddings = OpenAIEmbeddings()

docs = [
    Document(
        page_content="A bunch of scientists bring back dinosaurs and mayhem breaks loose",
        metadata={"year": 1993, "rating": 7.7, "genre": "action"},
    ),
    # 更多文档...
]

vectorstore = MongoDBAtlasVectorSearch.from_documents(
    docs,
    embeddings,
    collection=collection,
    index_name=INDEX_NAME,
)

创建向量搜索索引

在MongoDB Atlas中定义搜索索引：

{
  "mappings": {
    "dynamic": true,
    "fields": {
      "embedding": {
        "dimensions": 1536,
        "similarity": "cosine",
        "type": "knnVector"
      },
      "genre": {
        "type": "token"
      },
      "ratings": {
        "type": "number"
      },
      "year": {
        "type": "number"
      }
    }
  }
}

实现自查询检索器

使用以下代码实例化SelfQueryRetriever：

from langchain.chains.query_constructor.base import AttributeInfo
from langchain.retrievers.self_query.base import SelfQueryRetriever
from langchain_openai import OpenAI

metadata_field_info = [
    AttributeInfo(name="genre", description="The genre of the movie", type="string"),
    # 更多字段...
]

document_content_description = "Brief summary of a movie"
llm = OpenAI(temperature=0)

retriever = SelfQueryRetriever.from_llm(
    llm, vectorstore, document_content_description, metadata_field_info, verbose=True
)

测试检索器

试用检索器：

retriever.invoke("What are some movies about dinosaurs")
retriever.invoke("What are some highly rated movies (above 9)?")
retriever.invoke("I want to watch a movie about toys rated higher than 9")