从 Vectara 索引实现自动检索¶

本指南展示如何在 LlamaIndex 中结合 Vectara 执行自动检索功能。

通过自动检索技术，我们会在向 Vectara 提交检索查询前对其进行分析，识别可能的重写方案——将查询简化为更短的形式并附加元数据过滤条件。

例如，类似"2022年收入是多少"的查询可能被重写为"收入是多少"并附加"doc.year = 2022"的过滤条件。下面我们通过示例来演示其工作原理。

安装¶

如果您在 Colab 上打开此 Notebook，可能需要安装 LlamaIndex 🦙。

In [ ]:

Copied!

!pip install llama_index llama-index-llms-openai llama-index-indices-managed-vectara
!pip install llama_index llama-index-llms-openai llama-index-indices-managed-vectara

Requirement already satisfied: llama_index in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (0.10.37)
Requirement already satisfied: llama-index-llms-openai in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (0.1.19)
Requirement already satisfied: llama-index-indices-managed-vectara in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (0.1.3)
Requirement already satisfied: llama-index-agent-openai<0.3.0,>=0.1.4 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.1.5)
Requirement already satisfied: llama-index-cli<0.2.0,>=0.1.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.1.7)
Requirement already satisfied: llama-index-core<0.11.0,>=0.10.35 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.10.37)
Requirement already satisfied: llama-index-embeddings-openai<0.2.0,>=0.1.5 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.1.6)
Requirement already satisfied: llama-index-indices-managed-llama-cloud<0.2.0,>=0.1.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.1.3)
Requirement already satisfied: llama-index-legacy<0.10.0,>=0.9.48 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.9.48)
Requirement already satisfied: llama-index-multi-modal-llms-openai<0.2.0,>=0.1.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.1.4)
Requirement already satisfied: llama-index-program-openai<0.2.0,>=0.1.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.1.4)
Requirement already satisfied: llama-index-question-gen-openai<0.2.0,>=0.1.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.1.3)
Requirement already satisfied: llama-index-readers-file<0.2.0,>=0.1.4 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.1.6)
Requirement already satisfied: llama-index-readers-llama-parse<0.2.0,>=0.1.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama_index) (0.1.3)
Requirement already satisfied: llama-index-vector-stores-chroma<0.2.0,>=0.1.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.1.5)
Requirement already satisfied: PyYAML>=6.0.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (6.0.1)
Requirement already satisfied: SQLAlchemy>=1.4.49 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.11.0,>=0.10.35->llama_index) (2.0.14)
Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (3.9.5)
Requirement already satisfied: dataclasses-json in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (0.6.5)
Requirement already satisfied: deprecated>=1.2.9.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (1.2.14)
Requirement already satisfied: dirtyjson<2.0.0,>=1.0.8 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (1.0.8)
Requirement already satisfied: fsspec>=2023.5.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (2024.3.1)
Requirement already satisfied: httpx in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (0.27.0)
Requirement already satisfied: jsonpath-ng<2.0.0,>=1.6.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (1.6.1)
Requirement already satisfied: llamaindex-py-client<0.2.0,>=0.1.18 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (0.1.18)
Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (1.6.0)
Requirement already satisfied: networkx>=3.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (3.2.1)
Requirement already satisfied: nltk<4.0.0,>=3.8.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (3.8.1)
Requirement already satisfied: numpy in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (1.26.4)
Requirement already satisfied: openai>=1.1.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (1.30.1)
Requirement already satisfied: pandas in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (2.2.2)
Requirement already satisfied: pillow>=9.0.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (10.3.0)
Requirement already satisfied: requests>=2.31.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (2.31.0)
Requirement already satisfied: spacy<4.0.0,>=3.7.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (3.7.4)
Requirement already satisfied: tenacity<9.0.0,>=8.2.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (8.2.3)
Requirement already satisfied: tiktoken>=0.3.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (0.6.0)
Requirement already satisfied: tqdm<5.0.0,>=4.66.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (4.66.4)
Requirement already satisfied: typing-extensions>=4.5.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (4.11.0)
Requirement already satisfied: typing-inspect>=0.8.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (0.9.0)
Requirement already satisfied: wrapt in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-core<0.11.0,>=0.10.35->llama_index) (1.16.0)
Requirement already satisfied: beautifulsoup4<5.0.0,>=4.12.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama_index) (4.12.3)
Requirement already satisfied: bs4<0.0.3,>=0.0.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama_index) (0.0.2)
Requirement already satisfied: pymupdf<2.0.0,>=1.23.21 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama_index) (1.23.26)
Requirement already satisfied: pypdf<5.0.0,>=4.0.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-readers-file<0.2.0,>=0.1.4->llama_index) (4.2.0)
Requirement already satisfied: llama-parse<0.4.0,>=0.3.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-readers-llama-parse<0.2.0,>=0.1.2->llama_index) (0.3.5)
Requirement already satisfied: aiosignal>=1.1.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama_index) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama_index) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.9.4)
Requirement already satisfied: soupsieve>1.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from beautifulsoup4<5.0.0,>=4.12.3->llama-index-readers-file<0.2.0,>=0.1.4->llama_index) (2.5)
Requirement already satisfied: ply in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from jsonpath-ng<2.0.0,>=1.6.0->llama-index-core<0.11.0,>=0.10.35->llama_index) (3.11)
Requirement already satisfied: chromadb<0.5.0,>=0.4.22 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.4.24)
Requirement already satisfied: onnxruntime<2.0.0,>=1.17.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.18.0)
Requirement already satisfied: tokenizers<0.16.0,>=0.15.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.15.2)
Requirement already satisfied: pydantic>=1.10 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from llamaindex-py-client<0.2.0,>=0.1.18->llama-index-core<0.11.0,>=0.10.35->llama_index) (2.7.1)
Requirement already satisfied: anyio in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama_index) (4.3.0)
Requirement already satisfied: certifi in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama_index) (2024.2.2)
Requirement already satisfied: httpcore==1.* in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.0.5)
Requirement already satisfied: idna in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama_index) (2.10)
Requirement already satisfied: sniffio in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from httpx->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.3.1)
Requirement already satisfied: h11<0.15,>=0.13 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from httpcore==1.*->httpx->llama-index-core<0.11.0,>=0.10.35->llama_index) (0.14.0)
Requirement already satisfied: click in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from nltk<4.0.0,>=3.8.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (8.1.7)
Requirement already satisfied: joblib in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from nltk<4.0.0,>=3.8.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.4.2)
Requirement already satisfied: regex>=2021.8.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from nltk<4.0.0,>=3.8.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (2024.4.28)
Requirement already satisfied: distro<2,>=1.7.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from openai>=1.1.0->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.9.0)
Requirement already satisfied: PyMuPDFb==1.23.22 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from pymupdf<2.0.0,>=1.23.21->llama-index-readers-file<0.2.0,>=0.1.4->llama_index) (1.23.22)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from requests>=2.31.0->llama-index-core<0.11.0,>=0.10.35->llama_index) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from requests>=2.31.0->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.26.18)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.2.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (8.2.3)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.1.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (2.0.10)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (0.3.4)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (0.9.0)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (6.4.0)
Requirement already satisfied: jinja2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (3.1.4)
Requirement already satisfied: setuptools in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (68.2.2)
Requirement already satisfied: packaging>=20.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (23.2)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (3.4.0)
Requirement already satisfied: greenlet!=0.4.17 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.11.0,>=0.10.35->llama_index) (3.0.3)
Requirement already satisfied: mypy-extensions>=0.3.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from typing-inspect>=0.8.0->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.0.0)
Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from dataclasses-json->llama-index-core<0.11.0,>=0.10.35->llama_index) (3.21.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from pandas->llama-index-core<0.11.0,>=0.10.35->llama_index) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from pandas->llama-index-core<0.11.0,>=0.10.35->llama_index) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from pandas->llama-index-core<0.11.0,>=0.10.35->llama_index) (2024.1)
Requirement already satisfied: build>=1.0.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.1.1)
Requirement already satisfied: chroma-hnswlib==0.7.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.7.3)
Requirement already satisfied: fastapi>=0.95.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.110.0)
Requirement already satisfied: uvicorn>=0.18.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.27.1)
Requirement already satisfied: posthog>=2.4.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (3.5.0)
Requirement already satisfied: pulsar-client>=3.1.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (3.4.0)
Requirement already satisfied: opentelemetry-api>=1.2.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.21.0)
Requirement already satisfied: opentelemetry-exporter-otlp-proto-grpc>=1.2.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.21.0)
Requirement already satisfied: opentelemetry-instrumentation-fastapi>=0.41b0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.42b0)
Requirement already satisfied: opentelemetry-sdk>=1.2.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.21.0)
Requirement already satisfied: pypika>=0.48.9 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.48.9)
Requirement already satisfied: overrides>=7.3.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (7.7.0)
Requirement already satisfied: importlib-resources in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (6.1.2)
Requirement already satisfied: grpcio>=1.58.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.63.0)
Requirement already satisfied: bcrypt>=4.0.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (4.1.2)
Requirement already satisfied: kubernetes>=28.1.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (29.0.0)
Requirement already satisfied: mmh3>=4.0.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (4.1.0)
Requirement already satisfied: orjson>=3.9.12 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (3.9.15)
Requirement already satisfied: language-data>=1.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from langcodes<4.0.0,>=3.2.0->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.2.0)
Requirement already satisfied: coloredlogs in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from onnxruntime<2.0.0,>=1.17.0->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (15.0.1)
Requirement already satisfied: flatbuffers in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from onnxruntime<2.0.0,>=1.17.0->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (24.3.25)
Requirement already satisfied: protobuf in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from onnxruntime<2.0.0,>=1.17.0->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (4.25.3)
Requirement already satisfied: sympy in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from onnxruntime<2.0.0,>=1.17.0->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.12)
Requirement already satisfied: annotated-types>=0.4.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from pydantic>=1.10->llamaindex-py-client<0.2.0,>=0.1.18->llama-index-core<0.11.0,>=0.10.35->llama_index) (0.6.0)
Requirement already satisfied: pydantic-core==2.18.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from pydantic>=1.10->llamaindex-py-client<0.2.0,>=0.1.18->llama-index-core<0.11.0,>=0.10.35->llama_index) (2.18.2)
Requirement already satisfied: six>=1.5 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.16.0)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from thinc<8.3.0,>=8.2.2->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from thinc<8.3.0,>=8.2.2->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (0.1.4)
Requirement already satisfied: huggingface_hub<1.0,>=0.16.4 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from tokenizers<0.16.0,>=0.15.1->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.23.0)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from weasel<0.4.0,>=0.1.0->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (0.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from jinja2->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (2.1.5)
Requirement already satisfied: pyproject_hooks in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from build>=1.0.3->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.0.0)
Requirement already satisfied: starlette<0.37.0,>=0.36.3 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from fastapi>=0.95.2->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.36.3)
Requirement already satisfied: filelock in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers<0.16.0,>=0.15.1->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (3.14.0)
Requirement already satisfied: google-auth>=1.0.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (2.16.1)
Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.7.0)
Requirement already satisfied: requests-oauthlib in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (2.0.0)
Requirement already satisfied: oauthlib>=3.2.2 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (3.2.2)
Requirement already satisfied: marisa-trie>=0.7.7 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from language-data>=1.2->langcodes<4.0.0,>=3.2.0->spacy<4.0.0,>=3.7.1->llama-index-core<0.11.0,>=0.10.35->llama_index) (1.1.1)
Requirement already satisfied: importlib-metadata<7.0,>=6.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-api>=1.2.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (6.11.0)
Requirement already satisfied: backoff<3.0.0,>=1.10.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (2.2.1)
Requirement already satisfied: googleapis-common-protos~=1.52 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.63.0)
Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.21.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.21.0)
Requirement already satisfied: opentelemetry-proto==1.21.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.21.0)
Requirement already satisfied: opentelemetry-instrumentation-asgi==0.42b0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.42b0)
Requirement already satisfied: opentelemetry-instrumentation==0.42b0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.42b0)
Requirement already satisfied: opentelemetry-semantic-conventions==0.42b0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.42b0)
Requirement already satisfied: opentelemetry-util-http==0.42b0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.42b0)
Requirement already satisfied: asgiref~=3.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from opentelemetry-instrumentation-asgi==0.42b0->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (3.7.2)
Requirement already satisfied: monotonic>=1.5 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from posthog>=2.4.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.6)
Requirement already satisfied: httptools>=0.5.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.6.1)
Requirement already satisfied: python-dotenv>=0.13 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.0.0)
Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.19.0)
Requirement already satisfied: watchfiles>=0.13 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.21.0)
Requirement already satisfied: websockets>=10.4 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (12.0)
Requirement already satisfied: humanfriendly>=9.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from coloredlogs->onnxruntime<2.0.0,>=1.17.0->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (10.0)
Requirement already satisfied: mpmath>=0.19 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from sympy->onnxruntime<2.0.0,>=1.17.0->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (1.3.0)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (5.3.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.4.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (4.9)
Requirement already satisfied: zipp>=0.5 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from importlib-metadata<7.0,>=6.0->opentelemetry-api>=1.2.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (3.18.1)
Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /Users/ofer/miniconda3/envs/langchain/lib/python3.11/site-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes>=28.1.0->chromadb<0.5.0,>=0.4.22->llama-index-vector-stores-chroma<0.2.0,>=0.1.1->llama-index-cli<0.2.0,>=0.1.2->llama_index) (0.6.0)

In [ ]:

Copied!

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

In [ ]:

Copied!





from llama_index.core.schema import TextNode
from llama_index.core.indices.managed.types import ManagedIndexQueryMode
from llama_index.indices.managed.vectara import VectaraIndex
from llama_index.indices.managed.vectara import VectaraAutoRetriever

from llama_index.core.vector_stores import MetadataInfo, VectorStoreInfo

from llama_index.llms.openai import OpenAI
from llama_index.core.schema import TextNode
from llama_index.core.indices.managed.types import ManagedIndexQueryMode
from llama_index.indices.managed.vectara import VectaraIndex
from llama_index.indices.managed.vectara import VectaraAutoRetriever

from llama_index.core.vector_stores import MetadataInfo, VectorStoreInfo

from llama_index.llms.openai import OpenAI

定义示例数据¶

我们首先定义一个电影数据集：

每个节点描述一部电影。
text 字段描述电影内容，而 metadata 定义了特定元数据字段，如年份、导演、评分或类型。

在 Vectara 平台上，您需要将这些元数据字段定义为可过滤属性，以便基于它们进行筛选操作。

In [ ]:

Copied!





nodes = [
    TextNode(
        text=(
            "A pragmatic paleontologist touring an almost complete theme park on an island "
            + "in Central America is tasked with protecting a couple of kids after a power "
            + "failure causes the park's cloned dinosaurs to run loose."
        ),
        metadata={"year": 1993, "rating": 7.7, "genre": "science fiction"},
    ),
    TextNode(
        text=(
            "A thief who steals corporate secrets through the use of dream-sharing technology "
            + "is given the inverse task of planting an idea into the mind of a C.E.O., "
            + "but his tragic past may doom the project and his team to disaster."
        ),
        metadata={
            "year": 2010,
            "director": "Christopher Nolan",
            "rating": 8.2,
        },
    ),
    TextNode(
        text="Barbie suffers a crisis that leads her to question her world and her existence.",
        metadata={
            "year": 2023,
            "director": "Greta Gerwig",
            "genre": "fantasy",
            "rating": 9.5,
        },
    ),
    TextNode(
        text=(
            "A cowboy doll is profoundly threatened and jealous when a new spaceman action "
            + "figure supplants him as top toy in a boy's bedroom."
        ),
        metadata={"year": 1995, "genre": "animated", "rating": 8.3},
    ),
    TextNode(
        text=(
            "When Woody is stolen by a toy collector, Buzz and his friends set out on a "
            + "rescue mission to save Woody before he becomes a museum toy property with his "
            + "roundup gang Jessie, Prospector, and Bullseye. "
        ),
        metadata={"year": 1999, "genre": "animated", "rating": 7.9},
    ),
    TextNode(
        text=(
            "The toys are mistakenly delivered to a day-care center instead of the attic "
            + "right before Andy leaves for college, and it's up to Woody to convince the "
            + "other toys that they weren't abandoned and to return home."
        ),
        metadata={"year": 2010, "genre": "animated", "rating": 8.3},
    ),
]
nodes = [
    TextNode(
        text=(
            "A pragmatic paleontologist touring an almost complete theme park on an island "
            + "in Central America is tasked with protecting a couple of kids after a power "
            + "failure causes the park's cloned dinosaurs to run loose."
        ),
        metadata={"year": 1993, "rating": 7.7, "genre": "science fiction"},
    ),
    TextNode(
        text=(
            "A thief who steals corporate secrets through the use of dream-sharing technology "
            + "is given the inverse task of planting an idea into the mind of a C.E.O., "
            + "but his tragic past may doom the project and his team to disaster."
        ),
        metadata={
            "year": 2010,
            "director": "Christopher Nolan",
            "rating": 8.2,
        },
    ),
    TextNode(
        text="Barbie suffers a crisis that leads her to question her world and her existence.",
        metadata={
            "year": 2023,
            "director": "Greta Gerwig",
            "genre": "fantasy",
            "rating": 9.5,
        },
    ),
    TextNode(
        text=(
            "A cowboy doll is profoundly threatened and jealous when a new spaceman action "
            + "figure supplants him as top toy in a boy's bedroom."
        ),
        metadata={"year": 1995, "genre": "animated", "rating": 8.3},
    ),
    TextNode(
        text=(
            "When Woody is stolen by a toy collector, Buzz and his friends set out on a "
            + "rescue mission to save Woody before he becomes a museum toy property with his "
            + "roundup gang Jessie, Prospector, and Bullseye. "
        ),
        metadata={"year": 1999, "genre": "animated", "rating": 7.9},
    ),
    TextNode(
        text=(
            "The toys are mistakenly delivered to a day-care center instead of the attic "
            + "right before Andy leaves for college, and it's up to Woody to convince the "
            + "other toys that they weren't abandoned and to return home."
        ),
        metadata={"year": 2010, "genre": "animated", "rating": 8.3},
    ),
]

然后我们将样本数据加载到 Vectara 索引中。

In [ ]:

Copied!

import os

os.environ["VECTARA_API_KEY"] = "<YOUR_VECTARA_API_KEY>"
os.environ["VECTARA_CORPUS_ID"] = "<YOUR_VECTARA_CORPUS_ID>"
os.environ["VECTARA_CUSTOMER_ID"] = "<YOUR_VECTARA_CUSTOMER_ID>"

index = VectaraIndex(nodes=nodes)
import os

os.environ["VECTARA_API_KEY"] = ""
os.environ["VECTARA_CORPUS_ID"] = ""
os.environ["VECTARA_CUSTOMER_ID"] = ""

index = VectaraIndex(nodes=nodes)

定义 `VectorStoreInfo`¶

我们定义一个 VectorStoreInfo 对象，该对象包含了对 Vectara 索引所支持的元数据过滤器的结构化描述。这些信息后续将用于自动检索提示，使大语言模型能够推断出针对特定查询应使用的元数据过滤器。

In [ ]:

Copied!





vector_store_info = VectorStoreInfo(
    content_info="information about a movie",
    metadata_info=[
        MetadataInfo(
            name="genre",
            description="""
                The genre of the movie. 
                One of ['science fiction', 'fantasy', 'comedy', 'drama', 'thriller', 'romance', 'action', 'animated']
            """,
            type="string",
        ),
        MetadataInfo(
            name="year",
            description="The year the movie was released",
            type="integer",
        ),
        MetadataInfo(
            name="director",
            description="The name of the movie director",
            type="string",
        ),
        MetadataInfo(
            name="rating",
            description="A 1-10 rating for the movie",
            type="float",
        ),
    ],
)
vector_store_info = VectorStoreInfo(
    content_info="information about a movie",
    metadata_info=[
        MetadataInfo(
            name="genre",
            description="""
                The genre of the movie. 
                One of ['science fiction', 'fantasy', 'comedy', 'drama', 'thriller', 'romance', 'action', 'animated']
            """,
            type="string",
        ),
        MetadataInfo(
            name="year",
            description="The year the movie was released",
            type="integer",
        ),
        MetadataInfo(
            name="director",
            description="The name of the movie director",
            type="string",
        ),
        MetadataInfo(
            name="rating",
            description="A 1-10 rating for the movie",
            type="float",
        ),
    ],
)

运行自动检索¶

现在让我们创建一个 VectaraAutoRetriever 实例并尝试 retrieve() 方法：

In [ ]:

Copied!





from llama_index.indices.managed.vectara import VectaraAutoRetriever
from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-3.5-turbo", temperature=0)

retriever = VectaraAutoRetriever(
    index,
    vector_store_info=vector_store_info,
    llm=llm,
    verbose=True,
)
from llama_index.indices.managed.vectara import VectaraAutoRetriever
from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-3.5-turbo", temperature=0)

retriever = VectaraAutoRetriever(
    index,
    vector_store_info=vector_store_info,
    llm=llm,
    verbose=True,
)

In [ ]:

Copied!

retriever.retrieve("movie directed by Greta Gerwig")
retriever.retrieve("movie directed by Greta Gerwig")

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
Using query str: movie directed by Greta Gerwig
Using implicit filters: [('director', '==', 'Greta Gerwig')]
final filter string: (doc.director == 'Greta Gerwig')

Out[ ]:

[NodeWithScore(node=TextNode(id_='935c2319f66122e1c4189ccf32164b362d4b6bc2af4ede77fc47fd126546ba8d', embedding=None, metadata={'lang': 'eng', 'offset': '0', 'len': '79', 'year': '2023', 'director': 'Greta Gerwig', 'genre': 'fantasy', 'rating': '9.5'}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, text='Barbie suffers a crisis that leads her to question her world and her existence.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\n\n{content}', metadata_template='{key}: {value}', metadata_seperator='\n'), score=0.61976165)]

In [ ]:

Copied!

retriever.retrieve("a movie with a rating above 8")
retriever.retrieve("a movie with a rating above 8")

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
Using query str: movie with rating above 8
Using implicit filters: [('rating', '>', 8)]
final filter string: (doc.rating > '8')

Out[ ]:

[NodeWithScore(node=TextNode(id_='a55396c48ddb51e593676d37a18efa8da1cbab6fd206c9029ced924e386c12b5', embedding=None, metadata={'lang': 'eng', 'offset': '0', 'len': '129', 'year': '1995', 'genre': 'animated', 'rating': '8.3'}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, text="A cowboy doll is profoundly threatened and jealous when a new spaceman action figure supplants him as top toy in a boy's bedroom.", start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\n\n{content}', metadata_template='{key}: {value}', metadata_seperator='\n'), score=0.5414715),
 NodeWithScore(node=TextNode(id_='b1e87de59678f3f1831327948a716f6e5a217c9b8cee4c08d5555d337825fca8', embedding=None, metadata={'lang': 'eng', 'offset': '0', 'len': '220', 'year': '2010', 'director': 'Christopher Nolan', 'rating': '8.2'}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, text='A thief who steals corporate secrets through the use of dream-sharing technology is given the inverse task of planting an idea into the mind of a C.E.O., but his tragic past may doom the project and his team to disaster.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\n\n{content}', metadata_template='{key}: {value}', metadata_seperator='\n'), score=0.50233454),
 NodeWithScore(node=TextNode(id_='935c2319f66122e1c4189ccf32164b362d4b6bc2af4ede77fc47fd126546ba8d', embedding=None, metadata={'lang': 'eng', 'offset': '0', 'len': '79', 'year': '2023', 'director': 'Greta Gerwig', 'genre': 'fantasy', 'rating': '9.5'}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, text='Barbie suffers a crisis that leads her to question her world and her existence.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\n\n{content}', metadata_template='{key}: {value}', metadata_seperator='\n'), score=0.500028),
 NodeWithScore(node=TextNode(id_='a860d7045527b4659a1a1171f922e30bee754ddcf83e0ec8423238f9f634f118', embedding=None, metadata={'lang': 'eng', 'offset': '0', 'len': '209', 'year': '2010', 'genre': 'animated', 'rating': '8.3'}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, text="The toys are mistakenly delivered to a day-care center instead of the attic right before Andy leaves for college, and it's up to Woody to convince the other toys that they weren't abandoned and to return home.", start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\n\n{content}', metadata_template='{key}: {value}', metadata_seperator='\n'), score=0.47051564)]

我们同样可以在 VectaraAutoRetriever 中包含标准的 VectaraRetriever 参数。例如，若需要添加一个会被附加到查询本身任何额外筛选条件上的 filter，可以按如下方式实现：

从 Vectara 索引实现自动检索¶

安装¶

定义示例数据¶

定义 VectorStoreInfo¶

运行自动检索¶

定义 `VectorStoreInfo`¶