支持 Pydantic 输出的查询引擎¶

每个查询引擎都支持通过 RetrieverQueryEngine 的以下 response_mode 实现结构化响应集成：

• refine（精炼模式）
• compact（紧凑模式）
• tree_summarize（树状摘要模式）
• accumulate（累积模式，测试版，需额外解析转换为对象）
• compact_accumulate（紧凑累积模式，测试版，需额外解析转换为对象）

本示例将演示具体使用方法。

底层实现中，每个大语言模型（LLM）的响应都将以 pydantic 对象形式返回。若响应需要被精炼或摘要处理，则会先转换为 JSON 字符串供后续响应使用，最终响应结果仍以 pydantic 对象形式返回。

注意： 该功能理论上支持所有 LLM，但对非 OpenAI 模型的支持仍处于开发阶段，目前视为测试版功能。

安装¶

如果您在 Colab 上打开此 Notebook，可能需要安装 LlamaIndex 🦙。

In [ ]:

%pip install llama-index-llms-anthropic
%pip install llama-index-llms-openai

%pip install llama-index-llms-anthropic %pip install llama-index-llms-openai

In [ ]:

!pip install llama-index

!pip install llama-index

In [ ]:

import os
import openai

os.environ["OPENAI_API_KEY"] = "sk-..."
openai.api_key = os.environ["OPENAI_API_KEY"]

import os import openai os.environ["OPENAI_API_KEY"] = "sk-..." openai.api_key = os.environ["OPENAI_API_KEY"]

下载数据

In [ ]:

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

!mkdir -p 'data/paul_graham/' !wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

In [ ]:

from llama_index.core import SimpleDirectoryReader

documents = SimpleDirectoryReader("./data/paul_graham").load_data()

from llama_index.core import SimpleDirectoryReader documents = SimpleDirectoryReader("./data/paul_graham").load_data()

创建我们的 Pydantic 输出对象¶

In [ ]:

from typing import List
from pydantic import BaseModel


class Biography(BaseModel):
    """Data model for a biography."""

    name: str
    best_known_for: List[str]
    extra_info: str

from typing import List from pydantic import BaseModel class Biography(BaseModel): """Data model for a biography.""" name: str best_known_for: List[str] extra_info: str

创建索引+查询引擎（OpenAI）¶

当使用OpenAI时，将利用函数调用API来获取可靠的结构化输出。

In [ ]:

from llama_index.core import VectorStoreIndex
from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)

index = VectorStoreIndex.from_documents(
    documents,
)

from llama_index.core import VectorStoreIndex from llama_index.llms.openai import OpenAI llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1) index = VectorStoreIndex.from_documents( documents, )

In [ ]:

query_engine = index.as_query_engine(
    output_cls=Biography, response_mode="compact", llm=llm
)

query_engine = index.as_query_engine( output_cls=Biography, response_mode="compact", llm=llm )

In [ ]:

response = query_engine.query("Who is Paul Graham?")

response = query_engine.query("Who is Paul Graham?")

In [ ]:

print(response.name)
print(response.best_known_for)
print(response.extra_info)

print(response.name) print(response.best_known_for) print(response.extra_info)

Paul Graham
['working on Bel', 'co-founding Viaweb', 'creating the programming language Arc']
Paul Graham is a computer scientist, entrepreneur, and writer. He is best known for his work on Bel, a programming language, and for co-founding Viaweb, an early web application company that was later acquired by Yahoo. Graham also created the programming language Arc. He has written numerous essays on topics such as startups, programming, and life.

In [ ]:

# get the full pydanitc object
print(type(response.response))

# get the full pydanitc object print(type(response.response))

<class '__main__.Biography'>

创建索引+查询引擎（非OpenAI版本，Beta阶段）¶

当使用不支持函数调用的LLM时，我们依赖LLM自行生成JSON，然后将该JSON解析为正确的pydantic对象。

In [ ]:

import os

os.environ["ANTHROPIC_API_KEY"] = "sk-..."

import os os.environ["ANTHROPIC_API_KEY"] = "sk-..."

In [ ]:

from llama_index.core import VectorStoreIndex
from llama_index.llms.anthropic import Anthropic

llm = Anthropic(model="claude-instant-1.2", temperature=0.1)

index = VectorStoreIndex.from_documents(
    documents,
)

from llama_index.core import VectorStoreIndex from llama_index.llms.anthropic import Anthropic llm = Anthropic(model="claude-instant-1.2", temperature=0.1) index = VectorStoreIndex.from_documents( documents, )

In [ ]:

query_engine = index.as_query_engine(
    output_cls=Biography, response_mode="tree_summarize", llm=llm
)

query_engine = index.as_query_engine( output_cls=Biography, response_mode="tree_summarize", llm=llm )

In [ ]:

response = query_engine.query("Who is Paul Graham?")

response = query_engine.query("Who is Paul Graham?")

In [ ]:

print(response.name)
print(response.best_known_for)
print(response.extra_info)

print(response.name) print(response.best_known_for) print(response.extra_info)

Paul Graham
['Co-founder of Y Combinator', 'Essayist and programmer']
He is known for creating Viaweb, one of the first web application builders, and for founding Y Combinator, one of the world's top startup accelerators. Graham has also written extensively about technology, investing, and philosophy.

In [ ]:

# get the full pydanitc object
print(type(response.response))

# get the full pydanitc object print(type(response.response))

<class '__main__.Biography'>

累加示例（Beta 版）¶

使用 pydantic 对象进行累加操作需要额外的解析处理。目前这仍是一项测试功能，但已能实现 pydantic 对象的累加功能。

In [ ]:

from typing import List
from pydantic import BaseModel


class Company(BaseModel):
    """Data model for a companies mentioned."""

    company_name: str
    context_info: str

from typing import List from pydantic import BaseModel class Company(BaseModel): """Data model for a companies mentioned.""" company_name: str context_info: str

In [ ]:

from llama_index.core import VectorStoreIndex,
from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1)

index = VectorStoreIndex.from_documents(
    documents,
)

from llama_index.core import VectorStoreIndex, from llama_index.llms.openai import OpenAI llm = OpenAI(model="gpt-3.5-turbo", temperature=0.1) index = VectorStoreIndex.from_documents( documents, )

In [ ]:

query_engine = index.as_query_engine(
    output_cls=Company, response_mode="accumulate", llm=llm
)

query_engine = index.as_query_engine( output_cls=Company, response_mode="accumulate", llm=llm )

In [ ]:

response = query_engine.query("What companies are mentioned in the text?")

response = query_engine.query("What companies are mentioned in the text?")

在 accumulate 中，响应内容通过默认分隔符进行分隔，并在开头添加前缀。

In [ ]:

companies = []

# split by the default separator
for response_str in str(response).split("\n---------------------\n"):
    # remove the prefix --  every response starts like `Response 1: {...}`
    # so, we find the first bracket and remove everything before it
    response_str = response_str[response_str.find("{") :]
    companies.append(Company.parse_raw(response_str))

companies = [] # split by the default separator for response_str in str(response).split("\n---------------------\n"): # remove the prefix -- every response starts like `Response 1: {...}` # so, we find the first bracket and remove everything before it response_str = response_str[response_str.find("{") :] companies.append(Company.parse_raw(response_str))

In [ ]:

print(companies)

print(companies)

[Company(company_name='Yahoo', context_info='Yahoo bought us'), Company(company_name='Yahoo', context_info="I'd been meaning to since Yahoo bought us")]