路由查询引擎¶

在本教程中，我们将定义一个自定义的路由查询引擎，该引擎能够从多个候选查询引擎中选择一个来执行查询。

安装配置¶

如果您在 Colab 上打开此 Notebook，可能需要安装 LlamaIndex 🦙。

In [ ]:

%pip install llama-index-embeddings-openai
%pip install llama-index-llms-openai

%pip install llama-index-embeddings-openai %pip install llama-index-llms-openai

In [ ]:

!pip install llama-index

!pip install llama-index

In [ ]:

# NOTE: This is ONLY necessary in jupyter notebook.
# Details: Jupyter runs an event-loop behind the scenes.
#          This results in nested event-loops when we start an event-loop to make async queries.
#          This is normally not allowed, we use nest_asyncio to allow it for convenience.
import nest_asyncio

nest_asyncio.apply()

# NOTE: This is ONLY necessary in jupyter notebook. # Details: Jupyter runs an event-loop behind the scenes. # This results in nested event-loops when we start an event-loop to make async queries. # This is normally not allowed, we use nest_asyncio to allow it for convenience. import nest_asyncio nest_asyncio.apply()

全局模型¶

In [ ]:

import os

os.environ["OPENAI_API_KEY"] = "sk-..."

import os os.environ["OPENAI_API_KEY"] = "sk-..."

In [ ]:

from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.core import Settings

Settings.llm = OpenAI(model="gpt-3.5-turbo-1106", temperature=0.2)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")

from llama_index.llms.openai import OpenAI from llama_index.embeddings.openai import OpenAIEmbedding from llama_index.core import Settings Settings.llm = OpenAI(model="gpt-3.5-turbo-1106", temperature=0.2) Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")

加载数据¶

我们首先演示如何将文档转换为节点集合，并将其插入文档存储库。

In [ ]:

from llama_index.core import SimpleDirectoryReader

# load documents
documents = SimpleDirectoryReader("../data/paul_graham").load_data()

from llama_index.core import SimpleDirectoryReader # load documents documents = SimpleDirectoryReader("../data/paul_graham").load_data()

In [ ]:

from llama_index.core import Settings

# initialize settings (set chunk size)
Settings.chunk_size = 1024
nodes = Settings.node_parser.get_nodes_from_documents(documents)

from llama_index.core import Settings # initialize settings (set chunk size) Settings.chunk_size = 1024 nodes = Settings.node_parser.get_nodes_from_documents(documents)

In [ ]:

from llama_index.core import StorageContext

# initialize storage context (by default it's in-memory)
storage_context = StorageContext.from_defaults()
storage_context.docstore.add_documents(nodes)

from llama_index.core import StorageContext # initialize storage context (by default it's in-memory) storage_context = StorageContext.from_defaults() storage_context.docstore.add_documents(nodes)

在同一数据集上定义摘要索引与向量索引¶

In [ ]:

from llama_index.core import SummaryIndex
from llama_index.core import VectorStoreIndex

summary_index = SummaryIndex(nodes, storage_context=storage_context)
vector_index = VectorStoreIndex(nodes, storage_context=storage_context)

from llama_index.core import SummaryIndex from llama_index.core import VectorStoreIndex summary_index = SummaryIndex(nodes, storage_context=storage_context) vector_index = VectorStoreIndex(nodes, storage_context=storage_context)

定义查询引擎与设置元数据¶

In [ ]:

list_query_engine = summary_index.as_query_engine(
    response_mode="tree_summarize",
    use_async=True,
)
vector_query_engine = vector_index.as_query_engine()

list_query_engine = summary_index.as_query_engine( response_mode="tree_summarize", use_async=True, ) vector_query_engine = vector_index.as_query_engine()

In [ ]:

from llama_index.core.tools import QueryEngineTool


list_tool = QueryEngineTool.from_defaults(
    query_engine=list_query_engine,
    description=(
        "Useful for summarization questions related to Paul Graham eassy on"
        " What I Worked On."
    ),
)

vector_tool = QueryEngineTool.from_defaults(
    query_engine=vector_query_engine,
    description=(
        "Useful for retrieving specific context from Paul Graham essay on What"
        " I Worked On."
    ),
)

from llama_index.core.tools import QueryEngineTool list_tool = QueryEngineTool.from_defaults( query_engine=list_query_engine, description=( "Useful for summarization questions related to Paul Graham eassy on" " What I Worked On." ), ) vector_tool = QueryEngineTool.from_defaults( query_engine=vector_query_engine, description=( "Useful for retrieving specific context from Paul Graham essay on What" " I Worked On." ), )

定义路由查询引擎¶

现有多种选择器可供使用，每种都具有独特的属性特征。

LLM选择器利用大语言模型输出可解析的JSON数据，随后查询对应的索引。

Pydantic选择器（目前仅支持gpt-4-0613和默认的gpt-3.5-turbo-0613）通过OpenAI函数调用接口生成pydantic选择对象，而非解析原始JSON数据。

每种选择器类型都支持两种路由模式：单索引路由或多索引路由。

Pydantic单路选择器¶

该选择器底层采用OpenAI函数接口生成/解析pydantic对象来实现路由选择功能。

In [ ]:

from llama_index.core.query_engine import RouterQueryEngine
from llama_index.core.selectors import LLMSingleSelector, LLMMultiSelector
from llama_index.core.selectors import (
    PydanticMultiSelector,
    PydanticSingleSelector,
)


query_engine = RouterQueryEngine(
    selector=PydanticSingleSelector.from_defaults(),
    query_engine_tools=[
        list_tool,
        vector_tool,
    ],
)

from llama_index.core.query_engine import RouterQueryEngine from llama_index.core.selectors import LLMSingleSelector, LLMMultiSelector from llama_index.core.selectors import ( PydanticMultiSelector, PydanticSingleSelector, ) query_engine = RouterQueryEngine( selector=PydanticSingleSelector.from_defaults(), query_engine_tools=[ list_tool, vector_tool, ], )

In [ ]:

response = query_engine.query("What is the summary of the document?")
print(str(response))

response = query_engine.query("What is the summary of the document?") print(str(response))

The document provides a comprehensive account of the author's diverse experiences, including writing, programming, founding and running startups, and investing in early-stage companies. It covers the challenges, successes, and lessons learned in these ventures, as well as the author's personal and professional growth, interactions with colleagues, and evolving interests and priorities over time.

In [ ]:

response = query_engine.query("What did Paul Graham do after RICS?")
print(str(response))

response = query_engine.query("What did Paul Graham do after RICS?") print(str(response))

Paul Graham started painting after leaving Y Combinator. He wanted to see how good he could get if he really focused on it. After spending most of 2014 painting, he eventually ran out of steam and stopped working on it. He then started writing essays again and wrote a bunch of new ones over the next few months. Later, in March 2015, he started working on Lisp again.

LLMSingleSelector¶

利用 OpenAI（或任何其他大型语言模型）在底层解析生成的 JSON 数据，以选择用于路由的子索引。

In [ ]:

query_engine = RouterQueryEngine(
    selector=LLMSingleSelector.from_defaults(),
    query_engine_tools=[
        list_tool,
        vector_tool,
    ],
)

query_engine = RouterQueryEngine( selector=LLMSingleSelector.from_defaults(), query_engine_tools=[ list_tool, vector_tool, ], )

In [ ]:

response = query_engine.query("What is the summary of the document?")
print(str(response))

response = query_engine.query("What is the summary of the document?") print(str(response))

The document provides a comprehensive account of the author's professional journey, covering his involvement in various projects such as Viaweb, Y Combinator, and Hacker News, as well as his transition to focusing on writing essays and working on Y Combinator. It also delves into his experiences with the Summer Founders Program, the growth and challenges of Y Combinator, personal struggles, and his return to working on Lisp. The author reflects on the challenges and successes encountered throughout his career, including funding startups, developing a new version of Arc, and the impact of Hacker News. Additionally, the document touches on the author's interactions with colleagues, his time in Italy, experiences with painting, and the completion of a new Lisp called Bel. Throughout, the author shares insights and lessons learned from his diverse experiences.

In [ ]:

response = query_engine.query("What did Paul Graham do after RICS?")
print(str(response))

response = query_engine.query("What did Paul Graham do after RICS?") print(str(response))

Paul Graham started painting after leaving Y Combinator. He wanted to see how good he could get if he really focused on it. After spending most of 2014 painting, he eventually ran out of steam and stopped working on it. He then started writing essays again and wrote a bunch of new ones over the next few months. In March 2015, he started working on Lisp again.

In [ ]:

# [optional] look at selected results
print(str(response.metadata["selector_result"]))

# [optional] look at selected results print(str(response.metadata["selector_result"]))

selections=[SingleSelection(index=1, reason='The question is asking for specific context about what Paul Graham did after RICS, which would require retrieving specific information from his essay.')]

PydanticMultiSelector¶

若预期查询需路由至多个索引，则应使用多重选择器。该选择器会将查询同时发送至多个子索引，随后通过汇总索引聚合所有响应，最终生成完整答案。

In [ ]:

from llama_index.core import SimpleKeywordTableIndex

keyword_index = SimpleKeywordTableIndex(nodes, storage_context=storage_context)

keyword_tool = QueryEngineTool.from_defaults(
    query_engine=vector_query_engine,
    description=(
        "Useful for retrieving specific context using keywords from Paul"
        " Graham essay on What I Worked On."
    ),
)

from llama_index.core import SimpleKeywordTableIndex keyword_index = SimpleKeywordTableIndex(nodes, storage_context=storage_context) keyword_tool = QueryEngineTool.from_defaults( query_engine=vector_query_engine, description=( "Useful for retrieving specific context using keywords from Paul" " Graham essay on What I Worked On." ), )

In [ ]:

query_engine = RouterQueryEngine(
    selector=PydanticMultiSelector.from_defaults(),
    query_engine_tools=[
        list_tool,
        vector_tool,
        keyword_tool,
    ],
)

query_engine = RouterQueryEngine( selector=PydanticMultiSelector.from_defaults(), query_engine_tools=[ list_tool, vector_tool, keyword_tool, ], )

In [ ]:

# This query could use either a keyword or vector query engine, so it will combine responses from both
response = query_engine.query(
    "What were noteable events and people from the authors time at Interleaf"
    " and YC?"
)
print(str(response))

# This query could use either a keyword or vector query engine, so it will combine responses from both response = query_engine.query( "What were noteable events and people from the authors time at Interleaf" " and YC?" ) print(str(response))

The author's time at Interleaf involved working on software for creating documents and learning valuable lessons about what not to do. Notable individuals associated with Y Combinator during the author's time there include Jessica Livingston, Robert Morris, and Sam Altman, who eventually became the second president of YC. The author's time at Y Combinator included notable events such as the creation of the Summer Founders Program, which attracted impressive individuals like Reddit, Justin Kan, Emmett Shear, Aaron Swartz, and Sam Altman.

In [ ]:

# [optional] look at selected results
print(str(response.metadata["selector_result"]))

# [optional] look at selected results print(str(response.metadata["selector_result"]))

selections=[SingleSelection(index=0, reason='Summarization questions related to Paul Graham essay on What I Worked On.'), SingleSelection(index=2, reason='Retrieving specific context using keywords from Paul Graham essay on What I Worked On.')]