查询转换指南手册¶

用户查询在执行成为RAG查询引擎、智能体或其他处理流程的一部分之前，可以通过多种方式进行转换和分解。

本指南将展示不同的查询转换、分解方法以及相关工具集的发现技巧。每种技术可能适用于不同的应用场景！

为便于说明，我们将底层处理流程定义为"工具"。以下是不同的查询转换方式：

1. 路由选择：保持原查询不变，但识别出查询适用的工具子集。输出这些工具作为相关选项。
2. 查询重写：保持工具不变，但以多种不同方式重写查询，以便在相同工具上执行。
3. 子问题分解：将查询分解为针对不同工具（通过其元数据识别）的多个子问题。
4. ReAct智能体工具选择：根据初始查询确定：1) 选择的工具，2) 在工具上执行的查询。

本指南旨在展示如何将这些查询转换作为模块化组件使用。当然，这些组件都可以集成到更大的系统中（例如子问题生成器就是我们SubQuestionQueryEngine的一部分）——每个组件的详细指南链接如下。

欢迎查阅并分享您的想法！

In [ ]:

%pip install llama-index-question-gen-openai
%pip install llama-index-llms-openai

%pip install llama-index-question-gen-openai %pip install llama-index-llms-openai

In [ ]:

from IPython.display import Markdown, display


# define prompt viewing function
def display_prompt_dict(prompts_dict):
    for k, p in prompts_dict.items():
        text_md = f"**Prompt Key**: {k}<br>" f"**Text:** <br>"
        display(Markdown(text_md))
        print(p.get_template())
        display(Markdown("<br><br>"))

from IPython.display import Markdown, display # define prompt viewing function def display_prompt_dict(prompts_dict): for k, p in prompts_dict.items(): text_md = f"**Prompt Key**: {k}
" f"**Text:**
" display(Markdown(text_md)) print(p.get_template()) display(Markdown("

"))

路由机制¶

本示例展示如何通过查询语句选择相关工具集。

我们采用 selector 抽象层来选取相关工具（根据抽象层的不同，可能选择单个工具或多个工具组合）。

现有四种选择器类型：（LLM 或函数调用）x（单选或多选）的组合

In [ ]:

from llama_index.core.selectors import LLMSingleSelector, LLMMultiSelector
from llama_index.core.selectors import (
    PydanticMultiSelector,
    PydanticSingleSelector,
)

from llama_index.core.selectors import LLMSingleSelector, LLMMultiSelector from llama_index.core.selectors import ( PydanticMultiSelector, PydanticSingleSelector, )

In [ ]:

# pydantic selectors feed in pydantic objects to a function calling API
# single selector (pydantic, function calling)
# selector = PydanticSingleSelector.from_defaults()

# multi selector (pydantic, function calling)
# selector = PydanticMultiSelector.from_defaults()

# LLM selectors use text completion endpoints
# single selector (LLM)
# selector = LLMSingleSelector.from_defaults()
# multi selector (LLM)
selector = LLMMultiSelector.from_defaults()

# pydantic selectors feed in pydantic objects to a function calling API # single selector (pydantic, function calling) # selector = PydanticSingleSelector.from_defaults() # multi selector (pydantic, function calling) # selector = PydanticMultiSelector.from_defaults() # LLM selectors use text completion endpoints # single selector (LLM) # selector = LLMSingleSelector.from_defaults() # multi selector (LLM) selector = LLMMultiSelector.from_defaults()

In [ ]:

from llama_index.core.tools import ToolMetadata

tool_choices = [
    ToolMetadata(
        name="covid_nyt",
        description=("This tool contains a NYT news article about COVID-19"),
    ),
    ToolMetadata(
        name="covid_wiki",
        description=("This tool contains the Wikipedia page about COVID-19"),
    ),
    ToolMetadata(
        name="covid_tesla",
        description=("This tool contains the Wikipedia page about apples"),
    ),
]

from llama_index.core.tools import ToolMetadata tool_choices = [ ToolMetadata( name="covid_nyt", description=("This tool contains a NYT news article about COVID-19"), ), ToolMetadata( name="covid_wiki", description=("This tool contains the Wikipedia page about COVID-19"), ), ToolMetadata( name="covid_tesla", description=("This tool contains the Wikipedia page about apples"), ), ]

In [ ]:

display_prompt_dict(selector.get_prompts())

display_prompt_dict(selector.get_prompts())

Prompt Key: prompt
Text:

Some choices are given below. It is provided in a numbered list (1 to {num_choices}), where each item in the list corresponds to a summary.
---------------------
{context_list}
---------------------
Using only the choices above and not prior knowledge, return the top choices (no more than {max_outputs}, but only select what is needed) that are most relevant to the question: '{query_str}'


The output should be ONLY JSON formatted as a JSON instance.

Here is an example:
[
    {{
        choice: 1,
        reason: "<insert reason for choice>"
    }},
    ...
]

In [ ]:

selector_result = selector.select(
    tool_choices, query="Tell me more about COVID-19"
)

selector_result = selector.select( tool_choices, query="Tell me more about COVID-19" )

In [ ]:

selector_result.selections

selector_result.selections

Out[ ]:

[SingleSelection(index=0, reason='This tool contains a NYT news article about COVID-19'),
 SingleSelection(index=1, reason='This tool contains the Wikipedia page about COVID-19')]

在我们的专属路由器页面中了解更多关于路由抽象的内容。

查询重写¶

本节将介绍如何将单个查询改写成多个查询。改写后的所有查询均可用于检索器执行。

这是高级检索技术中的关键步骤。通过查询重写，您可以为[集成检索]和[结果融合]生成多个查询，从而获得更高质量的检索结果。

与子问题生成器不同，该方法仅通过提示调用实现，且独立于工具链存在。

查询改写（自定义）¶

本节将展示如何利用我们的LLM和提示抽象层，通过提示生成多个查询变体。

In [ ]:

from llama_index.core import PromptTemplate
from llama_index.llms.openai import OpenAI

query_gen_str = """\
You are a helpful assistant that generates multiple search queries based on a \
single input query. Generate {num_queries} search queries, one on each line, \
related to the following input query:
Query: {query}
Queries:
"""
query_gen_prompt = PromptTemplate(query_gen_str)

llm = OpenAI(model="gpt-3.5-turbo")


def generate_queries(query: str, llm, num_queries: int = 4):
    response = llm.predict(
        query_gen_prompt, num_queries=num_queries, query=query
    )
    # assume LLM proper put each query on a newline
    queries = response.split("\n")
    queries_str = "\n".join(queries)
    print(f"Generated queries:\n{queries_str}")
    return queries

from llama_index.core import PromptTemplate from llama_index.llms.openai import OpenAI query_gen_str = """\ You are a helpful assistant that generates multiple search queries based on a \ single input query. Generate {num_queries} search queries, one on each line, \ related to the following input query: Query: {query} Queries: """ query_gen_prompt = PromptTemplate(query_gen_str) llm = OpenAI(model="gpt-3.5-turbo") def generate_queries(query: str, llm, num_queries: int = 4): response = llm.predict( query_gen_prompt, num_queries=num_queries, query=query ) # assume LLM proper put each query on a newline queries = response.split("\n") queries_str = "\n".join(queries) print(f"Generated queries:\n{queries_str}") return queries

In [ ]:

queries = generate_queries("What happened at Interleaf and Viaweb?", llm)

queries = generate_queries("What happened at Interleaf and Viaweb?", llm)

Generated queries:
1. What were the major events or milestones in the history of Interleaf and Viaweb?
2. Who were the founders and key figures involved in the development of Interleaf and Viaweb?
3. What were the products or services offered by Interleaf and Viaweb?
4. Are there any notable success stories or failures associated with Interleaf and Viaweb?

In [ ]:

queries

queries

Out[ ]:

['1. What were the major events or milestones in the history of Interleaf and Viaweb?',
 '2. Who were the founders and key figures involved in the development of Interleaf and Viaweb?',
 '3. What were the products or services offered by Interleaf and Viaweb?',
 '4. Are there any notable success stories or failures associated with Interleaf and Viaweb?']

如需了解结合检索器的端到端实现详情，请参阅我们的融合检索器指南：

• 模块指南
• 从零构建融合检索器

查询重写（使用 QueryTransform）¶

本节将介绍如何通过 QueryTransform 类实现查询转换。

In [ ]:

from llama_index.core.indices.query.query_transform import HyDEQueryTransform
from llama_index.llms.openai import OpenAI

from llama_index.core.indices.query.query_transform import HyDEQueryTransform from llama_index.llms.openai import OpenAI

In [ ]:

hyde = HyDEQueryTransform(include_original=True)
llm = OpenAI(model="gpt-3.5-turbo")

query_bundle = hyde.run("What is Bel?")

hyde = HyDEQueryTransform(include_original=True) llm = OpenAI(model="gpt-3.5-turbo") query_bundle = hyde.run("What is Bel?")

这将生成一个查询包，其中不仅包含原始查询，还包括代表应被嵌入查询的 custom_embedding_strs。

In [ ]:

new_query.custom_embedding_strs

new_query.custom_embedding_strs

Out[ ]:

['Bel is a term that has multiple meanings and can be interpreted in various ways depending on the context. In ancient Mesopotamian mythology, Bel was a prominent deity and one of the chief gods of the Babylonian pantheon. He was often associated with the sky, storms, and fertility. Bel was considered to be the father of the gods and held great power and authority over the other deities.\n\nIn addition to its mythological significance, Bel is also a title that was used to address rulers and leaders in ancient Babylon. It was a term of respect and reverence, similar to the modern-day title of "king" or "emperor." The title of Bel was bestowed upon those who held significant political and military power, and it symbolized their authority and dominion over their subjects.\n\nFurthermore, Bel is also a common given name in various cultures around the world. It can be found in different forms and variations, such as Belinda, Isabel, or Bella. As a personal name, Bel often carries connotations of beauty, grace, and strength.\n\nIn summary, Bel can refer to a powerful deity in ancient Mesopotamian mythology, a title of respect for rulers and leaders, or a personal name with positive attributes. The meaning of Bel can vary depending on the specific context in which it is used.',
 'What is Bel?']

子问题生成¶

给定一组工具和用户查询，需同时确定：1) 需要生成的子问题集合，2) 每个子问题应调用的工具集。

我们通过两个示例进行说明：使用依赖函数调用的OpenAIQuestionGenerator，以及依赖提示工程的LLMQuestionGenerator。

In [ ]:

from llama_index.core.question_gen import LLMQuestionGenerator
from llama_index.question_gen.openai import OpenAIQuestionGenerator
from llama_index.llms.openai import OpenAI

from llama_index.core.question_gen import LLMQuestionGenerator from llama_index.question_gen.openai import OpenAIQuestionGenerator from llama_index.llms.openai import OpenAI

In [ ]:

llm = OpenAI()
question_gen = OpenAIQuestionGenerator.from_defaults(llm=llm)

llm = OpenAI() question_gen = OpenAIQuestionGenerator.from_defaults(llm=llm)

In [ ]:

display_prompt_dict(question_gen.get_prompts())

display_prompt_dict(question_gen.get_prompts())

Prompt Key: question_gen_prompt
Text:

You are a world class state of the art agent.

You have access to multiple tools, each representing a different data source or API.
Each of the tools has a name and a description, formatted as a JSON dictionary.
The keys of the dictionary are the names of the tools and the values are the descriptions.
Your purpose is to help answer a complex user question by generating a list of sub questions that can be answered by the tools.

These are the guidelines you consider when completing your task:
* Be as specific as possible
* The sub questions should be relevant to the user question
* The sub questions should be answerable by the tools provided
* You can generate multiple sub questions for each tool
* Tools must be specified by their name, not their description
* You don't need to use a tool if you don't think it's relevant

Output the list of sub questions by calling the SubQuestionList function.

## Tools
```json
{tools_str}
```

## User Question
{query_str}

In [ ]:

from llama_index.core.tools import ToolMetadata

tool_choices = [
    ToolMetadata(
        name="uber_2021_10k",
        description=(
            "Provides information about Uber financials for year 2021"
        ),
    ),
    ToolMetadata(
        name="lyft_2021_10k",
        description=(
            "Provides information about Lyft financials for year 2021"
        ),
    ),
]

from llama_index.core.tools import ToolMetadata tool_choices = [ ToolMetadata( name="uber_2021_10k", description=( "Provides information about Uber financials for year 2021" ), ), ToolMetadata( name="lyft_2021_10k", description=( "Provides information about Lyft financials for year 2021" ), ), ]

In [ ]:

from llama_index.core import QueryBundle

query_str = "Compare and contrast Uber and Lyft"
choices = question_gen.generate(tool_choices, QueryBundle(query_str=query_str))

from llama_index.core import QueryBundle query_str = "Compare and contrast Uber and Lyft" choices = question_gen.generate(tool_choices, QueryBundle(query_str=query_str))

输出结果为 SubQuestion Pydantic 对象。

In [ ]:

choices

choices

Out[ ]:

[SubQuestion(sub_question='What are the financials of Uber for the year 2021?', tool_name='uber_2021_10k'),
 SubQuestion(sub_question='What are the financials of Lyft for the year 2021?', tool_name='lyft_2021_10k')]

如需了解如何以更集成化的方式将此功能接入您的 RAG 流程，请参阅我们的 SubQuestionQueryEngine 文档。

基于 ReAct 提示的查询转换¶

ReAct 是一种流行的智能体框架，在此我们将展示如何利用其核心提示机制来实现查询转换。

我们使用 ReActChatFormatter 来生成供大语言模型(LLM)处理的输入消息集。

In [ ]:

from llama_index.core.agent import ReActChatFormatter
from llama_index.core.agent.react.output_parser import ReActOutputParser
from llama_index.core.tools import FunctionTool
from llama_index.core.llms import ChatMessage

from llama_index.core.agent import ReActChatFormatter from llama_index.core.agent.react.output_parser import ReActOutputParser from llama_index.core.tools import FunctionTool from llama_index.core.llms import ChatMessage

In [ ]:

def execute_sql(sql: str) -> str:
    """Given a SQL input string, execute it."""
    # NOTE: This is a mock function
    return f"Executed {sql}"


def add(a: int, b: int) -> int:
    """Add two numbers."""
    return a + b


tool1 = FunctionTool.from_defaults(fn=execute_sql)
tool2 = FunctionTool.from_defaults(fn=add)
tools = [tool1, tool2]

def execute_sql(sql: str) -> str: """Given a SQL input string, execute it.""" # NOTE: This is a mock function return f"Executed {sql}" def add(a: int, b: int) -> int: """Add two numbers.""" return a + b tool1 = FunctionTool.from_defaults(fn=execute_sql) tool2 = FunctionTool.from_defaults(fn=add) tools = [tool1, tool2]

在这里我们获取要传递给大语言模型（LLM）的输入提示信息。来看一下！

In [ ]:

chat_formatter = ReActChatFormatter()
output_parser = ReActOutputParser()
input_msgs = chat_formatter.format(
    tools,
    [
        ChatMessage(
            content="Can you find the top three rows from the table named `revenue_years`",
            role="user",
        )
    ],
)
input_msgs

chat_formatter = ReActChatFormatter() output_parser = ReActOutputParser() input_msgs = chat_formatter.format( tools, [ ChatMessage( content="Can you find the top three rows from the table named `revenue_years`", role="user", ) ], ) input_msgs

Out[ ]:

[ChatMessage(role=<MessageRole.SYSTEM: 'system'>, content='\nYou are designed to help with a variety of tasks, from answering questions     to providing summaries to other types of analyses.\n\n## Tools\nYou have access to a wide variety of tools. You are responsible for using\nthe tools in any sequence you deem appropriate to complete the task at hand.\nThis may require breaking the task into subtasks and using different tools\nto complete each subtask.\n\nYou have access to the following tools:\n> Tool Name: execute_sql\nTool Description: execute_sql(sql: str) -> str\nGiven a SQL input string, execute it.\nTool Args: {\'title\': \'execute_sql\', \'type\': \'object\', \'properties\': {\'sql\': {\'title\': \'Sql\', \'type\': \'string\'}}, \'required\': [\'sql\']}\n\n> Tool Name: add\nTool Description: add(a: int, b: int) -> int\nAdd two numbers.\nTool Args: {\'title\': \'add\', \'type\': \'object\', \'properties\': {\'a\': {\'title\': \'A\', \'type\': \'integer\'}, \'b\': {\'title\': \'B\', \'type\': \'integer\'}}, \'required\': [\'a\', \'b\']}\n\n\n## Output Format\nTo answer the question, please use the following format.\n\n```\nThought: I need to use a tool to help me answer the question.\nAction: tool name (one of execute_sql, add) if using a tool.\nAction Input: the input to the tool, in a JSON format representing the kwargs (e.g. {"input": "hello world", "num_beams": 5})\n```\n\nPlease ALWAYS start with a Thought.\n\nPlease use a valid JSON format for the Action Input. Do NOT do this {\'input\': \'hello world\', \'num_beams\': 5}.\n\nIf this format is used, the user will respond in the following format:\n\n```\nObservation: tool response\n```\n\nYou should keep repeating the above format until you have enough information\nto answer the question without using any more tools. At that point, you MUST respond\nin the one of the following two formats:\n\n```\nThought: I can answer without using any more tools.\nAnswer: [your answer here]\n```\n\n```\nThought: I cannot answer the question with the provided tools.\nAnswer: Sorry, I cannot answer your query.\n```\n\n## Current Conversation\nBelow is the current conversation consisting of interleaving human and assistant messages.\n\n', additional_kwargs={}),
 ChatMessage(role=<MessageRole.USER: 'user'>, content='Can you find the top three rows from the table named `revenue_years`', additional_kwargs={})]

接下来我们获取模型的输出。

In [ ]:

llm = OpenAI(model="gpt-4-1106-preview")

llm = OpenAI(model="gpt-4-1106-preview")

In [ ]:

response = llm.chat(input_msgs)

response = llm.chat(input_msgs)

最后我们使用 ReActOutputParser 将内容解析为结构化输出，并分析动作输入。

In [ ]:

reasoning_step = output_parser.parse(response.message.content)

reasoning_step = output_parser.parse(response.message.content)

In [ ]:

reasoning_step.action_input

reasoning_step.action_input

Out[ ]:

{'sql': 'SELECT * FROM revenue_years ORDER BY revenue DESC LIMIT 3'}