Vectara 托管索引¶

本笔记本将展示如何在 LlamaIndex 中使用 Vectara。请注意，本笔记本适用于 Vectara ManagedIndex 版本 >=0.4.0。

Vectara 是值得信赖的 AI 助手与智能体平台，专注于企业级关键任务应用的成熟解决方案。

Vectara 提供检索增强生成（RAG）的端到端托管服务，包含以下功能：

1. 集成式 API 处理输入数据，包括文档文本提取和基于机器学习的文本分块。

2. 最先进的 Boomerang 嵌入模型。每个文本块通过 Boomerang 编码为向量嵌入，并存储在 Vectara 内部知识（向量+文本）库中。因此，在 LlamaIndex 中使用 Vectara 时无需调用单独的嵌入模型——这一过程已在 Vectara 后端自动完成。

3. 查询服务自动将查询编码为嵌入向量，并通过混合搜索与多种重排序策略检索最相关文本片段，包括：

   • 多语言重排序器
   • 最大边际相关性（MMR）重排序器
   • 用户自定义函数重排序器
   • 链式重排序器可串联多种重排序方法，通过组合不同重排序策略的优势实现更精准的控制

4. 基于检索文档创建生成式摘要的选项，提供多种 LLM 摘要生成器（包括专为 RAG 任务训练的 Vectara Mockingbird），并支持引用标注。

更多 API 使用详情请参阅 Vectara API 文档。

使用 Vectara RAG 即服务构建应用的主要优势包括：

• 准确性与质量：Vectara 提供端到端平台，专注于消除幻觉、减少偏见并保障版权完整性。
• 安全性：平台提供访问控制（防范提示注入攻击），符合 SOC2 和 HIPAA 合规要求。
• 可解释性：通过清晰展示查询重构、LLM 提示、检索结果和智能体行为，便于排查不良结果。

快速开始¶

如果您在 Colab 上打开这个 Notebook，很可能需要先安装 LlamaIndex 🦙。

In [ ]:

!pip install llama-index llama-index-indices-managed-vectara

!pip install llama-index llama-index-indices-managed-vectara

要开始使用 Vectara，请先注册账号（若尚未注册），并按照我们的快速入门指南创建语料库和 API 密钥。

获取这些信息后，您可以将它们设置为环境变量 VECTARA_CORPUS_KEY 和 VECTARA_API_KEY。请确保您的 API 密钥同时具备查询和索引权限。

使用 LlamaIndex 和 Vectara 实现 RAG¶

有几种方法可以将数据索引到 Vectara 中，包括：

1. 使用 VectaraIndex 的 from_documents() 或 insert_file() 方法
2. 直接在 Vectara 控制台 上传文件
3. 使用 Vectara 的 文件上传 或 文档索引 API
4. 使用开源爬虫/索引项目 vectara-ingest
5. 使用我们的数据集成合作伙伴如 Airbyte、Unstructured 或 DataVolo

为此，我们将使用一组简单的小文档，因此直接使用 VectaraIndex 进行数据摄取就足够了。

让我们将《AI 权利法案》文档摄取到新创建的语料库中。

In [ ]:

from llama_index.indices.managed.vectara import VectaraIndex
import requests

url = "https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf"
response = requests.get(url)
local_path = "ai-bill-of-rights.pdf"
with open(local_path, "wb") as file:
    file.write(response.content)

index = VectaraIndex()
index.insert_file(
    local_path, metadata={"name": "AI bill of rights", "year": 2022}
)

from llama_index.indices.managed.vectara import VectaraIndex import requests url = "https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf" response = requests.get(url) local_path = "ai-bill-of-rights.pdf" with open(local_path, "wb") as file: file.write(response.content) index = VectaraIndex() index.insert_file( local_path, metadata={"name": "AI bill of rights", "year": 2022} )

Out[ ]:

'ai-bill-of-rights.pdf'

使用 Vectara 查询引擎执行单一查询¶

现在我们已经上传了文档（或之前已上传过文档），可以直接在 LlamaIndex 中提问。这将激活 Vectara 的 RAG 流程。

如需使用 Vectara 内置的 LLM 进行摘要生成，请确保在创建查询引擎时指定 summary_enabled=True 参数。示例如下：

In [ ]:

questions = [
    "What are the risks of AI?",
    "What should we do to prevent bad actors from using AI?",
    "What are the benefits?",
]

questions = [ "What are the risks of AI?", "What should we do to prevent bad actors from using AI?", "What are the benefits?", ]

In [ ]:

qe = index.as_query_engine(
    n_sentences_before=1,
    n_sentences_after=1,
    summary_enabled=True,
    summary_prompt_name="mockingbird-1.0-2024-07-16",
)
qe.query(questions[0]).response

qe = index.as_query_engine( n_sentences_before=1, n_sentences_after=1, summary_enabled=True, summary_prompt_name="mockingbird-1.0-2024-07-16", ) qe.query(questions[0]).response

Out[ ]:

'The risks of AI include biased data and discriminatory outcomes, opaque decision-making processes, and lack of public trust and understanding of algorithmic systems [1]. These risks can have significant impacts on individuals and communities, particularly those who are directly affected by AI systems [5]. To mitigate these risks, it is essential to identify and address potential risks before deployment, and to implement ongoing monitoring and mitigation strategies [2][6]. This includes risk assessments, auditing mechanisms, and public consultation to ensure that AI systems are designed and used in a responsible and transparent manner [2][6]. Additionally, the development of AI systems should be guided by principles that prioritize lawfulness, accuracy, and transparency, and that are regularly monitored and accountable [7].'

若希望以流式模式返回响应，只需设置 streaming=True

In [ ]:

qe = index.as_query_engine(
    n_sentences_before=1,
    n_sentences_after=1,
    summary_enabled=True,
    summary_prompt_name="mockingbird-1.0-2024-07-16",
    streaming=True,
)
response = qe.query(questions[0])

response.print_response_stream()

qe = index.as_query_engine( n_sentences_before=1, n_sentences_after=1, summary_enabled=True, summary_prompt_name="mockingbird-1.0-2024-07-16", streaming=True, ) response = qe.query(questions[0]) response.print_response_stream()

The risks of AI include biased data and discriminatory outcomes, opaque decision-making processes, and lack of public trust and understanding of algorithmic systems [1]. These risks can have significant impacts on individuals and communities, particularly those who are directly affected by AI systems [5]. To mitigate these risks, it is essential to identify and address potential risks before deployment, and to implement ongoing monitoring and mitigation strategies [2][6]. This includes risk assessments, auditing mechanisms, and public consultation to ensure that AI systems are designed and used in a responsible and transparent manner [2][6]. Additionally, the development of AI systems should be guided by principles that prioritize lawfulness, accuracy, and transparency, and that are regularly monitored and accountable [7].

使用 Vectara 聊天功能¶

Vectara 还支持简易的聊天模式。在此模式下，聊天历史记录由 Vectara 自动维护，您无需手动管理。只需调用 as_chat_engine 即可启用该功能。

（聊天模式始终使用 Vectara 的摘要生成功能，因此您无需像之前那样显式指定 summary_enabled=True）

In [ ]:

ce = index.as_chat_engine(n_sentences_before=1, n_sentences_after=1)

ce = index.as_chat_engine(n_sentences_before=1, n_sentences_after=1)

In [ ]:

for q in questions:
    print(f"Question: {q}\n")
    response = ce.chat(q).response
    print(f"Response: {response}\n")

for q in questions: print(f"Question: {q}\n") response = ce.chat(q).response print(f"Response: {response}\n")

Question: What are the risks of AI?

Response: The risks of AI include potential biases and discriminatory outcomes due to biased data, opaque decision-making processes, and lack of public trust and understanding of algorithmic systems. Mitigating these risks involves ongoing transparency, participatory design, explanations for stakeholders, and public consultation [1]. Industry is developing innovative solutions like risk assessments, auditing mechanisms, and monitoring tools to ensure the safety and efficacy of AI systems [2]. Identifying and mitigating risks before deployment is crucial, focusing on impacts on rights, opportunities, and communities, as well as risks from misuse of the system [6]. The Executive Order on Trustworthy AI in the Federal Government outlines principles for lawful, purposeful, accurate, safe, understandable, responsible, monitored, transparent, and accountable AI use [7].

Question: What should we do to prevent bad actors from using AI?

Response: To prevent bad actors from using AI, we should implement a set of principles and practices to ensure the safe and effective use of AI systems. This includes adhering to specific principles such as legality, respect for values, accuracy, reliability, safety, transparency, and accountability in the design and use of AI [2]. Additionally, entities should follow privacy and security best practices to prevent data leaks and employ audits and impact assessments to identify and mitigate algorithmic discrimination [3][4]. It is crucial to involve the public in discussions about the promises and potential harms of AI technologies to shape policies that protect against discrimination and ensure fairness in the use of automated systems [1][6][7]. By promoting transparency, ongoing monitoring, and public consultation, we can work towards building trust, understanding, and ethical use of AI while safeguarding against misuse by bad actors.

Question: What are the benefits?

Response: The benefits of AI include the potential to build innovative infrastructure, improve customer service through faster responses, and enhance decision-making processes. AI can also lead to transformative improvements in people's lives, protect individuals from potential harms, and ensure the ethical use of automated systems. By incorporating principles for responsible stewardship and trustworthy AI, companies and government agencies can create safe, effective, and transparent AI systems that respect values, ensure accuracy, and promote accountability [1][4][6][7].

当然，聊天功能同样支持流式传输：

In [ ]:

ce = index.as_chat_engine(
    n_sentences_before=1, n_sentences_after=1, streaming=True
)

ce = index.as_chat_engine( n_sentences_before=1, n_sentences_after=1, streaming=True )

In [ ]:

response = ce.stream_chat("Will artificial intelligence rule the government?")

response.print_response_stream()

response = ce.stream_chat("Will artificial intelligence rule the government?") response.print_response_stream()

Artificial intelligence will not rule the government. The government has established principles and guidelines for the ethical use of AI, ensuring it is used responsibly, lawfully, and in alignment with the nation's values. These principles emphasize safety, accountability, transparency, and regular monitoring of AI systems within the federal government [1] [2]. Additionally, there are specific considerations for law enforcement and national security activities, highlighting the need for oversight and adherence to existing policies and safeguards [3]. The government is focused on promoting equity, fairness, civil rights, and racial justice through the use of AI, guided by principles that protect the American public [5]. Transparency and accountability are key aspects to ensure that AI technologies are used in ways that respect people's rights and expectations [7].

代理式 RAG¶

Vectara 还提供了自研的 vectara-agentic 工具包，该工具基于 LlamaIndex 的诸多功能构建，可轻松实现代理式 RAG 应用。通过该工具包，您能创建具备 RAG 查询工具及其他自定义功能（例如调用 API 从财经网站获取数据）的 AI 助手。完整文档详见 vectara-agentic 官方文档。

下面演示如何使用 vectara-agentic 创建一个仅包含单个 RAG 工具的 ReAct 代理（创建 ReAct 代理时，需在环境变量中设置 VECTARA_AGENTIC_AGENT_TYPE="REACT"）。

目前 Vectara 尚未提供具备规划与工具调用能力的代理型 LLM，因此我们需要使用其他 LLM 作为代理推理的驱动引擎。

本示例采用 OpenAI 的 GPT-4o 模型。请确保环境变量中已配置 OPENAI_API_KEY，或指定其他 LLM 及对应密钥（支持的全部 LLM 列表详见环境配置文档）。

In [ ]:

!pip install -U vectara-agentic

!pip install -U vectara-agentic

In [ ]:

from vectara_agentic.agent import Agent
from IPython.display import display, Markdown

agent = Agent.from_corpus(
    tool_name="query_ai",
    data_description="AI regulations",
    assistant_specialty="artificial intelligence",
    vectara_reranker="mmr",
    vectara_rerank_k=50,
    vectara_summary_num_results=5,
    vectara_summarizer="mockingbird-1.0-2024-07-16",
    verbose=True,
)

response = agent.chat(
    "What are the risks of AI? What are the benefits? Compare and contrast and provide a summary with arguments for and against from experts."
)

display(Markdown(response))

from vectara_agentic.agent import Agent from IPython.display import display, Markdown agent = Agent.from_corpus( tool_name="query_ai", data_description="AI regulations", assistant_specialty="artificial intelligence", vectara_reranker="mmr", vectara_rerank_k=50, vectara_summary_num_results=5, vectara_summarizer="mockingbird-1.0-2024-07-16", verbose=True, ) response = agent.chat( "What are the risks of AI? What are the benefits? Compare and contrast and provide a summary with arguments for and against from experts." ) display(Markdown(response))

Failed to set up observer (No module named 'phoenix.otel'), ignoring
> Running step 21fe2d4d-c74c-45df-9921-94c7f9e4f670. Step input: What are the risks of AI? What are the benefits? Compare and contrast and provide a summary with arguments for and against from experts.
Thought: The current language of the user is: English. I need to use a tool to help me answer the question.
Action: query_ai
Action Input: {'query': 'risks and benefits of AI, expert opinions'}
Observation: 
                    Response: '''According to expert opinions, the risks of AI include biased data and discriminatory outcomes, opaque decision-making processes, and lack of public trust and understanding of algorithmic systems [1]. To mitigate these risks, experts emphasize the importance of ongoing transparency, value-sensitive and participatory design, explanations designed for relevant stakeholders, and public consultation [1]. Additionally, industry is providing innovative solutions to mitigate risks to the safety and efficacy of AI systems, including risk assessments, auditing mechanisms, and documentation procedures [3]. The National Institute of Standards and Technology (NIST) is developing a risk management framework to better manage risks posed to individuals, organizations, and society by AI [3]. Furthermore, the White House Office of Science and Technology Policy has led a year-long process to seek input from people across the country on the issue of algorithmic and data-driven harms and potential remedies [4].'''
                    References:
                    [1]: CreationDate='1663695035'; Producer='iLovePDF'; Title='Blueprint for an AI Bill of Rights'; Creator='Adobe Illustrator 26.3 (Macintosh)'; ModDate='1664808078'; name='AI bill of rights'; year='2022'; framework='llama_index'; title='Blueprint for an AI Bill of Rights'.
[3]: CreationDate='1663695035'; Producer='iLovePDF'; Title='Blueprint for an AI Bill of Rights'; Creator='Adobe Illustrator 26.3 (Macintosh)'; ModDate='1664808078'; name='AI bill of rights'; year='2022'; framework='llama_index'; title='Blueprint for an AI Bill of Rights'.
[4]: CreationDate='1663695035'; Producer='iLovePDF'; Title='Blueprint for an AI Bill of Rights'; Creator='Adobe Illustrator 26.3 (Macintosh)'; ModDate='1664808078'; name='AI bill of rights'; year='2022'; framework='llama_index'; title='Blueprint for an AI Bill of Rights'.

                
> Running step a2b4d751-9f91-4fd9-9004-e276da54b75f. Step input: None
Thought: I can answer without using any more tools. I'll use the user's language to answer
Answer: The risks and benefits of AI are widely discussed among experts, and there are several key points to consider.

**Risks of AI:**
1. **Bias and Discrimination:** AI systems can perpetuate and even amplify biases present in the data they are trained on, leading to discriminatory outcomes.
2. **Opaque Decision-Making:** The decision-making processes of AI systems can be difficult to understand, leading to a lack of transparency.
3. **Public Trust:** There is often a lack of public trust and understanding of how AI systems work, which can hinder their acceptance and use.

To mitigate these risks, experts suggest measures such as ensuring transparency, involving stakeholders in the design process, providing clear explanations, and conducting public consultations. Additionally, there are efforts to develop frameworks and guidelines, such as the National Institute of Standards and Technology (NIST) risk management framework, to manage these risks effectively.

**Benefits of AI:**
1. **Efficiency and Productivity:** AI can automate repetitive tasks, leading to increased efficiency and productivity in various industries.
2. **Innovation:** AI drives innovation by enabling new applications and solutions that were not possible before.
3. **Improved Decision-Making:** AI can process large amounts of data quickly, providing insights that can improve decision-making processes.

**Expert Opinions:**
Experts argue for the benefits of AI in terms of its potential to transform industries and improve quality of life. However, they also caution against the risks, emphasizing the need for responsible development and deployment of AI technologies. The balance between leveraging AI's benefits and managing its risks is crucial for its successful integration into society.

References:
- [Blueprint for an AI Bill of Rights](https://www.whitehouse.gov/ostp/ai-bill-of-rights/)
Time taken: 20.452504634857178

The risks and benefits of AI are widely discussed among experts, and there are several key points to consider.

Risks of AI:

1. Bias and Discrimination: AI systems can perpetuate and even amplify biases present in the data they are trained on, leading to discriminatory outcomes.
2. Opaque Decision-Making: The decision-making processes of AI systems can be difficult to understand, leading to a lack of transparency.
3. Public Trust: There is often a lack of public trust and understanding of how AI systems work, which can hinder their acceptance and use.

To mitigate these risks, experts suggest measures such as ensuring transparency, involving stakeholders in the design process, providing clear explanations, and conducting public consultations. Additionally, there are efforts to develop frameworks and guidelines, such as the National Institute of Standards and Technology (NIST) risk management framework, to manage these risks effectively.

Benefits of AI:

1. Efficiency and Productivity: AI can automate repetitive tasks, leading to increased efficiency and productivity in various industries.
2. Innovation: AI drives innovation by enabling new applications and solutions that were not possible before.
3. Improved Decision-Making: AI can process large amounts of data quickly, providing insights that can improve decision-making processes.

Expert Opinions: Experts argue for the benefits of AI in terms of its potential to transform industries and improve quality of life. However, they also caution against the risks, emphasizing the need for responsible development and deployment of AI technologies. The balance between leveraging AI's benefits and managing its risks is crucial for its successful integration into society.

References:

• Blueprint for an AI Bill of Rights