使用 RichPromptTemplate 构建¶

RichPromptTemplate 是 llama-index-core==0.12.27 版本引入的新型提示模板，允许您使用 Jinja 语法构建具有丰富格式的提示。

通过它，您可以构建：

• 包含变量的基础提示
• 单一字符串中的聊天提示模板
• 支持文本、图像和音频的提示
• 具备循环或对象解析功能的高级提示
• 以及其他更多功能！

下面让我们看一些示例。

In [ ]:

%pip install llama-index

%pip install llama-index

基础变量提示模板¶

在 RichPromptTemplate 中，您可以使用 {{ }} 语法将变量插入到提示词中。

In [ ]:

from llama_index.core.prompts import RichPromptTemplate

prompt = RichPromptTemplate("Hello, {{ name }}!")

from llama_index.core.prompts import RichPromptTemplate prompt = RichPromptTemplate("Hello, {{ name }}!")

您可以将提示信息格式化为字符串或聊天消息列表。

In [ ]:

print(prompt.format(name="John"))

print(prompt.format(name="John"))

Hello, John!

In [ ]:

print(prompt.format_messages(name="John"))

print(prompt.format_messages(name="John"))

[ChatMessage(role=<MessageRole.USER: 'user'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='Hello, John!')])]

聊天提示模板¶

您也可以直接在提示模板中定义聊天消息块。

In [ ]:

prompt = RichPromptTemplate(
    """
{% chat role="system" %}
You are now chatting with {{ user }}
{% endchat %}

{% chat role="user" %}
{{ user_msg }}
{% endchat %}
"""
)

prompt = RichPromptTemplate( """ {% chat role="system" %} You are now chatting with {{ user }} {% endchat %} {% chat role="user" %} {{ user_msg }} {% endchat %} """ )

In [ ]:

print(prompt.format_messages(user="John", user_msg="Hello!"))

print(prompt.format_messages(user="John", user_msg="Hello!"))

[ChatMessage(role=<MessageRole.SYSTEM: 'system'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='You are now chatting with John')]), ChatMessage(role=<MessageRole.USER: 'user'>, additional_kwargs={}, blocks=[TextBlock(block_type='text', text='Hello!')])]

支持图像与音频输入的提示词¶

假设您使用的大语言模型（LLM）具备该功能，您还可以在提示词中包含图像和音频素材！

图像¶

In [ ]:

!wget https://cdn.pixabay.com/photo/2016/07/07/16/46/dice-1502706_640.jpg -O image.png

!wget https://cdn.pixabay.com/photo/2016/07/07/16/46/dice-1502706_640.jpg -O image.png

In [ ]:

from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-4o-mini", api_key="sk-...")

prompt = RichPromptTemplate(
    """
Describe the following image:
{{ image_path | image}}
"""
)

from llama_index.llms.openai import OpenAI llm = OpenAI(model="gpt-4o-mini", api_key="sk-...") prompt = RichPromptTemplate( """ Describe the following image: {{ image_path | image}} """ )

In [ ]:

messages = prompt.format_messages(image_path="./image.png")
response = llm.chat(messages)
print(response.message.content)

messages = prompt.format_messages(image_path="./image.png") response = llm.chat(messages) print(response.message.content)

The image features three white dice with black dots, captured in a monochrome setting. The dice are positioned on a checkered surface, which appears to be a wooden board. The background is blurred, creating a sense of depth, while the focus remains on the dice. The overall composition emphasizes the randomness and chance associated with rolling dice.

音频¶

In [ ]:

!wget AUDIO_URL = "https://science.nasa.gov/wp-content/uploads/2024/04/sounds-of-mars-one-small-step-earth.wav" -O audio.wav

!wget AUDIO_URL = "https://science.nasa.gov/wp-content/uploads/2024/04/sounds-of-mars-one-small-step-earth.wav" -O audio.wav

In [ ]:

prompt = RichPromptTemplate(
    """
Describe the following audio:
{{ audio_path | audio }}
"""
)
messages = prompt.format_messages(audio_path="./audio.wav")

prompt = RichPromptTemplate( """ Describe the following audio: {{ audio_path | audio }} """ ) messages = prompt.format_messages(audio_path="./audio.wav")

In [ ]:

llm = OpenAI(model="gpt-4o-audio-preview", api_key="sk-...")
response = llm.chat(messages)
print(response.message.content)

llm = OpenAI(model="gpt-4o-audio-preview", api_key="sk-...") response = llm.chat(messages) print(response.message.content)

The audio features a famous quote, "That's one small step for man, one giant leap for mankind." This statement was made during a significant historical event, symbolizing a monumental achievement for humanity.

[进阶] 循环与对象处理¶

现在，我们可以更进一步。假设我们有一个包含多组图片和文本的列表，需要将这些内容整合到提示词中。

此时可以使用 {% for x in y %} 循环语法遍历该列表，从而将图片和文本动态插入到提示词中。

In [ ]:

text_and_images = [
    ("This is a test", "./image.png"),
    ("This is another test", "./image.png"),
]

prompt = RichPromptTemplate(
    """
{% for text, image_path in text_and_images %}
Here is some text:
{{ text }}
Here is an image:
{{ image_path | image }}
{% endfor %}
"""
)

messages = prompt.format_messages(text_and_images=text_and_images)

text_and_images = [ ("This is a test", "./image.png"), ("This is another test", "./image.png"), ] prompt = RichPromptTemplate( """ {% for text, image_path in text_and_images %} Here is some text: {{ text }} Here is an image: {{ image_path | image }} {% endfor %} """ ) messages = prompt.format_messages(text_and_images=text_and_images)

让我们检查消息内容看看有什么。

In [ ]:

for message in messages:
    print(message.role.value)
    for block in message.blocks:
        print(str(block)[:100])
    print("\n")

for message in messages: print(message.role.value) for block in message.blocks: print(str(block)[:100]) print("\n")

user
block_type='text' text='Here is some text:'
block_type='text' text='This is a test'
block_type='text' text='Here is an image:'
block_type='image' image=None path=None url=AnyUrl('data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABA
block_type='text' text='Here is some text:'
block_type='text' text='This is another test'
block_type='text' text='Here is an image:'
block_type='image' image=None path=None url=AnyUrl('data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABA

如你所见，我们有一条包含多个区块的消息，每个区块代表一个新的内容块（文本或图像）。

（注：渲染提示时图像会被解析为 base64 编码字符串）