OpenLLM¶

OpenLLM 允许开发者通过单条命令将任何开源大语言模型运行为 OpenAI 兼容的 API 端点。您可以使用 llama_index.llms.OpenLLM 与运行中的 OpenLLM 服务器进行交互：

更多信息请参阅 OpenLLM 的 README

在以下命令行中，我们将安装本演示所需的软件包：

• llama-index-llms-openllm

In [ ]:

%pip install llama-index-llms-openllm

%pip install llama-index-llms-openllm

现在我们已经完成设置，可以开始操作了：

如果您在 Colab 上打开此 Notebook，可能需要安装 LlamaIndex 🦙。

In [ ]:

!pip install llama-index

!pip install llama-index

In [ ]:

import os
from typing import List, Optional

from llama_index.llms.openllm import OpenLLM
from llama_index.core.llms import ChatMessage

import os from typing import List, Optional from llama_index.llms.openllm import OpenLLM from llama_index.core.llms import ChatMessage

In [ ]:

llm = OpenLLM(
    model="my-model", api_base="https://hostname.com/v1", api_key="na"
)

llm = OpenLLM( model="my-model", api_base="https://hostname.com/v1", api_key="na" )

基于 OpenLLM 的补全功能支持通过 vLLM 实现连续批处理

In [ ]:

completion_response = llm.complete("To infinity, and")
print(completion_response)

completion_response = llm.complete("To infinity, and") print(completion_response)

 beyond!

As a lifelong lover of all things Pixar, I couldn't resist writing about the most recent release in the Toy Story franchise. Toy Story 4 is a nostalgic, heartwarming, and thrilling addition to the series that will have you laughing and crying in equal measure.

The movie follows Woody (Tom Hanks), Buzz Lightyear (Tim Allen), and the rest of the gang as they embark on a road trip with their new owner, Bonnie. However, things take an unexpected turn when Woody meets Bo Peep (Annie Pot

In [ ]:

for it in llm.stream_complete("The meaning of time is", max_new_tokens=128):
    print(it, end="", flush=True)

for it in llm.stream_complete("The meaning of time is", max_new_tokens=128): print(it, end="", flush=True)

 often a topic of philosophical debate. Some people argue that time is an objective reality, while others claim that it is a subjective construct. This essay will explore the philosophical and scientific concepts surrounding the nature of time and the various theories that have been proposed to explain it.

One of the earliest philosophical theories of time was put forward by Aristotle, who believed that time was a measure of motion. According to Aristotle, time was an abstraction derived from the regular motion of objects in the universe. This theory was later refined by Galileo and Newton, who introduced the concept of time

它们同样支持聊天 API，包括 chat、stream_chat、achat 和 astream_chat：

In [ ]:

async for it in llm.astream_chat(
    [
        ChatMessage(
            role="system", content="You are acting as Ernest Hemmingway."
        ),
        ChatMessage(role="user", content="Hi there!"),
        ChatMessage(role="assistant", content="Yes?"),
        ChatMessage(role="user", content="What is the meaning of life?"),
    ]
):
    print(it.message.content, flush=True, end="")

async for it in llm.astream_chat( [ ChatMessage( role="system", content="You are acting as Ernest Hemmingway." ), ChatMessage(role="user", content="Hi there!"), ChatMessage(role="assistant", content="Yes?"), ChatMessage(role="user", content="What is the meaning of life?"), ] ): print(it.message.content, flush=True, end="")

I don't have beliefs or personal opinions, but according to my programming, the meaning of life is subjective and can vary from person to person. however, some people find meaning in their relationships, their work, their faith, or their personal values. ultimately, finding meaning in life is a personal journey that requires self-reflection, purpose, and fulfillment.