Oracle Cloud Infrastructure 数据科学服务¶

Oracle Cloud Infrastructure (OCI) 数据科学 是一个全托管、无服务器平台，供数据科学团队在 Oracle 云基础设施中构建、训练和管理机器学习模型。

该服务提供 AI 快捷操作，可用于在 OCI 数据科学中部署、评估和微调基础大语言模型。AI 快捷操作面向希望快速利用 AI 能力的用户，旨在通过提供简化、无需编码且高效的基础模型操作环境，将基础模型的适用范围扩展到更广泛的用户群体。AI 快捷操作可通过数据科学笔记本访问。

关于如何使用 AI 快捷操作在 OCI 数据科学中部署大语言模型的详细文档，请参阅此处和此处。

本笔记本将说明如何结合 LlamaIndex 使用 OCI 的数据科学模型。

安装配置¶

若您在 Colab 平台上打开此 Notebook，很可能需要先安装 LlamaIndex 🦙。

In [ ]:

%pip install llama-index-llms-oci-data-science

%pip install llama-index-llms-oci-data-science

In [ ]:

!pip install llama-index

!pip install llama-index

您还需要安装 oracle-ads SDK。

In [ ]:

!pip install -U oracle-ads

!pip install -U oracle-ads

认证¶

LlamaIndex 支持的认证方式与其他 OCI 服务相同，遵循标准 SDK 认证方法，具体包括 API 密钥、会话令牌、实例主体和资源主体。更多详细信息可参阅此处。请确保已配置访问 OCI 数据科学模型部署端点所需的策略。oracle-ads 工具可简化 OCI 数据科学平台内的认证流程。

基本用法¶

使用 OCI 数据科学 AI 提供的 LLM 与 LlamaIndex 结合时，您只需使用数据科学模型部署端点和模型 ID 初始化 OCIDataScience 接口即可。默认情况下，AI 快速操作中部署的所有模型都会获得 odsc-model 作为 ID，不过该 ID 在部署过程中可以修改。

调用 complete 方法并传入提示词¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = llm.complete("Tell me a joke")

print(response)

import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = llm.complete("Tell me a joke") print(response)

使用消息列表调用 chat 方法¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = llm.chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

print(response)

import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.base.llms.types import ChatMessage ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = llm.chat( [ ChatMessage(role="user", content="Tell me a joke"), ChatMessage( role="assistant", content="Why did the chicken cross the road?" ), ChatMessage(role="user", content="I don't know, why?"), ] ) print(response)

流式传输¶

使用 stream_complete 端点¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)

for chunk in llm.stream_complete("Tell me a joke"):
    print(chunk.delta, end="")

import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) for chunk in llm.stream_complete("Tell me a joke"): print(chunk.delta, end="")

使用 stream_chat 端点¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = llm.stream_chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

for chunk in response:
    print(chunk.delta, end="")

import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.base.llms.types import ChatMessage ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = llm.stream_chat( [ ChatMessage(role="user", content="Tell me a joke"), ChatMessage( role="assistant", content="Why did the chicken cross the road?" ), ChatMessage(role="user", content="I don't know, why?"), ] ) for chunk in response: print(chunk.delta, end="")

异步处理¶

使用提示词调用 acomplete¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = await llm.acomplete("Tell me a joke")

print(response)

import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = await llm.acomplete("Tell me a joke") print(response)

使用消息列表调用 achat¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = await llm.achat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

print(response)

import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.base.llms.types import ChatMessage ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = await llm.achat( [ ChatMessage(role="user", content="Tell me a joke"), ChatMessage( role="assistant", content="Why did the chicken cross the road?" ), ChatMessage(role="user", content="I don't know, why?"), ] ) print(response)

使用 astream_complete 端点¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)

async for chunk in await llm.astream_complete("Tell me a joke"):
    print(chunk.delta, end="")

import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) async for chunk in await llm.astream_complete("Tell me a joke"): print(chunk.delta, end="")

使用 astream_chat 端点¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = await llm.stream_chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

async for chunk in response:
    print(chunk.delta, end="")

import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.base.llms.types import ChatMessage ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", ) response = await llm.stream_chat( [ ChatMessage(role="user", content="Tell me a joke"), ChatMessage( role="assistant", content="Why did the chicken cross the road?" ), ChatMessage(role="user", content="I don't know, why?"), ] ) async for chunk in response: print(chunk.delta, end="")

配置模型¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)
response = llm.chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
    ]
)
print(response)

import ads from llama_index.llms.oci_data_science import OCIDataScience ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", temperature=0.2, max_tokens=500, timeout=120, context_window=2500, additional_kwargs={ "top_p": 0.75, "logprobs": True, "top_logprobs": 3, }, ) response = llm.chat( [ ChatMessage(role="user", content="Tell me a joke"), ] ) print(response)

函数调用¶

AI Quick Actions 提供预构建的服务容器，可极大简化大型语言模型的部署与服务流程。这些服务容器采用 vLLM（专为 LLM 设计的高吞吐、内存高效推理与服务引擎）或 TGI（面向热门开源 LLM 的高性能文本生成服务器）作为模型托管方案，所创建的终端节点全面兼容 OpenAI API 协议。这使得模型部署可直接替代现有使用 OpenAI API 的应用场景。若部署模型支持函数调用功能，则通过 llm 的 predict_and_call 函数与 LlamaIndex 工具集成后，可挂载任意工具并交由 LLM 自主决策是否调用（及调用哪些工具）。

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.tools import FunctionTool

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)


def multiply(a: float, b: float) -> float:
    print(f"---> {a} * {b}")
    return a * b


def add(a: float, b: float) -> float:
    print(f"---> {a} + {b}")
    return a + b


def subtract(a: float, b: float) -> float:
    print(f"---> {a} - {b}")
    return a - b


def divide(a: float, b: float) -> float:
    print(f"---> {a} / {b}")
    return a / b


multiply_tool = FunctionTool.from_defaults(fn=multiply)
add_tool = FunctionTool.from_defaults(fn=add)
sub_tool = FunctionTool.from_defaults(fn=subtract)
divide_tool = FunctionTool.from_defaults(fn=divide)

response = llm.predict_and_call(
    [multiply_tool, add_tool, sub_tool, divide_tool],
    user_msg="Calculate the result of `8 + 2 - 6`.",
    verbose=True,
)

print(response)

import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.tools import FunctionTool ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", temperature=0.2, max_tokens=500, timeout=120, context_window=2500, additional_kwargs={ "top_p": 0.75, "logprobs": True, "top_logprobs": 3, }, ) def multiply(a: float, b: float) -> float: print(f"---> {a} * {b}") return a * b def add(a: float, b: float) -> float: print(f"---> {a} + {b}") return a + b def subtract(a: float, b: float) -> float: print(f"---> {a} - {b}") return a - b def divide(a: float, b: float) -> float: print(f"---> {a} / {b}") return a / b multiply_tool = FunctionTool.from_defaults(fn=multiply) add_tool = FunctionTool.from_defaults(fn=add) sub_tool = FunctionTool.from_defaults(fn=subtract) divide_tool = FunctionTool.from_defaults(fn=divide) response = llm.predict_and_call( [multiply_tool, add_tool, sub_tool, divide_tool], user_msg="Calculate the result of `8 + 2 - 6`.", verbose=True, ) print(response)

使用 FunctionCallingAgent¶

In [ ]:

import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.tools import FunctionTool
from llama_index.core.agent import FunctionCallingAgent

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)


def multiply(a: float, b: float) -> float:
    print(f"---> {a} * {b}")
    return a * b


def add(a: float, b: float) -> float:
    print(f"---> {a} + {b}")
    return a + b


def subtract(a: float, b: float) -> float:
    print(f"---> {a} - {b}")
    return a - b


def divide(a: float, b: float) -> float:
    print(f"---> {a} / {b}")
    return a / b


multiply_tool = FunctionTool.from_defaults(fn=multiply)
add_tool = FunctionTool.from_defaults(fn=add)
sub_tool = FunctionTool.from_defaults(fn=subtract)
divide_tool = FunctionTool.from_defaults(fn=divide)

agent = FunctionCallingAgent.from_tools(
    tools=[multiply_tool, add_tool, sub_tool, divide_tool],
    llm=llm,
    verbose=True,
)
response = agent.chat(
    "Calculate the result of `8 + 2 - 6`. Use tools. Return the calculated result."
)

print(response)

import ads from llama_index.llms.oci_data_science import OCIDataScience from llama_index.core.tools import FunctionTool from llama_index.core.agent import FunctionCallingAgent ads.set_auth(auth="security_token", profile="") llm = OCIDataScience( model="odsc-llm", endpoint="https:///predict", temperature=0.2, max_tokens=500, timeout=120, context_window=2500, additional_kwargs={ "top_p": 0.75, "logprobs": True, "top_logprobs": 3, }, ) def multiply(a: float, b: float) -> float: print(f"---> {a} * {b}") return a * b def add(a: float, b: float) -> float: print(f"---> {a} + {b}") return a + b def subtract(a: float, b: float) -> float: print(f"---> {a} - {b}") return a - b def divide(a: float, b: float) -> float: print(f"---> {a} / {b}") return a / b multiply_tool = FunctionTool.from_defaults(fn=multiply) add_tool = FunctionTool.from_defaults(fn=add) sub_tool = FunctionTool.from_defaults(fn=subtract) divide_tool = FunctionTool.from_defaults(fn=divide) agent = FunctionCallingAgent.from_tools( tools=[multiply_tool, add_tool, sub_tool, divide_tool], llm=llm, verbose=True, ) response = agent.chat( "Calculate the result of `8 + 2 - 6`. Use tools. Return the calculated result." ) print(response)