MyMagic AI 大语言模型¶

简介¶

本笔记本演示了如何利用 MyMagicAI 对云存储桶中的海量数据进行批量推理。目前仅实现了 complete 和 acomplete 两个端点，可适用于多种应用场景，包括文本补全、摘要生成和内容提取等。
使用本笔记本前，您需要准备：

• 从 MyMagicAI 获取的 API 密钥（个人访问令牌）
• 已存储在云存储桶中的数据

请访问 MyMagicAI 官网 点击 "Get Started" 注册以获取 API 密钥。

配置¶

如需设置您的存储桶并为 MyMagic API 授予云存储的安全访问权限，请参考 MyMagic 文档。 若您在 Colab 上打开此 Notebook，可能需要安装 LlamaIndex 🦙。

In [ ]:

%pip install llama-index-llms-mymagic

%pip install llama-index-llms-mymagic

In [ ]:

!pip install llama-index

!pip install llama-index

In [ ]:

from llama_index.llms.mymagic import MyMagicAI

from llama_index.llms.mymagic import MyMagicAI

In [ ]:

llm = MyMagicAI(
    api_key="your-api-key",
    storage_provider="s3",  # s3, gcs
    bucket_name="your-bucket-name",
    session="your-session-name",  # files should be located in this folder on which batch inference will be run
    role_arn="your-role-arn",
    system_prompt="your-system-prompt",
    region="your-bucket-region",
    return_output=False,  # Whether you want MyMagic API to return the output json
    input_json_file=None,  # name of the input file (stored on the bucket)
    list_inputs=None,  # Option to provide inputs as a list in case of small batch
    structured_output=None,  # json schema of the output
)

llm = MyMagicAI( api_key="your-api-key", storage_provider="s3", # s3, gcs bucket_name="your-bucket-name", session="your-session-name", # files should be located in this folder on which batch inference will be run role_arn="your-role-arn", system_prompt="your-system-prompt", region="your-bucket-region", return_output=False, # Whether you want MyMagic API to return the output json input_json_file=None, # name of the input file (stored on the bucket) list_inputs=None, # Option to provide inputs as a list in case of small batch structured_output=None, # json schema of the output )

注意：如果上方将 return_output 设为 True，则 max_tokens 参数应至少设置为 100

In [ ]:

resp = llm.complete(
    question="your-question",
    model="chhoose-model",  # currently we support mistral7b, llama7b, mixtral8x7b, codellama70b, llama70b, more to come...
    max_tokens=5,  # number of tokens to generate, default is 10
)

resp = llm.complete( question="your-question", model="chhoose-model", # currently we support mistral7b, llama7b, mixtral8x7b, codellama70b, llama70b, more to come... max_tokens=5, # number of tokens to generate, default is 10 )

In [ ]:

# The response indicated that the final output is stored in your bucket or raises an exception if the job failed
print(resp)

# The response indicated that the final output is stored in your bucket or raises an exception if the job failed print(resp)

通过 acomplete 端点实现异步请求¶

对于异步操作，请采用以下方法。

In [ ]:

import asyncio

import asyncio

In [ ]:

async def main():
    response = await llm.acomplete(
        question="your-question",
        model="choose-model",  # supported models constantly updated and are listed at docs.mymagic.ai
        max_tokens=5,  # number of tokens to generate, default is 10
    )

    print("Async completion response:", response)

async def main(): response = await llm.acomplete( question="your-question", model="choose-model", # supported models constantly updated and are listed at docs.mymagic.ai max_tokens=5, # number of tokens to generate, default is 10 ) print("Async completion response:", response)

In [ ]:

await main()

await main()