Astra DB¶

DataStax Astra DB 是基于 Apache Cassandra 构建的无服务器向量数据库，通过易用的 JSON API 进行访问。

要运行本笔记本，您需要一个运行在云端的 DataStax Astra DB 实例（可在 datastax.com 免费获取）。

请确保已安装 llama-index 和 astrapy 依赖包：

In [ ]:

%pip install llama-index-vector-stores-astra-db
%pip install llama-index-embeddings-openai

%pip install llama-index-vector-stores-astra-db %pip install llama-index-embeddings-openai

In [ ]:

!pip install llama-index
!pip install "astrapy>=1.0"

!pip install llama-index !pip install "astrapy>=1.0"

请提供数据库连接参数及密钥：¶

In [ ]:

import os
import getpass

api_endpoint = input(
    "\nPlease enter your Database Endpoint URL (e.g. 'https://4bc...datastax.com'):"
)

token = getpass.getpass(
    "\nPlease enter your 'Database Administrator' Token (e.g. 'AstraCS:...'):"
)

os.environ["OPENAI_API_KEY"] = getpass.getpass(
    "\nPlease enter your OpenAI API Key (e.g. 'sk-...'):"
)

import os import getpass api_endpoint = input( "\nPlease enter your Database Endpoint URL (e.g. 'https://4bc...datastax.com'):" ) token = getpass.getpass( "\nPlease enter your 'Database Administrator' Token (e.g. 'AstraCS:...'):" ) os.environ["OPENAI_API_KEY"] = getpass.getpass( "\nPlease enter your OpenAI API Key (e.g. 'sk-...'):" )

导入所需依赖包：¶

In [ ]:

from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    StorageContext,
)
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.vector_stores.astra_db import AstraDBVectorStore

from llama_index.core import ( VectorStoreIndex, SimpleDirectoryReader, StorageContext, ) from llama_index.embeddings.openai import OpenAIEmbedding from llama_index.vector_stores.astra_db import AstraDBVectorStore

加载示例数据：¶

In [ ]:

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

!mkdir -p 'data/paul_graham/' !wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

读取数据：¶

In [ ]:

# load documents
documents = SimpleDirectoryReader("./data/paul_graham/").load_data()
print(f"Total documents: {len(documents)}")
print(f"First document, id: {documents[0].doc_id}")
print(f"First document, hash: {documents[0].hash}")
print(
    "First document, text"
    f" ({len(documents[0].text)} characters):\n{'='*20}\n{documents[0].text[:360]} ..."
)

# load documents documents = SimpleDirectoryReader("./data/paul_graham/").load_data() print(f"Total documents: {len(documents)}") print(f"First document, id: {documents[0].doc_id}") print(f"First document, hash: {documents[0].hash}") print( "First document, text" f" ({len(documents[0].text)} characters):\n{'='*20}\n{documents[0].text[:360]} ..." )

创建 Astra DB 向量存储对象：¶

In [ ]:

astra_db_store = AstraDBVectorStore(
    token=token,
    api_endpoint=api_endpoint,
    collection_name="astra_v_table",
    embedding_dimension=1536,
)

astra_db_store = AstraDBVectorStore( token=token, api_endpoint=api_endpoint, collection_name="astra_v_table", embedding_dimension=1536, )

从文档构建索引：¶

In [ ]:

embed_model = OpenAIEmbedding(model_name="text-embedding-3-small")

storage_context = StorageContext.from_defaults(vector_store=astra_db_store)

index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context, embed_model=embed_model
)

embed_model = OpenAIEmbedding(model_name="text-embedding-3-small") storage_context = StorageContext.from_defaults(vector_store=astra_db_store) index = VectorStoreIndex.from_documents( documents, storage_context=storage_context, embed_model=embed_model )

使用索引进行查询：¶

In [ ]:

query_engine = index.as_query_engine()
response = query_engine.query("Why did the author choose to work on AI?")

print(response.response)

query_engine = index.as_query_engine() response = query_engine.query("Why did the author choose to work on AI?") print(response.response)