tos-vectors

# TOS Vectors Skill

Comprehensive skill for managing vector storage, indexing, and similarity search using the TOS Vectors service - a cloud-based vector database optimized for AI applications.

## Quick Start

### Initialize Client

```python
import os
import tos

# Get credentials from environment
ak = os.getenv('TOS_ACCESS_KEY')
sk = os.getenv('TOS_SECRET_KEY')
account_id = os.getenv('TOS_ACCOUNT_ID')

# Configure endpoint and region
endpoint = 'https://tosvectors-cn-beijing.volces.com'
region = 'cn-beijing'

# Create client
client = tos.VectorClient(ak, sk, endpoint, region)
```

### Basic Workflow

```python
# 1. Create vector bucket (like a database)
client.create_vector_bucket('my-vectors')

# 2. Create vector index (like a table)
client.create_index(
    account_id=account_id,
    vector_bucket_name='my-vectors',
    index_name='embeddings-768d',
    data_type=tos.DataType.DataTypeFloat32,
    dimension=768,
    distance_metric=tos.DistanceMetricType.DistanceMetricCosine
)

# 3. Insert vectors
vectors = [
    tos.models2.Vector(
        key='doc-1',
        data=tos.models2.VectorData(float32=[0.1] * 768),
        metadata={'title': 'Document 1', 'category': 'tech'}
    )
]
client.put_vectors(
    vector_bucket_name='my-vectors',
    account_id=account_id,
    index_name='embeddings-768d',
    vectors=vectors
)

# 4. Search similar vectors
query_vector = tos.models2.VectorData(float32=[0.1] * 768)
results = client.query_vectors(
    vector_bucket_name='my-vectors',
    account_id=account_id,
    index_name='embeddings-768d',
    query_vector=query_vector,
    top_k=5,
    return_distance=True,
    return_metadata=True
)
```

## Core Operations

### Vector Bucket Management

**Create Bucket**
```python
client.create_vector_bucket(bucket_name)
```

**List Buckets**
```python
result = client.list_vector_buckets(max_results=100)
for bucket in result.vector_buckets:
    print(bucket.vector_bucket_name)
```

**Delete Bucket** (must be empty)
```python
client.delete_vector_bucket(bucket_name, account_id)
```

### Vector Index Management

**Create Index**
```python
client.create_index(
    account_id=account_id,
    vector_bucket_name=bucket_name,
    index_name='my-index',
    data_type=tos.DataType.DataTypeFloat32,
    dimension=128,
    distance_metric=tos.DistanceMetricType.DistanceMetricCosine
)
```

**List Indexes**
```python
result = client.list_indexes(bucket_name, account_id)
for index in result.indexes:
    print(f"{index.index_name}: {index.dimension}d")
```

### Vector Data Operations

**Insert Vectors** (batch up to 500)
```python
vectors = []
for i in range(100):
    vector = tos.models2.Vector(
        key=f'vec-{i}',
        data=tos.models2.VectorData(float32=[...]),
        metadata={'category': 'example'}
    )
    vectors.append(vector)

client.put_vectors(
    vector_bucket_name=bucket_name,
    account_id=account_id,
    index_name=index_name,
    vectors=vectors
)
```

**Query Similar Vectors** (KNN search)
```python
results = client.query_vectors(
    vector_bucket_name=bucket_name,
    account_id=account_id,
    index_name=index_name,
    query_vector=query_vector,
    top_k=10,
    filter={"$and": [{"category": "tech"}]},  # Optional metadata filter
    return_distance=True,
    return_metadata=True
)

for vec in results.vectors:
    print(f"Key: {vec.key}, Distance: {vec.distance}")
```

**Get Vectors by Keys**
```python
result = client.get_vectors(
    vector_bucket_name=bucket_name,
    account_id=account_id,
    index_name=index_name,
    keys=['vec-1', 'vec-2'],
    return_data=True,
    return_metadata=True
)
```

**Delete Vectors**
```python
client.delete_vectors(
    vector_bucket_name=bucket_name,
    account_id=account_id,
    index_name=index_name,
    keys=['vec-1', 'vec-2']
)
```

## Common Use Cases

### 1. Semantic Search
Build a semantic search system for documents:

```python
# Index documents
for doc in documents:
    embedding = get_embedding(doc.text)  # Your embedding model
    vector = tos.models2.Vector(
        key=doc.id,
        data=tos.models2.VectorData(float32=embedding),
        metadata={'title': doc.title, 'content': doc.text[:500]}
    )
    vectors.append(vector)

client.put_vectors(
    vector_bucket_name=bucket_name,
    account_id=account_id,
    index_name=index_name,
    vectors=vectors
)

# Search
query_embedding = get_embedding(user_query)
results = client.query_vectors(
    vector_bucket_name=bucket_name,
    account_id=account_id,
    index_name=index_name,
    query_vector=tos.models2.VectorData(float32=query_embedding),
    top_k=5,
    return_metadata=True
)
```

### 2. RAG (Retrieval Augmented Generation)
Retrieve relevant context for LLM prompts:

```python
# Retrieve relevant documents
question_embedding = get_embedding(user_question)
search_results = client.query_vectors(
    vector_bucket_name=bucket_name,
    account_id=account_id,
    index_name='knowledge-base',
    query_vector=tos.models2.VectorData(float32=question_embedding),
    top_k=3,
    return_metadata=True
)

# Build context
context = "\n\n".join([
    v.metadata.get('content', '') for v in search_results.vectors
])

# Generate answer with LLM
prompt = f"Context:\n{context}\n\nQuestion: {user_question}"
```

### 3. Recommendation System
Find similar items based on user preferences:

```python
# Query with metadata filtering
results = client.query_vectors(
    vector_bucket_name=bucket_name,
    account_id=account_id,
    index_name='products',
    query_vector=user_preference_vector,
    top_k=10,
    filter={"$and": [{"category": "electronics"}, {"price_range": "mid"}]},
    return_metadata=True
)
```

## Best Practices

### Naming Conventions
- **Bucket names**: 3-32 chars, lowercase letters, numbers, hyphens only
- **Index names**: 3-63 chars
- **Vector keys**: 1-1024 chars, use meaningful identifiers

### Batch Operations
- Insert up to 500 vectors per call
- Delete up to 100 vectors per call
- Use pagination for listing operations

### Error Handling
```python
try:
    result = client.create_vector_bucket(bucket_name)
except tos.exceptions.TosClientError as e:
    print(f'Client error: {e.message}')
except tos.exceptions.TosServerError as e:
    print(f'Server error: {e.code}, Request ID: {e.request_id}')
```

### Performance Tips
- Choose appropriate vector dimensions (balance accuracy vs performance)
- Use metadata filtering to reduce search space
- Use cosine similarity for normalized vectors
- Use Euclidean distance for absolute distances

## Important Limits

- **Vector buckets**: Max 100 per account
- **Vector dimensions**: 1-4096
- **Batch insert**: 1-500 vectors per call
- **Batch get/delete**: 1-100 vectors per call
- **Query TopK**: 1-30 results

## Additional Resources

For detailed API reference, see [REFERENCE.md](REFERENCE.md)
For complete workflows, see [WORKFLOWS.md](WORKFLOWS.md)
For example scripts, see the `scripts/` directory