memory-persistence

# 🧠 Memory System

A flexible memory system for AI agents with optional embedding support and multiple storage backends.

## Features

- **Private & Shared Memories** - Private by default, shared memories for multi-agent collaboration
- **Embedding Search** - Semantic search using sentence-transformers
- **Multiple Backends** - Local file / SQLite / GitHub / Gitee
- **LLM Summarization** - Auto-extract key info from conversations
- **Memory Maintenance** - Review, consolidate, tag suggestions
- **Templates** - Quick memory creation with templates

## Installation

```bash
pip install sentence-transformers scikit-learn pyyaml numpy
```

## Quick Start

### Python API

```python
from memory_system import MemoryManager

# Initialize (local storage)
mm = MemoryManager(backend='local')

# Add 
mm.add("User prefers dark theme", tags=["preference"])

# Search
results = mm.search("dark theme preference")

# List
entries = mm.list(tags=["preference"])
```

### CLI

```bash
# Add 
python3 memory_cli.py add "User feedback: slow page load" --tags "bug,performance"

# List
python3 memory_cli.py list

# Search
python3 memory_cli.py -e search "performance issue"

# Semantic search (with embedding)
python3 memory_cli.py -e search "dark mode"
```

## Private vs Shared Memory

| Type | Storage | Access | Use Case |
|------|---------|--------|----------|
| **Private** | `./memory_data/` | Current agent only | User preferences, personal notes |
| **Shared** | `./shared_memory/` | All agents | Team decisions, collaboration |

**Default**: All memories are private. Use `shared add` only when other agents need to know.

```python
# Private memory - user says "remember..."
mm.add("User name is Zhang San")

# Shared memory - user says "tell other agents..."
smm.add("Team decision: use React", agent_id="agent_a")
```

## Storage Backends

### Local (Default)
```python
mm = MemoryManager(backend='local')
```

### SQLite (High Performance)
```python
mm = MemoryManager(backend='sqlite', base_path='./memory.db')
```

### GitHub
```bash
export GITHUB_TOKEN="your_token"
```
```python
mm = MemoryManager(
    backend='github',
    repo='owner/repo',
    branch='main'
)
```

### Gitee
```bash
export GITEE_TOKEN="your_token"
```
```python
mm = MemoryManager(
    backend='gitee',
    repo='owner/repo',
    branch='master'
)
```

## Embedding & Semantic Search

Embedding is optional and auto-downloads on first use.

```python
# Enable embedding
mm = MemoryManager(backend='local', use_embedding=True)

# Add (auto-generates vector)
mm.add("User works from 9am to 6pm")

# Semantic search - finds similar content
results = mm.search("what time does user work")
```

CLI with embedding:
```bash
python3 memory_cli.py -e search "working hours"
```

## Shared Memory (Multi-Agent)

```python
from memory_system import SharedMemoryManager

# Initialize
smm = SharedMemoryManager(backend='local', shared_path='./shared_memory')

# Add shared memory (from an agent)
smm.add("Bug #123 fixed", agent_id='agent_b')

# List shared memories
shared = smm.list()

# By agent
by_agent = smm.get_by_agent('agent_b')
```

CLI:
```bash
# Add shared 
python3 memory_cli.py shared add "Team decision: use Vue" --agent "agent_a"

# List
python3 memory_cli.py shared list

# Search
python3 memory_cli.py -e shared search "Vue decision"
```

## Conversation Summarization

Auto-extract key information from conversation history.

```python
from memory_system import MemoryManager, MemorySummarizer, ConversationMemoryProcessor

mm = MemoryManager(use_embedding=True)
summarizer = MemorySummarizer()  # Auto-detects OpenClaw model
processor = ConversationMemoryProcessor(mm, summarizer, auto_save=True)

conversation = """
User: I prefer dark theme
Assistant: Changed to dark theme
User: Page loads slowly
Assistant: Optimized images
"""

memories = processor.process(conversation)
```

CLI:
```bash
python3 memory_cli.py summarize --file conversation.txt --save
```

## Memory Maintenance

```bash
# Generate report
python3 memory_cli.py maintenance report

# Review old memories
python3 memory_cli.py maintenance review --days 7

# Find similar memories
python3 memory_cli.py maintenance consolidate

# Suggest tags for untagged memories
python3 memory_cli.py maintenance suggest-tags

# Mark as outdated
python3 memory_cli.py maintenance outdated --mark <id> --reason "expired"
```

## Templates

Predefined formats for quick memory creation.

```bash
# List templates
python3 memory_cli.py template list

# Show template
python3 memory_cli.py template show task

# Use template
python3 memory_cli.py template use task \
  --field title="Complete report" \
  --field priority="high"
```

## Memory Groups

Organize memories into groups.

```bash
# Add to group
python3 memory_cli.py add "work task" --tags "work" --group "work"

# List groups
python3 memory_cli.py group list

# Show group
python3 memory_cli.py group show "work"
```

## Batch Operations

```bash
# Batch add tags
python3 memory_cli.py batch-add-tags id1,id2 --tags "important,priority"

# Batch delete (requires confirmation)
python3 memory_cli.py batch-delete id1,id2 --force
```

## API Reference

### MemoryManager

| Method | Description |
|--------|-------------|
| `add(content, tags, metadata, group)` | Add memory |
| `get(id)` | Get by ID |
| `delete(id)` | Delete |
| `list(tags, limit, offset)` | List with pagination |
| `search(query, tags, top_k, threshold)` | Search |
| `batch_delete(ids)` | Batch delete |
| `list_groups()` | List groups |
| `export_json(filepath)` | Export JSON |

### SharedMemoryManager

| Method | Description |
|--------|-------------|
| `add(content, agent_id, tags)` | Add shared memory |
| `list(tags)` | List shared |
| `get_by_agent(agent_id)` | By agent |
| `search(query)` | Search shared |

## Files Structure

```
memory_system/
├── memory_manager.py   # Core manager
├── shared_memory.py    # Shared 
├── summarizer.py      # LLM summarization
├── maintenance.py      # Maintenance tools
├── templates.py       # Templates
├── embedding.py       # Embedding handler
├── storage/           # Storage backends
│   ├── local.py
│   ├── sqlite.py
│   ├── github.py
│   └── gitee.py
└── memory_cli.py         # CLI entry (run with python3)
```

## Configuration

`config.yaml`:
```yaml
STORAGE_BACKEND: "local"

USE_EMBEDDING: false
EMBEDDING_MODEL: "sentence-transformers/all-MiniLM-L6-v2"

storage:
  local:
    base_path: "./memory_data"
  sqlite:
    base_path: "./memory.db"
  github:
    repo: "owner/repo"
    token_env: "GITHUB_TOKEN"
  gitee:
    repo: "owner/repo"
    token_env: "GITEE_TOKEN"
```

## License

MIT