cliproxy-media

# cliproxy-media

**Source:** https://github.com/bencoremans/site/tree/main/skills/cliproxy-media

Analyze images and PDFs via CLIProxyAPI (Claude Max subscription, zero extra cost).

## Setup

Set the endpoint to your CLIProxy instance:

```bash
export CLIPROXY_URL=http://your-host:8317/v1/messages
```

For Docker setups, replace `your-host` with your container hostname (e.g. `cliproxyapi`, `localhost`, or the container IP).

## Quick start

```bash
# Analyze an image
python3 skills/cliproxy-media/scripts/analyze.py /path/to/image.jpg "What is in this image?"

# Read a PDF
python3 skills/cliproxy-media/scripts/analyze.py /path/to/document.pdf "Give a summary"

# Compare multiple images
python3 skills/cliproxy-media/scripts/analyze.py img1.jpg img2.jpg "Compare these images"

# With streaming (output appears immediately)
python3 skills/cliproxy-media/scripts/analyze.py --stream image.jpg "Describe in detail"

# With system prompt
python3 skills/cliproxy-media/scripts/analyze.py --system "You are a medical expert" scan.jpg "What do you see?"

# With higher token limit
python3 skills/cliproxy-media/scripts/analyze.py --max-tokens 4096 document.pdf "Extensive analysis"
```

## What works ✅ / What doesn't ❌

### ✅ Supported file types

| Type | Format | Note |
|------|--------|------|
| Image | `.jpg` / `.jpeg` | Requires valid JPEG data |
| Image | `.png` | Fully supported |
| Image | `.gif` | Fully supported |
| Image | `.webp` | Fully supported |
| Document | `.pdf` | Base64-encoded, via `document` content type |
| Image via URL | `http://` / `https://` | Direct URL reference, no download needed |

**Multiple files at once:** Provide multiple paths before the question. Max ~100 per request (Anthropic limit).

### ❌ Not supported

- **Office files** (`.docx`, `.xlsx`, `.pptx`) — Workaround: convert to PDF
- **Audio** (`.mp3`, `.wav`, `.ogg`) — Use Whisper for transcription
- **Video** (`.mp4`, `.mov`, `.avi`) — Not supported by the model
- **Other document types** (`.txt`, `.html`, `.md` as document) — Send text directly as a string

## ⚠️ System prompt warning

CLIProxyAPI accepts **only** the array notation for system prompts. The string notation is **silently ignored** — the model does not see it, but you also won't get an error message!

```python
# ❌ DOES NOT WORK — ignored without error message
payload["system"] = "You are an expert."

# ✅ WORKS — always use array notation
payload["system"] = [{"type": "text", "text": "You are an expert."}]
```

The `--system` argument in `analyze.py` automatically uses the correct array notation.

## Configuration (env vars)

| Variable | Default | Description |
|----------|---------|-------------|
| `CLIPROXY_URL` | `http://localhost:8317/v1/messages` | Full endpoint URL |
| `CLIPROXY_MODEL` | `claude-sonnet-4-6` | Model to use |

Example:
```bash
export CLIPROXY_URL=http://localhost:8317/v1/messages
export CLIPROXY_MODEL=claude-opus-4-6
python3 skills/cliproxy-media/scripts/analyze.py image.jpg "question"
```

## Additional options

```
--stream          Streaming output via SSE (output appears immediately)
--system TEXT     System prompt (automatically sent as array)
--max-tokens N    Maximum output tokens (default: 1024)
--model MODEL     Model override (overrides CLIPROXY_MODEL)
--url URL         Endpoint override (overrides CLIPROXY_URL)
```

## Compatibility

This script works with any API that supports the Anthropic Messages format:

| Provider | Compatible | Note |
|----------|-----------|------|
| **CLIProxyAPI** | ✅ Yes | Primarily tested, system prompt array required |
| **OpenRouter** | ✅ Yes | Use Bearer token instead of `x-api-key: dummy` |
| **LiteLLM** | ✅ Yes | As proxy for Anthropic format |
| **Anthropic direct** | ✅ Yes | Use `ANTHROPIC_API_KEY` as x-api-key |

**Note for non-CLIProxy endpoints:** Some proxies do accept string notation for system prompts. Always use array notation for maximum compatibility.

## Known limitations of CLIProxyAPI

- `temperature` and `top_p` may **not** be used at the same time (HTTP 400)
- PDF as document with URL source does not work (`Unable to download the file`)
- Only `claude-sonnet-4-6` and `claude-opus-4-6` available (haiku is deprecated)
- `inference_geo` is always `not_available` in the response

## Direct Python API

If you want to call the script from your own Python code:

```python
import subprocess, json

result = subprocess.run(
    ["python3", "skills/cliproxy-media/scripts/analyze.py", "image.jpg", "Describe this"],
    capture_output=True, text=True
)
print(result.stdout)
```

Or use the built-in exec tool:
```
exec: python3 skills/cliproxy-media/scripts/analyze.py /path/to/image.jpg "question"
```