request-routing

# request-routing

Configure routing rules in ai-gateway to control which LLM provider handles each request.

## Routing Rule Format

Each rule has:
- A model glob pattern (e.g., `gpt-4*`, `claude-*`, `command-r*`)
- A primary provider
- An optional ordered fallback chain
- An active/disabled status

Rules are evaluated in database insertion order. First match wins.

## Admin API

### List rules

```bash
curl http://localhost:4081/admin/routes \
  -H "Authorization: Bearer gw_your_admin_key"
```

### Create a rule

```bash
curl -X POST http://localhost:4081/admin/routes \
  -H "Authorization: Bearer gw_your_admin_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "GPT-4 to OpenAI with Anthropic fallback",
    "model_pattern": "gpt-4*",
    "provider_id": "openai-provider-id",
    "fallback_ids": ["anthropic-provider-id"],
    "is_active": 1
  }'
```

### Update a rule

```bash
curl -X PATCH http://localhost:4081/admin/routes/<rule-id> \
  -H "Authorization: Bearer gw_your_admin_key" \
  -H "Content-Type: application/json" \
  -d '{"fallback_ids": ["anthropic-id", "cohere-id"]}'
```

### Delete a rule

```bash
curl -X DELETE http://localhost:4081/admin/routes/<rule-id> \
  -H "Authorization: Bearer gw_your_admin_key"
```

## Pattern Matching

Patterns use glob syntax:
- `*` matches any sequence of characters
- `?` matches a single character
- Literal strings match exactly

Examples:
- `gpt-4*` matches gpt-4o, gpt-4-turbo, gpt-4-vision-preview
- `claude-*` matches all Claude models
- `gpt-3.5-turbo` matches only that exact model
- `*` matches all models (useful as a catch-all last rule)

## Fallback Behavior

The gateway tries providers in order:
1. Primary provider
2. Fallback 1 (if primary fails)
3. Fallback 2 (if fallback 1 fails)
4. ... until all exhausted, then returns 502

Failure triggers:
- HTTP 5xx from provider
- Connection timeout (default 30s)
- Connection refused
- Provider 429 (provider-side rate limit, not gateway rate limit)

Non-retriable (no fallback):
- HTTP 400 invalid request
- HTTP 401 auth error
- HTTP 403 forbidden

## Common Patterns

### Route by cost tier

```json
[
  { "pattern": "gpt-4o",         "provider": "openai",    "comment": "premium" },
  { "pattern": "gpt-3.5-turbo",  "provider": "openai",    "comment": "budget"  },
  { "pattern": "claude-3-haiku", "provider": "anthropic", "comment": "cheap"   }
]
```

### High availability for critical model

```json
{
  "pattern": "gpt-4o",
  "primary": "openai",
  "fallbacks": ["anthropic-claude-3-5-sonnet", "cohere-command-r-plus"]
}
```

### Shadow routing (A/B test)

Create two rules for the same pattern with different providers, then toggle active state to switch traffic.