yaml-scenarios

# yaml-scenarios Skill

## When to Use

Use this skill when the user wants to:
- Write a new YAML scenario file for load-test-runner
- Add or modify request steps in an existing scenario
- Configure thresholds for pass/fail determination
- Use template variables in headers or request bodies
- Validate scenario YAML without running it
- Understand which fields are required vs optional

## Scenario Structure

A scenario YAML file has four top-level sections:

1. **Run parameters** - `name`, `target`, `ramp_up_seconds`, `duration_seconds`, `max_vus`
2. **Steps** - the request steps executed by virtual users
3. **Thresholds** - pass/fail criteria evaluated after the run
4. **Variables** (optional) - static values substituted into step fields

## Complete Example with All Fields

```yaml
name: Full API Test
description: Comprehensive scenario showing all available fields
target: https://api.example.com

ramp_up_seconds: 60
duration_seconds: 300
max_vus: 100

variables:
  TOKEN: abc123
  USER_ID: 42

steps:
  # Simple GET
  - name: Health check
    method: GET
    path: /health
    expected_status: 200
    weight: 5

  # GET with auth header using variable
  - name: Get user profile
    method: GET
    path: /users/${USER_ID}
    headers:
      Authorization: Bearer ${TOKEN}
    expected_status: 200
    weight: 20

  # POST with JSON body
  - name: Create order
    method: POST
    path: /orders
    headers:
      Authorization: Bearer ${TOKEN}
      Content-Type: application/json
    body: '{"user_id": ${USER_ID}, "item": "widget"}'
    expected_status: 201
    weight: 30

  # PUT
  - name: Update order status
    method: PUT
    path: /orders/1/status
    headers:
      Authorization: Bearer ${TOKEN}
      Content-Type: application/json
    body: '{"status": "shipped"}'
    expected_status: 200
    weight: 10

  # DELETE
  - name: Delete draft order
    method: DELETE
    path: /orders/draft
    headers:
      Authorization: Bearer ${TOKEN}
    expected_status: 204
    weight: 5

thresholds:
  p95_response_ms: 300
  error_rate: 0.005
  rps_min: 200
```

## Step Weight Explained

Weights determine how often each step is executed relative to others. Virtual users pick a step randomly, weighted by these values.

```
weight: 10  ->  10 / (10+40+30) = 12.5% of requests
weight: 40  ->  40 / (10+40+30) = 50.0% of requests
weight: 30  ->  30 / (10+40+30) = 37.5% of requests
```

If all steps have equal weight (or weight is omitted), they are distributed evenly.

## Template Variables

Variables defined under `variables:` are substituted using `${VAR_NAME}` syntax in:
- `path`
- `body`
- `headers` values

Variables can also be set from environment variables at run time. The server substitutes env vars for any variable not defined in the YAML:

```yaml
variables:
  TOKEN: default-value   # overridden by TOKEN env var if present
```

Runtime override via CLI (planned):
```
load-test run ./scenario.yaml --var TOKEN=my-token --var USER_ID=99
```

## Thresholds Reference

Thresholds are evaluated after the run completes (or at abort). All defined thresholds must pass for the overall run to be marked PASS.

| Threshold | Type | Description | Example |
|---|---|---|---|
| `p95_response_ms` | integer | p95 latency must be below this value (milliseconds) | `500` |
| `error_rate` | float | Error rate must be below this fraction | `0.01` (= 1%) |
| `rps_min` | number | Average RPS must be at or above this value | `100` |

Any subset of thresholds can be defined. A scenario with no thresholds always passes.

## Validation Rules

The scenario is validated before a run starts. Validation errors:

| Rule | Error |
|---|---|
| `name` missing | `name is required` |
| `target` missing or not a URL | `target must be a valid URL` |
| `ramp_up_seconds` >= `duration_seconds` | `ramp_up_seconds must be less than duration_seconds` |
| `max_vus` < 1 | `max_vus must be at least 1` |
| `steps` empty | `at least one step is required` |
| step `method` not in allowed set | `method must be GET, POST, PUT, PATCH, or DELETE` |
| step `path` does not start with `/` | `path must start with /` |
| step `weight` < 0 | `weight must be a positive integer` |
| `expected_status` not a valid HTTP status | `expected_status must be a valid HTTP status code` |
| `error_rate` threshold > 1.0 | `error_rate must be between 0 and 1` |

Run validation without executing:
```
load-test validate ./api-smoke.yaml
```

## Common Scenario Patterns

**Smoke test (quick sanity check)**
```yaml
ramp_up_seconds: 10
duration_seconds: 60
max_vus: 10
```

**Load test (realistic production traffic)**
```yaml
ramp_up_seconds: 60
duration_seconds: 300
max_vus: 100
```

**Stress test (find breaking point)**
```yaml
ramp_up_seconds: 120
duration_seconds: 600
max_vus: 500
```

**Soak test (sustained load over time)**
```yaml
ramp_up_seconds: 60
duration_seconds: 3600
max_vus: 50
```

## Validate a File

```bash
# Validate syntax and schema
load-test validate ./api-smoke.yaml

# Example output
ok  scenario   API Smoke Test
ok  target     https://api.example.com
ok  steps      3 steps defined
ok  thresholds 3 thresholds defined
ok  VALID
```

```bash
# Example failure output
FAIL  steps[1].path must start with /
FAIL  ramp_up_seconds (120) must be less than duration_seconds (60)
2 validation errors
```