consensus

## Overview

<!-- CANONICAL: shared/dispatch-convention.md -->
All subagent dispatches use disk-mediated dispatch. See `shared/dispatch-convention.md` for the full protocol.

Multi-model consensus dispatches high-stakes decision prompts to multiple LLM providers in parallel via an MCP server, then synthesizes the responses. It surfaces blind spots that single-model review misses. Entirely opt-in — without configuration, all skills behave exactly as before.

The consensus system is not a replacement for existing skill logic. It is an amplifier applied at specific, high-leverage decision points where the cost of a missed defect or a wrong judgment is disproportionately high. Skills that integrate consensus call it at defined moments (e.g., the stagnation judge in quality-gate, the Challenger in design review) and treat its output as additional signal, not as an override.

## Tool Interface

### `consensus_query` (MCP tool)

Dispatches a prompt to all configured models in parallel, collects responses, and returns a synthesized result.

#### Parameters

| Parameter  | Type   | Required | Description |
|------------|--------|----------|-------------|
| `prompt`   | string | Yes      | The decision prompt to send to each model. Should be self-contained and unambiguous. |
| `context`  | string | Yes      | Supporting context (artifact content, conversation history, prior findings). Sent verbatim to each model alongside the prompt. |
| `mode`     | enum   | Yes      | One of `"review"`, `"verdict"`, or `"investigate"`. Controls how responses are aggregated. |
| `metadata` | object | No       | Optional metadata for traceability. Fields: `artifact_type` (string), `round_number` (int), `score_progression` (list of ints). |

#### Return Value

| Field              | Type   | Description |
|--------------------|--------|-------------|
| `status`           | enum   | `"complete"` (all models responded), `"partial"` (some models responded, count >= min_models), `"unavailable"` (fewer than min_models responded or tool not configured). |
| `models_queried`   | int    | Number of models the query was dispatched to. |
| `models_responded` | int    | Number of models that returned a valid response within the timeout. |
| `synthesis`        | string | Aggregated result text, formatted according to the selected mode. |
| `agreements`       | list   | Findings or verdicts where 2+ models converged. |
| `disagreements`    | list   | Findings or verdicts where models explicitly contradicted each other. |
| `unique_findings`  | list   | Findings raised by exactly one model, with provenance. |
| `per_model`        | list   | Per-model detail. Each entry: `{ provider: string, model_id: string, content: string, responded: boolean }`. |

When `status` is `"unavailable"`, all list fields are empty and `synthesis` contains a short explanation (e.g., "Consensus unavailable: no models configured"). The caller should fall back to single-model behavior.

## Consensus Modes

### review

For adversarial review of artifacts (red-team rounds, design review, code review).

- Each model receives the prompt and context independently and reviews the artifact.
- The aggregator merges findings and deduplicates by semantic similarity (same root cause = one finding).
- **Confidence levels:**
  - **High** — 3 or more models independently identify the same finding.
  - **Medium** — 2 models independently identify the same finding.
  - **Low** — A single model identifies a finding with no corroboration.
- **Severity** uses Crucible's standard taxonomy: Fatal, Significant, Minor.
- Every finding carries **per-finding provenance** — the list of model IDs that surfaced it.
- Deduplicated findings are ordered by severity (Fatal first), then by confidence (High first).

### verdict

For binary or ternary decisions (stagnation judge, fix verification, pass/fail gates).

- Each model renders a verdict (e.g., `STAGNATION` or `PROGRESS`) along with structured reasoning.
- The aggregator reports the **verdict distribution** (e.g., `"STAGNATION: 3, PROGRESS: 1"`).
- **Confidence levels:**
  - **Unanimous** — All responding models agree.
  - **Supermajority** — 75%+ of responding models agree.
  - **Split** — No supermajority exists.
- Split decisions are flagged with `"requires_human_judgment": true` in the synthesis.
- The majority verdict is reported as the recommendation, but the caller decides how to act on splits.

### investigate

For design exploration (Challenger mode, architectural analysis).

- Each model investigates the prompt independently, surfacing risks, alternatives, or concerns.
- The aggregator merges findings and deduplicates by semantic similarity.
- Findings unique to a single model are marked with **provenance** and highlighted as potential blind-spot discoveries.
- Contradictory findings (where two models reach opposing conclusions about the same aspect) are flagged explicitly in the `disagreements` list with both positions and their reasoning.
- The synthesis groups results into: shared concerns, unique discoveries, and contradictions.

## Configuration

Consensus is configured via `.claude/consensus-config.yaml` in the project root. If this file is absent or `consensus.enabled` is `false`, all consensus calls return `status: "unavailable"` and skills fall back to single-model behavior with zero overhead.

### Schema

```yaml
consensus:
  enabled: true                # Master switch. Default: false.
  min_models: 2                # Minimum models that must respond for a valid result. Default: 2.
  timeout_seconds: 120         # Per-query timeout. Models that haven't responded are marked responded: false. Default: 120.
  models:
    - provider: anthropic      # Provider identifier. Shipped: "anthropic", "google". Planned: "openai", "deepseek".
      model: claude-sonnet-4-20250514  # Model identifier within the provider.
      api_key_env: ANTHROPIC_API_KEY   # Environment variable name holding the API key. NEVER a raw key.
      temperature: 0.6         # Sampling temperature for this model. Default: 0.6. Range: 0.0-1.0.
    - provider: google
      model: gemini-2.5-pro
      api_key_env: GOOGLE_API_KEY
      temperature: 0.6
  modes:
    review: true               # Enable consensus for review-mode calls. Default: true.
    verdict: true              # Enable consensus for verdict-mode calls. Default: true.
    investigate: true          # Enable consensus for investigate-mode calls. Default: true.
```

### Field Reference

| Field                       | Type    | Default | Description |
|-----------------------------|---------|---------|-------------|
| `consensus.enabled`         | boolean | `false` | Master switch. When false, all `consensus_query` calls immediately return `status: "unavailable"`. |
| `consensus.min_models`      | int     | `2`     | Minimum number of models that must respond for the result to be `"complete"` or `"partial"`. If fewer respond, status is `"unavailable"`. Must be >= 2. |
| `consensus.timeout_seconds` | int     | `120`   | Maximum time (seconds) to wait for all models to respond. Models exceeding this are marked `responded: false`. Must be >= 10 and <= 600. |
| `consensus.models`          | list    | `[]`    | List of model configurations. At least `min_models` entries required. |
| `consensus.models[].provider`    | string | —  | Provider identifier. Must be one of the shipped providers (`anthropic`, `google`) or a planned provider once available. |
| `consensus.models[].model`       | string | —  | Model identifier recognized by the provider's API. |
| `consensus.models[].api_key_env` | string | —  | Name of the environment variable containing the API key. The MCP server reads the key from the environment at runtime. Keys are NEVER stored in config files. |
| `consensus.models[].temperature` | float  | `0.6` | Sampling temperature. Lower values produce more deterministic responses. |
| `consensus.modes.review`     | boolean | `true`  | Whether `review` mode calls are dispatched to consensus. When false, review-mode calls return `"unavailable"`. |
| `consensus.modes.verdict`    | boolean | `true`  | Whether `verdict` mode calls are dispatched to consensus. |
| `consensus.modes.investigate`| boolean | `true`  | Whether `investigate` mode calls are dispatched to consensus. |

### Validation Rules

- If `consensus.enabled` is `true` but `consensus.models` has fewer entries than `min_models`, the MCP server logs a warning and treats consensus as unavailable.
- If a referenced `api_key_env` variable is not set in the environment, that model is skipped. If the remaining count drops below `min_models`, status is `"unavailable"`.
- Duplicate provider+model combinations are rejected at config load time.
- The `temperature` field is clamped to [0.0, 1.0].

## Graceful Degradation

The consensus system is designed to never break existing workflows. Degradation follows a strict hierarchy:

1. **Consensus available** — All configured models respond within the timeout. `status: "complete"`. Full multi-model synthesis is returned to the calling skill.

2. **Partially available** — Some models fail or time out, but the number of successful responses is >= `min_models`. `status: "partial"`. Aggregation proceeds with available responses. The `models_queried` and `models_responded` fields indicate the gap.

3. **Unavailable (configuration)** — No config file, `enabled: false`, no models configured, or no API keys in environment. `status: "unavailable"`. The calling skill receives an immediate response and proceeds with its existing single-model logic. Zero overhead, zero delay.

4. **MCP server not running** — The `consensus_query` tool is not present in the tool list. Skills that integrate consensus check for tool availability before attempting calls. If the tool is absent, the consensus step is skipped entirely. No errors, no retries.

In all cases, the calling skill's core logic is self-contained. Consensus is additive signal, never a gate.

## Cost Estimation

Consensus multiplies API costs by the number of models queried. The following table estimates cost multipliers relative to a single-model baseline, assuming prompts of roughly equal token count across models.

### Per-Query Cost Multiplier

| Models Configured | Multiplier |
|-------------------|------------|
| 2                 | 2x         |
| 3                 | 3x         |
| 4                 | 4x         |

### Estimated Impact on Quality Gate Runs

Quality gate integrates consensus at two points: the stagnation judge (verdict mode, every round) and periodic red-team review (review mode, periodic). The table below shows total additional API calls from consensus across different run lengths, assuming periodic red-team consensus fires on rounds 1, 4, 7, 10 (every 3rd round starting from round 1).

| Run Length | Stagnation Verdicts | Periodic Red-Team Reviews | Total Consensus Calls | With 2 Models | With 3 Models | With 4 Models |
|------------|---------------------|---------------------------|-----------------------|----------------|----------------|----------------|
| 5 rounds   | 5                   | 2 (rounds 1, 4)          | 7                     | 14 API calls   | 21 API calls   | 28 API calls   |
| 10 rounds  | 10                  | 4 (rounds 1, 4, 7, 10)   | 14                    | 28 API calls   | 42 API calls   | 56 API calls   |

Periodic application (rather than every-round) keeps cost manageable. A 10-round quality gate run with 2 models adds 28 API calls total — significant but bounded. Teams should start with 2 models and add more only after validating the signal-to-cost ratio.

### Cost Management Recommendations

- Start with 2 models to validate consensus value before scaling.
- Use mode-level toggles (`consensus.modes.verdict: false`) to disable consensus for lower-value decision points.
- Monitor `models_responded` in results to detect models that consistently time out (wasted cost).

## Red Flags

### Never

- **Use consensus on every red-team round.** Consensus is for periodic checkpoints (e.g., rounds 1, 4, 7), not every iteration. Every-round consensus is wasteful and slows feedback loops.
- **Treat single-model unique findings as less important than multi-model agreements.** Unique findings are often the most valuable — they represent blind spots. Confidence level indicates corroboration, not importance.
- **Pass consensus provenance to the next red-team reviewer.** This creates anchoring bias. The next reviewer must form independent judgments. Provenance is for the aggregator and the human operator, not for downstream reviewers.
- **Store API keys in config files.** Keys are referenced by environment variable name only. The `api_key_env` field holds the variable name, never the key itself.
- **Log prompt content or API keys.** The MCP server must not persist prompt text, context, or API credentials to disk or stdout. Structured metadata (model IDs, response times, token counts) may be logged.

### Always

- **Fall back gracefully when consensus is unavailable.** Every skill that calls `consensus_query` must have a code path that works without it. Consensus is additive, never required.
- **Include provenance metadata in consensus results.** Every finding and verdict must trace back to the model(s) that produced it. Provenance enables debugging, trust calibration, and cost attribution.
- **Check tool availability before attempting consensus calls.** Skills must verify that the `consensus_query` tool exists in the current tool list before calling it. If absent, skip the consensus step silently.

## Integration Points

### Active Integration

| Skill | Decision Point | Consensus Mode | Frequency |
|-------|---------------|---------------|-----------|
| quality-gate | Stagnation judge | verdict | Every judge dispatch |
| quality-gate | Red-team review | review | Rounds 1, 4, 7, 10, 13 |
| design | Challenger (deep dive) | investigate | Every deep-dive challenge |

### Excluded (with rationale)

| Skill | Decision Point | Why Excluded |
|-------|---------------|-------------|
| inquisitor | 5-dimension dispatch | Subagents write and run tests — requires local filesystem and execution environment. External models cannot participate. |
| quality-gate | Fix verifier | Mechanical check (did the fix address the finding?). Single-model Sonnet is sufficient. Multi-model adds cost without proportional value. |
| quality-gate | Fix agent | Requires filesystem access to modify code/docs. External models cannot write local files. |
| red-team | Standalone full-loop | When invoked directly (not via quality-gate), red-team runs its own stagnation loop. Consensus integration is at the quality-gate level, which owns the iteration. |
| design | Investigation agents | Domain Researcher, Impact Analyst all need filesystem access. External models cannot read the local codebase. Recon provides structural context but runs locally. |
| build | Task implementation | Code generation requires filesystem access and project context. |

### Reconsideration Criteria

An excluded decision point should be reconsidered for consensus integration if:
1. The task becomes self-contained (no filesystem dependency), OR
2. MCP servers gain filesystem-pass-through capabilities, OR
3. Empirical evidence shows the single-model output has systematic blind spots at that point

## MCP Server Implementation Guide

The `crucible-consensus` MCP server is a standalone Python process at `mcp-servers/crucible-consensus/`.

### Architecture

1. Reads `.claude/consensus-config.yaml` from the project directory
2. Validates API key environment variables are set
3. Exposes the `consensus_query` tool via MCP protocol
4. On each tool call:
   a. Dispatches the prompt + context to all configured models in parallel
   b. Waits up to `timeout_seconds` for each model
   c. Collects successful responses, discards failures
   d. If successful_count >= min_models: runs aggregation prompt, returns synthesis
   e. If successful_count < min_models: returns status "unavailable"

### Provider Adapters

Each provider adapter handles:
- API authentication (reading key from environment)
- Request formatting (model-specific API shape)
- Response extraction (model-specific response parsing)
- Error handling (HTTP errors, rate limits, malformed responses)

Shipped providers: anthropic, google.
Planned: openai, deepseek (deferred to follow-up).
Adding a new provider requires one adapter function following the BaseProvider protocol.

### Parallel Dispatch

Uses `asyncio.gather()` with per-model timeout. Each model call is independent. Failures are caught per-model and do not abort other model calls.

### Aggregation

After collecting responses, the aggregator:
1. Selects the mode-specific prompt template from `skills/consensus/aggregation-{mode}-prompt.md`
2. Injects all successful model responses (identified by provider name)
3. Dispatches to the first configured Anthropic model (primary aggregator)
4. Parses the structured JSON output from the aggregator response
5. Returns the parsed output as the tool response

### Testing

The MCP server is testable with mock model responses. No live API keys needed for the test suite.

Key test scenarios:
- All models succeed → consensus response
- N-1 models fail, 1 succeeds, min_models=2 → unavailable
- 2 of 4 models succeed, min_models=2 → partial consensus
- All models fail → unavailable
- Timeout handling (one model slow, others fast)
- Malformed response from one model (does not crash aggregation)
- `consensus.enabled: false` → immediate "unavailable" return