skill-selection-evals
Eval-only skill for measuring skill routing accuracy. Not invoked directly — contains selection evals that test whether the agent picks the correct skill for a given prompt.
Best use case
skill-selection-evals is best used when you need a repeatable AI agent workflow instead of a one-off prompt.
Eval-only skill for measuring skill routing accuracy. Not invoked directly — contains selection evals that test whether the agent picks the correct skill for a given prompt.
Teams using skill-selection-evals should expect a more consistent output, faster repeated execution, less prompt rewriting.
When to use this skill
- You want a reusable workflow that can be run more than once with consistent structure.
When not to use this skill
- You only need a quick one-off answer and do not need a reusable workflow.
- You cannot install or maintain the underlying files, dependencies, or repository context.
Installation
Claude Code / Cursor / Codex
Manual Installation
- Download SKILL.md from GitHub
- Place it in
.claude/skills/skill-selection-evals/SKILL.mdinside your project - Restart your AI agent — it will auto-discover the skill
How skill-selection-evals Compares
| Feature / Agent | skill-selection-evals | Standard Approach |
|---|---|---|
| Platform Support | Not specified | Limited / Varies |
| Context Awareness | High | Baseline |
| Installation Complexity | Unknown | N/A |
Frequently Asked Questions
What does this skill do?
Eval-only skill for measuring skill routing accuracy. Not invoked directly — contains selection evals that test whether the agent picks the correct skill for a given prompt.
Where can I find the source code?
You can find the source code on GitHub using the link provided at the top of the page.
SKILL.md Source
# Skill-Selection Evals This is not an executable skill. It contains evaluation data for measuring the accuracy of skill selection (routing) decisions. ## Purpose Crucible's 49 execution evals measure quality once a skill is invoked. Selection evals measure whether the **right skill gets invoked** in the first place. ## Eval Types - **Direct selection**: Given a prompt, does the agent pick the correct skill? - **Negative selection**: Given a prompt that sounds like skill X but is not, does the agent avoid the false positive? - **Context-dependent**: Same verb, different context, different correct skill. - **Cascade ordering**: Multi-skill tasks requiring correct invocation order. ## Boundaries Tested 1. **test-methodology** — TDD vs test-coverage vs adversarial-tester 2. **review-direction** — temper vs review-feedback 3. **adversarial-scope** — red-team vs inquisitor vs audit vs siege 4. **completion-claims** — verify vs finish 5. **bug-handling** — debugging vs verify vs audit ## Difficulty Ratings Each eval is rated easy/medium/hard based on routing ambiguity. This enables stratified baseline measurement — distinguishing between improvements that lift hard cases (high value) vs confirming easy cases already work (low signal). ## See Also - `evals/evals.json` — the eval data - `GRADING.md` — grading criteria and baseline measurement protocol