skill-comply
Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines
About this skill
The `skill-comply` agent skill provides a robust framework for validating the behavioral compliance of AI agents. It functions by first auto-generating expected behavioral sequences, or 'specs,' from any given Markdown file that defines rules or desired agent actions. Subsequently, it creates test scenarios across three prompt strictness levels: supportive, neutral, and competing, to thoroughly challenge the agent's adherence. The skill then executes the target agent using `claude -p`, meticulously capturing all tool call traces. These traces are then classified against the predefined specs using an advanced LLM-based approach (not regex) to identify compliance. Finally, it checks temporal ordering deterministically and generates comprehensive reports detailing compliance rates, full tool call timelines, and specific instances of non-adherence. This ensures agents reliably follow their intended programming and operational guidelines.
Best use case
This skill is essential for developers, QA engineers, and system architects who need to ensure AI agents consistently and reliably follow their defined behaviors, API specifications, or operational rules. It helps in verifying agent integrity and preventing unexpected or non-compliant actions.
Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines
Users will receive detailed compliance reports that quantify an agent's adherence to its definitions across various prompt conditions. The output includes specific compliance rates (e.g., 90% compliant), visual timelines of tool calls, classified behavioral sequences, and clear identification of any non-compliant actions, providing actionable insights to improve agent reliability.
Practical example
Example input
An agent definition or rule set specified in a Markdown file (e.g., `api_handling_rules.md`) that outlines expected tool calls and their sequences, along with a reference to the AI agent to be tested (e.g., `my_new_api_agent`). The skill would be invoked with parameters pointing to these resources.
Example output
```json
{
"compliance_report": {
"agent_name": "my_new_api_agent",
"spec_file": "api_handling_rules.md",
"overall_compliance_rate": "88%",
"scenario_results": {
"supportive_strictness": {
"compliance_rate": "95%",
"failed_scenarios": 2,
"details": [
{ "scenario": "Scenario 1", "status": "PASS", "timeline": "..." }
]
},
"neutral_strictness": {
"compliance_rate": "80%",
"failed_scenarios": 8,
"details": [
{ "scenario": "Scenario N", "status": "FAIL", "deviation": "Expected tool 'Foo', called 'Bar'", "timeline": "..." }
]
},
"competing_strictness": {
"compliance_rate": "65%",
"failed_scenarios": 15,
"details": [
{ "scenario": "Scenario C", "status": "FAIL", "deviation": "Missed expected 'Cleanup' step", "timeline": "..." }
]
}
},
"non_compliance_summary": [
"Detected unhandled error condition in 3 scenarios under neutral strictness.",
"Agent failed to use 'logging_tool' in 5 competing scenarios."
]
}
}
```
(Output also includes full tool call timelines and classified sequences, potentially in a more verbose format or separate files.)When to use this skill
- When developing new AI agents or skills, to validate their initial behavior against specifications.
- When updating existing AI agents, to ensure new changes haven't introduced regressions or compliance issues.
- Before deploying critical AI agents to production, to audit their adherence to performance, security, or ethical guidelines.
- For continuous integration/continuous deployment (CI/CD) pipelines to automate agent behavior testing.
When not to use this skill
- For very simple, non-critical agent tasks where formal compliance verification adds unnecessary overhead.
- When the 'rules' or 'definitions' are extremely vague or abstract, making automated classification impractical or unreliable.
- During very early-stage rapid prototyping where behavior is still highly fluid and formal testing is not yet beneficial.
- If the performance overhead of running multiple test scenarios and LLM-based classification outweighs the need for strict compliance validation.
Installation
Claude Code / Cursor / Codex
Manual Installation
- Download SKILL.md from GitHub
- Place it in
.claude/skills/skill-comply/SKILL.mdinside your project - Restart your AI agent — it will auto-discover the skill
How skill-comply Compares
| Feature / Agent | skill-comply | Standard Approach |
|---|---|---|
| Platform Support | Claude | Limited / Varies |
| Context Awareness | High | Baseline |
| Installation Complexity | easy | N/A |
Frequently Asked Questions
What does this skill do?
Visualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines
Which AI agents support this skill?
This skill is designed for Claude.
How difficult is it to install?
The installation complexity is rated as easy. You can find the installation instructions above.
Where can I find the source code?
You can find the source code on GitHub using the link provided at the top of the page.
Related Guides
AI Agents for Coding
Browse AI agent skills for coding, debugging, testing, refactoring, code review, and developer workflows across Claude, Cursor, and Codex.
Best AI Skills for Claude
Explore the best AI skills for Claude and Claude Code across coding, research, workflow automation, documentation, and agent operations.
ChatGPT vs Claude for Agent Skills
Compare ChatGPT and Claude for AI agent skills across coding, writing, research, and reusable workflow execution.
SKILL.md Source
# skill-comply: Automated Compliance Measurement Measures whether coding agents actually follow skills, rules, or agent definitions by: 1. Auto-generating expected behavioral sequences (specs) from any .md file 2. Auto-generating scenarios with decreasing prompt strictness (supportive → neutral → competing) 3. Running `claude -p` and capturing tool call traces via stream-json 4. Classifying tool calls against spec steps using LLM (not regex) 5. Checking temporal ordering deterministically 6. Generating self-contained reports with spec, prompts, and timelines ## Supported Targets - **Skills** (`skills/*/SKILL.md`): Workflow skills like search-first, TDD guides - **Rules** (`rules/common/*.md`): Mandatory rules like testing.md, security.md, git-workflow.md - **Agent definitions** (`agents/*.md`): Whether an agent gets invoked when expected (internal workflow verification not yet supported) ## When to Activate - User runs `/skill-comply <path>` - User asks "is this rule actually being followed?" - After adding new rules/skills, to verify agent compliance - Periodically as part of quality maintenance ## Usage ```bash # Full run uv run python -m scripts.run ~/.claude/rules/common/testing.md # Dry run (no cost, spec + scenarios only) uv run python -m scripts.run --dry-run ~/.claude/skills/search-first/SKILL.md # Custom models uv run python -m scripts.run --gen-model haiku --model sonnet <path> ``` ## Key Concept: Prompt Independence Measures whether a skill/rule is followed even when the prompt doesn't explicitly support it. ## Report Contents Reports are self-contained and include: 1. Expected behavioral sequence (auto-generated spec) 2. Scenario prompts (what was asked at each strictness level) 3. Compliance scores per scenario 4. Tool call timelines with LLM classification labels ### Advanced (optional) For users familiar with hooks, reports also include hook promotion recommendations for steps with low compliance. This is informational — the main value is the compliance visibility itself.
Related Skills
git-workflow
Git workflow patterns including branching strategies, commit conventions, merge vs rebase, conflict resolution, and collaborative development best practices for teams of all sizes.
kotlin-patterns
Coroutine'ler, null safety ve DSL builder'lar ile sağlam, verimli ve sürdürülebilir Kotlin uygulamaları oluşturmak için idiomatic Kotlin kalıpları, en iyi uygulamalar ve konvansiyonlar.
docker-patterns
Docker and Docker Compose patterns for local development, container security, networking, volume strategies, and multi-service orchestration. Use when setting up containerized development environments or reviewing Docker configurations.
verification-loop
A comprehensive verification system for Claude Code sessions.
workspace-surface-audit
Audit the active repo, MCP servers, plugins, connectors, env surfaces, and harness setup, then recommend the highest-value ECC-native skills, hooks, agents, and operator workflows. Use when the user wants help setting up Claude Code or understanding what capabilities are actually available in their environment.
ui-demo
Record polished UI demo videos using Playwright. Use when the user asks to create a demo, walkthrough, screen recording, or tutorial video of a web application. Produces WebM videos with visible cursor, natural pacing, and professional feel.
token-budget-advisor
Offers the user an informed choice about how much response depth to consume before answering. Use this skill when the user explicitly wants to control response length, depth, or token budget. TRIGGER when: "token budget", "token count", "token usage", "token limit", "response length", "answer depth", "short version", "brief answer", "detailed answer", "exhaustive answer", "respuesta corta vs larga", "cuántos tokens", "ahorrar tokens", "responde al 50%", "dame la versión corta", "quiero controlar cuánto usas", or clear variants where the user is explicitly asking to control answer size or depth. DO NOT TRIGGER when: user has already specified a level in the current session (maintain it), the request is clearly a one-word answer, or "token" refers to auth/session/payment tokens rather than response size.
santa-method
Multi-agent adversarial verification with convergence loop. Two independent review agents must both pass before output ships.
safety-guard
Use this skill to prevent destructive operations when working on production systems or running agents autonomously.
repo-scan
Cross-stack source code asset audit — classifies every file, detects embedded third-party libraries, and delivers actionable four-level verdicts per module with interactive HTML reports.
project-flow-ops
Operate execution flow across GitHub and Linear by triaging issues and pull requests, linking active work, and keeping GitHub public-facing while Linear remains the internal execution layer. Use when the user wants backlog control, PR triage, or GitHub-to-Linear coordination.
product-lens
Use this skill to validate the "why" before building, run product diagnostics, and pressure-test product direction before the request becomes an implementation contract.