skill-auditor

# Skill Auditor

Audits every `SKILL.md` in `skills/` for frontmatter correctness, structural completeness, API/version drift, and cross-file reference integrity. Produces a prioritized markdown report. Read-only — never applies fixes.

## When This Skill Activates

Use this skill when the user:
- Says "audit skills", "audit my skills", "run the skill auditor"
- Asks to "check the skill repo for drift", "review skills for consistency"
- Says "find broken skills", "validate skill frontmatter"
- Is planning a bulk cleanup of the skill repo
- References a specific category to audit (e.g., "audit the generators/ skills")

**Does NOT activate** for: creating new skills (use `skill-creator`), applying fixes (follow-up flow after this auditor reports), or auditing Swift code inside skills (use `ios/coding-best-practices` or `macos/coding-best-practices`).

## Scope

Resolve the invocation argument in this order:

1. **No argument** → audit all of `skills/**/SKILL.md`
2. **Category path** (e.g., `generators/`, `ios/`) → audit `skills/<arg>/**/SKILL.md`
3. **Single file path** (e.g., `skills/liquid-glass/SKILL.md`) → audit one file
4. **Fuzzy match** → if arg doesn't resolve, try `skills/<arg>/SKILL.md`; fall back to asking the user

## Process

### 1. Enumerate

Use `Glob` with pattern `skills/**/SKILL.md` from the repo root. Filter by scope if an argument was passed. Record the canonical file list — every subsequent step operates on this list.

### 2. Parse Frontmatter (Cached)

Read the first 15 lines of each `SKILL.md`. Parse:
- Whether `---` frontmatter block exists
- `name:`, `description:`, `allowed-tools:` field values

Cache this result. Checks C-01, H-01, H-03, L-02 all read from this cache — do not re-read.

### 3. Run Checks

Execute bulk `Grep` passes in parallel (single tool-call batch) wherever possible. Per-file operations come after. The 11 checks are in the table below; the order is as-listed.

### 4. Classify Aggregator vs Leaf

For each file, mark it as **aggregator** or **leaf** using the rules in the "Aggregator Detection" section below. Some checks relax for aggregators.

### 5. Rank and Emit

Group findings by severity (🔴 → 🟢), sort within each group by file path, print the report inline using the template in "Output Format".

## Checks

### 🔴 Critical

**C-01 · Missing frontmatter.** The file has no leading `---` YAML block.
- **Detection**: multiline `Grep` for `\A---\n[\s\S]*?\n---` across all SKILL.md. Files with no match → C-01.
- **Fix**: Add YAML frontmatter with `name`, `description`, `allowed-tools`.

### 🟠 High

**H-01 · Missing `allowed-tools` field.** Frontmatter exists but `allowed-tools:` key is absent.
- **Detection**: from cached frontmatter parse.
- **Fix**: Add `allowed-tools: [Read, Glob, Grep]` (adjust based on what the skill actually does).

**H-02 · Broken supporting-file reference.** The SKILL.md references a `*.md` file that does not exist on disk in the same directory.
- **Detection**: `Grep` each SKILL.md for `[a-z0-9][a-z0-9-]*\.md` matches; resolve each relative to the SKILL.md's directory; `ls` to confirm. Missing files → H-02. Ignore matches inside fenced code blocks.
- **Fix**: Create the file or remove the reference.

**H-03 · H1 title does not match `name:`.** The first `#` heading after the frontmatter, slugified (lowercase, spaces → `-`), differs from the `name:` field.
- **Detection**: from cached frontmatter parse + per-file line-after-frontmatter.
- **Fix**: Rename the H1 or the `name:` to match.

### 🟡 Medium

**M-01 · Missing "When This Skill Activates" section.** No `## When This Skill Activates` heading anywhere in the file.
- **Detection**: `Grep -L` for `^## When This Skill Activates` across all SKILL.md.
- **Fix**: Add section with 3–5 user trigger phrases. See `shared/skill-creator/SKILL.md` for the canonical format.

**M-02 · Outdated version reference (drift).** Mentions iOS 17–25, macOS 13–25, or Swift 5.x with **drift context** (treated as current/latest/target).
- **Detection (two-stage)**:
  1. **Stage 1** — `Grep` for `\biOS (1[7-9]|2[0-5])\b|\bSwift 5\.\d+\b|\bmacOS (1[3-9]|2[0-5])\b`, capturing line numbers.
  2. **Stage 2** — for each hit, examine ±2 surrounding lines. Classify:
     - **Flag M-02** if surrounding lines contain: `latest`, `newest`, `current`, `target`, `deployment target`, `requires`, `minimum`, `as of`, `new in`, `now supports`, `today`
     - **Suppress** if surrounding lines contain: `legacy`, `pre-`, `prior to`, `before`, `deprecated`, `old`, `migrate from`, `backport`, `fallback`, `if available`, `#available`, or the version mention has a trailing `+` (e.g., `iOS 17+`)
     - **Neither** → L-03 (ambiguous)
- **Known-current constants** (dated 2026-04-20, update per WWDC): iOS 26, macOS 26, Swift 6.x. Mentions of these with drift context are always clean.
- **Fix**: Update the reference to iOS 26 / macOS 26 / Swift 6.x, or annotate as legacy context with one of the suppression keywords.

**M-03 · Pre-`@Observable` pattern without deprecation callout.** Uses `@StateObject` or `ObservableObject` without acknowledging that `@Observable` is the current pattern.
- **Detection**: `Grep` for `@StateObject|ObservableObject` with line numbers; for each hit, secondary `Grep` of the same file for `@Observable|deprecated|legacy|pre-@Observable|migration|old pattern` within ±10 lines. No secondary match → M-03.
- **Fix**: Either replace with `@Observable` + `@State`, or add a migration note explaining why the older pattern is shown.

**M-04 · Oversized single-file skill.** `SKILL.md` exceeds 400 lines and its directory contains no sibling `.md` files.
- **Detection**: `wc -l` via Bash on each SKILL.md; if `>400`, check sibling file list via `ls` for any other `.md`. None → M-04.
- **Fix**: Modularize — extract sections into `patterns.md`, `templates.md`, `checklist.md`, or `examples.md` per `skill-creator` conventions.

### 🟢 Low

**L-01 · No ✅/❌ examples in prose.** The file has no `✅` or `❌` markers anywhere.
- **Detection**: `Grep -L` for `✅|❌`.
- **Fix**: Add at least one good/bad example pair.

**L-02 · Description length out of range.** `description:` is <20 or >300 characters.
- **Detection**: from cached frontmatter parse.
- **Fix**: Expand or shorten the description. Include a "use when…" clause to anchor activation.

**L-03 · Ambiguous version mention.** A version keyword matched stage 1 of the drift check but surrounding lines contained neither drift nor legacy context. User reviews manually.
- **Detection**: fallthrough from M-02 stage-2 classification.
- **Fix**: Add a drift or legacy keyword to disambiguate, or leave as-is if the context is clearly a one-off mention.

## Aggregator vs Leaf Detection

A SKILL.md is an **aggregator** if any of the following hold:
- It sits at depth 2 under the repo root — i.e., `skills/<category>/SKILL.md`
- It contains the heading `## Available Modules` or `## Available Skills`
- It links to `./<subdir>/SKILL.md` or contains two or more references of the form `skills/<category>/<subskill>/`

Otherwise it is a **leaf**.

### Relaxations for aggregators

| Check | Behaviour |
|---|---|
| M-01 (activation section) | Still enforced — aggregators must describe activation |
| M-03 (pre-@Observable) | Suppressed — aggregators are prose, not code |
| M-04 (>400 lines) | Suppressed — aggregators are allowed to be long when enumerating modules |
| L-01 (no ✅/❌ examples) | Suppressed — aggregators don't carry patterns |

Tag every finding in the report with `(aggregator)` or `(leaf)` so severity can be read at a glance.

## Output Format

Print the report inline to the conversation using this template. Use exact headings — downstream tooling may grep them.

```markdown
# Skill Audit Report — <YYYY-MM-DD> — <N> files scanned

## Summary
- 🔴 Critical: <count>
- 🟠 High: <count>
- 🟡 Medium: <count>
- 🟢 Low: <count>
- ✅ Files clean: <clean-count> / <N>

Scope: <all | category | single file>

## 🔴 Critical Findings
### C-01 · Missing frontmatter
- `skills/<path>/SKILL.md` (leaf) — **Fix:** Add YAML frontmatter with `name`, `description`, `allowed-tools`.

## 🟠 High Findings
### H-01 · Missing `allowed-tools` field
- `skills/liquid-glass/SKILL.md:1-4` (leaf) — **Fix:** Add `allowed-tools: [Read, Glob, Grep]`.
- `skills/macos/macos-tahoe-apis/SKILL.md:1-4` (leaf) — **Fix:** Same.

### H-02 · Broken supporting-file reference
- `skills/<path>/SKILL.md:<line>` — references `patterns.md`, not found. **Fix:** Create the file or remove the reference.

### H-03 · H1 title does not match `name:`
- `skills/<path>/SKILL.md` — `name: foo-bar`, H1 is `# Foo Bars`. **Fix:** Rename one to match.

## 🟡 Medium Findings
### M-01 · Missing "When This Skill Activates" section (<count>)
- `skills/design/liquid-glass/SKILL.md` (leaf)
- `skills/macos/coding-best-practices/SKILL.md` (leaf)
- [collapsed list of remaining offenders]
- **Fix:** Add section with 3–5 user trigger phrases.

### M-02 · Outdated version reference (<count>)
- `skills/<path>/SKILL.md:87` (leaf) — "latest iOS 17" in drift context. **Fix:** Update to iOS 26.

### M-03 · Pre-`@Observable` pattern without callout (<count>)
- `skills/<path>/SKILL.md:142` (leaf) — `@StateObject` without migration note. **Fix:** Replace with `@Observable` or add callout.

### M-04 · Oversized single-file skill (<count>)
- `skills/<path>/SKILL.md` — <NNN> lines, no sibling .md files. **Fix:** Modularize into `patterns.md` / `templates.md`.

## 🟢 Low Findings
### L-01 · No ✅/❌ examples
- `skills/<path>/SKILL.md` (leaf) — **Fix:** Add at least one good/bad example pair.

### L-02 · Description length out of range
- `skills/<path>/SKILL.md` — description is <N> chars. **Fix:** Expand/shorten to 20–300 chars.

### L-03 · Ambiguous version mention
- `skills/<path>/SKILL.md:<line>` — "iOS 18" with no drift/legacy context. **Fix:** Review manually.

## ✅ Clean Files
<collapsed list of files that passed all checks>

## Next Steps
- Review 🔴/🟠 findings first — they block skills from working as intended.
- Batch-fix 🟡 M-01 and M-02 mechanically — suggested find/replace scripts shown per-file.
- 🟢 findings are polish; address opportunistically.
```

## Implementation Notes

- **Parallelize bulk Greps** in a single tool-call batch where possible. Activation, frontmatter, `@StateObject`, and drift-stage-1 can all run at once.
- **Cache the frontmatter parse.** Four checks read it; only parse once.
- **Resolve `*.md` references inside fenced code blocks carefully** — they're examples, not references. Strip fenced blocks before running H-02's regex.
- **Progress pings** — for full-repo scans (148 files), emit a short "scanning <N>/<total>…" before each phase so the user sees liveness.
- **Idempotence** — the report body (everything below the `## Summary`) should be deterministic for a given repo state. Only the date in the H1 varies across runs.
- **Never write files.** This skill's `allowed-tools` lists Bash, but only for read-only ops (`wc`, `ls`). If you find yourself needing to write, stop and ask the user to invoke a fix flow separately.

## Non-Goals

- **No auto-fix.** The report includes one-line fix suggestions per finding; applying them is a separate user-initiated task.
- **No network calls.** This auditor does not verify URL liveness or check Apple doc availability.
- **No Swift compilation.** Code-block validity is beyond scope; patterns are matched textually.
- **No WWDC session cross-referencing.** A future `wwdc-to-skill-workflow` skill owns that.

## Maintenance

The drift heuristic hardcodes "known-current" version constants:

- iOS 26, macOS 26, Swift 6.x (dated 2026-04-20)

**Update these constants** when Apple ships a new major platform version (usually post-WWDC each June). The stage-1 regex ranges (`iOS 1[7-9]|2[0-5]`, `macOS 1[3-9]|2[0-5]`) must also be widened to include the newly-retired version.

## Verification

After running the auditor on the full repo, counts should fall within this tolerance band (baseline taken 2026-04-20, 149 total `SKILL.md` files):

| Finding | Expected |
|---|---|
| C-01 missing frontmatter | **9** — legacy batch in `app-store/` (keyword-optimizer, app-description-writer, screenshot-planner, review-response-writer) and `generators/` (accessibility-generator, ci-cd-setup, deep-linking, localization-setup, push-notifications) |
| H-01 missing `allowed-tools` | **9** — same set as C-01 (they lack frontmatter entirely) |
| H-02 broken supporting refs | 0 |
| H-03 H1 mismatch | flag outliers manually |
| M-01 missing activation section | **28** — includes the 9 C-01 offenders (which use `## When to Use` instead) plus 19 newer skills missing the section entirely |
| M-02 version drift | ~50 (±10) — stage-1 Grep returns ~228 raw hits across ~69 files; stage-2 keyword filtering narrows to this band |
| M-03 pre-`@Observable` no callout | ~29 (±5) |
| M-04 oversized single-file | TBD — flag outliers |

### Known legacy pattern: `## When to Use`

The 9 C-01 offenders all use `## When to Use` (non-canonical) rather than `## When This Skill Activates`. The auditor flags this as M-01 correctly; the suggested fix is to rename the heading when adding frontmatter. Do **not** widen the M-01 regex to accept `## When to Use` — the whole point is to normalize onto the canonical heading.

### Smoke tests

1. **Full-repo run** — invoke with no arg. Counts must fall within tolerance above.
2. **Scoped run** — invoke with `generators/`. Only `skills/generators/**` paths appear.
3. **Single-file run** — invoke with `skills/liquid-glass/SKILL.md`. Report has exactly one H-01 finding.
4. **Idempotence** — re-run immediately. Report body is byte-identical; only the date in the H1 may differ.
5. **Drift heuristic sanity** — spot-check 5 M-02 hits and 5 suppressed legacy mentions. If any legacy mention is mis-flagged, extend the suppression keyword list and re-run.

## References

- `/Users/ravishankar/Work/MyApps/claude-code-apple-skills/CLAUDE.md` — normative source for frontmatter schema, naming, emoji convention
- `skills/shared/skill-creator/SKILL.md` — companion meta-skill for creating new skills (this auditor only audits; it does not create)
- `skills/ios/SKILL.md` — canonical aggregator example