periodic-skill-ecosystem-housekeeping-audit

# Periodic Skill Ecosystem Housekeeping Audit

## When to Use

Use this when creating, maintaining, or reviewing a recurring audit for the skill ecosystem, especially when the task is to identify improvement areas in skill content, grouping/category structure, duplicate names, alias drift, oversized skills, missing required sections, or stale/low-quality skill metadata.

This skill complements `coordination/cross-agent-skill-audit`: that skill checks whether skills are visible across Hermes/Codex/Codex/Gemini; this one checks whether the skills themselves are healthy and whether the audit can run safely as housekeeping.

## Core Pattern

1. **Keep the scheduled path singular**
   - Prefer extending the existing scheduled housekeeping/curation job over adding a parallel cron path.
   - In workspace-hub, the recurring path is the skills curation wrapper plus its schedule documentation.
   - The audit should produce deterministic local artifacts; avoid auto-posting to GitHub unless the plan explicitly asks for writeful behavior.

2. **Make the schema append-only and stable**
   - Keep existing finding keys stable across versions.
   - Add new optional sections rather than renaming/removing old ones.
   - Include a policy/version contract in the JSON output.
   - Include source metadata such as `source_id: local-skill-filesystem` so future readers can distinguish local inventory from usage telemetry or remote APIs.

3. **Separate finding families**
   - Content quality: missing required sections, weak metadata, empty/placeholder sections, low-actionability descriptions.
   - Grouping/taxonomy: duplicate leaf names, alias drift, category naming inconsistencies, category collisions.
   - Size/maintainability: oversized `SKILL.md`, too many linked files, excessive category depth, too-large markdown sections.
   - Follow-up candidates: issue-worthy actionable items, not every low-confidence unresolved finding.

4. **Baseline and waiver all new finding families**
   - Route new v2 findings through the same baseline/waiver path as older findings.
   - Do not keep a parallel unwaived/unbaselined list for new families.
   - Report suppressed findings separately.
   - When upgrading v1 -> v2, allow compatible v1 baseline artifacts if the policy declares append-only compatibility. Otherwise the first v2 run can falsely mark every legacy finding as new.

5. **Make Markdown actionable**
   - Show summary counts for all finding families.
   - In detailed sections, prioritize new/high-confidence/unresolved findings and follow-up candidates.
   - Avoid flooding the Markdown report with low-confidence carry-forward noise.
   - If a local GitHub payload is generated, make it a payload file only; do not post automatically in the audit script.

6. **Use TDD and adversarial review for audit changes**
   - Start with RED tests for the schema and the new finding families.
   - Add regression tests for each adversarial review blocker before fixing it.
   - Run existing v1 tests and cron wrapper tests so a v2 audit does not regress older contracts.

## Regression Tests to Add

For a v2 housekeeping audit, include tests for:

- Stable optional JSON sections even when there are no findings.
- Policy-driven grouping/content/size signals.
- Local-only GitHub payload generation.
- No writeful `.Codex/state/skill-usage-report` or equivalent usage telemetry side effects.
- V2 findings are baseline-aware and can be carried forward.
- V2 findings can be waived and appear in `suppressed_findings`.
- Markdown report includes v2 summary counts and actionable v2 sections.
- Inventory records include a stable `source_id`.
- Alias-drift detector avoids false positives when only one canonical spelling exists.
- v1 -> v2 baseline continuity works when append-only compatibility is declared.
- Slash-based alias families such as `business/admin` are not compared against unrelated top-level categories such as `business`.

## Validation Checklist

Before commit/closeout:

```bash
uv run pytest tests/cron/test_skills_curation.py tests/skills/test_weekly_skills_audit.py tests/skills/test_weekly_skills_audit_v2.py -q
uv run --no-project python -m py_compile scripts/skills/weekly_skills_audit.py
uv run --no-project python scripts/cron/validate-schedule.py
bash scripts/cron/skills-curation.sh --dry-run
git diff --check -- config/skills/weekly-audit-policy.yaml docs/ops/scheduled-tasks.md scripts/skills/weekly_skills_audit.py tests/skills/test_weekly_skills_audit_v2.py
```

Also run a manual redirected artifact proof in a temp output directory and verify:

- JSON artifact exists.
- Markdown artifact exists.
- local `github-update-payload.md` exists if requested.
- audit exits with `0 errors`.
- no usage-report/state file is newly dirtied.

## Adversarial Review Prompts

Ask reviewers to specifically look for:

- New findings bypassing baseline or waiver logic.
- Markdown output omitting new finding families.
- Version upgrades breaking carry-forward semantics.
- Alias/grouping detectors mixing slash-family aliases with unrelated top-level categories.
- The audit script accidentally becoming writeful against GitHub or `.Codex/state`.

## Commit Hygiene Pitfall

Raw review artifacts often contain trailing whitespace or extra blank lines at EOF. If committing archived adversarial review artifacts under `scripts/review/results/`, normalize them before commit:

```python
from pathlib import Path
for p in Path('scripts/review/results').glob('*code-ISSUE-*.md'):
    lines = p.read_text(encoding='utf-8', errors='replace').splitlines()
    p.write_text('\n'.join(line.rstrip() for line in lines).rstrip() + '\n', encoding='utf-8')
```

Then run:

```bash
git diff --cached --check
```

Keep unrelated provider/state/report dirt unstaged when closing the issue.