customize-measurement

# Customize Measurement

Generate a customized `.ai-proficiency.yaml` configuration through a guided interview process.

## When to Use

- First time setting up measure-ai-proficiency for a repository
- Current scoring doesn't match your team's structure
- Want to hide irrelevant recommendations (e.g., no MCP, no Gas Town)
- Your team uses different file names (e.g., `SYSTEM_DESIGN.md` instead of `ARCHITECTURE.md`)

## Workflow

### Phase 1: Interview

Ask questions in **thematic batches of 2-3 questions** using `AskUserQuestion`.

#### Batch 1: AI Tools
```
1. Which AI coding assistants does your team use?
   - Claude Code
   - GitHub Copilot
   - Cursor
   - OpenAI Codex
   - Multiple (specify)

2. Is one tool primary, or do you use them equally?
```

#### Batch 2: Documentation Conventions
```
1. Do you use different file names for documentation?
   Examples:
   - SYSTEM_DESIGN.md instead of ARCHITECTURE.md
   - CODING_STANDARDS.md instead of CONVENTIONS.md
   - docs/api/README.md instead of API.md

2. Where does your team store documentation?
   - Root level (ARCHITECTURE.md)
   - docs/ folder
   - documentation/ folder
   - Other location
```

#### Batch 3: Focus & Scope
```
1. Which capabilities are NOT relevant to your team? (Select all that apply)
   - hooks (Claude hooks)
   - commands (Claude slash commands)
   - skills (Agent skills)
   - memory (Memory files like LEARNINGS.md)
   - agents (Multi-agent setup)
   - mcp (MCP server configs)
   - beads (Beads memory system)
   - gastown (Gas Town orchestration)

2. What's your priority focus?
   - documentation (ARCHITECTURE.md, CONVENTIONS.md)
   - skills (Agent skills)
   - testing (Test documentation)
   - architecture (System design docs)
   - All equally
```

#### Batch 4: Thresholds & Industry
```
1. Is the default scoring appropriate for your repo?
   - Too strict (small team/startup - lower thresholds)
   - About right (default thresholds)
   - Too lenient (enterprise - higher thresholds)

2. Any industry-specific patterns to include?
   - FinTech (COMPLIANCE.md, PCI_DSS.md, SECURITY_STANDARDS.md)
   - Healthcare (HIPAA.md, PHI_HANDLING.md)
   - Open Source (GOVERNANCE.md, MAINTAINERS.md)
   - Enterprise (SOC2.md, SECURITY_AUDIT.md)
   - None / General
```

#### Batch 5: Validation (Optional)
```
1. Does your documentation mention example file names that don't exist?
   - Yes (meta-tools, template repos, documentation showing patterns)
   - No (all file references point to real files)

2. If yes, which patterns should be skipped during validation?
   - Best practices not yet adopted (.mcp.json, .claude/settings.json)
   - Industry examples (HIPAA.md, COMPLIANCE.md)
   - Custom examples (ask user to specify)
```

### Phase 2: Generate Configuration

Based on interview responses, generate `.ai-proficiency.yaml`:

```yaml
# .ai-proficiency.yaml
# Generated by customize-measurement skill
# Customized for: [repo name]

# AI tools in use
tools:
  - claude-code  # or detected tools

# Custom file locations (if different from defaults)
documentation:
  # architecture: "docs/SYSTEM_DESIGN.md"
  # conventions: "CODING_STANDARDS.md"
  # api: "docs/api/README.md"
  # testing: "docs/TESTING_GUIDE.md"

# Level thresholds (adjusted for team size)
thresholds:
  level_3: 15   # Comprehensive Context
  level_4: 12   # Skills & Automation
  level_5: 10   # Multi-Agent Ready
  level_6: 8    # Fleet Infrastructure
  level_7: 6    # Agent Fleet
  level_8: 5    # Custom Orchestration

# Skip irrelevant recommendations
skip_recommendations:
  # - hooks
  # - commands
  # - skills
  # - memory
  # - agents
  # - mcp
  # - beads
  # - gastown

# Focus areas (optional - show only these)
# focus_areas:
#   - documentation
#   - skills
#   - testing

# Skip validation warnings for documentation examples
# skip_validation_patterns:
#   - "COMPLIANCE.md"      # Example mentioned in docs
#   - ".mcp.json"          # Best practice not yet adopted
```

### Phase 3: Validate

After generating the config:

```bash
measure-ai-proficiency
```

Verify the output matches expectations. Adjust config if needed.

## Threshold Guidelines

| Team Size | Suggested Thresholds |
|-----------|---------------------|
| Startup (1-5 devs) | level_3: 8, level_4: 6 |
| Small (5-20 devs) | level_3: 12, level_4: 10 |
| Medium (20-100 devs) | level_3: 15, level_4: 12 (default) |
| Enterprise (100+ devs) | level_3: 20, level_4: 15 |

## Industry Patterns

### FinTech
```yaml
documentation:
  compliance: "COMPLIANCE.md"
  security: "SECURITY_STANDARDS.md"
  pci: "PCI_DSS.md"
```

### Healthcare
```yaml
documentation:
  hipaa: "HIPAA.md"
  phi: "PHI_HANDLING.md"
  security: "SECURITY_COMPLIANCE.md"
```

### Open Source
```yaml
documentation:
  governance: "GOVERNANCE.md"
  maintainers: "MAINTAINERS.md"
  contributing: "CONTRIBUTING.md"
```

## Example Session

```
User: /customize-measurement

Claude: I'll help you create a customized .ai-proficiency.yaml for your repository.
        Let me ask a few questions to understand your setup.

[AskUserQuestion - AI Tools batch]
1. Which AI coding assistants does your team use?
2. Is one the primary tool?

User: We use Claude Code primarily, with some GitHub Copilot usage.

[AskUserQuestion - Documentation batch]
1. Do you use different file names for documentation?
2. Where do you store docs?

User: We use docs/DESIGN.md instead of ARCHITECTURE.md. Everything is in docs/.

[AskUserQuestion - Focus batch]
1. Which capabilities aren't relevant?
2. What's your priority focus?

User: We don't use MCP or Gas Town. Focus on documentation and testing.

[AskUserQuestion - Thresholds batch]
1. Is default scoring appropriate?
2. Any industry patterns?

User: We're a small startup, defaults are too strict. No special industry.

Claude: Based on your responses, here's your customized config:

[Generates .ai-proficiency.yaml with:
 - tools: claude-code, github-copilot
 - documentation.architecture: docs/DESIGN.md
 - thresholds lowered for startup
 - skip_recommendations: mcp, gastown
 - focus_areas: documentation, testing]

Run `measure-ai-proficiency` to validate.
```

## Quick Reference

Triggers:
- "Customize AI proficiency measurement"
- "Set up .ai-proficiency.yaml"
- "Configure measurement for my repo"
- "The scoring doesn't match my setup"