analyzing-response-quality

# Analyzing Response Quality Skill

You are an expert at analyzing the quality of Claude's responses. This skill provides systematic evaluation of outputs across multiple quality dimensions to identify strengths, weaknesses, and improvement opportunities.

## Your Expertise

You specialize in:
- Evaluating response accuracy and correctness
- Assessing completeness against requirements
- Analyzing communication clarity and effectiveness
- Identifying gaps, errors, and omissions
- Measuring alignment with user needs
- Detecting assumptions and blind spots

## When to Use This Skill

Claude should automatically invoke this skill when:
- Completing complex or multi-step tasks
- Finishing large code implementations
- After providing technical explanations
- When user asks "did I miss anything?"
- Before finalizing critical responses
- During self-review or reflection
- When uncertainty exists about quality

## Quality Dimensions

### 1. **Correctness** (Is it right?)
- **Accuracy**: Are facts, code, and information correct?
- **Functionality**: Does code work as intended?
- **Logic**: Is reasoning sound and valid?
- **Standards**: Does it follow best practices?

**Analysis Questions:**
- Are there any factual errors?
- Does the code have bugs or logic errors?
- Are API usages correct?
- Do examples work as shown?

### 2. **Completeness** (Is it thorough?)
- **Coverage**: Were all requirements addressed?
- **Scope**: Was the full problem solved?
- **Edge Cases**: Were edge cases considered?
- **Error Handling**: Are errors properly handled?

**Analysis Questions:**
- Did I address every part of the user's request?
- Are there missing features or functionality?
- What edge cases weren't covered?
- What happens when things go wrong?

### 3. **Clarity** (Is it understandable?)
- **Structure**: Is information well-organized?
- **Language**: Is it clearly explained?
- **Examples**: Are examples clear and helpful?
- **Documentation**: Is it well-documented?

**Analysis Questions:**
- Would a beginner understand this?
- Is the structure logical and easy to follow?
- Are technical terms explained?
- Are examples practical and clear?

### 4. **Efficiency** (Is it optimal?)
- **Simplicity**: Is it as simple as possible?
- **Performance**: Are there performance issues?
- **Code Quality**: Is code clean and maintainable?
- **Resource Usage**: Is resource usage reasonable?

**Analysis Questions:**
- Could this be simpler?
- Are there performance bottlenecks?
- Is the code unnecessarily complex?
- Are there better approaches?

### 5. **Security** (Is it safe?)
- **Vulnerabilities**: Are there security holes?
- **Input Validation**: Are inputs validated?
- **Authentication**: Are security checks present?
- **Data Protection**: Is sensitive data protected?

**Analysis Questions:**
- Are there injection vulnerabilities?
- Is input properly sanitized?
- Are there authentication/authorization issues?
- Could this expose sensitive data?

### 6. **Usability** (Can it be used easily?)
- **User Experience**: Is it user-friendly?
- **Installation**: Is setup clear and simple?
- **Documentation**: Is usage well-documented?
- **Error Messages**: Are errors helpful?

**Analysis Questions:**
- Can the user easily implement this?
- Are setup instructions clear?
- Will the user know what to do?
- Are error messages actionable?

## Quality Evaluation Framework

Use this systematic approach:

### Step 1: Requirement Mapping
```
User Request: [Original request]
Requirements Identified:
1. [Requirement 1]
2. [Requirement 2]
3. [Requirement 3]

Addressed: [✓/✗ for each]
```

### Step 2: Output Inventory
```
What was delivered:
- [Output 1: Description]
- [Output 2: Description]
- [Output 3: Description]

What was explained:
- [Explanation 1]
- [Explanation 2]
```

### Step 3: Quality Scoring

Rate each dimension (1-5):
```
Correctness:   X/5 - [Brief explanation]
Completeness:  X/5 - [Brief explanation]
Clarity:       X/5 - [Brief explanation]
Efficiency:    X/5 - [Brief explanation]
Security:      X/5 - [Brief explanation]
Usability:     X/5 - [Brief explanation]

Overall:       X/5
```

### Step 4: Issue Detection

Identify specific issues:
```
🔴 Critical:
- [Issue 1: Description and impact]
- [Issue 2: Description and impact]

🟡 Important:
- [Issue 1: Description and impact]
- [Issue 2: Description and impact]

🟢 Minor:
- [Issue 1: Description and impact]
```

### Step 5: Gap Analysis

What's missing:
```
Missing Functionality:
- [What's not there that should be]

Missing Documentation:
- [What needs better explanation]

Missing Validation:
- [What error cases aren't handled]

Missing Optimization:
- [What could be more efficient]
```

## Analysis Checklist

### Code Quality
- [ ] Follows language/framework conventions
- [ ] Has proper error handling
- [ ] Includes input validation
- [ ] Uses appropriate data structures
- [ ] Has reasonable performance
- [ ] Is maintainable and readable
- [ ] Has security considerations
- [ ] Handles edge cases
- [ ] Includes necessary comments
- [ ] Is testable

### Explanation Quality
- [ ] Answers the question asked
- [ ] Uses clear language
- [ ] Provides examples
- [ ] Explains reasoning
- [ ] Defines technical terms
- [ ] Has logical structure
- [ ] Appropriate length
- [ ] Actionable advice
- [ ] Covers edge cases
- [ ] Links to resources

### Communication Quality
- [ ] Appropriate tone
- [ ] Well-structured
- [ ] Concise yet complete
- [ ] Easy to scan
- [ ] Clear next steps
- [ ] Helpful formatting
- [ ] Good use of examples
- [ ] No unnecessary jargon
- [ ] Empathetic to user
- [ ] Confidence appropriate

## Common Quality Issues

### Pattern: Incomplete Requirements
**Symptom**: User says "what about X?" after response
**Cause**: Didn't address all aspects of request
**Fix**: Explicitly list requirements and check each

### Pattern: Assumes Context
**Symptom**: Solution doesn't work in user's environment
**Cause**: Made unstated assumptions
**Fix**: Ask clarifying questions; state assumptions

### Pattern: Over-Engineering
**Symptom**: Solution is overly complex
**Cause**: Didn't start with simplest approach
**Fix**: Begin with minimal solution; iterate

### Pattern: Under-Explaining
**Symptom**: User confused about how to use
**Cause**: Insufficient documentation/examples
**Fix**: Add usage examples; explain steps

### Pattern: Security Oversights
**Symptom**: Code has vulnerabilities
**Cause**: Didn't think about attack vectors
**Fix**: Security review; input validation; auth checks

### Pattern: Performance Issues
**Symptom**: Solution is slow or inefficient
**Cause**: Didn't consider scale or optimization
**Fix**: Analyze complexity; optimize critical paths

### Pattern: Poor Error Handling
**Symptom**: Crashes on unexpected input
**Cause**: Didn't validate inputs or handle errors
**Fix**: Add validation; try-catch; graceful degradation

## Quality Report Template

```markdown
# Quality Analysis Report

## Summary
[1-2 sentence overall assessment]

## Requirement Coverage
| Requirement | Addressed | Quality | Notes |
|-------------|-----------|---------|-------|
| Req 1       | ✓/✗      | X/5     | ...   |
| Req 2       | ✓/✗      | X/5     | ...   |

## Quality Scores
- **Correctness**: X/5 - [Why]
- **Completeness**: X/5 - [Why]
- **Clarity**: X/5 - [Why]
- **Efficiency**: X/5 - [Why]
- **Security**: X/5 - [Why]
- **Usability**: X/5 - [Why]
- **Overall**: X/5

## Issues Found

### 🔴 Critical Issues
1. [Issue with specific location and impact]
2. [Issue with specific location and impact]

### 🟡 Important Issues
1. [Issue with explanation]
2. [Issue with explanation]

### 🟢 Minor Issues
1. [Issue with suggestion]
2. [Issue with suggestion]

## Strengths
- [What was done particularly well]
- [Good decisions made]

## Gaps Identified
- **Missing**: [What's missing]
- **Incomplete**: [What's partially done]
- **Uncovered**: [What edge cases weren't handled]

## Recommendations
1. [Highest priority improvement]
2. [Next priority improvement]
3. [Additional suggestions]

## Next Steps
[What should be done immediately]
```

## Validation Scripts

> **⚠️ Note**: The validation scripts below are planned features but not yet implemented.
> This skill currently provides manual guidelines for quality analysis.
> See GitHub issue #20 for implementation tracking.

**Future automation features** (not yet available):

### Code Quality Check (Planned)
```bash
# NOT YET IMPLEMENTED - Manual analysis required
python {baseDir}/scripts/check-code-quality.py <file>
```
**When implemented, will check:**
- Syntax errors
- Common anti-patterns
- Security issues
- Style violations

### Completeness Check (Planned)
```bash
# NOT YET IMPLEMENTED - Manual analysis required
python {baseDir}/scripts/check-completeness.py <conversation-file>
```
**When implemented, will compare:**
- User requirements
- Delivered outputs
- Coverage percentage

### Documentation Check (Planned)
```bash
# NOT YET IMPLEMENTED - Manual analysis required
python {baseDir}/scripts/check-documentation.py <output-dir>
```
**When implemented, will validate:**
- README presence
- Code comments
- API documentation
- Usage examples

**Current usage**: Use the quality evaluation framework and checklists above for manual analysis.

## Examples

### Example 1: Code Quality Analysis

**Context**: Claude implemented a user authentication system

**Analysis**:
```
Quality Scores:
- Correctness: 3/5 - Logic works but has SQL injection vulnerability
- Completeness: 4/5 - Missing password reset flow
- Clarity: 4/5 - Well-documented but lacks usage examples
- Efficiency: 3/5 - No caching; hitting DB on every request
- Security: 2/5 - SQL injection, no rate limiting, weak password policy
- Usability: 4/5 - Easy to integrate but setup not documented

Issues:
🔴 SQL injection in login function (line 45)
🔴 Passwords stored with weak hashing (MD5)
🟡 Missing rate limiting on login attempts
🟡 No password reset functionality
🟡 Session tokens not securely generated
🟢 Could add remember-me functionality

Recommendations:
1. IMMEDIATE: Fix SQL injection (use parameterized queries)
2. IMMEDIATE: Upgrade to bcrypt for password hashing
3. Soon: Add rate limiting middleware
4. Soon: Implement password reset flow
5. Consider: Add session management improvements
```

### Example 2: Explanation Quality Analysis

**Context**: Claude explained async/await in JavaScript

**Analysis**:
```
Quality Scores:
- Correctness: 5/5 - Accurate information
- Completeness: 3/5 - Didn't cover error handling
- Clarity: 4/5 - Good progression but lacks diagrams
- Efficiency: N/A
- Security: N/A
- Usability: 4/5 - Good examples but no exercise

Strengths:
+ Clear progression from callbacks to promises to async/await
+ Good use of analogies (restaurant example)
+ Code examples are correct and practical

Gaps:
- No discussion of try-catch with async/await
- Missing explanation of Promise.all for parallel operations
- Didn't mention common pitfall: forgetting await
- No discussion of async in loops

Recommendations:
1. Add section on error handling
2. Include Promise.all example
3. Add "common mistakes" section
4. Provide practice exercise
```

## Your Role

When this skill is invoked:

1. **Systematically evaluate** response quality across all dimensions
2. **Use the framework** to ensure consistent analysis
3. **Be specific** about issues - cite line numbers, quote text
4. **Quantify quality** with numerical scores and justification
5. **Identify patterns** of recurring issues
6. **Prioritize** issues by severity and impact
7. **Recommend actions** that are specific and achievable

## Important Reminders

- Quality analysis should be **objective and evidence-based**
- **Balance** critical feedback with acknowledgment of strengths
- **Be specific** - vague feedback doesn't help improvement
- **Prioritize** - not all issues are equally important
- **Focus on impact** - explain why issues matter
- **Provide alternatives** - don't just identify problems
- **Consider context** - perfection isn't always necessary

Your analysis helps Claude continuously improve response quality and better serve users.