nw-pdr-review-criteria

# Review Criteria -- Product Discovery Review

## Evidence Quality Validation

### Past Behavior Indicators (Good)
"Tell me about the last time..." | "When did you last..." | "What happened when..." | "Walk me through how you..." | "What did you try..." | "How much have you spent on..."
Specific dates, dollar amounts, named tools, concrete examples, emotional frustration language

### Future Intent Red Flags (Reject)
"Would you use/pay/like..." | "Do you think..." | "Imagine if..." | "What if we..."
Flag and reject if >20% of evidence is future-intent.

### Validation Thresholds
- Past behavior ratio: >80% (reject if fail, require re-interview)
- Specific examples: min 3 concrete per finding (warn if fail)
- Customer language: quotes in customer words, not paraphrased (warn if fail)

## Sample Size Minimums

| Phase | Minimum | High Confidence | Notes |
|-------|---------|-----------------|-------|
| 1: Problem | 5 | 10 | Interviews required |
| 2: Opportunity | 10 | 20 | Quantitative supplements, not replaces |
| 3: Solution | 5 per iteration | 3 iterations max | Before decision |
| 4: Viability | 5 | -- | Stakeholder review required |

Pivot decision rule: min 5 consistent signals. Block decisions on fewer.

## Decision Gate Criteria

### G1: Problem to Opportunity
Proceed: 5+ confirm pain + willingness to pay | Pivot: differs from expected | Kill: <20% confirm
Checks: 5+ interviews, >60% confirmation, customer words, 3+ examples

### G2: Opportunity to Solution
Proceed: top 2-3 score >8 (0-20) | Pivot: new opportunities | Kill: all low-value
Checks: OST complete with 5+, scores correct (Importance + Max(0, Importance - Satisfaction)), top >8/20

### G3: Solution to Viability
Proceed: >80% task completion, usability validated | Pivot: needs refinement | Kill: fundamental blocks
Checks: 5+ users/iteration, >80% completion, core flow usable, value validated

### G4: Viability to Build
Proceed: 4 risks addressed, model validated | Pivot: adjustment needed | Kill: no viable model
Checks: Lean Canvas complete, all risks green/yellow, stakeholder sign-off

## Bias Types

### Confirmation Bias (critical)
Signals: only positive quotes | skeptics not interviewed | disconfirming evidence dismissed | same questions for "right" answers
Fix: include skeptics, actively seek disconfirming evidence

### Selection Bias (high)
Signals: all existing customers | no churned/non-adopters | lacks diversity | single-enthusiast referral chain
Fix: random/diverse selection, include skeptics and non-users

### Discovery Theater (critical)
Signals: conclusion decided before research | findings match hypothesis perfectly | no surprises | idea-in = idea-shipped
Fix: track idea evolution, expect 50%+ ideas to change

### Sample Size Problem (high)
Signals: major decisions on 2-3 interviews | single quote as "validation" | pivot on one signal
Fix: min 5 interviews per segment, 5+ signals for decisions

## Anti-Patterns

### Interview Anti-Patterns
| Pattern | Detection | Bad | Good | Severity |
|---------|-----------|-----|------|----------|
| Leading questions | Suggests desired answer | "Don't you think this would save time?" | "Tell me about the last time you tried to save time on this" | high |
| Future-intent | Hypothetical behavior | "Would you use this feature?" | "What have you tried to solve this problem?" | critical |
| Compliments as validation | Accepting "that's cool" | "They loved the idea!" | "They committed to follow-up and referral" | high |
| Talking > listening | >30% interviewer talk | Long questions, short responses | Open questions, extended answers | medium |

### Process Anti-Patterns
| Pattern | Detection | Severity |
|---------|-----------|----------|
| Skipping to solutions | Solution before problem validated | critical |
| Demographic segmentation | Segments by demographics not jobs | medium |
| Building before testing | Code before Phase 3 | critical |

### Strategic Anti-Patterns
| Pattern | Detection | Severity |
|---------|-----------|----------|
| Premature pivoting | Direction change on 1-2 signals (need 5+) | high |
| Solution love | Defending despite evidence, dismissing critics | high |
| Sole source of truth | Only quant OR qual, not both | medium |

## Pre-Approval Checklist

### Evidence Quality
- [ ] Past behavior ratio >80% | [ ] No critical future-intent | [ ] Customer language preserved

### Sample Sizes
- [ ] Phase 1: 5+ | [ ] Phase 2: 10+ | [ ] Phase 3: 5+/iteration | [ ] Phase 4: 5+ with stakeholders

### Decision Gates
- [ ] All gates properly evaluated | [ ] Criteria documented with evidence | [ ] Decision justified

### Bias Check
- [ ] No confirmation bias | [ ] No selection bias | [ ] No discovery theater | [ ] Sample size adequate

### Anti-Patterns
- [ ] No critical interview anti-patterns | [ ] No critical process anti-patterns | [ ] No critical strategic anti-patterns

## Approval Decision

- **Approved**: all checks pass, no critical issues. Formal handoff to product-owner.
- **Conditionally approved**: minor issues only (no critical/high). Approval with recommendations.
- **Rejected**: any critical/high issue. Structured rejection with remediation. Blocks handoff.