feature-metrics

# /feature-metrics - Define Success Metrics

Select trustworthy metrics using the STEDII framework.

## Context Routing Logic (Internal - for Claude)

**Automatic Context Checks:**
When this skill is invoked, immediately check:

| Source          | Files/Folders                                  | Search Terms                          | What to Extract                            |
| --------------- | ---------------------------------------------- | ------------------------------------- | ------------------------------------------ |
| Current PRD     | `thoughts/shared/pm/prds/*.md`                 | feature name from chat                | Hypothesis, problem statement, user impact |
| Business Info   | `thoughts/shared/pm/context/business-info-template.md` | business model, growth stage, metrics | Product strategy, current North Star       |
| Metrics Context | `thoughts/shared/pm/metrics/*.md`              | baseline numbers, historical data     | Current metric baselines, ranges           |
| Strategy        | `thoughts/shared/pm/frameworks/*.md`           | feature related to strategic pillar   | Strategic fit and expected outcomes        |
| Meetings        | `thoughts/shared/product/meeting-notes/*.md`   | feature name, "success metrics"       | Stakeholder expectations, past decisions   |

**Context Priority:**

1. Current PRD and feature context FIRST
2. Business model and strategy SECOND
3. Historical metrics and baselines THIRD
4. Stakeholder expectations FOURTH

**Cross-Skill Links:**

- If feature is part of larger product strategy → Link to `/write-prod-strategy`
- If testing this feature → Link to `/experiment-decision` and `/experiment-metrics`
- If metric is North Star related → Link to `/define-north-star`
- If sizing impact → Link to `/impact-sizing` for usage estimates
- If tracking retention → Link to `/retention-analysis` for cohort analysis

---

## When to Use

- Defining success criteria for a new feature
- Setting up an A/B test
- Creating a PRD metrics section
- Validating existing metrics

---

## Step 0: Understanding Current State

Before we define metrics, let me check what context already exists...

**Checking:**

- `thoughts/shared/pm/prds/` for any existing PRD for this feature
- `thoughts/shared/pm/context/business-info-template.md` for your product model
- `thoughts/shared/pm/metrics/` for historical baseline data
- `thoughts/shared/pm/frameworks/` for strategic context
- `thoughts/shared/product/meeting-notes/` for stakeholder expectations

**[If feature PRD exists]:** "I found your [Feature Name] PRD from [date]. It mentions [hypothesis/goal]. Let me use that as context."

**[If metrics exist]:** "I found historical data: [Metric] baselines are currently [values]. I'll use this as reference."

**Based on what I find, I'll show you:**

### What We Know About This Feature

**Strategic Context:**

- [How this feature fits into your Q# strategy / roadmap]
- [Expected user impact: # of users affected]
- [Business outcome: revenue/retention/engagement impact]

**Current Baselines:**

- [Relevant historical metrics for comparison]
- [Product stage: early-stage feature / mature feature / existing metric improvement]

**Success Expectations:**

- [From stakeholder meetings: what they're expecting]
- [From user research: what users need]
- [From business model: what drives your North Star]

### Questions to Clarify Before Selecting Metrics

1. **Feature Scope:** Is this a small UX improvement, new capability, or major feature overhaul?
2. **User Segment:** Who is this feature for? All users, specific segment, or internal teams?
3. **Impact Type:** Are we trying to drive growth, engagement, retention, monetization, or efficiency?
4. **Experiment Timeline:** How long can we run the test? (This affects which metrics we can use)
5. **Business Context:** What's more important right now - speed or certainty?

---

## STEDII Framework

Every good metric should pass these 6 criteria:

### S - Sensitive

Can the metric detect changes from your feature?

- Will it move meaningfully with expected impact?
- Is the sample size sufficient?

### T - Timely

How quickly does the metric respond?

- Can you measure it within your experiment window?
- Leading indicators > lagging indicators

### E - Easy to Understand

Can stakeholders interpret it?

- Avoid complex calculations
- Clear cause and effect

### D - Directional

Is improvement clear?

- Up = good or Down = good? Be explicit
- Avoid metrics where direction is ambiguous

### I - Implementable

Can you actually track it?

- Data exists or can be collected
- Engineering effort is reasonable

### I - Independent

Does it avoid external factors?

- Seasonality effects?
- Other experiments running?

---

## Quick Start Prompt

When PM types `/feature-metrics`, respond:

```
Let's define metrics for your feature. I'll use the STEDII framework.

Tell me:
1. What feature are we measuring?
2. What user behavior does it change?
3. What business outcome do we expect?

I'll help you select primary metrics, guardrails, and kill criteria.
```

---

## Metric Types

### Primary Metric

The one metric that defines success.

- Directly tied to feature goal
- Must pass all STEDII criteria
- Single source of truth for go/no-go

### Guardrail Metrics

Metrics that must NOT get worse.

- Protect against unintended harm
- Set acceptable ranges (not targets)
- Examples: page load time, error rate, support tickets

### Kill Criteria

When to stop the experiment early.

- Serious negative impact threshold
- Safety concerns
- Automatic rollback triggers

---

## Output Template

```markdown
# Feature Metrics: [Feature Name]

## Primary Metric

**Metric:** [Name]
**Definition:** [Exactly how it's calculated]
**Current baseline:** [X]
**Target:** [Y] ([+/- Z%])
**Timeline:** [When we expect to see impact]

**STEDII Check:**

- [x] Sensitive - [why]
- [x] Timely - [why]
- [x] Easy to understand - [why]
- [x] Directional - [up/down = good]
- [x] Implementable - [data source]
- [x] Independent - [controls for]

## Guardrail Metrics

| Metric     | Acceptable Range | Why It Matters     |
| ---------- | ---------------- | ------------------ |
| [Metric 1] | [range]          | [protects against] |
| [Metric 2] | [range]          | [protects against] |

## Kill Criteria

If any of these occur, immediately rollback:

- [Metric] drops below [threshold]
- [Metric] increases above [threshold]
- [Qualitative signal] occurs

## Measurement Plan

- **Data source:** [where data comes from]
- **Tracking:** [how it's implemented]
- **Dashboard:** [where to monitor]
- **Review cadence:** [how often to check]
```

---

## Common Metric Pairs

| Feature Type | Primary Metric      | Common Guardrails    |
| ------------ | ------------------- | -------------------- |
| Growth       | Signups, Activation | Retention, Quality   |
| Engagement   | DAU, Sessions       | Load time, Errors    |
| Revenue      | Conversion, ARPU    | Refunds, Churn       |
| Retention    | D7/D30 retention    | NPS, Support tickets |
| Efficiency   | Task completion     | Time on task, Errors |

---

## Output Integration

### Where Files Go

**Feature metrics definitions:**

- Active work: Add to PRD in `Strategic Fit` section
- When finalized: Reference in `/experiment-decision` for A/B testing approach
- Archive: Store final metrics in `thoughts/shared/pm/metrics/[feature-name]-baseline.md` for historical reference

### Link to Other Work

After defining metrics:

- **Reference in PRDs** - "Success is defined as [primary metric] reaching [target] based on STEDII framework"
- **Use in experiments** - Feature metrics become primary metric in `/experiment-decision`
- **Track progress** - Monitor against baseline in weekly status updates
- **Feed retention analysis** - If tracking retention, pass metric definitions to `/retention-analysis`

### Cross-Skill Integration

**Feeds into:**

- `/experiment-decision` - Primary metric determines test design and duration
- `/feature-results` - Use these metrics to measure actual impact post-launch
- `/impact-sizing` - Use guardrails to validate usage estimates
- `/metrics-framework` - This metric may become a leading indicator for North Star

**Pulls from:**

- `/define-north-star` - Ensure primary metric ladders up to North Star
- `/impact-sizing` - Usage estimates inform what metrics can detect changes
- [[business-info-template]] - Company metrics and baselines

---

## Tips

- **One primary metric** - Multiple "primary" metrics = no primary metric
- **Guardrails are not goals** - You're not trying to improve them, just protect them
- **Leading > Lagging** - Measure what you can act on quickly
- **Avoid vanity metrics** - Page views don't matter if nobody converts
- **Baseline matters** - Know your current numbers before running experiment
- **Time to signal** - Faster metrics (hours/days) beat slow metrics (months)

---

## Output Quality Self-Check

Before presenting output to the PM, verify:

- [ ] **File saved to correct location:** Output saved to `thoughts/shared/pm/metrics/feature-metrics-[feature-name]-[date].md`
- [ ] **Context routing table was checked:** Reviewed `thoughts/shared/pm/prds/` for feature context, `thoughts/shared/pm/context/business-info-template.md` for North Star metric, and `thoughts/shared/pm/metrics/` for existing dashboards and baselines
- [ ] **Metrics pass STEDII framework:** Each proposed metric is evaluated against all 6 STEDII dimensions (Sensitive, Timely, Easy to understand, Directional, Implementable, Independent) with pass/fail reasoning
- [ ] **Primary metric has baseline and target:** The primary metric includes a current baseline number and a specific target value with timeline (not "improve" or "increase")
- [ ] **Guardrail metrics defined:** At least 1 guardrail metric is specified with an acceptable range and explanation of what it protects against
- [ ] **Metrics ladder to North Star:** The output explicitly shows how the primary metric connects upward to the company's North Star metric from [[business-info-template]]
- [ ] **Data source identified for each metric:** Every metric names where the data comes from (e.g., "PostHog event: task_created" or "database query on users table")
- [ ] **Metric sensitivity estimated:** The output addresses whether the expected feature impact is large enough for the metric to detect, given current variance and traffic