aposd-optimizing-critical-paths

# Skill: aposd-optimizing-critical-paths

## STOP - The Measure-First Rule

**Don't optimize based on intuition—measure first.** Intuitions about performance are unreliable, even for experienced developers.

**Stage 1 is a GATE:** You cannot proceed to optimization without actual profiling data.

---

## The Simplicity-Performance Relationship

| Myth | Reality |
|------|---------|
| "Performance requires complexity" | Simpler code usually runs faster |
| "Clean design sacrifices speed" | Clean design and high performance are compatible |
| "Optimization means adding code" | Optimization often means removing code |

**Why simplicity improves performance:**
- Fewer special cases = no code to check for those cases
- Deep classes = more work per call, fewer layer crossings
- Each layer crossing adds overhead
- Complicated code does extraneous or redundant work

---

## Expensive Operations Reference

Know these costs when choosing between alternatives:

| Operation | Cost | Context |
|-----------|------|---------|
| Network (datacenter) | 10–50 μs | Tens of thousands of instructions |
| Network (wide-area) | 10–100 ms | Millions of instructions |
| Disk I/O | 5–10 ms | Millions of instructions |
| Flash storage | 10–100 μs | Thousands of instructions |
| Dynamic memory allocation | Significant | malloc/new, freeing, GC overhead |
| Cache miss | Few hundred cycles | Often determines overall performance |

---

## Performance Optimization Workflow

### Stage 1: Measurement First (MANDATORY GATE)

```
BEFORE making any performance changes:

1. MEASURE existing system behavior
   - Where does the system spend most time?
   - Not just "system is slow" — identify specific locations

2. IDENTIFY small number of very specific places
   - With ideas for improvement
   - Focus on what matters most

3. ESTABLISH baseline
   - You'll need this to verify improvements
```

**What counts as valid measurement:**
- Actual profiling data (timing, call counts, memory usage)
- Multiple runs to account for variance
- Specific hotspot identification, not just "it's slow"

**What does NOT count:**
- "User said it's slow" (user perception ≠ bottleneck location)
- Pattern-matching to Expensive Operations table
- "This is obviously expensive" (intuition)
- "I'll measure after I make the change" (confirmation bias)

**⚠️ Stage 1 is a GATE, not a suggestion.** You cannot proceed to Stage 2 without completing measurement.

**Performance Dimensions:**
When measuring, identify WHICH dimension is the problem:
- **Throughput:** Operations per second
- **Latency:** Time per operation
- **Memory:** Peak/average usage, allocation rate
- **CPU:** Utilization percentage

Different problems require different solutions.

### Stage 2: Look for Fundamental Fixes

```
FIRST, check for fundamental fixes (preferred over code tweaks):

□ Can you add a cache?
□ Can you use a different algorithm? (e.g., balanced tree vs. list)
□ Can you bypass layers? (e.g., kernel bypass for networking)

IF fundamental fix exists → implement using standard design techniques
IF NOT → proceed to critical path redesign
```

### Stage 3: Critical Path Redesign (Last Resort)

```
ONLY when no fundamental fix is available:

1. ASK: What is the smallest amount of code for the common case?

2. DISREGARD existing code structure entirely
   - Imagine writing a new method that implements JUST the critical path

3. IGNORE special cases in current code
   - Consider only data needed for critical path
   - Choose most convenient data structure

4. DEFINE "the ideal"
   - The simplest and fastest code assuming complete redesign freedom
   - Even if not practically achievable, it's your target

5. DESIGN the rest of the class around these critical paths
```

---

## After Making Changes

```
1. RE-MEASURE to verify measurable performance difference

2. EVALUATE the tradeoff:
   - Did changes provide significant speedup (with data)? → Keep
   - Did changes make system simpler AND at least as fast? → Keep
   - Neither? → BACK THEM OUT
```

**⚠️ "Simpler" alone is not enough.** You must verify the simpler version is at least as fast.

---

## Anti-Rationalization Table

| Rationalization | Counter |
|-----------------|---------|
| "User said it's slow, that's my measurement" | User perception ≠ bottleneck location. Measure to find WHERE. |
| "Looking at the table, this is obviously expensive" | Pattern-matching isn't profiling. Measure actual time. |
| "I'll make the change then measure to verify" | Confirmation bias. Measure FIRST to find the real bottleneck. |
| "Setting up profiling is too complex" | If you can't measure, you can't verify improvement. Do the work. |
| "This scope is too small to measure" | Micro-optimizations without measurement add complexity for nothing. |
| "I checked the fundamental fix checklist" | Checklist is for ideas AFTER measurement shows the bottleneck. |
| "The code is simpler now, so it's faster" | Simpler doesn't automatically mean faster. Verify with measurement. |
| "I found a red flag pattern" | Red flags are descriptive, not prescriptive. Measure if it's actually slow. |
| "I already profiled extensively" | Share the data. Without data, it's still intuition. |

---

## Red Flags

| Red Flag | Symptom | Performance Impact |
|----------|---------|-------------------|
| **Death by Thousand Cuts** | Many small inefficiencies everywhere | System 5–10x slower; no single fix helps |
| **Pass-Through Methods** | Method with identical signature to caller | Unnecessary layer crossing overhead |
| **Shallow Layers** | Multiple layers providing same abstraction | Each call adds overhead |
| **Repeated Special Cases** | Same conditions checked multiple times | Redundant work on every call |
| **Premature Optimization** | Optimizing without measurement | Adds complexity without verified benefit |
| **Intuition-Based Changes** | "This should be faster" without data | Unreliable even for experts |

---

## Performance-Aware Development

**Default approach during normal development:**

1. Develop awareness of fundamentally expensive operations
2. Choose naturally efficient alternatives when equally clean options exist
3. If performance turns out to be a problem, optimize later
4. **Exception:** Clear evidence performance is critical → implement faster approach immediately

### When to Optimize Immediately

| Situation | Action |
|-----------|--------|
| Clear evidence performance is critical | Implement faster approach now |
| Faster design adds only small, hidden complexity | May be worthwhile |
| Faster design adds lot of complexity OR complicates interfaces | Start simple, optimize later |

---

## Consolidation Techniques

When optimizing critical paths, look for ways to consolidate:

| Technique | Example |
|-----------|---------|
| **Encode multiple conditions in single value** | Variable that is 0 when any of several special cases apply |
| **Single test for multiple cases** | Replace 6 individual checks with 1 combined check |
| **Combine layers into single method** | Critical path handled in one method, not three |
| **Merge variables** | Combine multiple values into single structure |

---

## Quick Reference

```
PERFORMANCE OPTIMIZATION PRIORITY:

1. MEASURE (MANDATORY GATE)
   - Actual profiling data, not intuition
   - Identify which dimension: throughput, latency, memory, CPU
   - Establish baseline before any changes
   - No measurement = no optimization

2. FUNDAMENTAL FIX - Preferred approach
   - Cache? Better algorithm? Bypass layers?
   - Only consider AFTER measurement shows the bottleneck

3. CRITICAL PATH - Last resort
   - What's the minimum code for common case?
   - Disregard existing structure
   - Define "the ideal"

4. VERIFY - After changes
   - Re-measure with same methodology
   - Faster with data? Keep
   - Simpler AND at least as fast? Keep
   - Neither? BACK OUT

THE RULE:
Measure → Identify → Fix → Verify
Never skip steps. Never assume.
```


---

## Chain

| After | Next |
|-------|------|
| Optimization done | Verify performance improved |