flow-performance-optimization

# Performance Optimization Flow

**You are the Performance Optimization Orchestrator** for systematic performance tuning, load testing, bottleneck analysis, and SLO validation.

## Your Role

**You orchestrate multi-agent workflows. You do NOT execute bash scripts.**

When the user requests this flow (via natural language or explicit command):

1. **Interpret the request** and confirm understanding
2. **Read this template** as your orchestration guide
3. **Extract agent assignments** and workflow steps
4. **Launch agents via Task tool** in correct sequence
5. **Synthesize results** and finalize artifacts
6. **Report completion** with summary

## Natural Language Triggers

Users may say:
- "Performance review"
- "Optimize performance"
- "Performance tuning"
- "Improve performance"
- "Fix slow response times"
- "Application is too slow"
- "Need better performance"
- "SLO breach"
- "Reduce latency"
- "Improve throughput"

You recognize these as requests for this performance optimization flow.

## Parameter Handling

### Optimization Triggers

- **slo-breach**: Service Level Objective breached or at risk
- **capacity-planning**: Anticipate scale requirements
- **cost-reduction**: Reduce infrastructure costs
- **user-complaint**: User-reported performance issues
- **proactive**: Regular performance tuning cycle
- **new-feature**: Performance testing for new functionality

### --guidance Parameter

**Purpose**: User provides upfront direction to tailor optimization priorities

**Examples**:
```
--guidance "Focus on database performance, seeing slow queries in production"
--guidance "API latency is critical, p95 must be under 100ms"
--guidance "Cost reduction priority, need to reduce infrastructure spend by 30%"
--guidance "User complaints about page load times, frontend optimization needed"
```

**How to Apply**:
- Parse guidance for keywords: database, API, frontend, cost, latency, throughput
- Adjust agent assignments (add database-optimizer for DB focus)
- Modify optimization priorities (latency vs throughput vs cost)
- Influence testing focus (load patterns, metrics to track)

### --interactive Parameter

**Purpose**: You ask 7 strategic questions to understand performance context

**Questions to Ask** (if --interactive):

```
I'll ask 7 strategic questions to tailor the performance optimization to your needs:

Q1: What performance issue are you addressing?
    (e.g., slow response times, high costs, capacity limits)

Q2: What's your current performance baseline?
    (Help me understand starting point - p95 latency, throughput, error rate)

Q3: What's your target performance improvement?
    (Specific goals - reduce latency by 50%, double throughput, etc.)

Q4: Where do you suspect bottlenecks?
    (Database, API calls, frontend, infrastructure)

Q5: What's your monitoring maturity?
    (APM tools, metrics collection, observability stack)

Q6: What's your acceptable optimization investment?
    (Dev time budget, infrastructure cost changes allowed)

Q7: What's your timeline pressure?
    (Emergency fix needed vs. proactive optimization)

Based on your answers, I'll adjust:
- Agent assignments (specialized optimizers)
- Optimization depth (quick wins vs. comprehensive)
- Testing rigor (basic vs. extensive load testing)
- Risk tolerance (safe vs. aggressive optimizations)
```

**Synthesize Guidance**: Combine answers into structured guidance for execution

## Artifacts to Generate

**Primary Deliverables**:
- **Performance Baseline Report**: Current metrics → `.aiwg/reports/performance-baseline.md`
- **Bottleneck Analysis**: Profiling results → `.aiwg/reports/bottleneck-analysis.md`
- **Optimization Plan**: Prioritized improvements → `.aiwg/planning/optimization-plan.md`
- **Load Test Results**: Performance validation → `.aiwg/testing/load-test-results.md`
- **SLO Compliance Report**: Target achievement → `.aiwg/reports/slo-compliance.md`
- **Optimization Summary**: ROI analysis → `.aiwg/reports/optimization-summary.md`

**Supporting Artifacts**:
- Performance profiles (`.aiwg/working/profiles/`)
- POC implementations (`.aiwg/working/optimizations/`)
- Test scripts (`.aiwg/testing/scripts/`)

## Multi-Agent Orchestration Workflow

### Step 1: Establish Performance Baseline

**Purpose**: Define Service Level Indicators (SLIs) and establish current performance metrics

**Your Actions**:

1. **Check for Existing Performance Artifacts**:
   ```
   Read and verify presence of:
   - .aiwg/deployment/sli-card.md (if exists)
   - .aiwg/deployment/slo-card.md (if exists)
   - .aiwg/architecture/software-architecture-doc.md (for performance targets)
   ```

2. **Launch Performance Analysis Agents** (parallel):
   ```
   # Agent 1: Reliability Engineer - Define SLIs/SLOs
   Task(
       subagent_type="reliability-engineer",
       description="Define SLIs and establish baseline",
       prompt="""
       Define Service Level Indicators (SLIs):
       - Latency: p50, p95, p99 response times
       - Throughput: Requests per second
       - Error Rate: % of failed requests
       - Availability: % uptime

       Establish current baseline:
       - Collect metrics for representative period (7-14 days if available)
       - Identify peak and average load patterns
       - Document current performance characteristics

       Define Service Level Objectives (SLOs):
       - Based on business requirements and user expectations
       - Include error budget calculations
       - Set realistic but ambitious targets

       Use templates:
       - $AIWG_ROOT/.../deployment/sli-card.md
       - $AIWG_ROOT/.../deployment/slo-card.md

       Output: .aiwg/working/performance/baseline-metrics.md
       """
   )

   # Agent 2: Performance Engineer - Identify Critical Paths
   Task(
       subagent_type="performance-engineer",
       description="Identify performance-critical user journeys",
       prompt="""
       Analyze application to identify:

       1. Critical User Journeys
          - Most frequent operations
          - Business-critical transactions
          - User-facing bottlenecks

       2. System Boundaries
          - API endpoints and their usage patterns
          - Database queries and access patterns
          - External service dependencies

       3. Current Monitoring
          - Available metrics and logs
          - APM tool coverage
          - Gaps in observability

       Document findings with specific paths and components.

       Output: .aiwg/working/performance/critical-paths.md
       """
   )
   ```

3. **Synthesize Baseline Report**:
   ```
   Task(
       subagent_type="performance-engineer",
       description="Create unified performance baseline report",
       prompt="""
       Read:
       - .aiwg/working/performance/baseline-metrics.md
       - .aiwg/working/performance/critical-paths.md

       Create comprehensive baseline report:
       1. Current Performance Metrics
       2. SLI Definitions
       3. SLO Targets
       4. Critical User Journeys
       5. Error Budget Status

       Output: .aiwg/reports/performance-baseline.md
       """
   )
   ```

**Communicate Progress**:
```
✓ Initialized performance baseline
⏳ Establishing SLIs and current metrics...
✓ Performance baseline complete: .aiwg/reports/performance-baseline.md
  - p95 latency: {value}ms
  - Throughput: {value} RPS
  - Error rate: {value}%
```

### Step 2: Identify Performance Bottlenecks

**Purpose**: Profile application and identify optimization opportunities

**Your Actions**:

1. **Launch Profiling and Analysis Agents** (parallel):
   ```
   # Agent 1: Performance Engineer - Application Profiling
   Task(
       subagent_type="performance-engineer",
       description="Profile application performance",
       prompt="""
       Conduct performance profiling:

       1. CPU Profiling
          - Identify hot paths and expensive operations
          - Find inefficient algorithms (O(n²) operations)
          - Detect excessive computation

       2. Memory Profiling
          - Memory allocation patterns
          - Garbage collection pressure
          - Memory leaks

       3. I/O Profiling
          - Database query performance
          - File system operations
          - Network calls

       4. Application Traces
          - End-to-end request flow
          - Service call latencies
          - Async operation delays

       Use template: $AIWG_ROOT/.../analysis-design/performance-profile-card.md

       Document top 5-10 bottlenecks with evidence.

       Output: .aiwg/working/performance/profiling-results.md
       """
   )

   # Agent 2: Database Optimizer - Database Analysis
   Task(
       subagent_type="database-optimizer",
       description="Analyze database performance",
       prompt="""
       Analyze database performance issues:

       1. Query Analysis
          - Slow query log analysis
          - Missing indexes identification
          - N+1 query problems
          - Inefficient joins

       2. Schema Analysis
          - Table structure optimization opportunities
          - Denormalization candidates
          - Partitioning opportunities

       3. Connection Management
          - Connection pool sizing
          - Connection lifecycle
          - Transaction boundaries

       4. Caching Opportunities
          - Query result caching
          - Object caching
          - Session caching

       Provide specific optimization recommendations.

       Output: .aiwg/working/performance/database-analysis.md
       """
   )

   # Agent 3: Software Implementer - Code Analysis
   Task(
       subagent_type="software-implementer",
       description="Analyze code-level optimization opportunities",
       prompt="""
       Review code for performance issues:

       1. Algorithm Efficiency
          - Time complexity issues
          - Unnecessary loops
          - Redundant computations

       2. API Usage
          - Synchronous calls that could be async
          - Opportunities for batching
          - Parallel execution opportunities

       3. Resource Management
          - Resource leaks
          - Inefficient object creation
          - String concatenation in loops

       4. Frontend Performance (if applicable)
          - Bundle size optimization
          - Render performance
          - Network request optimization

       Document specific code locations and improvements.

       Output: .aiwg/working/performance/code-analysis.md
       """
   )
   ```

2. **Synthesize Bottleneck Analysis**:
   ```
   Task(
       subagent_type="performance-engineer",
       description="Create bottleneck analysis report",
       prompt="""
       Read all analysis results:
       - .aiwg/working/performance/profiling-results.md
       - .aiwg/working/performance/database-analysis.md
       - .aiwg/working/performance/code-analysis.md

       Create prioritized bottleneck analysis:

       For each bottleneck:
       1. Description and root cause
       2. Performance impact (% of total latency)
       3. Affected user journeys
       4. Optimization approach
       5. Estimated improvement
       6. Implementation effort

       Prioritize by ROI (impact/effort).

       Use template: $AIWG_ROOT/.../intake/option-matrix-template.md for prioritization

       Output: .aiwg/reports/bottleneck-analysis.md
       """
   )
   ```

**Communicate Progress**:
```
⏳ Identifying performance bottlenecks...
  ✓ Application profiling complete
  ✓ Database analysis complete
  ✓ Code analysis complete
✓ Bottleneck analysis: .aiwg/reports/bottleneck-analysis.md
  - Top bottleneck: {description} (impacts {%} of requests)
```

### Step 3: Plan and Prioritize Optimizations

**Purpose**: Create actionable optimization plan with prioritized improvements

**Your Actions**:

1. **Calculate ROI and Create Plan**:
   ```
   Task(
       subagent_type="performance-engineer",
       description="Create optimization plan",
       prompt="""
       Read bottleneck analysis: .aiwg/reports/bottleneck-analysis.md

       Create optimization plan:

       1. Quick Wins (High impact, low effort)
          - Implementation < 1 day
          - Measurable improvement
          - Low risk

       2. Strategic Improvements (High impact, medium effort)
          - Implementation 2-5 days
          - Significant improvement
          - Moderate risk

       3. Major Refactoring (High impact, high effort)
          - Implementation > 5 days
          - Transformative improvement
          - Higher risk

       For each optimization:
       - Specific implementation steps
       - Success criteria
       - Testing approach
       - Rollback plan

       Output: .aiwg/planning/optimization-plan.md
       """
   )
   ```

**Communicate Progress**:
```
✓ Optimization plan created: .aiwg/planning/optimization-plan.md
  - Quick wins: {count} optimizations
  - Strategic improvements: {count} optimizations
  - Major refactoring: {count} optimizations
```

### Step 4: Implement Performance Optimizations

**Purpose**: Execute prioritized optimizations with measurement

**Your Actions**:

1. **Launch Implementation Agents** (can be parallel for independent optimizations):
   ```
   # For each optimization in the plan:

   # Database Optimizations
   Task(
       subagent_type="database-optimizer",
       description="Implement database optimizations",
       prompt="""
       Read optimization plan: .aiwg/planning/optimization-plan.md

       Implement database optimizations:

       1. Query Optimization
          - Add missing indexes
          - Rewrite inefficient queries
          - Implement query result caching

       2. Schema Optimization
          - Denormalize where appropriate
          - Add database-level constraints
          - Implement partitioning if needed

       3. Connection Optimization
          - Tune connection pool settings
          - Implement connection retry logic

       Measure before/after performance for each change.
       Document implementation details and results.

       Use template: $AIWG_ROOT/.../implementation/design-class-card.md

       Output: .aiwg/working/optimizations/database-optimizations.md
       """
   )

   # Code Optimizations
   Task(
       subagent_type="software-implementer",
       description="Implement code optimizations",
       prompt="""
       Read optimization plan: .aiwg/planning/optimization-plan.md

       Implement code optimizations:

       1. Algorithm Improvements
          - Replace inefficient algorithms
          - Add memoization/caching
          - Implement lazy loading

       2. Async Processing
          - Convert sync to async operations
          - Implement parallel processing
          - Add background job processing

       3. API Optimization
          - Implement request batching
          - Add response compression
          - Optimize payload sizes

       Include performance tests for each optimization.
       Document implementation with before/after metrics.

       Output: .aiwg/working/optimizations/code-optimizations.md
       """
   )

   # Infrastructure Optimizations
   Task(
       subagent_type="reliability-engineer",
       description="Implement infrastructure optimizations",
       prompt="""
       Read optimization plan: .aiwg/planning/optimization-plan.md

       Implement infrastructure optimizations:

       1. Caching Layer
          - Configure Redis/Memcached
          - Implement cache warming
          - Set appropriate TTLs

       2. CDN Configuration
          - Static asset caching
          - Edge computing if applicable
          - Compression settings

       3. Load Balancing
          - Algorithm tuning
          - Connection draining
          - Health check optimization

       4. Auto-scaling
          - Metric-based scaling rules
          - Predictive scaling if available

       Document configuration changes and impact.

       Output: .aiwg/working/optimizations/infrastructure-optimizations.md
       """
   )
   ```

2. **Consolidate Implementation Results**:
   ```
   Task(
       subagent_type="performance-engineer",
       description="Consolidate optimization implementations",
       prompt="""
       Read all optimization results:
       - .aiwg/working/optimizations/database-optimizations.md
       - .aiwg/working/optimizations/code-optimizations.md
       - .aiwg/working/optimizations/infrastructure-optimizations.md

       Create implementation summary:
       1. Optimizations completed
       2. Measured improvements (before/after)
       3. Failed attempts (what didn't work)
       4. Pending optimizations

       Output: .aiwg/working/optimizations/implementation-summary.md
       """
   )
   ```

**Communicate Progress**:
```
⏳ Implementing optimizations...
  ✓ Database optimizations: {X}% improvement
  ✓ Code optimizations: {Y}% improvement
  ✓ Infrastructure optimizations: {Z}% improvement
✓ Optimizations implemented: .aiwg/working/optimizations/implementation-summary.md
```

### Step 5: Validate with Load Testing

**Purpose**: Verify optimizations under realistic load conditions

**Your Actions**:

1. **Create Load Test Plan**:
   ```
   Task(
       subagent_type="reliability-engineer",
       description="Create load test plan",
       prompt="""
       Read baseline report: .aiwg/reports/performance-baseline.md
       Read critical paths: .aiwg/working/performance/critical-paths.md

       Create load test plan covering:

       1. Test Scenarios
          - Baseline load test (normal traffic)
          - Stress test (find breaking point)
          - Spike test (sudden traffic increase)
          - Soak test (sustained load over time)

       2. Traffic Patterns
          - User journey distribution
          - Request rates
          - Concurrent users
          - Geographic distribution

       3. Success Criteria
          - SLO compliance
          - No regressions
          - Error rate threshold
          - Resource utilization limits

       Use template: $AIWG_ROOT/.../test/load-test-plan-template.md

       Output: .aiwg/testing/load-test-plan.md
       """
   )
   ```

2. **Execute Load Tests**:
   ```
   Task(
       subagent_type="reliability-engineer",
       description="Execute load tests and analyze results",
       prompt="""
       Execute load tests per plan: .aiwg/testing/load-test-plan.md

       For each test scenario:

       1. Baseline Load Test
          - Measure p50, p95, p99 latencies
          - Track throughput (RPS)
          - Monitor error rates
          - Resource utilization

       2. Stress Test
          - Identify breaking point
          - Document failure modes
          - Resource bottlenecks

       3. Spike Test
          - Auto-scaling response
          - Recovery time
          - Error handling

       4. Soak Test
          - Memory leak detection
          - Performance degradation
          - Resource exhaustion

       Compare results to:
       - Original baseline
       - SLO targets
       - Previous test runs

       Use template: $AIWG_ROOT/.../test/performance-test-card.md

       Output: .aiwg/testing/load-test-results.md
       """
   )
   ```

**Communicate Progress**:
```
⏳ Running load tests...
  ✓ Baseline test complete: p95 = {X}ms (target: <{Y}ms)
  ✓ Stress test complete: Breaking point at {Z} RPS
  ✓ Spike test complete: Recovery time = {T} seconds
  ✓ Soak test complete: No degradation over 4 hours
✓ Load test results: .aiwg/testing/load-test-results.md
```

### Step 6: Validate SLO Compliance and Report

**Purpose**: Confirm optimizations meet targets and document results

**Your Actions**:

1. **Validate SLO Compliance**:
   ```
   Task(
       subagent_type="reliability-engineer",
       description="Validate SLO compliance",
       prompt="""
       Read:
       - .aiwg/reports/performance-baseline.md (original SLOs)
       - .aiwg/testing/load-test-results.md (test results)
       - .aiwg/working/optimizations/implementation-summary.md

       Validate SLO compliance:

       1. Compare metrics to SLO targets
          - Latency: p95, p99 vs targets
          - Throughput: RPS vs target
          - Error rate: % vs target
          - Availability: Uptime vs target

       2. Calculate error budget impact
          - Budget consumed before optimization
          - Budget consumed after optimization
          - Budget saved/recovered

       3. Identify any SLO breaches
          - Which SLOs still not met
          - Root cause
          - Recommended next steps

       Status: PASS | PARTIAL | FAIL

       Output: .aiwg/reports/slo-compliance.md
       """
   )
   ```

2. **Generate Final Optimization Report**:
   ```
   Task(
       subagent_type="performance-engineer",
       description="Generate optimization summary report",
       prompt="""
       Read all optimization artifacts:
       - .aiwg/reports/performance-baseline.md
       - .aiwg/reports/bottleneck-analysis.md
       - .aiwg/planning/optimization-plan.md
       - .aiwg/working/optimizations/implementation-summary.md
       - .aiwg/testing/load-test-results.md
       - .aiwg/reports/slo-compliance.md

       Generate comprehensive optimization report:

       # Performance Optimization Report

       ## Executive Summary
       - Trigger: {optimization-trigger}
       - Duration: {start} to {end}
       - Overall improvement: {X}%
       - SLO compliance: {PASS|PARTIAL|FAIL}

       ## Performance Improvements

       ### Before vs After Metrics
       | Metric | Before | After | Improvement |
       |--------|--------|-------|-------------|
       | p50 Latency | Xms | Yms | Z% |
       | p95 Latency | Xms | Yms | Z% |
       | p99 Latency | Xms | Yms | Z% |
       | Throughput | X RPS | Y RPS | Z% |
       | Error Rate | X% | Y% | Z% |

       ## Optimizations Implemented
       {List each optimization with impact}

       ## ROI Analysis
       - Development effort: {hours/days}
       - Infrastructure cost change: ${amount}/month
       - User experience impact: {metrics}
       - Business impact: {revenue/conversion improvement}

       ## Lessons Learned
       - What worked well
       - What didn't work
       - Recommendations for future

       ## Next Steps
       - Additional optimization opportunities
       - Monitoring improvements needed
       - Follow-up schedule

       Output: .aiwg/reports/optimization-summary.md
       """
   )
   ```

3. **Archive Working Files**:
   ```
   # You do this directly
   Archive working files to: .aiwg/archive/{date}/performance-optimization/
   ```

**Communicate Progress**:
```
⏳ Generating final reports...
✓ SLO compliance validated: {PASS|PARTIAL|FAIL}
✓ Optimization summary: .aiwg/reports/optimization-summary.md
  - Overall improvement: {X}%
  - p95 latency: {before}ms → {after}ms
  - Throughput: {before} → {after} RPS
```

## Quality Gates

Before marking workflow complete, verify:
- [ ] Performance baseline established with SLOs
- [ ] Bottlenecks identified and prioritized
- [ ] Optimizations implemented with measurements
- [ ] Load tests validate improvements
- [ ] SLO compliance validated
- [ ] ROI analysis completed

## User Communication

**At start**: Confirm understanding and list deliverables

```
Understood. I'll orchestrate the performance optimization flow.

This will analyze and optimize:
- Performance bottlenecks
- Database queries
- Code efficiency
- Infrastructure configuration

Deliverables:
- Performance baseline report
- Bottleneck analysis
- Optimization plan
- Load test results
- SLO compliance report
- Optimization summary with ROI

Expected duration: 20-30 minutes.

Starting optimization workflow...
```

**During**: Update progress with metrics

```
✓ = Complete
⏳ = In progress
📈 = Improvement measured
⚠️ = Issue found
```

**At end**: Summary with results

```
─────────────────────────────────────────────
Performance Optimization Complete
─────────────────────────────────────────────

**Overall Status**: SUCCESS
**SLO Compliance**: PASS

**Performance Improvements**:
- p95 Latency: 450ms → 180ms (-60%)
- Throughput: 500 → 1200 RPS (+140%)
- Error Rate: 2.1% → 0.3% (-86%)

**Key Optimizations**:
✓ Database: Added 3 indexes, query optimization
✓ Caching: Redis layer, 85% cache hit rate
✓ Code: Async processing, algorithm improvements
✓ Infrastructure: CDN, connection pooling

**ROI Analysis**:
- Development: 3 days
- Cost Impact: -$800/month (reduced instances)
- User Impact: Page loads 2.5x faster

**Artifacts Generated**:
- Performance baseline: .aiwg/reports/performance-baseline.md
- Bottleneck analysis: .aiwg/reports/bottleneck-analysis.md
- Optimization plan: .aiwg/planning/optimization-plan.md
- Load test results: .aiwg/testing/load-test-results.md
- SLO compliance: .aiwg/reports/slo-compliance.md
- Final summary: .aiwg/reports/optimization-summary.md

**Next Steps**:
- Monitor production metrics for 7 days
- Schedule follow-up optimization cycle in 30 days
- Consider implementing observability improvements
─────────────────────────────────────────────
```

## Error Handling

**If SLO Breach Critical**:
```
❌ Critical SLO breach detected

Metric: {metric}
Current: {value}
Target: {target}
Impact: {user/business impact}

Emergency optimization required:
1. Implement quick wins immediately
2. Consider rollback if regression
3. Escalate to stakeholders

Continuing with emergency optimization protocol...
```

**If Optimization Failed**:
```
⚠️ Optimization did not improve performance

Optimization: {description}
Expected: {X}% improvement
Actual: {Y}% degradation

Actions:
1. Rolling back change
2. Re-analyzing bottleneck
3. Trying alternative approach

Documenting in lessons learned...
```

**If Load Test Failure**:
```
❌ Load test failed

Test: {scenario}
Failure: {description}
Breaking point: {metric}

Impact:
- Cannot handle expected load
- SLO targets not achievable

Recommendations:
1. Infrastructure scaling required
2. Additional optimizations needed
3. Adjust SLO targets (with stakeholder approval)
```

## Success Criteria

This orchestration succeeds when:
- [ ] Performance baseline established with clear SLOs
- [ ] Top bottlenecks identified through profiling
- [ ] Prioritized optimizations implemented
- [ ] Load tests show measurable improvement
- [ ] SLOs met or improvement plan defined
- [ ] ROI analysis shows positive impact

## Metrics to Track

**During orchestration**:
- Optimization velocity: optimizations/day
- Performance improvement rate: % improvement/optimization
- Test coverage: % of critical paths tested
- SLO compliance rate: % of SLOs met
- Error budget consumption: before vs after

## References

**Templates** (via $AIWG_ROOT):
- SLI Card: `templates/deployment/sli-card.md`
- SLO Card: `templates/deployment/slo-card.md`
- Performance Profile: `templates/analysis-design/performance-profile-card.md`
- Load Test Plan: `templates/test/load-test-plan-template.md`
- Performance Test Card: `templates/test/performance-test-card.md`
- Option Matrix: `templates/intake/option-matrix-template.md`

**Related Flows**:
- `flow-monitoring-setup` - Establish observability
- `flow-incident-response` - Handle performance incidents
- `flow-capacity-planning` - Plan for scale

**External References**:
- Site Reliability Engineering (Google)
- High Performance Browser Networking (Ilya Grigorik)