analyze-pr-performance

# Analyze PR Performance

Analyze code review pipeline performance for a specific PR.

## Usage

Run the analyze-pr-performance CLI script with the provided arguments:

```bash
npx ts-node scripts/analyze-pr-performance.cli.ts $ARGUMENTS
```

## Arguments

- `prNumber` (required): The PR number to analyze
- `orgId` (required): The organization ID

## Options

- `--days=N`: Number of days to search back (default: 7)
- `--legacy`: Also search in legacy collection (observability_logs)
- `--env=PATH`: Path to .env file (e.g., `--env=.env.prod`)

## Examples

```bash
# Analyze performance for PR #558 in production
/analyze-pr-performance 558 04bd288b-595a-4ee1-87cd-8bbbdc312b3c --env=.env.prod

# Analyze with extended date range
/analyze-pr-performance 723 97442318-9d2a-496b-a0d2-b45fb --days=14 --env=.env.prod

# Analyze with legacy logs included
/analyze-pr-performance 701 97442318-9d2a-496b-a0d2-b45fb --legacy --env=.env.prod
```

## What it analyzes

1. **Pipeline identification**: Finds the pipelineId and correlationId for the PR

2. **Stage times**: Shows duration of each pipeline stage:
   - ValidateNewCommitsStage
   - ResolveConfigStage
   - FetchChangedFilesStage
   - PRLevelReviewStage
   - FileAnalysisStage
   - CreateFileCommentsStage
   - UpdateCommentsAndGenerateSummaryStage
   - And all other stages...

3. **LLM calls**: Details of each LLM operation:
   - Operation name (analyzeCodeWithAI, selectReviewMode, kodyRulesAnalyzeCodeWithAI, etc.)
   - Duration
   - Model used
   - Token counts (input/output)

4. **Summary metrics**:
   - Total pipeline duration
   - Total LLM calls count
   - Total tokens (input/output)
   - Slow calls count (> 60s)
   - Models used

5. **Bottlenecks**: Highlights stages and LLM calls taking > 60 seconds

6. **Pipeline status**: Whether the pipeline completed, failed, or is unknown

## Output

The script outputs:
- Pipeline and correlation IDs
- Organization and repository info
- Stage times table with duration and percentage of total
- LLM calls table with model, tokens, and duration
- Summary metrics
- Bottleneck list (stages and LLM calls > 60s)
- Final pipeline status

## How to Respond

- Identify the slowest stages and explain why they might be slow
- Look for LLM calls that are taking too long (> 2 minutes is concerning)
- Check if multiple slow LLM calls are running sequentially vs in parallel
- Note any patterns (e.g., all selectReviewMode calls are slow = possible model issue)
- Suggest potential optimizations if bottlenecks are clear
- If FileAnalysisStage is slow, it's usually due to many files or large files being analyzed
- If PRLevelReviewStage is slow, check the KodyRules and PR-level analysis calls
- Compare token counts to durations - high tokens with proportional time is expected, low tokens with high time indicates API latency issues
- Report the pipeline status and check if it completed successfully