llm-tuning-patterns

# LLM Tuning Patterns

Evidence-based patterns for configuring LLM parameters, based on APOLLO and Godel-Prover research.

## Pattern

Different tasks require different LLM configurations. Use these evidence-based settings.

## Theorem Proving / Formal Reasoning

Based on APOLLO parity analysis:

| Parameter | Value | Rationale |
|-----------|-------|-----------|
| max_tokens | 4096 | Proofs need space for chain-of-thought |
| temperature | 0.6 | Higher creativity for tactic exploration |
| top_p | 0.95 | Allow diverse proof paths |

### Proof Plan Prompt

Always request a proof plan before tactics:

```
Given the theorem to prove:
[theorem statement]

First, write a high-level proof plan explaining your approach.
Then, suggest Lean 4 tactics to implement each step.
```

The proof plan (chain-of-thought) significantly improves tactic quality.

### Parallel Sampling

For hard proofs, use parallel sampling:
- Generate N=8-32 candidate proof attempts
- Use best-of-N selection
- Each sample at temperature 0.6-0.8

## Code Generation

| Parameter | Value | Rationale |
|-----------|-------|-----------|
| max_tokens | 2048 | Sufficient for most functions |
| temperature | 0.2-0.4 | Prefer deterministic output |

## Creative / Exploration Tasks

| Parameter | Value | Rationale |
|-----------|-------|-----------|
| max_tokens | 4096 | Space for exploration |
| temperature | 0.8-1.0 | Maximum creativity |

## Anti-Patterns

- **Too low tokens for proofs**: 512 tokens truncates chain-of-thought
- **Too low temperature for proofs**: 0.2 misses creative tactic paths
- **No proof plan**: Jumping to tactics without planning reduces success rate

## Source Sessions

- This session: APOLLO parity - increased max_tokens 512->4096, temp 0.2->0.6
- This session: Added proof plan prompt for chain-of-thought before tactics