experimental-design

# Experimental Design Skill

Design rigorous, reproducible experiments across scientific disciplines.

## When to Use

- "Design an experiment to test..."
- "How many samples do I need?"
- "What controls should I include?"
- "Help me plan a clinical trial"
- "Is this experimental design valid?"
- Power analysis and sample size calculation

## When NOT to Use

- Running the actual experiment (use code-execution)
- Analyzing collected data (use scipy-analysis + statsmodels-stats)
- Writing up results (use paper-writing)
- Literature review (use literature-search)

## Design Components

### 1. Research Question and Hypotheses
- State clear, testable research question
- Formulate H0 and H1 (see hypothesis-gen skill)
- Define primary and secondary outcomes

### 2. Study Design Selection

| Design | When to Use | Strengths | Weaknesses |
|--------|------------|-----------|------------|
| RCT | Causal inference needed | Gold standard causality | Expensive, ethical limits |
| Factorial | Multiple factors | Tests interactions | Complex analysis |
| Crossover | Within-subject comparison | Reduced variability | Carryover effects |
| Quasi-experimental | Randomization impossible | Practical feasibility | Weaker causality |
| Observational (cohort) | Long-term outcomes | Natural setting | Confounding |
| Case-control | Rare outcomes | Efficient for rare events | Recall bias |

### 3. Power Analysis

```python
# Sample size calculation template (using scipy/statsmodels)
from statsmodels.stats.power import TTestIndPower
analysis = TTestIndPower()
n = analysis.solve_power(
    effect_size=0.5,   # Cohen's d (small=0.2, medium=0.5, large=0.8)
    alpha=0.05,         # Significance level
    power=0.80,         # Statistical power (commonly 0.80 or 0.90)
    ratio=1.0,          # Ratio of group sizes (n2/n1)
    alternative='two-sided'
)
print(f"Required sample size per group: {int(n) + 1}")
```

Key parameters:
- **Effect size**: Expected magnitude of difference
- **Alpha**: Type I error rate (usually 0.05)
- **Power**: 1 - Type II error rate (usually 0.80-0.95)
- **Attrition**: Add 10-20% for expected dropout

### 4. Variable Control

- **Independent variables**: What you manipulate
- **Dependent variables**: What you measure
- **Confounding variables**: What could bias results
- **Control strategies**: Randomization, blocking, matching, blinding

### 5. Randomization

- Simple randomization (coin flip)
- Block randomization (balanced groups)
- Stratified randomization (balance key covariates)
- Cluster randomization (group-level assignment)

### 6. Blinding

- Single-blind: Participants unaware of assignment
- Double-blind: Participants and researchers unaware
- Triple-blind: Including data analysts

## Reproducibility Checklist

- [ ] Protocol pre-registered (OSF, ClinicalTrials.gov, PROSPERO)
- [ ] All materials/reagents specified with catalog numbers
- [ ] Detailed step-by-step procedure written
- [ ] Statistical analysis plan pre-specified
- [ ] Data management plan documented
- [ ] Raw data sharing plan established
- [ ] Code availability ensured
- [ ] Sample size justified with power analysis
- [ ] Randomization method specified
- [ ] Blinding procedures documented
- [ ] Inclusion/exclusion criteria defined
- [ ] Primary endpoint pre-specified

## Output Format

```
## Experimental Design: [Title]

**Research Question**: [Clear question]
**Design Type**: [RCT/Factorial/etc.]

### Participants/Samples
- Population: [target population]
- Inclusion: [criteria]
- Exclusion: [criteria]
- Sample Size: N=[total] ([n] per group) — Power=[X], alpha=[X], effect=[X]

### Groups
- Experimental: [treatment description]
- Control: [control description]
- Blinding: [single/double/triple/none]

### Variables
- IV: [variables]
- DV: [primary + secondary outcomes]
- Controls: [confounds and how addressed]

### Procedure
1. [Step-by-step protocol]

### Analysis Plan
- Primary: [statistical test]
- Secondary: [additional analyses]
- Multiple comparison correction: [method]

### Timeline
- [Phase 1]: [duration]
- [Phase 2]: [duration]

### Ethics
- IRB/IACUC requirements: [details]
- Consent procedure: [details]
```