experiment-design

# Experiment Design

Scientific experiment planning, power analysis, and protocol development.

## Design Selection Guide

| Research Question | Recommended Design |
|---|---|
| Does X cause Y? | RCT (gold standard) |
| Does X cause Y? (can't randomize) | Quasi-experiment, natural experiment |
| How do factors interact? | Factorial design |
| Which version performs better? | A/B test |
| What is the prevalence/association? | Cross-sectional survey |
| How does outcome change over time? | Longitudinal / cohort study |
| What is the lived experience? | Qualitative (interviews, ethnography) |
| Does intervention work in practice? | Pragmatic trial |

## Power Analysis & Sample Size

```python
source /Users/zhangmingda/clawd/.venv/bin/activate
python3 << 'EOF'
from scipy import stats
import numpy as np

# --- Two-sample t-test ---
def sample_size_ttest(effect_size, alpha=0.05, power=0.80):
    """Cohen's d effect sizes: small=0.2, medium=0.5, large=0.8"""
    from scipy.stats import norm
    z_alpha = norm.ppf(1 - alpha/2)
    z_beta = norm.ppf(power)
    n = 2 * ((z_alpha + z_beta) / effect_size) ** 2
    return int(np.ceil(n))

# --- Chi-square test ---
def sample_size_chi2(effect_size, alpha=0.05, power=0.80, df=1):
    """Cohen's w effect sizes: small=0.1, medium=0.3, large=0.5"""
    from scipy.stats import norm, chi2
    z_beta = norm.ppf(power)
    z_alpha = norm.ppf(1 - alpha)
    n = ((z_alpha + z_beta) / effect_size) ** 2
    return int(np.ceil(n))

# --- Correlation ---
def sample_size_correlation(r, alpha=0.05, power=0.80):
    from scipy.stats import norm
    z_alpha = norm.ppf(1 - alpha/2)
    z_beta = norm.ppf(power)
    z_r = 0.5 * np.log((1+r)/(1-r))  # Fisher's z
    n = ((z_alpha + z_beta) / z_r) ** 2 + 3
    return int(np.ceil(n))

# Examples
print(f"t-test (d=0.5): n={sample_size_ttest(0.5)} per group")
print(f"t-test (d=0.3): n={sample_size_ttest(0.3)} per group")
print(f"Chi-square (w=0.3): n={sample_size_chi2(0.3)}")
print(f"Correlation (r=0.3): n={sample_size_correlation(0.3)}")
EOF
```

## Key Design Principles

### Controls
- **Positive control**: Known to produce effect (validates method works)
- **Negative control**: Known to produce no effect (validates baseline)
- **Placebo control**: Inert treatment (controls for expectation effects)
- **Active control**: Existing standard treatment (for superiority/non-inferiority)

### Randomization
- **Simple**: Coin flip / random number
- **Block**: Ensures equal groups per block
- **Stratified**: Randomize within strata (age, sex, severity)
- **Cluster**: Randomize groups, not individuals

### Blinding
- **Single-blind**: Participants don't know assignment
- **Double-blind**: Participants and researchers don't know
- **Triple-blind**: Participants, researchers, and analysts don't know

### Bias Mitigation
| Bias | Mitigation |
|------|-----------|
| Selection bias | Random sampling, clear inclusion criteria |
| Allocation bias | Random assignment, concealed allocation |
| Performance bias | Blinding, standardized protocols |
| Detection bias | Blinded outcome assessment |
| Attrition bias | ITT analysis, minimize dropout |
| Reporting bias | Pre-registration, analysis plan |

## Study Protocol Template

```markdown
# Study Protocol: [Title]

## 1. Background & Rationale
## 2. Objectives & Hypotheses
  - Primary: 
  - Secondary:
## 3. Study Design
  - Type: [RCT / quasi-experiment / observational / ...]
  - Duration:
## 4. Participants
  - Population:
  - Inclusion criteria:
  - Exclusion criteria:
  - Sample size: N = [calculated], power = 0.80, α = 0.05
## 5. Intervention / Exposure
## 6. Outcome Measures
  - Primary:
  - Secondary:
## 7. Randomization & Blinding
## 8. Data Collection Procedures
## 9. Statistical Analysis Plan
  - Primary analysis:
  - Secondary analyses:
  - Handling of missing data:
## 10. Ethical Considerations
  - IRB/Ethics approval:
  - Informed consent:
  - Data privacy:
## 11. Timeline
## 12. Budget
```

## Pre-registration

Recommend pre-registration for confirmatory studies:
- **OSF**: osf.io (general)
- **ClinicalTrials.gov**: clinical trials
- **PROSPERO**: systematic reviews
- **AsPredicted**: aspredicted.org (quick)

## Tips
- Always justify sample size with power analysis
- Pre-register hypotheses and analysis plan
- Plan for 10-20% attrition in sample size calculation
- Document all deviations from protocol
- Consider pilot study for novel methods