academic-reviewer

# Academic Manuscript Reviewer

This skill provides expert guidance for reviewing academic manuscripts with methodological rigor, focusing on causal identification, research design, and constructive critique.

## When to Use This Skill

Use this skill when:
- Asked to review an academic paper or manuscript
- Evaluating a research design or empirical strategy
- Providing feedback on causal identification approaches
- Critiquing experimental designs (field, survey, or lab experiments)
- Assessing quasi-experimental methods (DiD, RDD, IV, etc.)

## Review Process

Follow these steps to conduct a thorough review:

### 1. Initial Reading and Understanding

- Read the paper carefully to understand the research question, argument, methods, and findings
- Identify the core empirical strategy and identification approach
- Consider the theoretical contribution and stakes

### 2. Consult Reference Materials

Before writing the review, consult these reference documents as needed:

- **[Reviewer Profile](references/reviewer-profile.md)**: Detailed guidance on methodological approach, common critiques, and reviewing philosophy
- **[Review Structure](references/review-structure.md)**: Recommended structure for organizing your review
- **[Checklists](references/checklists.md)**: Manuscript checklist and reviewer self-checklist to ensure thorough evaluation
- **[Review Examples](references/review-examples.md)**: Example reviews demonstrating the style and approach

### 3. Evaluate Core Elements

**Identification Strategy (Primary Focus)**:
- Assess the credibility of causal identification
- Evaluate whether identifying assumptions are justified
- Check for plausible alternative explanations

**Research Design**:
- For DiD: check parallel trends, treatment timing endogeneity, estimator choice
- For RDD: check covariate balance, bandwidth selection, interpretation
- For IV: check exclusion restriction, instrument strength
- For experiments: check external validity, statistical power, realistic scenarios

**Statistical Inference**:
- Check subgroup analysis (are interactions formally tested?)
- Assess multiple comparison issues
- Evaluate clustering of standard errors
- Identify "bad controls" or post-treatment bias

**Theory-Empirics Link**:
- Can the design distinguish the proposed mechanism from alternatives?
- Is there a clear theoretical payoff (especially for top journals)?

### 4. Structure Your Review

Follow this format:

1. **Metadata**: Journal, dates, recommendation
2. **Summary**: Concise paragraph (~100 words) of paper's core elements
3. **Overall Assessment**: High-level verdict with strengths before reservations
4. **Major Issues**: 2-4 fundamental flaws (numbered list)
5. **Minor Issues**: Less critical suggestions (numbered list)
6. **Recommendation**: Clear decision justified by major issues

### 5. Maintain Appropriate Tone

- Be constructive but firm on methodological issues
- Praise genuine strengths and author efforts
- Provide actionable suggestions when possible
- Avoid ad hominem criticism
- Distinguish between fatal flaws and suggestions for improvement

## Key Principles

**Primacy of Identification**: The quality of causal identification strategy is paramount. Clever questions or novel data cannot compensate for flawed identification.

**Methodological Fluency**: Cite relevant methodological literature (Goodman-Bacon on DiD, Cattaneo et al. on RDD, Hainmueller et al. on conjoints, etc.).

**Skepticism of Survey Experiments**: Be particularly skeptical of survey experiments on sensitive topics (corruption, clientelism) due to social desirability bias and external validity concerns.

**Subgroup Analysis**: Consistently check whether apparent differences across subgroups are formally tested with interaction terms.

**Statistical Power**: Assess whether studies (especially field experiments) are adequately powered to detect plausible effect sizes.

## Self-Check Before Submitting

Use the reviewer self-checklist (in references/checklists.md) to ensure:
- Requests for controls are justified by confounding arguments
- Literature suggestions explain why citations are essential
- Comments about framing relate to scientific validity
- No unnecessary or ad hominem criticism
- Positive reviews adequately explain contributions