project-referee

Critiques ML conference papers with reviewer-style feedback. Use when users want to anticipate reviewer concerns, identify weaknesses, check claim-evidence gaps, or find missing citations.

148 stars

Best use case

project-referee is best used when you need a repeatable AI agent workflow instead of a one-off prompt.

Critiques ML conference papers with reviewer-style feedback. Use when users want to anticipate reviewer concerns, identify weaknesses, check claim-evidence gaps, or find missing citations.

Teams using project-referee should expect a more consistent output, faster repeated execution, less prompt rewriting.

When to use this skill

  • You want a reusable workflow that can be run more than once with consistent structure.

When not to use this skill

  • You only need a quick one-off answer and do not need a reusable workflow.
  • You cannot install or maintain the underlying files, dependencies, or repository context.

Installation

Claude Code / Cursor / Codex

$curl -o ~/.claude/skills/project-referee/SKILL.md --create-dirs "https://raw.githubusercontent.com/sundial-org/skills/main/skills/project-referee/SKILL.md"

Manual Installation

  1. Download SKILL.md from GitHub
  2. Place it in .claude/skills/project-referee/SKILL.md inside your project
  3. Restart your AI agent — it will auto-discover the skill

How project-referee Compares

Feature / Agentproject-refereeStandard Approach
Platform SupportNot specifiedLimited / Varies
Context Awareness High Baseline
Installation ComplexityUnknownN/A

Frequently Asked Questions

What does this skill do?

Critiques ML conference papers with reviewer-style feedback. Use when users want to anticipate reviewer concerns, identify weaknesses, check claim-evidence gaps, or find missing citations.

Where can I find the source code?

You can find the source code on GitHub using the link provided at the top of the page.

SKILL.md Source

# Project Referee

## Review Workflow

### Step 1: Assess Manuscript Stage

Ask or determine the manuscript stage:
- **Early draft**: Focus on novelty, feasibility, relevance, structure
- **Mid-stage draft**: Focus on development trajectory, completeness, preliminary results
- **Final submission**: Apply full review criteria (see [references/reviewer-instructions.md](references/reviewer-instructions.md))

### Step 2: Stage-Specific Review

**Early Draft Protocol:**
1. Assess core contribution novelty and significance
2. Evaluate method feasibility with current technology
3. Check alignment with venue scope
4. Review logical structure and completeness
5. Identify major missing components
6. Suggest high-level direction

**Mid-Stage Protocol:**
1. Evaluate development progress
2. Check section completeness
3. Assess preliminary results against claims
4. Review literature coverage
5. Identify specific weaknesses
6. Provide targeted refinements

**Final Submission Protocol:**
Apply complete review process. See [references/reviewer-instructions.md](references/reviewer-instructions.md) for the full reviewer form.

### Step 3: Search for Related Work

Proactively search for missing citations:
- Recent work in the paper's domain
- Papers with similar methods or contributions
- Relevant benchmarks and baselines
- Use WebSearch to find recent publications

### Step 4: Provide Structured Feedback

Use the output format in [references/reviewer-instructions.md](references/reviewer-instructions.md):
- Summary, Strengths, Weaknesses, Questions
- Soundness (1-4), Presentation (1-4), Contribution (1-4)
- Overall Score (1-10), Confidence (1-5)

## Neuro-Symbolic Papers

For papers combining LLMs with symbolic reasoning, see [references/neuro-symbolic-review-criteria.md](references/neuro-symbolic-review-criteria.md) for additional criteria:
- Symbolic formulation quality
- Solver selection and integration
- Faithfulness and verification
- Comparison to baselines (Logic-LM, LINC)

## References

- [references/reviewer-instructions.md](references/reviewer-instructions.md) - Full reviewer form
- [references/neuro-symbolic-review-criteria.md](references/neuro-symbolic-review-criteria.md) - Neuro-symbolic specific criteria

Related Skills

cs448b-visualization

148
from sundial-org/skills

Data visualization design based on Stanford CS448B. Use for: (1) choosing chart types, (2) selecting visual encodings, (3) critiquing visualizations, (4) building D3.js visualizations, (5) designing interactions/animations, (6) choosing colors, (7) visualizing networks, (8) visualizing text. Covers Bertin, Mackinlay, Cleveland & McGill.

training-data-curation

148
from sundial-org/skills

Guidelines for creating high-quality datasets for LLM post-training (SFT/DPO/RLHF). Use when preparing data for fine-tuning, evaluating data quality, or designing data collection strategies.

tinker

148
from sundial-org/skills

Fine-tune LLMs using the Tinker API. Covers supervised fine-tuning, reinforcement learning, LoRA training, vision-language models, and both high-level Cookbook patterns and low-level API usage.

tinker-training-cost

148
from sundial-org/skills

Calculate training costs for Tinker fine-tuning jobs. Use when estimating costs for Tinker LLM training, counting tokens in datasets, or comparing Tinker model training prices. Tokenizes datasets using the correct model tokenizer and provides accurate cost estimates.

skill

148
from sundial-org/skills

Find, install, create, improve, and publish AI agent skills through the Sundial ecosystem. Use when the user wants to find or search for skills, install a skill, create a new skill, improve or evaluate an existing skill, or publish a skill to Sundial Hub. Trigger phrases include "find a skill", "install skill", "create a skill", "make a skill", "improve this skill", "evaluate skill", "publish skill", "push skill", "search for skills".

skill-to-card

148
from sundial-org/skills

End-to-end workflow that creates a skill from a description and attached files, publishes it to Sundial as a private skill, generates a trading card (front + back with QR code), and sends it to a printer. Use when the user wants to create a skill and get a printed trading card, or says "skill to card", "create and print a skill card", "make me a skill with a card".

neuro-symbolic-reasoning

148
from sundial-org/skills

Neuro-symbolic AI combining LLMs with symbolic solvers. Use when exploring neuro-symbolic approaches (ideation, no code) or implementing solver integrations (code).

icml-reviewer

148
from sundial-org/skills

Paper reviewer that evaluates machine learning research projects following official ICML reviewer guidelines. Provides comprehensive reviews with actionable feedback across all key dimensions: claims/evidence, relation to prior work, originality, significance, clarity, and reproducibility. Also provides formative feedback on incomplete drafts, proposals, and research code repositories. MANDATORY TRIGGERS: review paper, ICML review, paper review, evaluate paper, research paper feedback, ML paper review, conference review, academic review, paper critique, NeurIPS review, ICLR review, project proposal, research proposal, paper draft, early feedback, incomplete paper, work in progress, WIP review, review repo, review codebase, research project review

cs-research-methodology

148
from sundial-org/skills

Conduct a literature review and develop a CS research proposal. Use when asked to review a research area, find gaps in existing work, and propose a novel research contribution. The output is a research proposal identifying an assumption to challenge (the "bit flip") and how to validate it.

commit-splitter

148
from sundial-org/skills

Split large sets of uncommitted changes into logical, well-organized commits. Use when the user has many uncommitted changes and wants structured commits, or proactively suggest when detecting a large diff that would benefit from splitting.

codex

148
from sundial-org/skills

Run OpenAI's Codex CLI agent in non-interactive mode using `codex exec`. Use when delegating coding tasks to Codex, running Codex in scripts/automation, or when needing a second agent to work on a task in parallel.

ai-co-scientist

148
from sundial-org/skills

Transform Claude Code into an AI Scientist that orchestrates research workflows using tree-based hypothesis exploration. Triggers on "research project", "scientific experiment", "run experiments", "AI scientist", "tree search experimentation", "systematic study".