Codex

auto-test-execution

Automatically execute tests when code-generating agents modify source files, enforcing the execute-before-return pattern

104 stars

Best use case

auto-test-execution is best used when you need a repeatable AI agent workflow instead of a one-off prompt.

It is a strong fit for teams already working in Codex.

Automatically execute tests when code-generating agents modify source files, enforcing the execute-before-return pattern

Teams using auto-test-execution should expect a more consistent output, faster repeated execution, less prompt rewriting.

When to use this skill

  • You want a reusable workflow that can be run more than once with consistent structure.

When not to use this skill

  • You only need a quick one-off answer and do not need a reusable workflow.
  • You cannot install or maintain the underlying files, dependencies, or repository context.

Installation

Claude Code / Cursor / Codex

$curl -o ~/.claude/skills/auto-test-execution/SKILL.md --create-dirs "https://raw.githubusercontent.com/jmagly/aiwg/main/.agents/skills/auto-test-execution/SKILL.md"

Manual Installation

  1. Download SKILL.md from GitHub
  2. Place it in .claude/skills/auto-test-execution/SKILL.md inside your project
  3. Restart your AI agent — it will auto-discover the skill

How auto-test-execution Compares

Feature / Agentauto-test-executionStandard Approach
Platform SupportCodexLimited / Varies
Context Awareness High Baseline
Installation ComplexityUnknownN/A

Frequently Asked Questions

What does this skill do?

Automatically execute tests when code-generating agents modify source files, enforcing the execute-before-return pattern

Which AI agents support this skill?

This skill is designed for Codex.

Where can I find the source code?

You can find the source code on GitHub using the link provided at the top of the page.

Related Guides

SKILL.md Source

# auto-test-execution

Automatically execute tests when code-generating agents write to source files, enforcing the execute-before-return pattern.

## Triggers


Primary phrases matched automatically from skill description. No additional alternate expressions defined.

## Purpose

This skill enforces the MetaGPT executable feedback pattern: code-generating agents must execute tests before returning results to the user. It activates automatically when agents modify source code files.

## Behavior

When triggered, this skill:

1. **Detect modified files**:
   - Track which source files the agent has written to
   - Identify the relevant test framework

2. **Find related tests**:
   - Look for test files matching the modified source
   - Convention: `src/foo/bar.ts` -> `test/unit/foo/bar.test.ts`
   - If no tests exist, prompt agent to generate them

3. **Execute tests**:
   - Run the project's test command focused on relevant tests
   - Capture results: passed, failed, errors

4. **Handle results**:
   - All pass: Allow agent to return results
   - Failures: Trigger debug-and-retry loop (max 3 attempts)
   - Persistent failures: Escalate with debug memory context

5. **Update debug memory**:
   - Record session in `.aiwg/ralph/debug-memory/sessions/`
   - Extract patterns for future reference

## Activation Conditions

```yaml
activation:
  always_active_for:
    - software-implementer
    - debugger
    - test-engineer

  triggered_by:
    - file_write:
        patterns:
          - "src/**/*.ts"
          - "src/**/*.js"
          - "src/**/*.py"
          - "**/*.go"
          - "**/*.rs"

  skip_when:
    - test_files_only: true
    - documentation_only: true
    - configuration_only: true
```

## Integration

This skill uses:
- `project-awareness`: Detect test framework and configuration
- Debug memory at `.aiwg/ralph/debug-memory/` for pattern learning

## References

- @$AIWG_ROOT/agentic/code/frameworks/sdlc-complete/rules/executable-feedback.md - Feedback rules
- @$AIWG_ROOT/agentic/code/addons/ralph/docs/executable-feedback-guide.md - Guide
- @$AIWG_ROOT/agentic/code/addons/ralph/schemas/debug-memory.yaml - Memory schema
- @.aiwg/research/findings/REF-013-metagpt.md - Research foundation

Related Skills

pytest-runner

104
from jmagly/aiwg

Execute Python tests with pytest, supporting fixtures, markers, coverage, and parallel execution. Use for Python test automation.

vitest-runner

104
from jmagly/aiwg

Execute JavaScript/TypeScript tests with Vitest, supporting coverage, watch mode, and parallel execution. Use for JS/TS test automation.

test-sync

104
from jmagly/aiwg

Detect orphaned tests, obsolete assertions, and test-code misalignment. Use for test suite maintenance, cleanup, and traceability validation.

Codex

test-coverage

104
from jmagly/aiwg

Analyze test coverage reports, identify gaps by priority, and recommend test improvements with trend tracking

Codex

mutation-test

104
from jmagly/aiwg

Run mutation testing to validate test quality beyond code coverage. Use when assessing test effectiveness, finding weak tests, or validating test suite quality.

Codex

issue-auto-sync

104
from jmagly/aiwg

Detect issue references in commits and artifacts and automatically update or close linked tracker issues

Codex

generate-tests

104
from jmagly/aiwg

Generate comprehensive test suite for $ARGUMENTS following project testing conventions and best practices.

Codex

flow-test-strategy-execution

104
from jmagly/aiwg

Orchestrate comprehensive test strategy with test suite execution, coverage validation, defect triage, and regression analysis

Codex

execution-mode

104
from jmagly/aiwg

Set or report the reproducibility mode governing how AIWG workflows execute, from standard to fully auditable

Codex

devkit-test

104
from jmagly/aiwg

Auto-fix discoverable issues

Codex

auto-provenance

104
from jmagly/aiwg

Generate W3C PROV-compliant provenance records automatically when agents create or modify artifacts

Codex

aiwg-orchestrate

104
from jmagly/aiwg

Route structured artifact work to AIWG workflows via MCP with zero parent context cost