sync-tutorial-to-e2e-tests

Match tutorial script blocks to e2e pytest functions and add missing tests

226 stars

Best use case

sync-tutorial-to-e2e-tests is best used when you need a repeatable AI agent workflow instead of a one-off prompt.

Match tutorial script blocks to e2e pytest functions and add missing tests

Teams using sync-tutorial-to-e2e-tests should expect a more consistent output, faster repeated execution, less prompt rewriting.

When to use this skill

  • You want a reusable workflow that can be run more than once with consistent structure.

When not to use this skill

  • You only need a quick one-off answer and do not need a reusable workflow.
  • You cannot install or maintain the underlying files, dependencies, or repository context.

Installation

Claude Code / Cursor / Codex

$curl -o ~/.claude/skills/sync-tutorial-to-e2e-tests/SKILL.md --create-dirs "https://raw.githubusercontent.com/imbue-ai/mngr/main/.claude/skills/sync-tutorial-to-e2e-tests/SKILL.md"

Manual Installation

  1. Download SKILL.md from GitHub
  2. Place it in .claude/skills/sync-tutorial-to-e2e-tests/SKILL.md inside your project
  3. Restart your AI agent — it will auto-discover the skill

How sync-tutorial-to-e2e-tests Compares

Feature / Agentsync-tutorial-to-e2e-testsStandard Approach
Platform SupportNot specifiedLimited / Varies
Context Awareness High Baseline
Installation ComplexityUnknownN/A

Frequently Asked Questions

What does this skill do?

Match tutorial script blocks to e2e pytest functions and add missing tests

Where can I find the source code?

You can find the source code on GitHub using the link provided at the top of the page.

SKILL.md Source

Default arguments (if none provided): `libs/mngr/imbue/mngr/resources/mega_tutorial.sh libs/mngr/imbue/mngr/e2e`

Your task is to ensure that every command block in a tutorial shell script has a corresponding pytest function.

## Step 1: Run the matcher

Run the tutorial matcher script to find unmatched blocks and functions:

```bash
uv run python scripts/tutorial_matcher.py $ARGUMENTS
```

If the output says everything is matched, you are done.

## Step 2: Understand the context

Read the tutorial script file and the test directory to understand the overall structure and conventions used in the existing tests.

Pay close attention to:
- How existing test functions are structured (fixtures, assertions, setup/teardown)
- What the tutorial script is demonstrating (the commands, their arguments, expected behavior)
- The patterns used for `write_tutorial_block()` calls (the block must appear verbatim in the call argument)

## Step 3: Handle unmatched pytest functions

Handle these FIRST, before adding new tests, because some of these may pair up with unmatched script blocks.

For each pytest function that doesn't correspond to any script block, compare its `write_tutorial_block()` call against the list of unmatched script blocks. If there is a script block that mostly matches (e.g., a command was renamed, a flag was added, or a line was changed), the script block was likely modified after the test was written. In that case, update the `write_tutorial_block()` argument to exactly reproduce the current script block, and update the test logic to match the new behavior. This also resolves that script block, so it no longer needs a new test in step 4.

If no script block is even a close match, the block was removed from the script entirely. Remove the test function.

## Step 4: Add tests for remaining unmatched script blocks

After step 3, some script blocks may still lack tests.

The priority is coverage, not perfection -- a separate step will improve test quality later. What matters is that each tutorial block has a corresponding test function with the correct `write_tutorial_block()` call and at least a basic assertion.

Add tests to the appropriate existing test file, or create a new file if the blocks belong to a distinct section (e.g., `test_create_remote.py` for "CREATING AGENTS REMOTELY" blocks).

### Requirements for each test function

Each function MUST call `e2e.write_tutorial_block("""...""")` as its first statement, with the **exact** text of the script block (the matcher checks this). The block text will be dedented and stripped automatically, so indent it naturally with the surrounding Python code. Example:
```python
@pytest.mark.release
def test_foo(e2e: E2eSession, agent_name: str) -> None:
    e2e.write_tutorial_block("""
        # comment from tutorial
        mngr create my-task --some-flag
        # another comment
    """)
    result = e2e.run("mngr create my-task --some-flag")
    assert result.exit_code == 0
```

Other requirements:
- Decorate with `@pytest.mark.release`
- Use `e2e: E2eSession` as the fixture type
- Run the actual command from the block (not just `--help`)
- At least one basic assertion (exit code check is fine)
- Follow existing patterns in the directory for style and fixtures

## Step 5: Verify

Re-run the matcher to confirm everything is matched:

```bash
uv run python scripts/tutorial_matcher.py $ARGUMENTS
```

Do NOT run the tests locally -- these are e2e tests and may be too expensive to run locally. They will be validated in CI.

Related Skills

writing-ratchet-tests

226
from imbue-ai/mngr

Write ratchet tests to prevent accumulation of code anti-patterns. Use when asked to create a "ratchet test" for tracking and preventing specific code patterns (e.g., TODO comments, inline imports, broad exception handling).

writing-specs

226
from imbue-ai/mngr

Write high quality specifications or design docs for a program. Use any time you are asked to write, improve, or update specs / design docs (e.g., files in a `specs/` folder).

writing-docs

226
from imbue-ai/mngr

Write high quality, user-facing documentation. Use any time you need to write, improve, or update a significant amount of user-facing documentation (e.g., files in a "docs/" folder or README file).

wait-for-agent

226
from imbue-ai/mngr

Wait for another agent to enter WAITING state, then execute follow-up instructions

update-issues-in-repo

226
from imbue-ai/mngr

Convert a file containing identified issues into a tracked file in current_tasks/. Use after running identify-* commands to create a local record of current issues.

triage-backlog

226
from imbue-ai/mngr

Interactively triage the user's local engineering backlog file into GitHub issues. Use when the user wants to process their raw thought notes / ticket backlog into proper GitHub issues.

think-of-something-to-fix

226
from imbue-ai/mngr

Come up with good ideas about what to fix. Use when you have to fix something, but you're not sure what.

minds-dev-iterate

226
from imbue-ai/mngr

Set up and iterate on the minds app stack (desktop client, workspace server, mngr, forever-claude-template) with a running Docker agent

message-agent

226
from imbue-ai/mngr

Send a message to another mngr agent. Use when you need to communicate with a peer agent.

identify-style-issues

226
from imbue-ai/mngr

Identify divergences from the style guide in the $1 library

identify-outdated-docstrings

226
from imbue-ai/mngr

Identify outdated docstrings in the $1 library

identify-inconsistencies

226
from imbue-ai/mngr

Identify inconsistencies in the $1 library