app-comprehensive-test-generator

Skill purpose

This Skill analyzes an application's codebase to enumerate all practical user-flow and edge-case scenarios (starting from a prioritized sample set), writes human-readable scenario .md files, automatically or manually executes those scenarios using available MCP(s) or by provisioning recommended MCP stubs, validates results, and compiles a consolidated QA report (app.qa.report.md) describing failures, reproducible steps, severity, and suggested fixes.















Step-by-step instructions Claude must follow

1. Gather context
  - Ask the user for repository location, runtime environment, available MCP connections, authentication details, and which sample scope to start with (default: core user flows: sign-up, login, search/browse, add-to-cart, checkout/payment).
  - If code is provided, fetch and index relevant files (routes/controllers, API endpoints, UI flows, business logic, tests). If not provided, request access or a zip.

2. Static analysis and feature extraction
  - Parse source code to extract endpoints, routes, UI screens, form fields, auth flows, third-party integrations, database operations, and configuration flags.
  - Build a feature map listing actions (e.g., create-account, login, forgot-password, search, filter, sort, add-to-cart, apply-coupon, checkout), inputs, preconditions, and side effects.

3. Scenario enumeration
  - For chosen sample scope, generate scenarios covering:
    - Happy path(s)
    - Input boundary conditions (empty, long, invalid formats)
    - Authorization/permission variations (unauthenticated, low-privilege user)
    - Error and retry flows (network failure, service timeout, DB error)
    - Integration edge cases (payment declines, third-party API rate-limit)
    - Concurrency and state races where applicable
  - Use reasonable combinatorial pruning to avoid explosion: prioritize by user impact and probability. Default: enumerate full set for core flows and provide counts.

4. Produce scenario .md files
  - For each scenario, create a .md file in a scenarios/ directory with a consistent template:
    - Title
    - Scope and priority
    - Preconditions (state, test data)
    - Steps (precise reproducible steps, API requests or UI actions)
    - Expected result / acceptance criteria
    - Cleanup steps
  - Number or tag scenarios for traceability (e.g., S001-signup-happy, S002-signup-invalid-email).

5. Select or provision MCP(s)
  - Detect connected MCP(s) and their capabilities (API-driven, headless browser, device farm). If none or insufficient, recommend and optionally generate a minimal MCP stub/config (e.g., Playwright/Playwright config, Cypress project, Postman collection, or a Python test harness) tailored to the app.
  - Ask user permission before creating new MCP artifacts in the repo.

6. Test execution (manual trigger)
  - Provide a command or CI job to run a selected scenario or the full suite (examples: npm run test:scenario S001, python tests/run.py --scenario S001, playwright test --project=...).
  - When user triggers execution, run scenarios via selected MCP(s) and capture outputs: logs, screenshots, HTTP traces, DB diffs, exit codes, timing.

7. Result validation and triage
  - Compare actual results to expected acceptance criteria.
  - For each failing scenario, collect: failure summary, raw evidence (logs, request/response bodies, screenshots), reproducible steps, severity (blocker/major/minor), likely root cause hints, and suggested fix or mitigation.

8. Produce app.qa.report.md
  - Summarize run: date/time, environment, MCPs used, scenarios executed, pass/fail counts.
  - For each failing scenario include:
    - Scenario id & title
    - Short description
    - Steps to reproduce (concise)
    - Evidence links or inline snippets
    - Severity and suggested fix
  - Attach or link generated scenario .md files and any test artifacts (screenshots, log files).

9. Iteration and expansion
  - Offer to expand coverage gradually: from sample scope to entire app, using feedback to refine pruning and priority.
  - Maintain scenario files and test harness in repo so they can be run by developers.

Usage examples

- Example 1: User provides repo and asks to start with core flows
  - Claude: analyzes repo, extracts features, creates scenarios/S001-signup-happy.md ... S010-checkout-payment-decline.md, provisions a Playwright config, describes commands to run tests, and waits for manual run. After run, Claude produces app.qa.report.md summarizing results.

- Example 2: User has an MCP connected (API-driven test runner)
  - Claude: inspects MCP capabilities, maps scenarios to MCP tasks, executes selected scenarios, collects logs and screenshots, produces app.qa.report.md with failures.

- Example 3: User wants only scenario generation
  - Claude: runs steps 1–4, outputs scenarios/*.md files and a summary, but does not provision or execute tests until prompted.

Scenario .md template (example)

Title: S001 - Signup: happy path
Priority: P0
Preconditions:
- Clean DB or test account seed
- Email delivery stubbed
Steps:
1. Go to POST /api/signup with payload {email: user@example.com, password: P@ssw0rd}
2. Confirm email via GET /api/confirm?token=... (use token from intercepted emails)
Expected:
- 201 Created with user id
- User can login at POST /api/login
Cleanup:
- Delete created user from test DB

Best practices

- Start small: run core flows first and confirm stability before expanding to full-surface tests.
- Keep scenario .md files human-readable and small; reference test data factories.
- Use environment isolation (test DB, feature flags off) to avoid flakiness and destructive side effects.
- Collect rich evidence (HTTP traces, DB snapshots, screenshots) to speed triage.
- Tag scenarios by area and priority to allow selective execution (smoke/regression/extended).
- Limit combinatorial explosion with heuristics: focus on high-impact permutations first.

Commands and integrations suggestions

- Playwright (E2E browser): playwright.config.js, tests/scenarios/*.spec.ts
- Cypress: cypress/e2e/scenarios/*.cy.js
- API harness: tests/api/scenarios/*.py using requests and pytest paramization
- Postman: collection.json with scenario folders and example environments

Permissions and safety

- Request explicit permission before creating or modifying files in the repo or provisioning new MCP artifacts.
- By default, operate in a non-destructive test mode (use test databases, mocked payment gateways).

When to ask clarifying questions

- If repo access or environment details are missing, ask for them before analysis.
- If the user wants a different starting scope or automatic scheduling (CI), confirm preferred behavior.

Related files or templates (optional to generate on user approval)

- A minimal Playwright starter (playwright.config.js) and example test harness can be generated upon request.