opentestai

# OpenTestAI - Open Source AI Testing Agents

Created and open-sourced by [Testers.AI](https://testers.ai)

Analyze application artifacts (screenshots, network logs, console logs, DOM) using **33 specialized AI testing agent profiles**, **diverse user persona panels**, and **automated test case generation**. Each agent is an expert in a specific testing domain. Issues are reported only when there is **high confidence** they are real bugs. Persona feedback provides UX/product insight from the perspective of diverse simulated users. Test case generation creates comprehensive, prioritized test suites.

> **Modular version** — This is the lightweight core file. Detailed content is split across companion files in this directory:
>
> - **`tester-profiles.md`** — All 33 tester profiles with IDs, images, check types, expertise, and full prompts
> - **`bug-detection.md`** — Bug detection mode: prompt templates, artifact instructions, JSON schema, output templates (chat/MD/HTML), deduplication, and STEP 4-5 logic
> - **`persona-mode.md`** — Persona feedback mode: persona images, analysis prompt, JSON schema, output templates (chat/MD/HTML)
> - **`testcase-mode.md`** — Test case generation mode: generation prompt, JSON schema, output templates (chat/MD/HTML)
> - **`output-config.md`** — Shared output configuration: filename conventions, screenshot embedding, auto-open, end-of-run summary
>
> Read ALL companion files before executing any mode.

---

## Three Modes

This skill has three modes that can be run independently or in any combination:

1. **Bug Detection Mode** — Find high-confidence bugs using 33 specialized testers
2. **Persona Feedback Mode** — Generate diverse user persona feedback for UX/product insight
3. **Test Case Generation Mode** — Create comprehensive, prioritized test cases for any page

If the user asks to "check for bugs", "test this page", "find issues", etc. → run **Bug Detection Mode**.
If the user asks for "persona feedback", "user feedback", "what would users think", "persona panel", etc. → run **Persona Feedback Mode**.
If the user asks to "generate test cases", "create tests", "write test cases", "test suite", etc. → run **Test Case Generation Mode**.
If the user asks for "full analysis" → run **all three modes**.

## Speed Modes (Bug Detection)

Bug detection supports two speed modes:

| Mode | Trigger | Testers Used |
|---|---|---|
| **Quick Check** | "quick check", "fast check", "quick test" | Only the 4 always-run testers (Mia, Sophia, Leila, Sharon) + artifact-specific testers. Skips content-signal testers. Fastest execution. |
| **Deep Check** | "deep check", "thorough check", "full test" (default) | All always-run testers + artifact-specific testers + content-signal testers. Full coverage. |

If the user doesn't specify, default to **Deep Check**.

## Compliance Mode

When the user asks for a compliance-focused audit, run ONLY the compliance testers with deeper analysis:

| Trigger | Testers |
|---|---|
| "GDPR audit", "privacy audit", "data protection check" | Fatima (privacy), Alejandro (gdpr) |
| "accessibility audit", "a11y audit", "WCAG audit", "508 compliance" | Sophia (accessibility), Mei (wcag) |
| "security audit", "OWASP audit", "pentest" | Tariq (security, owasp) |
| "compliance audit", "full compliance check" | Sophia, Mei, Fatima, Alejandro, Tariq — all compliance testers |

In compliance mode, testers should apply **stricter standards** and report issues at confidence >= 5 (instead of the normal >= 7), since compliance violations carry legal/regulatory risk even when uncertain.

## Diff / Comparison Mode

When the user says "compare to last run", "diff", "what changed", or "regression check":

1. **Load the previous report** — read the most recent `opentestai-report-*.json` file from the current working directory (find by timestamp in filename)
2. **Run a new analysis** on the current page/artifact
3. **Compare results** and categorize each issue as:
   - 🆕 **New** — issue found in current run but NOT in previous run
   - ✅ **Fixed** — issue in previous run but NOT in current run
   - 🔄 **Recurring** — issue found in BOTH runs (match by `bug_title` similarity or same `tester` + similar `bug_type`)
4. **Output a diff report** with sections for New, Fixed, and Recurring issues
5. Add a `"diff_status"` field to each issue in the JSON output: `"new"`, `"fixed"`, or `"recurring"`

This is essential for CI/regression workflows — it shows what got better, what got worse, and what remains.

---

## Targeted Testing — Single Tester or Area

The user can request a specific tester or testing area instead of the full automatic selection. When the user specifies a tester or area, **ONLY run that tester(s)** — skip the normal artifact-type and content-signal selection logic.

### How to Detect Targeted Requests

Look for these patterns in the user's message:

| User says... | Action |
|---|---|
| "run **Tariq**" / "have **Tariq** check this" / "security test" | Run **only Tariq** (security, owasp) |
| "check **accessibility**" / "a11y audit" / "WCAG check" | Run **only Sophia** (accessibility) + **Mei** (wcag) |
| "**security** audit" / "OWASP check" / "pentest this" | Run **only Tariq** (security, owasp) |
| "check **privacy**" / "GDPR audit" / "cookie consent check" | Run **only Fatima** (privacy) + **Alejandro** (gdpr) |
| "check the **console logs**" / "analyze console" | Run **only Diego** (console-logs) + **Jason** (javascript) |
| "check **network**" / "API issues" | Run **only Marcus** (networking) |
| "check **UI**" / "UX review" / "design review" | Run **only Mia** (ui-ux, forms) |
| "check **content**" / "copy review" | Run **only Leila** (content) |
| "check **mobile**" / "responsive check" | Run **only Zanele** (mobile) |
| "check **forms**" | Run **only Mia** (forms) + **Yuki** (signup) |
| "check **checkout**" / "payment flow" | Run **only Mateo** (checkout) + **Amara** (shopping-cart) |
| "check **errors**" / "error handling" | Run **only Sharon** (error-messages) + **Rajesh** (system-errors) |
| "run **Mia** and **Sophia**" | Run **only Mia and Sophia** |

### Matching Rules

1. **Tester name match**: If the user mentions a tester by name (e.g., "Tariq", "Sophia", "Diego"), run ONLY that tester.
2. **Multiple tester names**: If the user names multiple testers (e.g., "run Mia, Sophia, and Tariq"), run ONLY those testers.
3. **Check type / area match**: If the user mentions a check type keyword or area (e.g., "security", "accessibility", "mobile", "console-logs"), look up the relevant tester(s) in the Check Type Mappings table (see `tester-profiles.md`) and run ONLY those.
4. **Specialty match**: If the user describes an area that maps to a tester's specialty (e.g., "WCAG compliance" → Mei, "GDPR" → Alejandro, "AI chatbot" → Pete), run ONLY that tester.
5. **Combination**: The user can combine targeted testers with a mode, e.g., "run Tariq on this screenshot" or "have Sophia check the DOM".

### When NOT Targeted

If the user does NOT specify a tester or area (e.g., just says "check this page for bugs" or "test this"), use the normal automatic selection logic (always-run testers + artifact-specific + content-signal testers).

---

## STEP 1: Determine Available Artifacts

Detect what the user wants tested from their message. Supported artifact types:

| Artifact Type | Description | How to Obtain |
|---|---|---|
| `screenshot` | Screenshot image of the application | User uploads image, or capture via IDE browser tools (take screenshot) |
| `network_logs` | HTTP requests/responses | User pastes logs, or capture via IDE browser tools (network requests) |
| `console_logs` | Browser console messages | User pastes logs, or capture via IDE browser tools (console messages) |
| `dom` | DOM / accessibility tree | User pastes HTML, or capture via IDE browser tools (page snapshot / read page) |
| `page_text` | Text content from the page | User pastes text, or capture via IDE browser tools (get page text) |

### URL Auto-Capture (IMPORTANT)

**If the user provides a URL** (e.g., "test https://bing.com" or "check this page for bugs" while a page is open), **automatically capture ALL available artifacts** without asking. Do NOT ask the user what to provide — just grab everything:

```
1. Navigate to the URL (if not already there)
2. Take a screenshot of the page
3. Capture console log messages
4. Capture network request/response traffic
5. Read the page DOM / accessibility tree
6. Get the page text content
```

Capture as many artifact types as available. More artifacts = better coverage. Each artifact type unlocks different testers.

Use whatever browser automation tools are available in the IDE (e.g., Playwright MCP, Chrome DevTools, built-in browser tools).

### If Browser Automation Tools are NOT Available

Only if browser tools are not available, ask the user to provide artifacts directly (paste text, upload screenshot). But always try browser automation tools first.

---

## STEP 2: Load Tester Profiles

Read all 33 tester profiles from **`tester-profiles.md`** in this directory. That file contains each tester's identity, specialty, profile image URL, check types, expertise, full analysis prompt, and the check type mappings table.

---

## STEP 3: Select Relevant Testers

See the **Tester Selection Logic** section in `tester-profiles.md` for always-run testers, artifact-specific testers, and content-signal tester mapping tables.

**FIRST**: Check if the user specified a particular tester or area (see "Targeted Testing" section above). If so, ONLY run the specified tester(s) — skip ALL automatic selection.

**OTHERWISE**: Use the automatic selection rules from `tester-profiles.md`.

---

## STEP 4-5: Run Tester Prompts & Collect Results

See **`bug-detection.md`** for the full prompt template, artifact-type-specific instructions, confidence calibration scale, JSON output schema, deduplication rules, and all output templates (chat, markdown, HTML).

---

## MODE 2: Persona Feedback

See **`persona-mode.md`** for the full persona analysis prompt, persona images list, JSON schema, and all output templates (chat, markdown, HTML).

---

## MODE 3: Test Case Generation

See **`testcase-mode.md`** for the full test case generation prompt, JSON schema with priority scoring, and all output templates (chat, markdown, HTML).

---

## STEP 6: Write Output Files

See **`output-config.md`** for filename conventions, screenshot embedding, auto-open HTML, and end-of-run summary template.

---

## Example Workflow

### User says: "Check this page for bugs" (with a screenshot uploaded)

1. **Artifacts available**: screenshot
2. **Select testers** from `tester-profiles.md`: Mia (UI/UX), Sophia (accessibility), Leila (content), Tariq (security), Fatima (privacy), Sharon (errors) + any content-signal testers
3. **Run each tester's prompt** (from `bug-detection.md`) with the screenshot
4. **Collect issues** with `bug_confidence >= 7`
5. **Write** JSON, MD, HTML files using conventions from `output-config.md`
6. **Display** results in chat with tester identity
7. **Tell user** where the report files are saved

### User says: "Full analysis of this page" (all three modes)

1. **Run Bug Detection Mode** (find bugs)
2. **Run Persona Feedback Mode** (generate personas)
3. **Run Test Case Generation Mode** (create test cases)
4. **Output all** reports (12 files total: 4 per mode × 3 modes — .json, .md, .html, + chat)
5. **Display all** in chat
6. **Show end-of-run summary** with all absolute file paths

---

## Important Notes

- **High confidence only**: Never report speculative issues. If uncertain, don't report it.
- **No false positives**: It is always better to report 0 issues than to report issues that aren't real.
- **Tester identity**: Each issue must include the `tester` name, `byline`, and `image_url` (profile image) of the agent who found it.
- **Show tester avatar**: ALWAYS display the tester's profile image alongside their issue in ALL outputs — chat (inline markdown images), markdown files, and HTML files.
- **Actionable fixes**: Every issue must include a `suggested_fix` and `prompt_to_fix_this_issue` that an engineer or AI can use.
- **Deduplication**: If multiple testers find the same issue, consolidate into one entry with the highest confidence.
- **Empty results are OK**: Return `[]` if no high-confidence issues are found. This is a valid and good result.
- **Branding**: Always show the OpenTest.AI logo (`https://opentest.ai/img/otai.png`) and brand name at the top, with note that testing is provided by Testers.AI (`https://testers.ai/img/t_logo.png`) in the header.
- **Four outputs required**: Every mode MUST produce chat output, `.json` file, `.md` file, and `.html` file.
- **Absolute paths required**: ALWAYS use absolute file paths when writing files and when showing the user where files are saved. Absolute paths ensure links are clickable in the IDE.
- **Self-contained**: Read ALL companion files (`tester-profiles.md`, `bug-detection.md`, `persona-mode.md`, `testcase-mode.md`, `output-config.md`) before executing.
- **Persona diversity**: Always ensure personas are diverse in age, gender, and race. Always include at least two archetype personas (fangirl, fanboy, skeptic, technoob, technologist).
- **Persona voice**: All persona comments must be written in **first-person voice** ("I think...", "I would..."), never third-person.
- **Persona images**: All persona profile images use the URL pattern `https://testers.ai/img/profiles/{image_name}`. Each image should only be used once per panel.
- **Test case specificity**: Test cases must reference actual elements visible on the page — real button text, real link text, real form fields. Do NOT generate generic test cases.
- **Test case coverage**: Prioritize critical user journeys, revenue-impacting features, and business-critical functionality. Include edge cases and error scenarios.
- **Test case format**: Test cases must be returned as a valid JSON array with no extra text, markdown, or formatting outside the JSON.
- **Test case count**: Default to 10 test cases unless the user specifies a different number.