a11y-checker-ci

# A11y Checker CI

Automated accessibility testing in CI/CD pipelines with comprehensive reporting.

## Overview

To enforce accessibility standards in continuous integration, this skill configures automated WCAG compliance checks using industry-standard tools and generates detailed reports for every pull request.

## When to Use

Use this skill when:
- Adding accessibility testing to CI/CD pipelines
- Enforcing WCAG compliance in automated builds
- Generating accessibility reports for pull requests
- Setting up quality gates based on accessibility
- Automating accessibility audits
- Tracking accessibility improvements over time
- Ensuring new features meet accessibility standards

## Supported Tools

### @axe-core/playwright

Industry-standard accessibility testing engine with Playwright integration.

**Advantages:**
- Comprehensive WCAG rule coverage
- Fast execution in parallel with E2E tests
- Detailed violation reporting
- Active maintenance and updates

### pa11y-ci

Command-line accessibility testing tool for multiple URLs.

**Advantages:**
- Simple configuration
- Standalone execution (no browser automation needed)
- Multiple URL scanning
- Custom rule configuration

## Implementation Steps

### 1. Choose Testing Approach

To select the appropriate tool:

**Use @axe-core/playwright when:**
- Already using Playwright for E2E tests
- Need integration with existing test suites
- Want to test dynamic/authenticated pages
- Require detailed test context

**Use pa11y-ci when:**
- Need simple URL-based scanning
- Want standalone accessibility checks
- Testing static pages or public URLs
- Prefer configuration-based approach

### 2. Install Dependencies

For @axe-core/playwright:
```bash
npm install -D @axe-core/playwright
```

For pa11y-ci:
```bash
npm install -D pa11y-ci
```

### 3. Create Test Configuration

#### Option A: @axe-core/playwright

Create test file using `assets/a11y-test.spec.ts`:

```typescript
import { test, expect } from '@playwright/test'
import AxeBuilder from '@axe-core/playwright'

test.describe('Accessibility Tests', () => {
  test('homepage meets WCAG standards', async ({ page }) => {
    await page.goto('/')

    const accessibilityScanResults = await new AxeBuilder({ page })
      .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
      .analyze()

    expect(accessibilityScanResults.violations).toEqual([])
  })
})
```

#### Option B: pa11y-ci

Create configuration using `assets/pa11y-config.json`:

```json
{
  "defaults": {
    "timeout": 30000,
    "chromeLaunchConfig": {
      "executablePath": "/usr/bin/chromium-browser",
      "args": ["--no-sandbox"]
    },
    "standard": "WCAG2AA",
    "runners": ["axe", "htmlcs"],
    "ignore": []
  },
  "urls": [
    "http://localhost:3000",
    "http://localhost:3000/entities",
    "http://localhost:3000/timeline"
  ]
}
```

### 4. Generate Report Script

Create report generator using `scripts/generate_a11y_report.py`:

```bash
python scripts/generate_a11y_report.py \
  --input test-results/a11y-results.json \
  --output accessibility-report.md \
  --format github
```

The script generates markdown reports with:
- Executive summary with pass/fail status
- Violation count by severity (critical, serious, moderate, minor)
- Detailed violation list with:
  - Rule ID and description
  - WCAG criteria
  - Impact level
  - Affected elements
  - Remediation guidance
- Historical comparison (if available)

### 5. Configure CI Pipeline

#### GitHub Actions

Use template from `assets/github-actions-a11y.yml`:

```yaml
name: Accessibility Tests

on:
  pull_request:
    branches: [main, master]
  push:
    branches: [main, master]

jobs:
  a11y:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Build application
        run: npm run build

      - name: Start server
        run: npm start &

      - name: Wait for server
        run: npx wait-on http://localhost:3000 -t 60000

      - name: Run accessibility tests
        run: npm run test:a11y

      - name: Generate report
        if: always()
        run: |
          python scripts/generate_a11y_report.py \
            --input test-results/a11y-results.json \
            --output accessibility-report.md \
            --format github

      - name: Comment PR
        if: github.event_name == 'pull_request' && always()
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs')
            const report = fs.readFileSync('accessibility-report.md', 'utf8')

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: report
            })

      - name: Upload report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: accessibility-report
          path: |
            accessibility-report.md
            test-results/

      - name: Fail on violations
        if: failure()
        run: exit 1
```

#### GitLab CI

Use template from `assets/gitlab-ci-a11y.yml`:

```yaml
accessibility-test:
  stage: test
  image: mcr.microsoft.com/playwright:v1.40.0-focal
  script:
    - npm ci
    - npm run build
    - npm start &
    - npx wait-on http://localhost:3000 -t 60000
    - npm run test:a11y
    - python scripts/generate_a11y_report.py
      --input test-results/a11y-results.json
      --output accessibility-report.md
      --format gitlab
  artifacts:
    when: always
    paths:
      - accessibility-report.md
      - test-results/
    reports:
      junit: test-results/junit.xml
  only:
    - merge_requests
    - main
```

### 6. Add Package Scripts

Add to package.json:

```json
{
  "scripts": {
    "test:a11y": "playwright test a11y.spec.ts",
    "test:a11y:ci": "playwright test a11y.spec.ts --reporter=json",
    "pa11y": "pa11y-ci --config .pa11yci.json"
  }
}
```

## Report Format

### Executive Summary

```markdown
# Accessibility Test Report

**Status:** [ERROR] Failed
**Total Violations:** 12
**Pages Tested:** 5
**WCAG Level:** AA
**Date:** 2025-01-15

## Summary by Severity

- [CRITICAL] Critical: 2
- [SERIOUS] Serious: 5
- [MODERATE] Moderate: 3
- [MINOR] Minor: 2
```

### Violation Details

```markdown
## Violations

### [CRITICAL] Critical (2)

#### 1. Form elements must have labels (form-field-multiple-labels)

**WCAG Criteria:** 3.3.2 (Level A)
**Impact:** Critical
**Occurrences:** 3 elements

**Description:**
Form fields should have exactly one associated label element.

**Affected Elements:**
- Line 45: `<input type="text" name="entity-name">`
- Line 67: `<input type="email" name="user-email">`
- Line 89: `<select name="entity-type">`

**How to Fix:**
Add a `<label>` element with a `for` attribute matching the input's `id`:

\`\`\`html
<label for="entity-name">Entity Name</label>
<input id="entity-name" type="text" name="entity-name">
\`\`\`

**More Info:** https://dequeuniversity.com/rules/axe/4.7/label

---
```

### Historical Comparison

```markdown
## Progress

| Metric | Previous | Current | Change |
|--------|----------|---------|--------|
| Total Violations | 15 | 12 | [OK] -3 |
| Critical | 3 | 2 | [OK] -1 |
| Serious | 7 | 5 | [OK] -2 |
| Moderate | 4 | 3 | [OK] -1 |
| Minor | 1 | 2 | [ERROR] +1 |
```

## Quality Gates

### Blocking Violations

To fail builds on specific violations, configure thresholds:

```typescript
const results = await new AxeBuilder({ page }).analyze()

// Fail on any critical violations
const critical = results.violations.filter(v => v.impact === 'critical')
expect(critical).toHaveLength(0)

// Allow up to 5 moderate violations
const moderate = results.violations.filter(v => v.impact === 'moderate')
expect(moderate.length).toBeLessThanOrEqual(5)
```

### Configuration File

Use `assets/a11y-thresholds.json`:

```json
{
  "thresholds": {
    "critical": 0,
    "serious": 0,
    "moderate": 5,
    "minor": 10
  },
  "allowedViolations": [
    "color-contrast"
  ],
  "ignoreSelectors": [
    "#third-party-widget",
    "[data-testid='external-embed']"
  ]
}
```

## Advanced Configuration

### Custom Rules

To disable or configure specific rules:

```typescript
const results = await new AxeBuilder({ page })
  .disableRules(['color-contrast'])
  .withRules({
    'custom-rule': { enabled: true }
  })
  .analyze()
```

### Page-Specific Tests

Test different page types:

```typescript
const pages = [
  { url: '/', name: 'Homepage' },
  { url: '/entities', name: 'Entity List' },
  { url: '/timeline', name: 'Timeline View' }
]

for (const { url, name } of pages) {
  test(`${name} accessibility`, async ({ page }) => {
    await page.goto(url)
    const results = await new AxeBuilder({ page }).analyze()
    expect(results.violations).toEqual([])
  })
}
```

### Authenticated Pages

Test pages requiring authentication:

```typescript
test.use({ storageState: 'auth.json' })

test('dashboard accessibility', async ({ page }) => {
  await page.goto('/dashboard')
  const results = await new AxeBuilder({ page }).analyze()
  expect(results.violations).toEqual([])
})
```

## Report Customization

### Custom Templates

Create custom report templates in `assets/report-templates/`:

- `github-template.md` - GitHub PR comments
- `gitlab-template.md` - GitLab MR comments
- `slack-template.md` - Slack notifications
- `html-template.html` - HTML reports

### Report Destinations

Configure report distribution:

```python
python scripts/generate_a11y_report.py \
  --input results.json \
  --output-dir reports/ \
  --formats github gitlab slack html \
  --slack-webhook $SLACK_WEBHOOK \
  --github-token $GITHUB_TOKEN
```

## Monitoring and Tracking

### Historical Data

Store results for trend analysis:

```bash
# Save results with timestamp
python scripts/save_a11y_results.py \
  --input test-results/a11y-results.json \
  --database a11y-history.db

# Generate trend report
python scripts/generate_trend_report.py \
  --database a11y-history.db \
  --days 30 \
  --output a11y-trends.md
```

### Metrics Dashboard

Generate metrics for dashboards:

```json
{
  "timestamp": "2025-01-15T10:30:00Z",
  "commit": "abc123",
  "branch": "feature/new-ui",
  "violations": {
    "critical": 2,
    "serious": 5,
    "moderate": 3,
    "minor": 2
  },
  "wcagCompliance": {
    "a": false,
    "aa": false,
    "aaa": false
  },
  "pagesTested": 5,
  "totalElements": 1247,
  "testedElements": 1247
}
```

## Resources

Consult the following resources for detailed information:

- `scripts/generate_a11y_report.py` - Report generator
- `scripts/save_a11y_results.py` - Historical data storage
- `scripts/generate_trend_report.py` - Trend analysis
- `assets/a11y-test.spec.ts` - Playwright test template
- `assets/pa11y-config.json` - pa11y-ci configuration
- `assets/github-actions-a11y.yml` - GitHub Actions workflow
- `assets/gitlab-ci-a11y.yml` - GitLab CI configuration
- `assets/a11y-thresholds.json` - Violation thresholds
- `references/wcag-criteria.md` - WCAG standards reference
- `references/common-violations.md` - Common issues and fixes

## Best Practices

- Run accessibility tests on every pull request
- Set appropriate thresholds for violations
- Generate readable reports for developers
- Track accessibility metrics over time
- Test authenticated and dynamic pages
- Include accessibility in definition of done
- Review and update ignored rules periodically
- Provide remediation guidance in reports
- Celebrate accessibility improvements