upgrade-integration

# UpGrade A/B Testing Integration for EdTech

You are helping integrate [UpGrade](https://github.com/CarnegieLearningWeb/UpGrade), Carnegie Learning's open-source A/B testing platform designed specifically for educational technology.

## When This Skill Applies

- Adding A/B testing or experiments to educational apps
- Integrating feature flags in learning platforms
- Connecting EdTech apps to LMS via LTI with experiment support
- Setting up learning analytics with xAPI + experimentation
- Debugging experiment assignment issues

## Critical Context

UpGrade is NOT a generic A/B testing tool. It's designed for education with features like:
- **Group-level assignment** (whole classrooms get same condition)
- **Consistency rules** (students don't get re-randomized)
- **Educational context metadata** (grade, subject, proficiency)

**ALWAYS** ask these questions before implementing:

1. What's the assignment unit? (Individual student vs classroom vs school)
2. What happens if a student changes classes mid-experiment?
3. How will teachers access both conditions (or will they)?
4. Is the app used in LMS iframes? (Cookie issues!)

## Quick Reference

### Installation

```bash
# JavaScript/TypeScript
npm install upgrade_client_lib

# Java (Maven)
<dependency>
  <groupId>com.carnegielearning</groupId>
  <artifactId>upgrade-client</artifactId>
</dependency>
```

### Core Integration Pattern

```typescript
import UpgradeClient from 'upgrade_client_lib/dist/browser';
import { MARKED_DECISION_POINT_STATUS } from 'upgrade_client_lib';

// 1. Initialize (MUST complete before getDecisionPointAssignment)
const client = new UpgradeClient(userId, hostUrl, context);
await client.init(groupData, workingGroupData);

// 2. Get assignment at decision point
const assignment = await client.getDecisionPointAssignment('feature_name', 'target');
const condition = assignment.getCondition();
const payload = assignment.getPayload();

// 3. Mark that condition was applied
client.markDecisionPoint('feature_name', MARKED_DECISION_POINT_STATUS.CONDITION_APPLIED);

// 4. Log outcomes
client.log('score', 85);
client.log('completion_time_ms', 45000);
```

### Three SDK Modes

| Mode | Use Case | Requires init()? |
|------|----------|------------------|
| **Standard** | Pre-registered users with stored groups | Yes |
| **Ephemeral** | Runtime-provided groups, no DB lookup | No |
| **Merged** | Combine session + stored groups | Yes |

## Integration Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                         LMS                                  │
│                  (Canvas, Moodle, etc.)                     │
└─────────────────────┬───────────────────────────────────────┘
                      │ LTI 1.3 Launch
                      │ (user_id, context_id, roles)
                      ▼
┌─────────────────────────────────────────────────────────────┐
│                    Your EdTech App                          │
│  ┌─────────────────────────────────────────────────────┐   │
│  │ Identity Mapper                                      │   │
│  │ LTI user_id → UpGrade userId + altUserIds           │   │
│  └─────────────────────┬───────────────────────────────┘   │
│                        │                                    │
│  ┌─────────────────────▼───────────────────────────────┐   │
│  │ UpGrade SDK                                          │   │
│  │ - init(groupData)                                    │   │
│  │ - getDecisionPointAssignment()                       │   │
│  │ - markDecisionPoint()                                │   │
│  │ - log()                                              │   │
│  └─────────────────────┬───────────────────────────────┘   │
└─────────────────────────┼───────────────────────────────────┘
                          │ HTTPS
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                    UpGrade Backend                          │
│            (Self-hosted or Carnegie Learning)               │
└─────────────────────────────────────────────────────────────┘
```

## CRITICAL: Known Pitfalls

Before writing ANY integration code, read `resources/pitfalls-and-edge-cases.md`. The most dangerous issues:

### 1. Third-Party Cookie Blocking (CRITICAL)

If your app runs in an LMS iframe, Safari and Chrome WILL block your session cookies by default.

**Symptoms:**
- User gets re-randomized on every page load
- `init()` seems to work but assignments are inconsistent
- Works in development, fails in production LMS

**Solutions:**
- Set `SameSite=None; Secure` on all cookies
- Use localStorage with user-provided identifier (not session-based)
- Consider opening in new window instead of iframe

### 2. Race Condition: init() vs getDecisionPointAssignment()

```typescript
// WRONG - race condition
const client = new UpgradeClient(userId, hostUrl, context);
client.init(groupData); // async, not awaited!
const assignment = await client.getDecisionPointAssignment('feature');
// assignment may be NO_CONDITION_ASSIGNED
```

```typescript
// CORRECT
const client = new UpgradeClient(userId, hostUrl, context);
await client.init(groupData); // AWAIT this!
const assignment = await client.getDecisionPointAssignment('feature');
```

### 3. LTI Identity Mismatch

LTI provides `user_id` which is opaque and LMS-specific. Your app may use email or internal IDs.

```typescript
// Map all identities together
await client.init(groupData);
client.setAltUserIds([
  ltiUserId,           // from LTI launch
  userEmail,           // from your auth
  internalDatabaseId   // your user table PK
]);
```

### 4. Group Assignment Drift

Student transfers from Mrs. Smith's class (Treatment A) to Mr. Jones' class (Treatment B).

**Default behavior:** Student KEEPS original assignment (consistency rule)

**Problem:** Student now sees different UI than classmates, causing confusion.

**Solution:** Decide upfront:
- Accept drift (experimental purity)
- Re-assign on class change (classroom consistency)
- Exclude transferred students from analysis

### 5. Shared Device / Chromebook Cart Problem

```typescript
// WRONG - userId persists from previous student
const client = new UpgradeClient(getUserIdFromLocalStorage(), ...);

// CORRECT - validate identity on each session
const client = new UpgradeClient(getCurrentAuthenticatedUserId(), ...);
// Clear any cached assignments if userId changed
```

## Decision Point Patterns for Education

See `resources/decision-point-patterns.md` for detailed examples:

| Pattern | Example | Assignment Unit |
|---------|---------|-----------------|
| **UI Variant** | New vs old problem interface | Individual |
| **Pedagogical** | Worked examples vs practice problems | Classroom |
| **Adaptive Algorithm** | Mastery threshold 80% vs 90% | Individual |
| **Content Ordering** | Teach fractions before decimals vs after | School |
| **Feedback Timing** | Immediate vs delayed feedback | Classroom |

## Code Templates

See `templates/` directory for:
- `react-hook.tsx` - React hook with proper error handling
- `nextjs-middleware.ts` - Next.js API route pattern
- `vanilla-js.js` - Framework-agnostic implementation
- `lti-launch-handler.ts` - LTI 1.3 launch with UpGrade init

## Logging Best Practices

```typescript
// DO: Use consistent types
client.log('score', 85);              // number
client.log('completed', true);         // boolean
client.log('response', 'correct');     // string

// DON'T: Mix types for same metric
client.log('score', 85);
client.log('score', '85');  // String! Will corrupt analysis

// DON'T: Log before marking decision point
client.log('score', 85);  // Not associated with condition!
client.markDecisionPoint('quiz', MARKED_DECISION_POINT_STATUS.CONDITION_APPLIED);

// DO: Mark first, then log
client.markDecisionPoint('quiz', MARKED_DECISION_POINT_STATUS.CONDITION_APPLIED);
client.log('score', 85);  // Correctly associated
```

## Troubleshooting Decision Tree

```
Assignment returns NO_CONDITION_ASSIGNED
├── Did you await init()?
│   └── No → Add await before init()
├── Is experiment in ENROLLING state?
│   └── No → Check UpGrade UI, start experiment
├── Does user match segment criteria?
│   └── No → Check inclusion/exclusion rules
├── Is context correct?
│   └── No → Verify context string matches experiment config
└── Check network tab for failed requests
```

```
User gets different condition than classmates
├── Is assignment unit set to GROUP?
│   └── No → Change to GROUP in experiment settings
├── Was groupData provided to init()?
│   └── No → Pass { classId: 'xxx' } to init()
├── Did student join after experiment started?
│   └── Yes → Check POST_EXPERIMENT_RULE
└── Is student in multiple groups?
    └── Yes → Check group priority/override logic
```

## External Resources

- [UpGrade GitHub](https://github.com/CarnegieLearningWeb/UpGrade)
- [UpGrade Documentation](https://upgrade-platform.gitbook.io/docs/)
- [JavaScript SDK](https://github.com/CarnegieLearningWeb/UpGrade/tree/develop/clientlibs/js)
- [LTI 1.3 Spec](https://www.imsglobal.org/spec/lti/v1p3)
- [xAPI Spec](https://xapi.com/overview/)

## When to Escalate

Recommend the developer contact UpGrade support or file a GitHub issue if:
- Assignment algorithm seems incorrect (statistical anomaly)
- Data export shows impossible values
- Backend returns 5xx errors consistently
- Need custom consistency rules not supported by SDK