migration-helper

# Convex Migration Helper

Safely migrate Convex schemas and data when making breaking changes.

## When to Use

- Adding new required fields to existing tables
- Changing field types or structure
- Splitting or merging tables
- Renaming fields
- Migrating from nested to relational data

## Migration Principles

1. **No Automatic Migrations**: Convex doesn't automatically migrate data
2. **Additive Changes are Safe**: Adding optional fields or new tables is safe
3. **Breaking Changes Need Code**: Required fields, type changes need migration code
4. **Zero-Downtime**: Write migrations to keep app running during migration

## Safe Changes (No Migration Needed)

- Adding optional fields
- Adding new tables
- Adding indexes

## Breaking Changes (Migration Required)

### Adding Required Field

**Solution**: Add as optional first, backfill data, then make required.

```typescript
// Step 1: Add as optional
users: defineTable({
  name: v.string(),
  email: v.optional(v.string()),
})

// Step 2: Create migration
export const backfillEmails = internalMutation({
  args: {},
  handler: async (ctx) => {
    const users = await ctx.db.query("users").collect();
    for (const user of users) {
      if (!user.email) {
        await ctx.db.patch(user._id, {
          email: `user-${user._id}@example.com`,
        });
      }
    }
  },
});

// Step 3: Run migration via dashboard or CLI
// npx convex run migrations:backfillEmails

// Step 4: Make field required (after all data migrated)
users: defineTable({
  name: v.string(),
  email: v.string(),
})
```

### Renaming Field

```typescript
// Step 1: Add new field (optional)
// Step 2: Copy data with internalMutation
// Step 3: Update schema (remove old field)
// Step 4: Update all code to use new field name
```

## Migration Patterns

### Batch Processing

For large tables, process in batches:

```typescript
export const migrateBatch = internalMutation({
  args: {
    cursor: v.optional(v.string()),
    batchSize: v.number(),
  },
  handler: async (ctx, args) => {
    const items = await ctx.db.query("largeTable").take(args.batchSize);

    for (const item of items) {
      await ctx.db.patch(item._id, { /* migration logic */ });
    }

    return {
      processed: items.length,
      hasMore: items.length === args.batchSize,
    };
  },
});
```

### Dual-Write Pattern

For zero-downtime migrations, write to both old and new structure during transition.

### Scheduled Migration

Use cron jobs for gradual migration:

```typescript
import { cronJobs } from "convex/server";
import { internal } from "./_generated/api";

const crons = cronJobs();
crons.interval(
  "migrate-batch",
  { minutes: 5 },
  internal.migrations.migrateBatch,
  { batchSize: 100 }
);
export default crons;
```

## Migration Checklist

- [ ] Identify breaking change
- [ ] Add new structure as optional/additive
- [ ] Write migration function (internal mutation)
- [ ] Test migration on sample data
- [ ] Run migration in batches if large dataset
- [ ] Verify migration completed (all records updated)
- [ ] Update application code to use new structure
- [ ] Deploy new code
- [ ] Remove old fields from schema
- [ ] Clean up migration code

## Common Pitfalls

1. **Don't make field required immediately**: Always add as optional first
2. **Don't migrate in a single transaction**: Batch large migrations
3. **Don't forget to update queries**: Update all code using old field
4. **Don't delete old field too soon**: Wait until all data migrated
5. **Test thoroughly**: Verify migration on dev environment first

Source: https://github.com/get-convex/convex-agent-plugins