file-processing

# file-processing

Patterns for secure file upload, processing, and cleanup in Node.js Express servers.

## When to use

- Configuring multer for file uploads with type and size validation
- Managing temporary files with automatic cleanup
- Building file processing pipelines (upload, process, download)
- Streaming file downloads without blocking
- Implementing secure file storage with non-guessable paths

## Setup: multer with validation

```typescript
import multer from 'multer';
import path from 'node:path';
import crypto from 'node:crypto';

const storage = multer.diskStorage({
  destination: (req, file, cb) => {
    const jobId = crypto.randomUUID();
    const dir = path.join(process.env.TEMP_DIR ?? './tmp', jobId);
    fs.mkdirSync(dir, { recursive: true });
    (req as any).jobId = jobId;
    cb(null, dir);
  },
  filename: (req, file, cb) => {
    cb(null, `input-${Date.now()}.pdf`);
  },
});

const fileFilter = (req: Request, file: Express.Multer.File, cb: multer.FileFilterCallback) => {
  if (file.mimetype !== 'application/pdf') {
    cb(new Error('Only PDF files are accepted'));
    return;
  }
  cb(null, true);
};

export const upload = multer({
  storage,
  fileFilter,
  limits: {
    fileSize: (parseInt(process.env.MAX_UPLOAD_MB ?? '100')) * 1024 * 1024,
  },
});
```

## Temp file cleanup

```typescript
import fs from 'node:fs';
import path from 'node:path';

const TEMP_DIR = process.env.TEMP_DIR ?? './tmp';
const TTL_MS = (parseInt(process.env.JOB_TTL_MINUTES ?? '60')) * 60 * 1000;

export function scheduleCleanup(): void {
  setInterval(() => {
    const entries = fs.readdirSync(TEMP_DIR, { withFileTypes: true });
    for (const entry of entries) {
      if (!entry.isDirectory()) continue;
      const metaPath = path.join(TEMP_DIR, entry.name, 'meta.json');
      if (!fs.existsSync(metaPath)) continue;
      const meta = JSON.parse(fs.readFileSync(metaPath, 'utf8'));
      if (Date.now() - new Date(meta.created_at).getTime() > TTL_MS) {
        fs.rmSync(path.join(TEMP_DIR, entry.name), { recursive: true });
      }
    }
  }, 15 * 60 * 1000); // Run every 15 minutes
}
```

## Streaming download

```typescript
app.get('/api/download/:token', (req, res) => {
  const jobDir = path.join(TEMP_DIR, req.params.token);
  const metaPath = path.join(jobDir, 'meta.json');

  if (!fs.existsSync(metaPath)) {
    res.status(404).json({ error: 'File not found or expired' });
    return;
  }

  const meta = JSON.parse(fs.readFileSync(metaPath, 'utf8'));
  const outputPath = path.join(jobDir, 'output.pdf');

  res.setHeader('Content-Disposition', `attachment; filename="${meta.filename}"`);
  res.setHeader('Content-Type', 'application/pdf');

  const stream = fs.createReadStream(outputPath);
  stream.pipe(res);

  // Delete after streaming
  stream.on('end', () => {
    fs.rmSync(jobDir, { recursive: true });
  });
});
```

## Security checklist

- Validate MIME type AND file extension (not just MIME)
- Name output files by UUID, not user-supplied names
- Set Content-Disposition on downloads (prevent inline execution)
- Limit upload size via multer limits
- Never log file contents; log only metadata (size, page count)
- Clean temp files after download and on TTL expiry