http-proxy

# http-proxy Skill

## When to Use

Use this skill when building a Node.js HTTP reverse proxy that needs to:
- Forward requests transparently to an upstream
- Intercept and record request/response pairs
- Measure accurate end-to-end latency
- Handle both buffered and streaming responses
- Redact sensitive headers before storage

## Core Proxy Pattern

The proxy uses Node's built-in `http`/`https` modules to avoid buffering entire responses in memory for large or streaming upstreams.

```typescript
import http from 'http';
import https from 'https';
import { URL } from 'url';

function forwardRequest(
  clientReq: http.IncomingMessage,
  clientRes: http.ServerResponse,
  upstreamBase: string,
  onComplete: (record: RequestRecord) => void
) {
  const startMs = Date.now();
  const upstreamUrl = new URL(clientReq.url ?? '/', upstreamBase);
  const isHttps = upstreamUrl.protocol === 'https:';
  const transport = isHttps ? https : http;

  // Forward all headers except Host
  const headers = { ...clientReq.headers };
  delete headers['host'];
  headers['host'] = upstreamUrl.host;

  const options: http.RequestOptions = {
    hostname: upstreamUrl.hostname,
    port: upstreamUrl.port || (isHttps ? 443 : 80),
    path: upstreamUrl.pathname + upstreamUrl.search,
    method: clientReq.method,
    headers,
  };

  const proxyReq = transport.request(options, (proxyRes) => {
    // Forward status and response headers to client
    clientRes.writeHead(proxyRes.statusCode ?? 502, proxyRes.headers);

    const chunks: Buffer[] = [];
    proxyRes.on('data', (chunk: Buffer) => {
      chunks.push(chunk);
      clientRes.write(chunk); // stream to client immediately
    });
    proxyRes.on('end', () => {
      clientRes.end();
      const latencyMs = Date.now() - startMs;
      onComplete({
        status: proxyRes.statusCode ?? 0,
        latencyMs,
        responseBody: Buffer.concat(chunks),
        responseHeaders: proxyRes.headers,
      });
    });
  });

  proxyReq.on('error', (err) => {
    clientRes.writeHead(502, { 'content-type': 'application/json' });
    clientRes.end(JSON.stringify({ error: 'Bad Gateway', message: err.message }));
  });

  // Collect request body and forward it
  const reqChunks: Buffer[] = [];
  clientReq.on('data', (chunk: Buffer) => {
    reqChunks.push(chunk);
    proxyReq.write(chunk);
  });
  clientReq.on('end', () => {
    proxyReq.end();
  });
}
```

## Latency Measurement

Measure from the moment the proxy receives the first byte of the client request to the moment the last byte of the upstream response is sent to the client.

```typescript
// Start timer when first byte arrives (request 'data' event or before)
const startMs = process.hrtime.bigint();

// Stop timer after last response byte is written
proxyRes.on('end', () => {
  const latencyNs = process.hrtime.bigint() - startMs;
  const latencyMs = Number(latencyNs / 1_000_000n);
});
```

Use `process.hrtime.bigint()` for nanosecond precision. Convert to milliseconds before storing.

## Body Truncation

Never store unbounded bodies. Truncate at a configurable byte limit:

```typescript
const MAX_BODY_BYTES = parseInt(process.env.MAX_BODY_BYTES ?? '65536', 10);

function collectBody(chunks: Buffer[]): { body: string; truncated: boolean } {
  const full = Buffer.concat(chunks);
  const truncated = full.byteLength > MAX_BODY_BYTES;
  const slice = truncated ? full.subarray(0, MAX_BODY_BYTES) : full;
  return { body: slice.toString('utf8'), truncated };
}
```

Append `\n[truncated]` to the stored body string when truncated.

## Header Redaction

```typescript
const REDACT_HEADERS = new Set(
  (process.env.PROXY_REDACT_HEADERS ?? 'authorization,cookie,set-cookie')
    .split(',')
    .map((h) => h.trim().toLowerCase())
);

function redactHeaders(
  headers: http.IncomingHttpHeaders
): Record<string, string> {
  return Object.fromEntries(
    Object.entries(headers).map(([key, value]) => [
      key,
      REDACT_HEADERS.has(key.toLowerCase()) ? '[REDACTED]' : String(value),
    ])
  );
}
```

## Path Ignore Patterns

Use the `minimatch` package for glob matching:

```typescript
import { minimatch } from 'minimatch';

const IGNORE_GLOBS = (process.env.PROXY_IGNORE_PATHS ?? '')
  .split(',')
  .map((p) => p.trim())
  .filter(Boolean);

function shouldIgnorePath(pathname: string): boolean {
  return IGNORE_GLOBS.some((glob) => minimatch(pathname, glob));
}
```

## Endpoint Pattern Normalization

```typescript
const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
const MONGO_ID_RE = /^[0-9a-f]{24}$/i;
const DIGITS_RE = /^\d+$/;
const HEX_RE = /^[0-9a-f]{6,}$/i;

export function normalizeEndpoint(urlPath: string): string {
  const segments = urlPath.split('/');
  return segments
    .map((seg) => {
      if (!seg) return seg;
      if (DIGITS_RE.test(seg)) return ':id';
      if (UUID_RE.test(seg)) return ':id';
      if (MONGO_ID_RE.test(seg)) return ':id';
      if (HEX_RE.test(seg)) return ':id';
      return seg;
    })
    .join('/');
}
```

## SSE for Real-Time Events

The admin API exposes an SSE endpoint so the dashboard can receive new request records without polling:

```typescript
import express from 'express';

const clients = new Set<express.Response>();

// SSE endpoint
app.get('/api/live', (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  res.flushHeaders();

  clients.add(res);
  req.on('close', () => clients.delete(res));
});

// Call this after inserting each new record
function broadcastRequest(record: Partial<RequestRecord>) {
  const data = JSON.stringify({
    id: record.id,
    method: record.method,
    path: record.path,
    status: record.status,
    latency_ms: record.latency_ms,
  });
  for (const client of clients) {
    client.write(`event: request\ndata: ${data}\n\n`);
  }
}
```

## Percentile Calculation in SQLite

SQLite does not have a built-in percentile function. Compute percentiles with a window function approach:

```sql
WITH ranked AS (
  SELECT
    latency_ms,
    ROW_NUMBER() OVER (ORDER BY latency_ms) AS rn,
    COUNT(*) OVER () AS total
  FROM requests
  WHERE timestamp > datetime('now', '-1 hour')
)
SELECT
  MAX(CASE WHEN rn <= CAST(total * 0.50 AS INT) THEN latency_ms END) AS p50,
  MAX(CASE WHEN rn <= CAST(total * 0.95 AS INT) THEN latency_ms END) AS p95,
  MAX(CASE WHEN rn <= CAST(total * 0.99 AS INT) THEN latency_ms END) AS p99
FROM ranked;
```

## Proxy vs API Server Architecture

Keep the proxy server and the admin API server in separate Express (or raw http) instances but the same process:

```typescript
// proxy.ts
const proxyServer = http.createServer(handleProxyRequest);
proxyServer.listen(PROXY_PORT);

// server.ts
const apiApp = express();
apiApp.listen(API_PORT);
```

This avoids routing conflicts and makes it easy to expose only one port externally (the proxy).

## Error Handling

- `ECONNREFUSED`: upstream is not listening. Return 502, log the error.
- `ETIMEDOUT`: upstream took too long. Return 504, log with elapsed time.
- `ENOTFOUND`: DNS resolution failed for the upstream hostname. Return 502.
- `ECONNRESET`: upstream closed the connection mid-response. Return 502.

All errors should still call `onComplete` with the appropriate status code so the request is recorded for debugging purposes.