nw-der-review-criteria

# Data Engineer Review Criteria

Evaluation criteria for each review dimension. Load when performing reviews.

## Dimension 1: Research Citation Quality

Evaluate whether recommendations trace to specific evidence.

**Checks**: Each major recommendation cites a specific research finding | Citations are accurate (finding number matches content) | Vendor-specific claims have multiple independent sources | General best practices distinguished from research-validated guidance

**Scoring**: 10: All cited, verified | 7: Most cited, 1-2 missing on non-critical points | 4: Major recommendations lack citations | 0: No citations

## Dimension 2: Security Coverage

Evaluate defense-in-depth for data layer.

**Checks**: Encryption at rest (TDE) | Encryption in transit (TLS) | Access control model (RBAC/ABAC) | SQL injection prevention (parameterized queries) | OWASP/NIST standards referenced | Credential handling (no hardcoded secrets)

**Scoring**: 10: All 6 checks with standard references | 7: 4-5 checks | 4: 2-3 checks, missing encryption or injection prevention | 0: Security not mentioned

## Dimension 3: Trade-off Analysis

Evaluate balanced presentation of alternatives.

**Checks**: Multiple technology options (minimum 2) | Pros/cons for each | Context factors identified (scale, consistency, latency, cost) | Recommendation justified by context fit | Limitations acknowledged

**Scoring**: 10: Comprehensive trade-offs with context-driven justification | 7: Trade-offs present, some alternatives missing | 4: Single recommendation without alternatives | 0: Prescriptive with no analysis

## Dimension 4: Technical Accuracy

**Checks**: SQL/NoSQL syntax correct for specified DB | Architecture patterns appropriate for use case (OLTP vs OLAP, write-heavy vs read-heavy) | Optimization strategies valid for target DB | Normalization level appropriate for workload | Index type matches query patterns (B-tree for range, hash for equality) | CAP trade-offs correctly applied

**Scoring**: 10: All technical claims verified | 7: Minor syntax/edge-case issues | 4: Significant errors affecting recommendations | 0: Fundamentally incorrect guidance

## Dimension 5: Completeness

**Checks**: Scaling strategy (vertical, horizontal, sharding, replication) | Performance characteristics (query patterns, bottlenecks) | Data governance when applicable (lineage, quality, MDM) | Compliance when personal/regulated data involved (GDPR, CCPA, HIPAA) | Backup/recovery for production designs | Monitoring/observability

**Scoring**: 10: All applicable aspects covered | 7: Core covered, 1-2 peripheral missing | 4: Major gaps (missing scaling or governance for production) | 0: Only immediate question, no broader context

## Dimension 6: Bias Detection

**Checks**: No single-vendor preference without justification | No latest-technology bias (new tech only when justified) | Contradictory evidence acknowledged | Open-source and commercial both considered | Technology maturity/community factored in | Cost mentioned

**Scoring**: 10: Demonstrably balanced with explicit trade-offs | 7: Generally balanced, minor preferences | 4: Clear vendor/tech bias | 0: Single-vendor advocacy

## Dimension 7: Implementability

**Checks**: Schema designs include column types, constraints, indexes | Architecture specifies integration points/APIs | Security has concrete steps (not just "use encryption") | Migration path described if changing systems | Dependencies/prerequisites identified | Handoff to next agent clear

**Scoring**: 10: Downstream agent proceeds without clarification | 7: Minor clarifications needed | 4: Significant implementation details missing | 0: Abstract guidance, no actionable content

## Severity Classification Guide

**Blocker**: Prevents downstream work or introduces security vulnerability. Examples: missing encryption for PII, wrong DB choice for workload, SQL syntax errors in migrations.

**Major**: Significantly reduces quality or misses important considerations. Examples: missing trade-off analysis, no scaling strategy for production, incomplete security coverage.

**Minor**: Improvement that does not block progress. Examples: missing citation on secondary recommendation, single alternative not considered, minor syntax variation.

**Suggestion**: Enhancement that adds polish. Examples: additional index for edge-case query, governance for future compliance, alternative monitoring approach.