iac-terraform-modules-eng

# Terraform & OpenTofu Module Library

Production-ready Terraform and OpenTofu module patterns for multi-cloud infrastructure and 40+ providers including AWS, Azure, GCP, Kubernetes, Cloudflare, Vault, Grafana, and more. Modules are compatible with both Terraform and OpenTofu.

## Purpose

Create reusable, well-tested Terraform and OpenTofu modules for common infrastructure patterns across cloud providers, SaaS platforms, and on-premises virtualization. Support both Terraform (HashiCorp BSL) and OpenTofu (MPL 2.0) workflows.

## When to Use

- Build reusable infrastructure components for AWS, Azure, GCP
- Configure Kubernetes clusters with Helm, Talos, or managed services
- Set up CI/CD pipelines with GitHub, GitLab, or Buildkite
- Manage databases: MongoDB Atlas, CockroachDB, Elastic, Pinecone
- Configure networking: Cloudflare, DNS, CDN, VPN (ZeroTier)
- Implement secrets management with HashiCorp Vault
- Deploy to PaaS platforms: Vercel, Heroku, DigitalOcean, Linode, Vultr
- Configure observability with Grafana stack
- Manage on-premises infrastructure: vSphere, VMC, Proxmox
- Orchestrate workflows with Ansible, Kestra, or Prefect
- Implement feature flags with Flagsmith
- Establish organizational Terraform/OpenTofu standards
- Migrate existing Terraform configurations to OpenTofu
- Leverage OpenTofu-specific features (early evaluation, provider-defined functions)
- Generate and maintain module documentation (terraform-docs)

## Module Structure

```
terraform-modules/
├── aws/
│   ├── vpc/
│   ├── eks/
│   ├── rds/
│   └── s3/
├── azure/
│   ├── vnet/
│   ├── aks/
│   └── storage/
└── gcp/
    ├── vpc/
    ├── gke/
    └── cloud-sql/
```

## Standard Module Pattern

```
module-name/
├── main.tf              # Main resources
├── variables.tf         # Input variables
├── outputs.tf           # Output values
├── versions.tf          # Provider versions
├── README.md            # Documentation (generated by terraform-docs)
├── .terraform-docs.yml  # terraform-docs configuration
├── examples/            # Usage examples
│   └── complete/
│       ├── main.tf
│       └── variables.tf
└── tests/               # Terratest files
    └── module_test.go
```

## AWS VPC Module Example

**main.tf:**
```hcl
locals {
  nat_gateway_count = var.create_nat_gateway ? (var.single_nat_gateway ? 1 : length(var.availability_zones)) : 0
}

resource "aws_vpc" "main" {
  cidr_block           = var.cidr_block
  enable_dns_hostnames = var.enable_dns_hostnames
  enable_dns_support   = var.enable_dns_support

  tags = merge(
    { Name = var.name },
    var.tags
  )
}

# Public Subnets
resource "aws_subnet" "public" {
  count                   = length(var.public_subnet_cidrs)
  vpc_id                  = aws_vpc.main.id
  cidr_block              = var.public_subnet_cidrs[count.index]
  availability_zone       = var.availability_zones[count.index]
  map_public_ip_on_launch = true

  tags = merge(
    {
      Name = "${var.name}-public-${count.index + 1}"
      Tier = "public"
    },
    var.tags
  )
}

# Private Subnets
resource "aws_subnet" "private" {
  count             = length(var.private_subnet_cidrs)
  vpc_id            = aws_vpc.main.id
  cidr_block        = var.private_subnet_cidrs[count.index]
  availability_zone = var.availability_zones[count.index]

  tags = merge(
    {
      Name = "${var.name}-private-${count.index + 1}"
      Tier = "private"
    },
    var.tags
  )
}

# Internet Gateway
resource "aws_internet_gateway" "main" {
  count  = var.create_internet_gateway ? 1 : 0
  vpc_id = aws_vpc.main.id

  tags = merge(
    { Name = "${var.name}-igw" },
    var.tags
  )
}

# Elastic IPs for NAT Gateways
resource "aws_eip" "nat" {
  count  = local.nat_gateway_count
  domain = "vpc"

  tags = merge(
    { Name = "${var.name}-nat-eip-${count.index + 1}" },
    var.tags
  )

  depends_on = [aws_internet_gateway.main]
}

# NAT Gateways
resource "aws_nat_gateway" "main" {
  count         = local.nat_gateway_count
  allocation_id = aws_eip.nat[count.index].id
  subnet_id     = aws_subnet.public[count.index].id

  tags = merge(
    { Name = "${var.name}-nat-${count.index + 1}" },
    var.tags
  )

  depends_on = [aws_internet_gateway.main]
}

# Public Route Table
resource "aws_route_table" "public" {
  count  = var.create_internet_gateway ? 1 : 0
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.main[0].id
  }

  tags = merge(
    { Name = "${var.name}-public-rt" },
    var.tags
  )
}

resource "aws_route_table_association" "public" {
  count          = length(var.public_subnet_cidrs)
  subnet_id      = aws_subnet.public[count.index].id
  route_table_id = aws_route_table.public[0].id
}

# Private Route Tables (one per NAT Gateway)
resource "aws_route_table" "private" {
  count  = local.nat_gateway_count > 0 ? length(var.private_subnet_cidrs) : 0
  vpc_id = aws_vpc.main.id

  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.main[var.single_nat_gateway ? 0 : count.index].id
  }

  tags = merge(
    { Name = "${var.name}-private-rt-${count.index + 1}" },
    var.tags
  )
}

resource "aws_route_table_association" "private" {
  count          = local.nat_gateway_count > 0 ? length(var.private_subnet_cidrs) : 0
  subnet_id      = aws_subnet.private[count.index].id
  route_table_id = aws_route_table.private[count.index].id
}
```

**variables.tf:**
```hcl
variable "name" {
  description = "Name of the VPC"
  type        = string
}

variable "cidr_block" {
  description = "CIDR block for VPC"
  type        = string
  validation {
    condition     = can(regex("^([0-9]{1,3}\\.){3}[0-9]{1,3}/[0-9]{1,2}$", var.cidr_block))
    error_message = "CIDR block must be valid IPv4 CIDR notation."
  }
}

variable "availability_zones" {
  description = "List of availability zones"
  type        = list(string)
}

variable "public_subnet_cidrs" {
  description = "CIDR blocks for public subnets"
  type        = list(string)
  default     = []
}

variable "private_subnet_cidrs" {
  description = "CIDR blocks for private subnets"
  type        = list(string)
  default     = []
}

variable "enable_dns_hostnames" {
  description = "Enable DNS hostnames in VPC"
  type        = bool
  default     = true
}

variable "enable_dns_support" {
  description = "Enable DNS support in VPC"
  type        = bool
  default     = true
}

variable "create_internet_gateway" {
  description = "Create an Internet Gateway for public subnets"
  type        = bool
  default     = true
}

variable "create_nat_gateway" {
  description = "Create NAT Gateway(s) for private subnets"
  type        = bool
  default     = true
}

variable "single_nat_gateway" {
  description = "Use a single NAT Gateway for all AZs (cost savings)"
  type        = bool
  default     = false
}

variable "tags" {
  description = "Additional tags"
  type        = map(string)
  default     = {}
}
```

**versions.tf:**
```hcl
# Compatible with both Terraform >= 1.5.0 and OpenTofu >= 1.6.0
terraform {
  required_version = ">= 1.5.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}
```

**versions.tofu.tf (OpenTofu-specific, optional):**
```hcl
# Use for OpenTofu-specific features like early evaluation
terraform {
  required_version = ">= 1.8.0"
}
```

**outputs.tf:**
```hcl
output "vpc_id" {
  description = "ID of the VPC"
  value       = aws_vpc.main.id
}

output "vpc_cidr_block" {
  description = "CIDR block of VPC"
  value       = aws_vpc.main.cidr_block
}

output "public_subnet_ids" {
  description = "IDs of public subnets"
  value       = aws_subnet.public[*].id
}

output "private_subnet_ids" {
  description = "IDs of private subnets"
  value       = aws_subnet.private[*].id
}

output "internet_gateway_id" {
  description = "ID of the Internet Gateway"
  value       = try(aws_internet_gateway.main[0].id, null)
}

output "nat_gateway_ids" {
  description = "IDs of NAT Gateways"
  value       = aws_nat_gateway.main[*].id
}

output "public_route_table_id" {
  description = "ID of public route table"
  value       = try(aws_route_table.public[0].id, null)
}

output "private_route_table_ids" {
  description = "IDs of private route tables"
  value       = aws_route_table.private[*].id
}
```

## Best Practices

1. **Use semantic versioning** for modules
2. **Document all variables** with descriptions
3. **Provide examples** in examples/ directory
4. **Use validation blocks** for input validation
5. **Output important attributes** for module composition
6. **Pin provider versions** in versions.tf
7. **Use locals** for computed values
8. **Implement conditional resources** with count/for_each
9. **Test modules** with Terratest
10. **Tag all resources** consistently

## OpenTofu Compatibility

OpenTofu is an open-source fork of Terraform (MPL 2.0 licensed) that maintains HCL compatibility while adding new features. All modules in this library work with both tools.

### Key Differences

| Feature | Terraform | OpenTofu |
|---------|-----------|----------|
| License | BSL 1.1 | MPL 2.0 |
| State Encryption | Enterprise only | Built-in (1.7+) |
| Early Evaluation | No | Yes (1.8+) |
| Provider-defined Functions | Limited | Extended support |
| CLI Command | `terraform` | `tofu` |
| Registry | registry.terraform.io | registry.opentofu.org |

### Migration from Terraform to OpenTofu

```bash
# Install OpenTofu
brew install opentofu

# Initialize (uses existing .terraform.lock.hcl)
tofu init

# Validate configuration
tofu validate

# Plan (state file is compatible)
tofu plan

# Apply
tofu apply
```

### OpenTofu-Specific Features

**State Encryption (1.7+):**
```hcl
terraform {
  encryption {
    key_provider "pbkdf2" "main" {
      passphrase = var.state_encryption_passphrase
    }

    method "aes_gcm" "main" {
      keys = key_provider.pbkdf2.main
    }

    state {
      method = method.aes_gcm.main
    }
  }
}
```

**Early Evaluation (1.8+):**
```hcl
# Variables can be used in backend configuration
terraform {
  backend "s3" {
    bucket = var.state_bucket  # Works in OpenTofu 1.8+
    key    = var.state_key
    region = var.aws_region
  }
}
```

## Module Composition

```hcl
module "vpc" {
  source = "../../modules/aws/vpc"

  name               = "production"
  cidr_block         = "10.0.0.0/16"
  availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]

  public_subnet_cidrs = [
    "10.0.1.0/24",
    "10.0.2.0/24",
    "10.0.3.0/24"
  ]

  private_subnet_cidrs = [
    "10.0.11.0/24",
    "10.0.12.0/24",
    "10.0.13.0/24"
  ]

  create_nat_gateway = true
  single_nat_gateway = false  # HA: one NAT per AZ

  tags = {
    Environment = "production"
    ManagedBy   = "terraform"
  }
}

module "rds" {
  source = "../../modules/aws/rds"

  identifier     = "production-db"
  engine         = "postgres"
  engine_version = "15.4"
  instance_class = "db.t3.large"

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnet_ids

  tags = {
    Environment = "production"
  }
}

module "eks" {
  source = "../../modules/aws/eks"

  cluster_name    = "production"
  cluster_version = "1.29"

  vpc_id          = module.vpc.vpc_id
  subnet_ids      = module.vpc.private_subnet_ids

  node_groups = {
    default = {
      instance_types = ["t3.large"]
      min_size       = 2
      max_size       = 10
      desired_size   = 3
    }
  }

  tags = {
    Environment = "production"
  }
}
```

## Module Documentation

Generate README.md automatically from module code using terraform-docs:

```bash
# Install terraform-docs
brew install terraform-docs

# Generate documentation
terraform-docs markdown table . > README.md

# Generate with custom config
terraform-docs -c .terraform-docs.yml .
```

**.terraform-docs.yml:**
```yaml
formatter: markdown table

sections:
  show:
    - header
    - inputs
    - outputs
    - providers
    - requirements

content: |-
  {{ .Header }}

  ## Usage

  ```hcl
  module "example" {
    source = "path/to/module"
    # ... variables
  }
  ```

  {{ .Requirements }}
  {{ .Providers }}
  {{ .Inputs }}
  {{ .Outputs }}
```

See `references/terraform-docs.md` for complete patterns including pre-commit hooks, CI/CD integration, and custom templates.

## Reference Files

### Module Tooling
- `references/terraform-docs.md` - Module documentation generation, templates, pre-commit hooks

### OpenTofu
- `references/opentofu.md` - OpenTofu migration, state encryption, early evaluation, registry

### Cloud Providers
- `references/aws-modules.md` - AWS module patterns (VPC, EKS, RDS, S3, ALB, Lambda)
- `references/azure-modules.md` - Azure module patterns (VNet, AKS, SQL, Storage)
- `references/gcp-modules.md` - GCP module patterns (VPC, GKE, Cloud SQL, Cloud Run)

### CI/CD & Version Control
- `references/cicd-providers.md` - GitHub, GitLab, Buildkite provider patterns

### Kubernetes & Containers
- `references/kubernetes-providers.md` - Kubernetes, Helm, Talos, Coder provider patterns

### Databases & Data
- `references/database-providers.md` - MongoDB Atlas, CockroachDB, Elastic, Pinecone, Atlas, Airbyte patterns

### Networking & CDN
- `references/networking-providers.md` - Cloudflare, DNS, NS1, ZeroTier, Fastly, Akamai patterns

### Security
- `references/security-providers.md` - HashiCorp Vault, TLS provider patterns

### Utility Providers
- `references/utility-providers.md` - Template, Time, Local, HTTP provider patterns

### Orchestration & Feature Flags
- `references/orchestration-providers.md` - Ansible AAP, Kestra, Prefect, Flagsmith patterns

### Observability
- `references/observability-providers.md` - Grafana, Grafana Cloud, Synthetic Monitoring, Adaptive Metrics

### Platform as a Service
- `references/platform-providers.md` - Vercel, Heroku, DigitalOcean, Linode, Vultr patterns

### Virtualization
- `references/virtualization-providers.md` - vSphere, VMware Cloud, Proxmox patterns

## Testing

```go
// tests/vpc_test.go
package test

import (
    "testing"
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestVPCModule(t *testing.T) {
    terraformOptions := &terraform.Options{
        TerraformDir: "../examples/complete",
    }

    defer terraform.Destroy(t, terraformOptions)
    terraform.InitAndApply(t, terraformOptions)

    vpcID := terraform.Output(t, terraformOptions, "vpc_id")
    assert.NotEmpty(t, vpcID)
}
```

## Related Skills

- `multi-cloud-architecture` - For architectural decisions
- `cost-optimization` - For cost-effective designs