terraform-stacks

# Terraform Stacks

Terraform Stacks simplify infrastructure provisioning and management at scale by providing a configuration layer above traditional Terraform modules. Stacks enable declarative orchestration of multiple components across environments, regions, and cloud accounts.

## Core Concepts

**Stack**: A complete unit of infrastructure composed of components and deployments that can be managed together.

**Component**: An abstraction around a Terraform module that defines infrastructure pieces. Each component specifies a source module, inputs, and providers.

**Deployment**: An instance of all components in a stack with specific input values. Use deployments for different environments (dev/staging/prod), regions, or cloud accounts.

**Stack Language**: A separate HCL-based language (not regular Terraform HCL) with distinct blocks and file extensions.

## File Structure

Terraform Stacks use specific file extensions:

- **Component configuration**: `.tfcomponent.hcl`
- **Deployment configuration**: `.tfdeploy.hcl`
- **Provider lock file**: `.terraform.lock.hcl` (generated by CLI)

All configuration files must be at the root level of the Stack repository. HCP Terraform processes all files in dependency order.

### Recommended File Organization

```
my-stack/
├── .terraform-version               # The required Terraform version for this Stack
├── variables.tfcomponent.hcl        # Variable declarations
├── providers.tfcomponent.hcl        # Provider configurations
├── components.tfcomponent.hcl       # Component definitions
├── outputs.tfcomponent.hcl          # Stack outputs
├── deployments.tfdeploy.hcl         # Deployment definitions
├── .terraform.lock.hcl              # Provider lock file (generated)
└── modules/                         # Local modules (optional - only if using local modules)
    ├── s3/
    └── compute/
```

**Note**: The `modules/` directory is only required when using local module sources. Components can reference modules from:

- Local file paths: `./modules/vpc`
- Public registry: `terraform-aws-modules/vpc/aws`
- Private registry: `app.terraform.io/<org-name>/vpc/aws`
- Git: `git::https://github.com/org/repo.git//path?ref=v1.0.0`

HCP Terraform processes all `.tfcomponent.hcl` and `.tfdeploy.hcl` files in dependency order.

## Required Terraform version (.terraform-version)

Use Terraform v1.13.x or later to access the Stacks CLI plugin and to run
terraform stacks CLI commands. Begin by adding a .terraform-version file to
your Stack's root directory to specify the Terraform version required for your
Stack. For example, the following file specifies Terraform v1.14.5:

```
1.14.5
```

## Component Configuration (.tfcomponent.hcl)

### Variable Block

Declare input variables for the Stack configuration. Variables must define a `type` field and do not support the `validation` argument.

```hcl
variable "aws_region" {
  type        = string
  description = "AWS region for deployments"
  default     = "us-west-1"
}

variable "identity_token" {
  type        = string
  description = "OIDC identity token"
  ephemeral   = true  # Does not persist to state file
}

variable "instance_count" {
  type     = number
  nullable = false
}
```

**Important**: Use `ephemeral = true` for credentials and tokens (identity tokens, API keys, passwords) to prevent them from persisting in state files. Use `stable` for longer-lived values like license keys that need to persist across runs.

### Required Providers Block

```hcl
required_providers {
  aws = {
    source  = "hashicorp/aws"
    version = "~> 6.0"
  }
  random = {
    source  = "hashicorp/random"
    version = "~> 3.5.0"
  }
}
```

### Provider Block

Provider blocks differ from traditional Terraform:

1. Support `for_each` meta-argument
2. Define aliases in the block header (not as an argument)
3. Accept configuration through a `config` block

**Single Provider Configuration:**

```hcl
provider "aws" "this" {
  config {
    region = var.aws_region
    assume_role_with_web_identity {
      role_arn           = var.role_arn
      web_identity_token = var.identity_token
    }
  }
}
```

**Multiple Provider Configurations with for_each:**

```hcl
provider "aws" "configurations" {
  for_each = var.regions

  config {
    region = each.value
    assume_role_with_web_identity {
      role_arn           = var.role_arn
      web_identity_token = var.identity_token
    }
  }
}
```

**Authentication Best Practice**: Use **workload identity** (OIDC) as the preferred authentication method for Stacks. This approach:

- Avoids long-lived static credentials
- Provides temporary, scoped credentials per deployment run
- Integrates with cloud provider IAM (AWS IAM Roles, Azure Managed Identities, GCP Service Accounts)
- Eliminates need for platform-managed environment variables

Configure workload identity using `identity_token` blocks and `assume_role_with_web_identity` in provider configuration. For detailed setup instructions for AWS, Azure, and GCP, see: https://developer.hashicorp.com/terraform/cloud-docs/dynamic-provider-credentials

### Component Block

Each Stack requires at least one component block. Add a component for each module to include in the Stack. Components reference modules from local paths, registries, or Git.

```hcl
component "vpc" {
  source  = "app.terraform.io/my-org/vpc/aws"  # Local, registry, or Git URL
  version = "2.1.0"          # For registry modules

  inputs = {
    cidr_block  = var.vpc_cidr
    name_prefix = var.name_prefix
  }

  providers = {
    aws = provider.aws.this
  }
}
```

See `references/component-blocks.md` for examples of dependencies, for_each, public registry modules, Git sources, and more.

**Key Points:**

- Reference outputs: `component.<name>.<output>` or `component.<name>[key].<output>` for for_each
- Dependencies inferred automatically from component references
- Aggregate with for expressions: `[for x in component.s3 : x.bucket_name]`
- For components with `for_each`, reference specific instances: `component.<name>[each.value].<output>`
- Provider references are normal values: `provider.<type>.<alias>` or `provider.<type>.<alias>[each.value]`

### Output Block

Outputs require a `type` argument and do not support `preconditions`:

```hcl
output "vpc_id" {
  type        = string
  description = "VPC ID"
  value       = component.vpc.vpc_id
}

output "endpoint_urls" {
  type      = map(string)
  value     = {
    for region, comp in component.api : region => comp.endpoint_url
  }
  sensitive = false
}
```

### Locals Block

Locals blocks work the same in both `.tfcomponent.hcl` and `.tfdeploy.hcl` files:

```hcl
locals {
  common_tags = {
    Environment = var.environment
    ManagedBy   = "Terraform Stacks"
    Project     = var.project_name
  }

  region_config = {
    for region in var.regions : region => {
      name_suffix = "${var.environment}-${region}"
    }
  }
}
```

### Removed Block

Use to safely remove components from a Stack. HCP Terraform requires the component's providers to remove it.

```hcl
removed {
  from   = component.old_component
  source = "./modules/old-module"

  providers = {
    aws = provider.aws.this
  }
}
```

## Deployment Configuration (.tfdeploy.hcl)

### Identity Token Block

Generate JWT tokens for OIDC authentication with cloud providers:

```hcl
identity_token "aws" {
  audience = ["aws.workload.identity"]
}

identity_token "azure" {
  audience = ["api://AzureADTokenExchange"]
}
```

Reference tokens in deployments using `identity_token.<name>.jwt`

### Store Block

Access HCP Terraform variable sets within Stack deployments:

```hcl
store "varset" "aws_credentials" {
  id       = "varset-ABC123"  # Alternatively use: name = "varset_name"
  source   = "tfc-cloud-shared"
  category = "terraform"      # Alternatively use: category = "env" for environment variables
}

deployment "production" {
  inputs = {
    aws_access_key = store.varset.aws_credentials.AWS_ACCESS_KEY_ID
  }
}
```

Use to centralize credentials and share variables across Stacks. See `references/deployment-blocks.md` for details.

### Deployment Block

Define deployment instances (minimum 1, maximum 20 per Stack):

```hcl
deployment "production" {
  inputs = {
    aws_region     = "us-west-1"
    instance_count = 3
    role_arn       = local.role_arn
    identity_token = identity_token.aws.jwt
  }
}

# Create multiple deployments for different environments
deployment "development" {
  inputs = {
    aws_region     = "us-east-1"
    instance_count = 1
    name_suffix    = "dev"
    role_arn       = local.role_arn
    identity_token = identity_token.aws.jwt
  }
}
```

**To destroy a deployment**: Set `destroy = true`, upload configuration, approve destroy run, then remove the deployment block. See `references/deployment-blocks.md` for details.

### Deployment Group Block

Group deployments together for shared settings (HCP Terraform Premium tier feature). Free/standard tiers use default groups named `{deployment-name}_default`.

```hcl
deployment_group "canary" {
  auto_approve_checks = [deployment_auto_approve.safe_changes]
}

deployment "dev" {
  inputs = { /* ... */ }
  deployment_group = deployment_group.canary
}
```

Multiple deployments can reference the same group. See `references/deployment-blocks.md` for details.

### Deployment Auto-Approve Block

Define rules to automatically approve deployment plans (HCP Terraform Premium tier feature):

```hcl
deployment_auto_approve "safe_changes" {
  deployment_group = deployment_group.canary

  check {
    condition = context.plan.changes.remove == 0
    reason    = "Cannot auto-approve plans with resource deletions"
  }
}
```

**Available context variables**: `context.plan.applyable`, `context.plan.changes.add/change/remove/total`, `context.success`

**Note:** `orchestrate` blocks are deprecated. Use `deployment_group` and `deployment_auto_approve` instead.

See `references/deployment-blocks.md` for all context variables and patterns.

### Publish Output and Upstream Input Blocks

Link Stacks together by publishing outputs from one Stack and consuming them in another:

```hcl
# In network Stack - publish outputs
publish_output "vpc_id_network" {
  type  = string
  value = deployment.network.vpc_id
}

# In application Stack - consume outputs
upstream_input "network_stack" {
  type   = "stack"
  source = "app.terraform.io/my-org/my-project/networking-stack"
}

deployment "app" {
  inputs = {
    vpc_id = upstream_input.network_stack.vpc_id_network
  }
}
```

See `references/linked-stacks.md` for complete documentation and examples.

## Terraform Stacks CLI

**Note**: Terraform Stacks is Generally Available (GA) as of Terraform CLI v1.13+. Stacks now count toward Resources Under Management (RUM) for HCP Terraform billing.

### Initialize and Validate

```bash
terraform stacks init              # Download providers, modules, generate lock file
terraform stacks providers-lock    # Regenerate lock file (add platforms if needed)
terraform stacks validate          # Check syntax without uploading
```

### Deployment Workflow

**Important**: No `plan` or `apply` commands. Upload configuration triggers deployment runs automatically.

```bash
# 1. Upload configuration (triggers deployment runs)
terraform stacks configuration upload

# 2. Monitor deployments
terraform stacks deployment-run list                          # List runs (non-interactive)
terraform stacks deployment-group watch -deployment-group=... # Stream status updates

# 3. Approve deployments (if auto-approve not configured)
terraform stacks deployment-run approve-all-plans -deployment-run-id=...
terraform stacks deployment-group approve-all-plans -deployment-group=...
terraform stacks deployment-run cancel -deployment-run-id=...  # Cancel if needed
```

### Configuration Management

```bash
terraform stacks configuration list                    # List configuration versions
terraform stacks configuration fetch -configuration-id=...  # Download configuration
terraform stacks configuration watch                   # Monitor upload status
```

### Other Commands

```bash
terraform stacks create              # Create new Stack (interactive)
terraform stacks fmt                 # Format Stack files
terraform stacks list                # Show all Stacks
terraform stacks version             # Display version
terraform stacks deployment-group rerun -deployment-group=...  # Rerun deployment
```

## Monitoring Deployments with HCP Terraform API

For programmatic monitoring in automation, CI/CD, or non-interactive environments (like AI agents), use the HCP Terraform API instead of CLI watch commands. The API provides endpoints for:

- Configuration status and validation
- Deployment group summaries
- Deployment run status
- Deployment step details (plan/apply)
- Error diagnostics with file locations and code snippets
- Stack outputs via artifacts endpoint

**Key points:**

- CLI watch commands stream indefinitely and don't work in automation
- Use artifacts endpoint to retrieve Stack outputs: `GET /api/v2/stack-deployment-steps/{step-id}/artifacts?name=apply-description`
- Diagnostics endpoint requires `stack_deployment_step_id` query parameter
- Artifacts endpoint returns HTTP 307 redirect (use `curl -L`)

For complete API workflow, authentication, polling best practices, and example scripts, see `references/api-monitoring.md`.

## Common Patterns

**Component Dependencies**: Dependencies are automatically inferred when one component references another's output (e.g., `subnet_ids = component.vpc.private_subnet_ids`).

**Multi-Region Deployment**: Use `for_each` on providers and components to deploy across multiple regions. Each region gets its own provider configuration and component instances.

**Deferred Changes**: Stacks support deferred changes to handle dependencies where values are only known after apply. This enables complex multi-component deployments where some resources depend on runtime values from other components (cluster endpoints, generated passwords, etc.).

For complete examples including multi-region deployments, component dependencies, deferred changes patterns, and linked Stacks, see `references/examples.md`.

## Best Practices

1. **Component Granularity**: Create components for logical infrastructure units that share a lifecycle
2. **Module Compatibility**:
   - Modules used with Stacks cannot include provider blocks (configure providers in Stack configuration)
   - **Test public registry modules** before using in production Stacks - some modules may have compatibility issues
   - Consider using raw resources for critical infrastructure if module compatibility is uncertain
   - Example: Some terraform-aws-modules versions have been found to have compatibility issues with Stacks (e.g., ALB and ECS modules)
3. **State Isolation**: Each deployment has its own isolated state
4. **Input Variables**: Use variables for values that differ across deployments; use locals for shared values
5. **Provider Lock Files**: Always generate and commit `.terraform.lock.hcl` to version control
6. **Naming Conventions**: Use descriptive names for components and deployments
7. **Deployment Groups**: You can organize deployments into deployment groups. Deployment groups enable auto-approval rules, logical organization, and provide a foundation for scaling. Deployment groups are an HCP Terraform Premium tier feature
8. **Testing**: Test Stack configurations in dev/staging deployments before production

## Troubleshooting

**Circular Dependencies**: Refactor to break circular references or use intermediate components.

**Deployment Destruction**: Cannot destroy from UI. Set `destroy = true` in deployment block, upload configuration, and HCP Terraform creates a destroy run.

**Empty Diagnostics**: Add required `stack_deployment_step_id` query parameter to diagnostics API requests.

**Module Compatibility**: Test public registry modules before production use. Some modules may have compatibility issues with Stacks.

## References

For detailed documentation, see:

- `references/component-blocks.md` - Complete component block reference with all arguments and syntax
- `references/deployment-blocks.md` - Complete deployment block reference with all configuration options
- `references/linked-stacks.md` - Publish outputs and upstream inputs for linking Stacks together
- `references/examples.md` - Complete working examples for multi-region and component dependencies
- `references/api-monitoring.md` - Full API workflow for programmatic monitoring and automation
- `references/troubleshooting.md` - Detailed troubleshooting guide for common issues and solutions