Remote State & Backends In Depth

Why remote state is a team requirement, not a nice-to-have#

The providers, resources & state page introduced the state file and showed a bare-bones S3 backend block. This page goes further: setting the backend up correctly (including the bootstrapping problem nobody warns you about), what a real locking conflict looks like, how to inspect and repair state safely, and how to share values between separate Terraform configurations.

A local terraform.tfstate file works for a solo experiment and nothing more. The moment a second person, or a CI pipeline, might run terraform apply against the same infrastructure, a local file causes two concrete failures:

• Two applies can race. Nothing stops a teammate from running apply at the same moment you do. Both read the same "current" state, both compute a plan against it, and whichever finishes last overwrites the other's state update — Terraform's own bookkeeping about what exists can end up wrong even though the underlying cloud resources are fine.
• Nobody else has the file. If state only exists on your laptop, a teammate's plan has no way to know what you've already created — it will try to create everything again from scratch, or fail confusingly when the cloud API rejects the duplicate.

A remote backend fixes both problems: it stores the state file somewhere every team member and every CI job can reach, and (for backends that support it) uses locking to guarantee only one apply runs at a time against a given state file.

Setting up an S3 backend with DynamoDB locking#

The classic, still very common setup on AWS is an S3 bucket for the state file itself plus a DynamoDB table used purely to coordinate locks:

# backend.tf
terraform {
  backend "s3" {
    bucket         = "myapp-terraform-state"
    key            = "prod/network/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-locks"
    encrypt        = true
  }
}

• bucket — the S3 bucket holding the state file. One bucket is commonly shared across many projects/environments, each using a different key.
• key — the path within the bucket for this particular state file. Using a path like prod/network/terraform.tfstate lets one bucket hold many independent states side by side (see multi-environment strategies for how this fits into a larger layout).
• dynamodb_table — the table used for locking, covered below.
• encrypt — turns on server-side encryption for the state object at rest, which matters because state routinely contains sensitive values.

The bootstrapping problem#

There's a chicken-and-egg problem here worth naming explicitly: the S3 bucket and DynamoDB table are themselves infrastructure — so can Terraform create them too? Not inside the same configuration that's going to use them as its backend, because terraform init needs the backend to already exist before it can do anything at all.

The standard fix is a tiny, separate bootstrap configuration, applied once with plain local state, whose only job is creating the backend resources:

# bootstrap/main.tf — run once, with local state, before anything else
resource "aws_s3_bucket" "terraform_state" {
  bucket = "myapp-terraform-state"

  lifecycle {
    prevent_destroy = true
  }
}

resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_dynamodb_table" "terraform_locks" {
  name         = "terraform-locks"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }
}

A few details matter:

• prevent_destroy = true on the state bucket is a real safety net — an accidental terraform destroy run against the bootstrap config should not be able to delete the bucket every other configuration depends on.
• Versioning on the bucket means every past revision of the state file is retained, so a corrupted or bad state write can be recovered by restoring a previous object version — genuinely useful when something goes wrong with terraform state surgery.
• The DynamoDB table's partition key must be named exactly LockID, type String — this is a hard requirement of Terraform's S3 backend locking implementation, not a naming choice you get to make.

Every other configuration in the organization then simply points at these already-existing resources in its own backend "s3" {} block — it never manages them.

A newer alternative: native S3 locking#

As of Terraform 1.10, the S3 backend can lock directly against S3 itself (using conditional writes), removing the DynamoDB dependency entirely:

terraform {
  backend "s3" {
    bucket       = "myapp-terraform-state"
    key          = "prod/network/terraform.tfstate"
    region       = "us-east-1"
    use_lockfile = true
    encrypt      = true
  }
}

This is worth knowing about, but the DynamoDB approach above remains extremely common in existing codebases and in older Terraform versions, which is why it's the one to understand thoroughly.

What a state locking conflict actually looks like#

Every plan and apply acquires the lock first, does its work, then releases it. If a second apply starts while the first is still running, it doesn't corrupt anything — it simply fails fast with an error telling you exactly who's holding the lock:

$ terraform apply

Error: Error acquiring the state lock

Error message: ConditionalCheckFailedException: The conditional request failed
Lock Info:
  ID:        7b3f2a1c-4e5d-6f7a-8b9c-0d1e2f3a4b5c
  Path:      myapp-terraform-state/prod/network/terraform.tfstate
  Operation: OperationTypeApply
  Who:       ali@ci-runner-042
  Version:   1.9.2
  Created:   2026-08-26 09:14:22 UTC
  Info:

Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and
try again.

The correct response almost always is: wait for the other operation to finish, then retry. The lock is exactly doing its job here.

force-unlock — a last resort#

Occasionally a process is killed (a CI job timing out, a laptop losing its network connection mid-apply) without releasing its lock — a genuinely stale lock, where the "Created" timestamp is old and you've confirmed nothing is actually still running. Only then:

terraform force-unlock 7b3f2a1c-4e5d-6f7a-8b9c-0d1e2f3a4b5c

Running this against a lock that's still legitimately in use is exactly how two applies end up racing against the same state — treat it as a break-glass command, not a routine one.

Inspecting and repairing state with terraform state#

Terraform ships a family of subcommands for looking at, and carefully editing, what's recorded in state — useful once a config grows beyond a handful of resources:

# List every resource address currently tracked in state
terraform state list

# Show the full recorded attributes for one resource
terraform state show aws_s3_bucket.app_backups

# Rename a resource's address in state without destroying/recreating it
# (e.g. after renaming the resource block itself in your .tf files)
terraform state mv aws_s3_bucket.old_name aws_s3_bucket.new_name

# Remove a resource from state WITHOUT destroying the real infrastructure
# — Terraform simply "forgets" it; useful when a resource is being handed
# off to be managed by a different configuration.
terraform state rm aws_s3_bucket.app_backups

# Download the current remote state as raw JSON, for manual inspection
terraform state pull > state-snapshot.json

state mv in particular is the tool for the extremely common case of refactoring a .tf file — renaming a resource, or moving it into a module — without Terraform interpreting that as "destroy the old one, create a new one."

Sharing values between configurations: terraform_remote_state#

Splitting infrastructure into multiple, independently-applied configurations (a network config, an app config, a database config — see multi-environment strategies for why this split is often a good idea) means one config sometimes needs an output value from another. The terraform_remote_state data source reads another configuration's state file directly:

# In the "app" configuration, reading an output from the "network" configuration
data "terraform_remote_state" "network" {
  backend = "s3"

  config = {
    bucket = "myapp-terraform-state"
    key    = "prod/network/terraform.tfstate"
    region = "us-east-1"
  }
}

resource "aws_instance" "app" {
  subnet_id = data.terraform_remote_state.network.outputs.private_subnet_id
  # ...
}

This only works for values the source configuration explicitly exposed with an output block — it's another reason to treat a configuration's outputs as its public interface to the rest of the organization, not an afterthought.

Comparing backend approaches#

Common mistakes#

• Trying to create the S3 bucket and DynamoDB table for a backend inside the same configuration that uses them as its backend — this is the bootstrapping chicken-and-egg problem; use a separate one-time bootstrap config instead.
• Naming the DynamoDB table's partition key anything other than exactly LockID — Terraform's S3 backend locking implementation depends on that exact attribute name.
• Running terraform force-unlock reflexively when a plan/apply fails on a lock, instead of first confirming the lock is genuinely stale — force-unlocking a lock that's still legitimately held reintroduces the exact race condition locking exists to prevent.
• Forgetting to enable versioning on the state bucket, then having no way to recover from a bad state write short of manually reconstructing it.
• Using terraform state rm when the intent was actually terraform destroy — state rm only makes Terraform forget the resource; the real infrastructure keeps running, unmanaged, and can silently keep costing money.