Store state in a remote backend with locking, encryption, version recovery, least-privilege access, and audit visibility. Separate state by operational ownership and blast radius. Allow one controlled apply path per state, review saved plans, scan for drift, and treat every state export, backup, and plan artifact as sensitive. Test restoration before an incident.
Understand what Terraform state controls
Terraform state maps resource addresses in configuration to real objects and records metadata Terraform needs to plan changes. The current Terraform state documentation recommends remote storage for collaboration and warns against storage without secure access control and locking.
State is not merely a cache that can be deleted and recreated safely. Losing a binding can make Terraform propose a duplicate resource, while an incorrect binding can make it update or destroy the wrong object. A production state design therefore has four jobs:
- Consistency: serialize writers and reject stale operations.
- Confidentiality: restrict and audit reads as carefully as writes.
- Availability: keep versioned, restorable snapshots with a tested recovery procedure.
- Containment: prevent one failed plan or compromised identity from reaching unrelated infrastructure.
.terraform.lock.hcl; do not commit terraform.tfstate, state backups, saved plan files, the .terraform/ directory, or backend credentials.Choose a backend by operational guarantees
“Remote” is only the location. The production decision is whether the complete backend arrangement provides locking, durable history, encryption, access separation, audit events, reliable availability, and a recovery path your team can operate. Terraform backends have different capabilities, and locking is optional rather than universal.
| Option | Best fit | Operational trade-off |
|---|---|---|
| Managed Terraform platform | Teams that want state, runs, permissions, policies, and audit workflow together | Less control over the service boundary; platform availability and pricing become dependencies |
| Cloud object backend | Teams already operating a cloud with mature identity, encryption, logging, and backup controls | The team owns bucket hardening, locking configuration, CI serialization, monitoring, and recovery |
| Local state | Disposable learning or isolated experiments with no valuable infrastructure | No practical collaboration or centralized recovery; secrets and loss risk remain on one workstation |
Do not choose a backend only because it is easy to initialize. Start from recovery time, permitted readers and writers, compliance boundaries, expected run concurrency, and the impact of an unavailable backend. Endurance Softwares' Terraform development services and cloud infrastructure engineering cover the wider module, identity, delivery, and operations design around this decision.
Configure a locked, versioned S3 backend
This example uses Amazon S3 because its current Terraform backend supports an opt-in S3 lockfile. Replace every identifier with infrastructure created and governed for your organization:
terraform {
backend "s3" {
bucket = "example-terraform-state"
key = "payments/production/terraform.tfstate"
region = "us-east-1"
encrypt = true
use_lockfile = true
}
}According to the current Terraform S3 backend reference, use_lockfile = true enables S3-based state locking. DynamoDB-based locking is deprecated, so a new design should not adopt it as the default. The documentation also strongly recommends S3 bucket versioning for recovery from accidental deletion or human error.
The bucket should exist before this root module initializes. Bootstrap it with a separate, tightly controlled process or a small independent state; a configuration cannot depend on the backend it is still trying to create. Enable versioning at the bucket level, set encryption and transport controls, block public access, log access through the organization's audit system, and prevent ordinary Terraform identities from changing those protections. AWS documents that S3 Versioning retains object variants, which enables recovery from overwrites and deletions but also requires an intentional retention and cost policy.
Grant only the required object-path permissions
The runtime identity needs access to the state object and, when lockfiles are enabled, the corresponding .tflock object. It should not receive broad administration of the backend bucket. Separate bootstrap administration, state read/write, and audit or recovery roles. For production, scope each runtime identity to the exact environment prefix it operates.
Do not hard-code cloud access keys, temporary credentials, or KMS secrets in the backend block or pass secrets in command history. Terraform's backend configuration guidance explains that backend configuration is copied into .terraform/ and saved plan files; use the provider's environment, workload identity, or standard credential chain instead.
Treat state and plan files as secret-bearing artifacts
A value marked sensitive is redacted from normal CLI and UI output, but that label alone does not keep the value out of state. Terraform's current sensitive-data guidance says state and plan files can contain credentials and other sensitive resource attributes. Protect their full lifecycle:
At rest
Encrypt backend objects and backups. Control encryption-key administration separately from state use where the threat model requires it.
In transit
Require secure transport. Prefer short-lived workload identities over copied, long-lived credentials on developer machines and runners.
During processing
Keep plans and exports out of public logs, generic build artifacts, chat, tickets, and unencrypted local folders. Delete temporary copies according to policy.
Restrict read access: a state reader may recover database endpoints, internal identifiers, generated tokens, or provider-returned attributes even when they were hidden in terminal output. Review provider support for ephemeral values or write-only resource arguments before relying on them, and avoid placing secrets in infrastructure configuration when the destination system can retrieve them directly from a secret manager.
The .terraform.lock.hcl file is different. It records provider dependency selections and checksums and should normally be reviewed and committed. HashiCorp's dependency lock file documentation describes this reproducibility role. It does not lock remote state and does not prevent concurrent applies.
For application-side secret boundaries, see our environment variables and secret management guide.
Split state by ownership and blast radius
One large state makes cross-resource references convenient, but every plan refreshes a wider graph, every writer receives broader permissions, and one error can affect more systems. Hundreds of tiny states create dependency coordination, duplicated provider setup, and operational overhead. Choose boundaries deliberately rather than treating one state per repository or one workspace per environment as a rule.
| Boundary signal | Prefer a separate state when | Coordinate through |
|---|---|---|
| Environment | Production requires distinct credentials, approvals, or recovery | Versioned modules and explicit environment configuration |
| Lifecycle | Network foundations change far less often than application services | Provider lookups, stable identifiers, or a narrow output contract |
| Ownership | Different teams operate resources and need independent delivery | Documented interfaces and consumer-safe changes |
| Risk | Identity, state storage, DNS, or data services need stronger controls | Privileged pipelines and change windows appropriate to that system |
| Scale | Plan duration, provider throttling, or lock contention is materially slowing delivery | A dependency graph and ordered rollout automation |
Avoid exposing a full state merely to share one value. Prefer provider data sources or a deliberately published, non-sensitive interface when practical. If remote-state outputs are used, grant the consumer only the access actually required and remember that backend access often reveals more than the declared outputs.
development and production does not create security. Use separate identities, scoped backend paths, apply permissions, and approval policies that enforce the boundary.Make CI/CD the single production writer
Backend locking prevents simultaneous state-writing operations, but it is not a complete deployment workflow. A production pipeline should also serialize runs per state, verify the exact configuration and dependency lock file, produce a reviewable plan, and apply only the approved plan artifact from a trusted context.
terraform fmt -check
terraform init -input=false
terraform validate
terraform plan \
-input=false \
-lock-timeout=5m \
-out=tfplan
# Apply the reviewed artifact in the same trusted workflow.
terraform apply -input=false -lock-timeout=5m tfplanThe five-minute lock timeout is an example, not a universal setting. It lets a valid writer wait for ordinary contention without disabling locking. Set it from observed plan and apply durations, and surface queue time so persistent contention becomes an architecture signal.
- Pull request: run formatting, validation, policy and security checks; create a plan with read-only or planning permissions where the provider allows it.
- Review: show human-readable changes without publishing raw plan JSON. Require ownership approval for destructive or privileged resources.
- Apply: use the reviewed saved plan from an immutable revision and a short-lived production identity. Serialize by the exact state key, not merely by repository.
- Record: retain commit, actor, plan summary, run identifier, state path, and result in the audit system without retaining exposed secret values.
Avoid automatic production apply from an unreviewed scheduled plan. Avoid routine -lock=false, ad hoc applies from laptops, and two automation systems that can both write the same state. Coordinate application delivery with infrastructure changes: our Kubernetes deployment strategies guide explains how rollout safety also depends on traffic, health, and data compatibility.
Detect drift without normalizing it blindly
A normal Terraform plan refreshes managed objects before comparing configuration and proposed changes. For scheduled detection, -detailed-exitcode distinguishes no change, error, and a non-empty plan. HashiCorp's current terraform plan reference defines exit codes 0, 1, and 2 respectively.
set +e
terraform plan \
-input=false \
-lock-timeout=5m \
-detailed-exitcode \
-out=tfplan
exit_code=$?
set -e
case "$exit_code" in
0) echo "No drift or configuration changes" ;;
1) echo "Terraform plan failed"; exit 1 ;;
2) echo "Review the proposed changes"; exit 2 ;;
esacExit code 2 does not prove unauthorized drift. It can represent a committed configuration change, a provider's normalization behavior, a data-dependent value, or an external modification. Route the plan to review with the configuration revision, provider versions, state identity, and timestamp. Do not upload the raw tfplan file to an artifact store that lacks the state's confidentiality controls.
Reconcile based on intent
- If the real-world change is unauthorized or mistaken, update infrastructure back toward reviewed configuration through the normal plan and apply path.
- If an emergency change is intentional, encode the desired result in configuration and review the normal plan before applying.
- If only Terraform's records should accept an intentional external change, review
terraform plan -refresh-onlybefore any refresh-only apply. - If a resource was created outside Terraform but should now be managed, add configuration and use the supported import workflow; do not paste a fabricated object into state.
The standalone terraform refresh command is deprecated because it updates state automatically. Prefer reviewable planning modes and make drift resolution an engineering decision, not a scheduled overwrite.
Migrate backends as a controlled state change
A backend migration changes where the control-plane record lives. Schedule a write freeze for the affected state, verify both source and destination access, and communicate who owns the migration. Terraform requires reinitialization after backend changes and supports copying state with terraform init -migrate-state.
Prepare
Confirm the current state key and workspace, stop all writers, capture a protected backup, and test destination locking, encryption, permissions, versioning, and audit logs.
Migrate
Change the backend configuration on a dedicated revision, run the migration interactively from a trusted environment, and preserve the complete command output.
Verify
Run state list and a normal plan, confirm expected lineage and resources, then update every runner before reopening writes.
The terraform init reference distinguishes -migrate-state, which attempts to copy existing state, from -reconfigure, which disregards the existing backend configuration and does not migrate it. Do not substitute one because it appears to clear an initialization error.
Keep the previous backend read-only and protected until verification and the recovery window complete. Then remove obsolete writer permissions deliberately. A state copy made for migration is sensitive; store and expire it under the same controls as the active state.
Recover from locks and damaged state without guessing
Most recovery mistakes happen when urgency turns an unclear symptom into manual JSON editing. Pause writers first, preserve evidence, identify the last successful operation, and decide whether the problem is a live lock, stale lock, unavailable backend, incorrect binding, or damaged snapshot.
| Symptom | First checks | Safe direction |
|---|---|---|
| Lock cannot be acquired | Active CI jobs, lock owner, operation, timestamp, backend health | Wait for the writer or stop it cleanly; force-unlock only a confirmed stale lock |
| Plan proposes duplicates | State key, workspace, backend initialization, resource addresses | Stop; restore the correct backend context or use reviewed import/state operations |
| Resource rename causes replacement | Address change and provider plan | Use a declarative moved block or reviewed state mv where appropriate |
| State object was overwritten | Backend object versions, serial, lineage, last successful apply | Preserve current data, select the verified prior version, and rehearse restore steps |
| Backend write failed | Terraform output and any local recovery state it produced | Protect the recovery file immediately; follow backend-specific recovery guidance |
Terraform locks supported backends automatically for operations that can write state. Its state-locking guidance warns that force-unlock should unlock only your own failed lock after confirming no writer still holds it. The lock ID is a safeguard, not evidence that unlocking is safe.
Use terraform state subcommands instead of editing JSON directly, make an additional protected backup before any state surgery, and peer-review the exact addresses. HashiCorp's state recovery workflow covers unlock, pull, and push operations. A recovery plan should specify who can restore backend versions, how lineage and serial are verified, how writes are frozen, and what plan proves recovery succeeded.
Terraform state management checklist
✓ Every valuable state uses a supported remote backend
✓ State locking is enabled and never routinely bypassed
✓ Backend objects, versions, backups, and plans are encrypted
✓ Read and write permissions are scoped to exact state paths
✓ Workload identities are short-lived and audited
✓ State boundaries match ownership, lifecycle, and blast radius
✓ One serialized production workflow owns each state
✓ Saved plans come from immutable reviewed revisions
✓ Drift scans distinguish changes from execution errors
✓ Backend migrations freeze writers and verify a normal plan
✓ Lock and state recovery procedures are documented
✓ Restoration is rehearsed with non-production state
Frequently asked questions
Should Terraform state be committed to Git?
No. State can contain sensitive values and changes outside the code-review model. Commit Terraform configuration and the dependency lock file, while storing state in a protected backend that supports the required collaboration and recovery controls.
Does an S3 backend lock Terraform state automatically?
Not merely because S3 stores the state. In the current S3 backend, S3 lockfile support is opt-in with use_lockfile = true. Confirm the runtime identity also has the documented permissions for the .tflock object.
Is .terraform.lock.hcl the state lock?
No. It records provider dependency selections and checksums for reproducible installation. Backend state locking coordinates state-writing operations. The two mechanisms solve different problems.
How many Terraform state files should a platform use?
There is no universal number. Split where permissions, ownership, lifecycle, failure impact, or plan scale require independence; combine resources that genuinely need one atomic operational lifecycle. Document dependencies between states.
When is terraform force-unlock safe?
Only after confirming the referenced lock is stale and no process still owns the operation. Stop automation, inspect the lock metadata and backend, record the decision, and use the exact lock ID. Unlocking a live writer can create competing state changes.
Build infrastructure delivery around recoverable state
Endurance Softwares helps teams design Terraform modules, cloud foundations, access boundaries, CI/CD workflows, container platforms, observability, and recovery practices for SaaS and custom software systems. If your infrastructure-as-code workflow is moving from individual use to production operations, discuss the architecture with our engineering team.
