Reset the digest of a terraform state in one command

Caveat: This solution only applies on states stored on AWS and relying on DynamoDB, the default, to store the lock/checksum. YMMV on other platforms, but the logic is the same.

Usually, running a terraform command is easy and, even if your plan contains errors, returns everything in a clean state. Sometimes, it fails badly¹ and you get the following error:

Error: Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional request failed
        status code: 400, request id: P5T1PGB8VQDI4GKIIT8G5B9K0VVV4KQNSO5AEMVJF66Q9ASUAAJG
Lock Info:
  ID:        c18b397f-817b-b5dc-53cb-cd24ab01f789
  Path:      path/terraform.tfstate
  Operation: OperationTypeApply
  Who:       user@box
  Version:   0.11.7
  Created:   2018-08-03 12:24:03.5267495 +0000 UTC
  Info:

Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the "-lock=false"
flag, but this is not recommended.

This error is easily fixed by running the force-unlock command with the ID given in message.

But sometimes, even after running the force-unlock command, you still get an error:

Error loading state: state data in S3 does not have the expected content.

This may be caused by unusually long delays in S3 processing a previous state
update. Please wait for a minute or two and try again. If this problem
persists, and neither S3 nor DynamoDB are experiencing an outage, you may need
to manually verify the remote state and update the Digest value stored in the
DynamoDB table to the following value: 9355476ad083c709fe8da56ca7b60b73uri

If you're not familiar with DynamoDB, or don't have direct access to table due to IAM restrictions, here is a single aws command to run to reset the digest :

$ aws dynamodb update-item --table-name terraform-states-lock --key '{"LockID": {"S": "<LOCKID_PATH>"}}' --attribute-updates '{"Digest": {"Value": {"S": "<DIGEST>"},"Action": "PUT"}}' --return-values UPDATED_NEW | jq '.Attributes.RuleSetVersion.S'

You need to change the values of <LOCKID_PATH> and <DIGEST> to match your own, of course.

Often, it is due to a Ctrl+C done at the wrong time. Sh** happens. ↩

Reset the digest of a terraform state in one command

About Pierre-Yves Gillier

Comments