问题
I am trying to apply my infra changes with terraform's docker image from my CI pipeline using gitlab-ci. However, i am getting some timing related errors. I kind of suspect some kind of time drift somewhere?
I have tried setting timezone and installing NTP and synching the timezone. I verified that the time and TZ has changed from the default UTC to my local timezone GMT+8. However, the error still occurs.
Notice that the time elpased went from 20s to 8h so it seems to be there are some problem with the internal clock?
I am kind of confused if this is a terraform issue or docker issue or both? Any help will be greatly appreciated. Thank you.
module.ecs_singapore.network.aws_nat_gateway.main: Still creating... (20s elapsed) module.ecs_singapore.aws_alb.main: Still creating... (20s elapsed)
module.ecs_singapore.network.aws_nat_gateway.main: Still creating... (8h0m7s elapsed) module.ecs_singapore.aws_alb.main: Still creating... (8h0m8s elapsed)
Failed to save state: failed to upload state: RequestTimeTooSkewed: The difference between the request time and the current time is too large. status code: 403, request id: 0B0026E03C66248C, host id: n4DuSL4lWCUY0uXCLecwNtOyszV/Pr6ucFMZYxr4TjMuPWNHz2UYtuziy5dysFpJncWdwss1XTo=
Error releasing the state lock!
Error message: failed to retrieve lock info: InvalidSignatureException: Signature not yet current: 20181206T223022Z is still later than 20181206T144544Z (20181206T143044Z + 15 min.) status code: 400, request id: G5CBO6G31177HOF9IB22HKEQDBVV4KQNSO5AEMVJF66Q9ASUAAJG
Terraform acquires a lock when accessing your state to prevent others running Terraform to potentially modify the state at the same time. An error occurred while releasing this lock. This could mean that the lock did or did not release properly. If the lock didn't release properly, Terraform may not be able to run future commands since it'll appear as if the lock is held.
回答1:
I had faced this issue once, few things to note which can help -
Most importantly, you need to maintain the same timezone on your host machine & inside the container both. If changing timezone doesn't work, change host & container timezone to UTC itself.
Are you using STS? If yes, you will need to fetch/refresh the token again after timezone sync/change.
I am quite certain that this is a timezone issue which mostly occurs when we try to run a terraform binary inside the docker container in case of a timezone conflict on your host machine & docker container.
回答2:
The root cause for me was an incorrect time setting in my ESXi host. That wrong time is being synched to all the VMs running in it, which resulted in this issue. Everything work fine once i set it back to the right UTC time.
来源：https://stackoverflow.com/questions/53740432/terraform-apply-failed-due-to-mismatch-time