Terraform terraform_remote_state Partial Configuration

Question

My team relies heavily on S3 remote state from within Terraform. We use the -backend-config feature of the CLI to specify the S3 configuration when initializing projects, so our actual terraform code looks like:

terraform {
  backend "s3" {}
}

The above works great as long as all the S3 attributes are specified on the CLI with -backend-config.

We would like to use a similar strategy for referencing these states elsewhere in our configurations. Since the parameters for the backend are dynamic and specified on the CLI, we are looking to do the same.

data "terraform_remote_state" "dns" {
  backend = "s3"
  config {
    key = "configurations/production/dns/terraform.tfstate"
  }
}

In the above example, we've omitted the required region and bucket parameters, which of course causes plan/apply to fail (with not a valid region:).

Is there a method by which we can specify the region and bucket for remote state references from the CLI instead of hard-coding them?

Answer 1

The backend block is rather special because it gets processed so early in Terraform's workflow, and thus it doesn't have access to normal Terraform features such as variables. That's why it has its own special mechanism for configuring it.

The terraform_remote_state data source, on the other hand, is just a regular data source and so any normal interpolation strategy can be used with it. To pass settings from the CLI, for example, you could use variables:

variable "dns_state_region" {
}

variable "dns_state_key" {
}

data "terraform_remote_state" "dns" {
  backend = "s3"
  config {
    region = "${var.dns_state_region}"
    key    = "${var.dns_state_key}"
  }
}

You can then pass these to the terraform plan command:

$ terraform plan \
    -var="dns_state_region=us-west-1" \
    -var="dns_state_key=configurations/production/dns/terraform.tfstate"