Terraform stalls while trying to get IP addresses of multiple instances?

Question

So I am using terraform to provision ec2 instances and openstack instances. I am trying to reference the IP addresses of the instances I am creating because I need to run commands that use them (to set up consul). However after adding references to these variables terraform just stalls out and does absolutely nothing after I run a terraform apply or terraform plan:

Here is a sample of the resource block for what I am trying to run:

resource "aws_instance" "consul" {
  count = 3
  ami = "ami-ce5a9fa3"
  instance_type = "t2.micro"
  key_name = "ansible_aws"
  tags {
    Name = "consul"
  }

  connection {
    user = "ubuntu"
    private_key="${file("/home/ubuntu/.ssh/id_rsa")}"
    agent = true
    timeout = "3m"
  }

  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y curl",
      "echo ${aws_instance.consul.0.private_ip} >> /home/ubuntu/test.txt",
      "echo ${aws_instance.consul.1.private_ip} >> /home/ubuntu/test.txt",
      "echo ${aws_instance.consul.2.private_ip} >> /home/ubuntu/test.txt"
    ]
  }
}

Update: so I tried running a similar command with my openstack cloud and got the same problem:

  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y curl",
      "echo ${openstack_compute_instance_v2.consul.0.network.0.fixed_ip_v4}",
      "echo ${openstack_compute_instance_v2.consul.1.network.1.fixed_ip_v4}",
      "echo ${openstack_compute_instance_v2.consul.2.network.2.fixed_ip_v4}"
    ]
  }

So I have found if instead I use only one of the IP addresses then the other instances wont even be created until my first instance is created, like in the block below:

  provisioner "remote-exec" {
    inline = [
      "echo ${openstack_compute_instance_v2.consul.0.network.0.fixed_ip_v4}",
    ]
  }

I need all my instances to be created at the same time and have access to the IP addresses of all the other instances created as soon as they are created.

Answer 1

So taking a longer look at this it looks like it is actually an issue with Terraform where it gets locked into a dependency cycle.

That same issue also includes a workaround for this by using a null_resource to then connect to all of the provisioned instances and run a script against them.

For your use case you could use something like this:

resource "aws_instance" "consul" {
  count = 3
  ami = "ami-ce5a9fa3"
  instance_type = "t2.micro"
  key_name = "ansible_aws"
  tags {
    Name = "consul"
  }
}

resource "null_resource" "configure-consul-ips" {
  count = 3

  connection {
    user = "ubuntu"
    private_key="${file("/home/ubuntu/.ssh/id_rsa")}"
    agent = true
    timeout = "3m"
  }

  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y curl",
      "sudo echo '${join("\n", aws_instance.consul.*.private_ip)}' > /home/ubuntu/test.txt"
    ]
  }
}

This should bring up 3 instances and then connect to them, install curl and then create a file with a new line separated list of the private IP addresses of the 3 instances.