Running Multiple Instances Of A Module With Varying Fields

Gruntwork Customers
, ,
1

I am having a very hard time adjusting to Terragrunt. In my past experience I would go to the Terraform Registry and find a widely used module, and then I might make slight modifications, and then I’d use the module. For example, for an IAM policy module, I would use this widely used module. Then in my environments, I would call instances of that module based on the README. Example:

#########################################
# IAM policy
#########################################
module "iam_policy" {
  source = "../../modules/iam-policy"

  name        = "example"
  path        = "/"
  description = "My example policy"

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "ec2:Describe*"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}
EOF

  tags = {
    PolicyDescription = "Policy created using heredoc policy"
  }
}

And maybe there’d be 200 different instances of this module where each has a different policy. But, in terragrunt, it seems that I can define one inputs to that module when really i need 200+ varying inputs (200+ instances of the module). In order to achieve that, I find myself doing all sorts of hacky things when all i want to do is create many instances of this module with different policies for each. Another example is what I asked regarding a SNS topic module.

The TLDR of what I’m asking is how are people handling these cases in Terragrunt where you want to simply run many variations of a module like you may have done in native Terraform? I am finding myself doing very hacky things with locals and for_each loops. I feel that I must be missing something and would like advice. All of the examples I’ve found are extremely basic cases with 1 instance of a module represented by 1 set of inputs.

2

Hi @Howard_Roark ,

Typically we recommend creating a wrapper module in this case that Terragrunt calls. What you want to do is design a Terraform module that identifies a use case that should be replicated in each of your environments, and then use terragrunt to deploy that module, instead of directly calling the module on the registry. This gives you the flexibility of using the looping constructs available in terraform with for_each and as such. The modules on the registry are typically designed for use in a larger Terraform module rather than representing a directly deployable unit, so they will tend to feel much more granular than what you need.

Alternatively, for one off use cases where you have dynamic data, you can use generate blocks to generate a terraform module on the fly. For example, if you had a folder that contains separate policy files, you can do something like the following:

locals {
  datafiles = compact(split("\n", run_cmd("ls", "/folder/that/contains/datafiles.yaml")))
  data = [for datafile in local.datafiles : yamldecode(file(datafile))]
}

generate "main" {
  path      = "provider.tf"
  if_exists = "overwrite"
  contents = <<EOF
%{ for policy in local.data }
module "iam_policy_${policy.name}" {
  source = "../../modules/iam-policy"

  name        = "${policy.name}"
  path        = "/"
  description = "${policy.description}"

  policy = <<EOF
${policy.json}
EOF
}
%{ endfor }
EOF
}

In the second approach, you are relying on terragrunt to generate a terraform module dynamically that replicates all the module calls you want to make.

Hope this helps!

Best regards,
Yori

2 Likes