I’ve got resources that I need to describe via variables in order to populate a template. If I need to create many similar resources, I inevitably want to put that description within a list, so that I can iterate over the list via count and cause a single resource declaration to create n resources. Doing this seems to be entirely broken in terraform. When I have a single resource, I just declare a variable for each parameter that may change from environment to environment, and then I have a separate vars.tf file in each environment if I’m going to have multipe parallel environments. That’s simple enough. But if I have multiple similar resources in the same environnment, how do you manage that problem? It’s fine, if less than optimal, if there’s a static number and I just have to define separate variables for each instance - assuming the number is small. But as soon as the number of resources is dynamic, how does one accomplish this?
You can’t have a list of maps. Because none of the functions for extracting values from lists or maps will work on a compound variable. You can’t reasonably have m lists instead of a map for each resource with m keys, both because that would be INCREDIBLY error prone, requiring me to maintain multiple lists AND ensure that updates to a list never accidentally re-order any of the elements, since I cannot refer to them by name.
You can’t have n maps of m keys, because n may be a large number and I would have to refer to each map with a different variable name, so no way to use a single resource declaration with count to access multiple maps - unless you can put the maps in a list, which you cannot do if you want to get any values back out.
But more to the point, if the resource is at all complex, the odds of the set of descriptive values being able to exist in a flat structure are pretty much nil, so you actually need separate lists and maps for each nested element, and that would quickly become impossible to manage.
I asked on the terraform google group, and the response was to use an external tool to RENDER a template dynamically with each resource explicitly listed and all variables already populated so that HCL doesn’t have to do any work. But that is a hugely problematic kludge. Not only does it mean that I have to pre-render my templates - something I probably cannot do when using terragrunt because my template rendering is all too likely to be dependent on outputs of other modules/templates that are executed first, but that’s also an incredibly painful extra step when my vars and templates are separated into separate repos when following terragrunt best-practices.
So does anyone have a solution that works in a terragrunt context? I find it mindboggling that terraform doesn’t support it’s own configuration language in any meaningful way - why define a configuration language that supports nested structures if you can’t write functions/operators which are able to interact with those nested structures? It’s hard to think of any other language out there which uses a square bracket for dereferencing a nested structure which does not support chaining the damn things together - my_list_of_maps[index][“key”] is not valid syntax in HCL, even though my_list[index] is valid and my_map[“key”] is valid, for some incredibly stupid reason. It’s honestly a little difficult to understand how you would even write a parser that does NOT understand that syntax, since any recursive traversal of a parse tree ought to function correctly when it sees that. Never mind that literally every other language in widespread use does exactly that.
Is there some way to have terragrunt run an external tool to render a template prior to running terraform, so that I can at least avoid a manual pre-render step in the template/modules repo?
I can’t possibly be the first person to want to do something like this. I find it mindboggling that there isn’t a standard mechanism for doing this within a tool intended to do the job that terraform is (apparently) intended to do.