I’ve been getting to grips with terragrunt quite well as I’ve been using it extensively for the last 6 months. For most things it is great, but there are a couple of things I cannot figure out.
If I want to use a module from the terraform registry, is there any benefit at all in being able to pull that with one line of code using source other than just being able to change the version at any time? The reason I am asking is that I have tried two ways in using modules but neither seems to be the perfect solution.
There is the method you have outlined in your example for Consul here: https://github.com/gruntwork-io/terragrunt-infrastructure-modules-example/tree/master/consul which requires you to build a main.tf, variables.tf and outputs.tf and basically extrapolate out the variables again, so that you can define in your terraform.tfvars “root module” file along with the terragrunt config.
When using this method, which requires you to at a minimum specify the required variables, what happens if you want to then go ahead and override a non-required variable in your terraform.tfvars file? Do you need to then add this variable to the module block in main.tf and also call out the variable block in variables.tf before you can use it in your tfvars file?
If so, then isn’t it much easier just to download the entire module from github and just use it locally (minus the ability to easily update the version)?
Pulling the registry/github module directly into your terraform.tfvars file and not using the module locally at all. This means you save time by being able to run the module fairly quickly, but as far as I can see there is no way to use data.terraform_remote_state with this method. I tried creating another .tf file alongside the terraform.tfvars file so that terragrunt can pull it in while building the .cache but this did not work because we are using data to override user-defined variables.
to download the entire module from github and just use it locally
Do you mean clone the repo locally and modify the module’s main.tf file to include the provider and remote state configurations?
If so, one better way to handle this is to fork the repo and reference that in the terragrunt tfvars file. That way, it is linked in github so you can merge in any changes made to the source repo to keep up to date, but have the flexibility to define the provider and the state backend blocks in the module code.
That said, this only applies if you are using the module directly with no other resources being created. In practice, most of the modules need surrounding resources that you might want to create with the module. In those cases, it is more robust to wrap the module in another module that provides the additional necessary components.
Note that modules are an abstraction layer, and like all abstraction layers, you need to decide how deep you want to go and what abstractions you want to provide in your code. The open source registry modules and our Gruntwork modules provide one layer of abstraction, but are meant to be generalized and cover a wide range of use cases. However, in most cases, you will want to be opinionated and constrained about how to call those modules in your code. The second layer of abstraction in the module like https://github.com/gruntwork-io/terragrunt-infrastructure-modules-example/tree/master/consul is our recommended way to inject your opinions in how to call the underlying registry module.
Yes I mean to clone the repo locally. We are not yet using GitHub in my company, we are still on SVN but plan to migrate to Github in the coming months.
Forking the repo makes sense. I get what you are saying about wrapping the module in another module, but this means that I need to call out all variables I need, locally, again, so that I can use the variables from the original registry/github module. Of course yes this lets me define the provider and backend, along with data, but I find this really long winded. If I simply copied the code from github and used it locally I would be able to insert the provider and backend into the modified code along with any data.
I just thought that the purpose of using the registry or github was so fire out infrastructure quickly and easily by using only a few lines of code. It seems unless you want to use those modules in a very vanilla way with no custom data or whatever else, you must make a local copy of the module regardless, whether this is a simply copy and paste, or wrapping it in another module as shown in your Consul example. I guess I misunderstood!