Variable precedence and overrides between parent and child tfvars

Gruntwork Customers
1

In my experience it would appear that variables with the same name/key defined in both a parent and child tfvars file will always use the parent’s definition as the override value. This strikes me as odd. Generally I would envision the parent var being of a more generic nature and the child var being able to override that.

Consider this example…

live
├── test
│   ├── thing1
│   │   └── terraform.tfvars
│   └── terraform.tfvars

If I have foo = "chinchilla" defined in live/test/terraform.tfvars and foo = "bar" defined in live/test/thing1/terraform.tfvars (along with the usual include { path = "${find_in_parent_folders()}" } stuff), my var value (what is picked up by my terraform module) is set to "chinchilla". I would like/expect that the child (thing1) value (bar) would be the override here but that is not what I am seeing.

  1. Is this the intended behavior?
  2. Is there a way/hack around this to where I can use the child’s value as the override?
2

Hm, the only thing that should be getting “included” from the parent is the Terragrunt configuration; not the other variables in there. Those shouldn’t have any impact at all.

Could you share your log output?

3

Thanks Jim.
I probably should have mentioned we’re doing some var file inclusion using required_var_files and optional_var_files in the parent tfvar file.
e.g.

terragrunt = {
  terraform {
    extra_arguments "root_tfvars" {
      commands = ["${get_terraform_commands_that_need_vars()}"]

      required_var_files = [
        # literally _this_ file (env parent dir)
        "${get_parent_tfvars_dir()}/terraform.tfvars",
      ]

      optional_var_files = [
        # common.tfvars file in 'live' root dir
        "${get_parent_tfvars_dir()}/../common.tfvars",
      ]

      arguments = [
        "-lock-timeout=20m",
        # Set's runtime derived var used by tag module
        "-var", "tfstate_key=${path_relative_to_include()}/terraform.tfstate",
      ]
    }
  }

  remote_state {
    backend = "s3"
    config {
      encrypt        = true
      bucket         = "test-env.terraform"
      key            = "${path_relative_to_include()}/terraform.tfstate"
      region         = "us-west-2"
      dynamodb_table = "terraform-lock-table"
    }
  }
}

# If changing any of these variables, do a search/replace across this entire
# repo and ensure that any called modules reflect the correct variable names.

################################################################################
# The variables below are different between envs
################################################################################

foo = "chinchilla"
4

Ah, that’s the issue then. Check out the docs on Terraform variable precedence. Since you’re explicitly specify the parent terraform.tfvars file with -var-file, it’ll take precedence over the terraform.tfvars file in the folder where your app is defined. To reverse that, you’ll have to specify both .tfvars files via -var-file.

5

Boy, if I had looked closely at the output of the run I would have noticed how terragrunt is turning the required_var_files and optional_var_files into -var-file cmd line args. doh!

One thing I would maybe suggest (or perhaps should I file a feature request?) is that the optional_var_files get placed before the required_var_files in the command string so that vars defined optionally don’t override the required ones. IDK… you guys may have thought that through and have very strong reasons/opinions for doing it the way it is currently done. For now I will move all of my hierarchical var files into required_var_files in the particular order I need them processed in and not use the optional_var_files arg.

Thanks for clearing this up for me!

Just to close the loop for anyone watching, the fix for me to get the desired hiearchical var override precedence in place I changed my terraform section in the parent terraform.tfvars file to look like this:

  terraform {
    extra_arguments "root_tfvars" {
      commands = ["${get_terraform_commands_that_need_vars()}"]

      required_var_files = [
        # common.tfvars file in 'live' root dir
        "${get_parent_tfvars_dir()}/../common.tfvars",
        # literally _this_ file (env parent dir)
        "${get_parent_tfvars_dir()}/terraform.tfvars",
        # the calling/child var file (gives it override)
        "terraform.tfvars"
      ]

      arguments = [
        "-lock-timeout=20m",
        # Set's runtime derived var used by tag module
        "-var", "tfstate_key=${path_relative_to_include()}/terraform.tfstate",
      ]
    }
  }
6

That’s a good point. Not sure that we handled that in any intentional way. PR to tweak the order would be very welcome!

Thanks for sharing!

7

This post really made it click for me on how variables work with Terragrunt/Terraform. Thanks.