Incorporating new updates to modules

Gruntwork Customers
1

Hey guys – I was wondering if there was documentation or a tutorial somewhere to walk one through how to incorporate new fixes to the modules into local instances? I imagine this is mostly just git work, but I’d love to be able to refer back to a document whenever I need to merge/update?

Thanks a bunch!

-charles.

2

Our monthly newsletter lists the updates we’ve made to our code over the last month (we recently started publishing the newsletter on our blog). There are several types of updates you may want to incorporate:

Terraform updates

If you are using Gruntwork’s Terraform modules, then in your infrastructure-modules repo, you most likely have some Terraform code that uses a Gruntwork module similar to this:

module "vpc" {
  source = "git::git@github.com:gruntwork-io/module-vpc.git//modules/vpc-app?ref=v0.3.2"

  # ... (other params omitted) ...
}

When we release a new version of the relevant module—in this case, vpc-app—we will publish it at a new version number, such as v0.3.3 or v0.4.0. To incorporate these changes into your code, you will need to:

  1. Bump the version number for the module in your infrastructure-modules repo.
  2. Apply any other changes described in the release notes (e.g., see the module-vpc releases page).
  3. Test the change. Typically, you do this by deploying the new code in a sandbox environment.
  4. If things work well, release a new version of your infrastructure-modules repo (i.e., by creating a new git tag).
  5. Use the new version of infrastructure-modules in the corresponding terraform.tfvars files in infrastructure-live. Typically, you’d deploy the new version first in pre-prod environments (e.g., stage), check that everything works well, and if it does, deploy the exact same code in prod.

Other types of updates

If you are using other types of Gruntwork modules, such as ssh-iam for managing SSH access with IAM groups, you are probably installing those modules in your AMIs in a Packer template via the Gruntwork Installer:

{
  "provisioners": [{
    {
    "type": "shell",
    "inline": [
      "gruntwork-install --binary-name 'ssh-iam' --repo 'https://github.com/gruntwork-io/module-security' --tag v0.6.6"
    ]
  }]
}

When we release a new version of ssh-iam, you’ll want to bump the corresponding version number in the tag parameter of that Packer template, build a new AMI (by using packer build), and deploy that AMI in each environment (as always, testing first in pre-prod before prod) by plugging the new AMI ID into the corresponding terraform.tfvars files in infrastructure-live.

Please note that some modules offer zero-downtime deployment for AMIs just by running terragrunt apply: e.g., the asg-rolling-deploy will deploy a new AMI in your ASG automatically when you run apply. However, with other modules, you may need extra steps. For example, with the ecs-cluster module, after running apply, you need to run a the roll-out-ecs-cluster-update.py script (instructions) to deploy new ECS nodes and migrate the Docker containers to them.

3

Thanks, Jim – this was helpful.

What about the cases where the infrastructure-modules tfm code that pulls in modules changes over time? Basically, I’m looking for a good way for my infrastructure-modules repo to track updates to infrastructure-modules-acme, so that I make sure my infrastructure-modules remains current and aligned with new updates to modules.

Should I just git rebase off of infrastructure-modules-acme? Any suggestions?

Thanks!

-charles.

4

You could try rebasing off the Acme repos, but the Acme repos are just one example of how the code in the IaC Library can be assembled, and your own versions of those repos are likely different enough to make rebasing tricky.

FWIW, just about all updates to the Acme repos are bug fixes or new features that are announced in the newsletter, so following along with that will get you most of the way there. You can also periodically check the commit history in the Acme repos and make analogous updates to your own code.

In general, pulling in new infrastructure code versions is not a well solved problem, as before, almost everyone had their own custom infra, and there were no updates to pull! So, this is actually a good problem to have :slight_smile:

We have some ideas on how to significantly improve the updating process, but it’s a pretty major undertaking, and not likely something we’ll get to this year.