Relation with hashicorp, specifically for registry.terraform.io

Gruntwork Customers
1

I’m wondering what’s the deal with modules in the registry? Is it the same production-ready modules your customers are using?

I’ve started to poke around modules for nomad and consul and they are… not very robust, especially server configs.

2

Can you be a bit more specific about what issues you are seeing?

3

Specifically, the config management done in a way that it’s almost impossible to extend it.

For example, if I want to add an option to enable script run on consul node:

Did you consider any config management to manage settings and apps installation? Chef, ansible or any other.

Thanks!

4

We looked into a number of different ways to manage this code, including Chef, Puppet, Ansible, and many others. You can read about it here. In general, there are trade-offs with any set of tools.

That said, if you have suggestions on how to manage this configuration management code so it’s more extensible, we’re very open to feedback!

5

That’s interesting. I haven’t decided yet what to do with consul/nomad configurations.

I think it should be a config management system, so it abstracts away all those details that ‘run-script’ and other similar scripts are doing in the repo. With usual change/apply/test lifecycle, packer could run all that in the build.

6

So, there are two types of scripts: one to install all dependencies, the other to configure & run the code during boot.

How would you use Chef/Puppet/etc to improve either of those so you could “extend” the code better?

Jim

7

With ansible, chef or what not.
Packer can run different provisioners and configure the required type of instance (server, agent etc).
It can run any sorts of integrating tests too. And those configs can be extended to run on different OS.

This doesn’t scale…
if [[ ! -f “/usr/bin/supervisorctl” ]]; then
sudo ln -s /usr/local/bin/supervisorctl /usr/bin/supervisorctl
fi
if [[ ! -f “/usr/bin/supervisord” ]]; then
sudo ln -s /usr/local/bin/supervisord /usr/bin/supervisord
fi

Another consideration is relatively easy local dev cycles: vagrant up --provision can run exactly same playbooks on a local VM. Not a dealbreaker, but a very nice thing to have.

8

We use our install scripts in Packer templates too.

In what sense?

You can run a local VM or Docker container with our scripts too.

It sounded to me like your original complaint was that the config management is impossible to “extend.” Can you give a concrete example of the type of extension you have in mind and how Chef/Ansible/etc would make that possible where bash would not?

To be clear, I’m not against other cfg mgmt tools, but introducing a dependency on a new toolset has a significant cost of its own (every user has to learn it, we have to keep up with new versions, we have to install clients/agents, etc), so it needs to offer something very compelling for this use case to be worth it.

9

If there is more than one developer working on the codebase.

For example, if few instances need to have script checks enabled (optionally): Agents - Configuration File Reference | Consul | HashiCorp Developer

Trivial with ansible, can add a line to a template and set variable
enable_script_checks = {{ enable_script_checks | default('False') | bool | lower }}

10

We aren’t trying to generate the entire config file here, as there are a very large number of settings and permutations. Instead, we try to fill in reasonable defaults for a few of the parameters. For the rest, you can simply include your own config file on the server in the right folder, and it gets picked up automatically: https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/run-consul#overriding-the-configuration

11

I see. Yes, pretty easy to add a file. But things like consul are almost never run by themselves - there is tons of other little apps and utilities to configure - from logs shipment to firewalls and dns.

Anyways, I was simply wondering about having real config management.
Seems like it is a legit case.

12

There’s no reason you can’t do that. You can run our Consul bash scripts from any cfg mgmt tool you might be using or as part of a Packer template (as in this example).

That’s the nice thing about simple scripts: it’s fairly straightforward to fit them into whatever tools you’re already using. On the other hand, if we had picked, say, Ansible, then anyone using Chef or Puppet would more or less automatically refuse to touch this code.