How to process terraform apply logs for catching errors in CI

I’m running terragrunt in CI to automatically apply changes once they’re merged, and am having trouble collecting useful data when an apply failure is encountered. the problem is that run-all apply output is a little messy; it’s difficult to programmatically parse out error messages. i guess an ideal configuration would be if each module wrote its apply output to a separate file, and then if terragrunt wrote a json object with the return code of each module and the filepath of the apply output file. anyone know how i might achieve this? i thought about maybe piping the apply output to tee via the extra_arguments block, but you aren’t allowed to run arbitrary bash in the terraform invocation so that won’t work. there isn’t anything like -apply-output that could be passed to terraform via extra_arguments either… anyone have any ideas? maybe set an alias for terraform in a before_hook? or potentially invoke script in a before_hook then end the script run in an after_hook? I’m gonna try the latter option, will update this thread if i get something working

okay, script is a no go - i get dropped into the script section and terragrunt won’t run until i exit it. i did have a little progress doing this in a before_hook:

mkfifo ${SERVICENAME}.pipe
tee < ${SERVICENAME}.pipe ${SERVICENAME}.txt &
exec &> ${SERVICENAME}.pipe
exec 2>&1
rm ${SERVICENAME}.pipe

which creates the file i want, but nothing is written to it.

Hi @spacerainbow000

Unfortunately, this is something that terragrunt currently doesn’t support. The hooks are run as subprocesses without a shell, which makes it difficult to hijack the streams from the parent processes.

We have an open ticket to address this (The apply-all and destroy-all commands should save log output to file · Issue #74 · gruntwork-io/terragrunt · GitHub) which we are planning on prioritizing, so your best bet is to subscribe to that ticket and wait for that to be implemented, or if you are up for it, open a PR.

Best regards,
Yori

i actually kinda like the workaround here: The apply-all and destroy-all commands should save log output to file · Issue #74 · gruntwork-io/terragrunt · GitHub might give that a try