Terraform state is messed up after moving folders

Hello Gruntwork,

Could really use some help with this. I’ve set the TERRAGRUNT_DOWNLOAD env variable to a folder TERRAGRUNT_DOWNLOAD=/Users/username/Documents/.terragrunt and because of this I think I am having problems with state.

I manage multiple regions in AWS and before I had all my regions under one Regions folder:

├── Regions
│   └── us-east-2\ (Ohio-Prod)
│       └── terragrunt.hcl
├── main.tf
├── terragrunt.hcl
└── variables.tf

I then had a source block in each regions terragrunt.hcl file to “…/…” which would then recognize the main.tf file and provision the shared resources I need (vpc, security groups etc.)

Today, I moved the the regions folder and put them under the account folders like so:

│   ├── Development
│   │   └── ca-central-1\ (Canada-ProdAdmin)
│   │       └── terragrunt.hcl
│   ├── Production
│   │   ├── ap-southeast-2\ (Sydney-Prod)
│   │   │   └── terragrunt.hcl
│   │   ├── eu-central-1(Frankfurt-Prod)
│   │   │   └── terragrunt.hcl
│   │   ├── eu-west-2\ (London-Prod)
│   │   │   └── terragrunt.hcl
│   │   ├── us-east-1\ (Virginia-Prod)
│   │   │   └── terragrunt.hcl
│   │   ├── us-west-1\ (Cali-Prod)
│   │   │   └── terragrunt.hcl
│   │   └── us-west-2\ (Oregon-Prod)
│   │       └── terragrunt.hcl
│   └── Staging
│       └── us-east-1\ (Virginia-Staging)
│           └── terragrunt.hcl

Now, when I run a terragrunt plan, it doesn’t detect the state at all anymore and treats it as a new deployment. Is there anyway I can fix this (besides going back to the old folder scheme)? Migrate the state somehow? (Another way I am thinking is going back to the older folder structure, migrating to a s3 remote backend and then changing the folder scheme)

On an off note, would this be the best practice on managing different regions in AWS while creating the same infrastructure using terragrunt?

Unfortunately, terragrunt is very sensitive to the folder structure due to the way it manages the state files, even with remote state. There are ways to navigate this, with the steps being slightly easier with remote state than local, but is nonetheless painful.

Can you clarify how you were managing the state previously? Were they all local?

On an off note, would this be the best practice on managing different regions in AWS while creating the same infrastructure using terragrunt?

Yes this is typically what we recommend for a multiregion deployment.

Yori

Hello Yori,

Appreciate the quick response. Yes, state was managed locally before moving the folders around. How do you suggest I try and resolve this? Should I move back to the old folder structure?

Regards,
Alexander

Hi Alexander,

The first thing to do is make sure you still have the state files. I would suggest going back to the old folder structure to verify this. Terragrunt has certain modes where it will wipe the cache, and if you are using local state, this is the equivalent to losing all the state! It is more than likely that you still have the state files, but in any case this is a good sanity check.

Once you verify the state files are still intact, I would suggest backing up the terragrunt cache directory before doing anything else to make sure you don’t have any risk of losing the state.

Next, I will suggest updating one module at a time to the new folder structure. This is more tedious, but is much easier to guarantee the state files remain intact. To do this while keeping state, for each module do the following:

  1. Dump out the state to a known, temporary location using terragrunt state pull > $PATH_TO_TEMP_LOCATION
  2. Move the module to the location in the refactored folder structure.
  3. In the new folder structure, import the state using terragrunt state push $PATH_TO_TEMP_LOCATION

Hope this helps!

Best regards,
Yori

Hello Yori,

THANK YOU so much. You saved me :slight_smile: Following your steps everything works great…

I’m curious if I can delete the folder I set my env of TERRAGRUNT_DOWNLOAD to now that I’ve set up remote backend? How do you manage your terragrunt cache and do you set this env variable or do you keep it stock configuration?

Regards,
Alex

I’m curious if I can delete the folder I set my env of TERRAGRUNT_DOWNLOAD to now that I’ve set up remote backend?

If you have a remote state backend, yes you should be able to safely delete the dir. However, I recommend confirming this by taking a backup of the dir, wiping it, and running terragrunt plan-all to make sure. Assuming you don’t have any perpetual diffs in your modules, this should return no changes for all your modules. At that point you can clean up your backup.

How do you manage your terragrunt cache and do you set this env variable or do you keep it stock configuration?

We typically keep the stock terragrunt cache directory as it is easier to navigate and find the corresponding terraform module dir as opposed to it being centralized. Sometimes terraform emits an errored.tfstate file when there is an error pushing the state. When this happens, we need to be able to find the file and run terragrunt state push so knowing exactly which terraform module dir corresponds to the terragrunt dir helps.