Alb plan/apply not "sticking"?

This message is extracted from a ticket originally emailed to support@gruntwork.io. Names and URLs have been removed where appropriate.

In my upgrade process, I’m going through alb’s (we have alb-internal & alb-public) and even though I terragrunt plan/apply:

➜  alb-internal git:(master) ✗ terragrunt apply
[terragrunt] [/Users/chad.small/Documents/workspace/infrastructure-live/dev/us-east-1/auto-01/networking/alb-internal] 2017/10/11 17:27:43 Running command: terraform --version
[terragrunt] 2017/10/11 17:27:44 Reading Terragrunt config file at /Users/chad.small/Documents/workspace/infrastructure-live/dev/us-east-1/auto-01/networking/alb-internal/terraform.tfvars
[terragrunt] 2017/10/11 17:27:44 Terraform files in /var/folders/n4/80hgcshd5_5990h8ssbn22gh0000gp/T/terragrunt/k5NzkryHKYusZH-I8VPSGsZt2kg/x2QcOf4pnGwF8_hmHxpUqhKfPeE/networking/alb are up to date. Will not download again.
[terragrunt] 2017/10/11 17:27:44 Copying files from /Users/chad.small/Documents/workspace/infrastructure-live/dev/us-east-1/auto-01/networking/alb-internal into /var/folders/n4/80hgcshd5_5990h8ssbn22gh0000gp/T/terragrunt/k5NzkryHKYusZH-I8VPSGsZt2kg/x2QcOf4pnGwF8_hmHxpUqhKfPeE/networking/alb
[terragrunt] 2017/10/11 17:27:44 Setting working directory to /var/folders/n4/80hgcshd5_5990h8ssbn22gh0000gp/T/terragrunt/k5NzkryHKYusZH-I8VPSGsZt2kg/x2QcOf4pnGwF8_hmHxpUqhKfPeE/networking/alb
[terragrunt] 2017/10/11 17:27:44 Remote state is already configured for backend s3
[terragrunt] 2017/10/11 17:27:44 Running command: terraform apply -lock-timeout=20m
data.terraform_remote_state.vpc: Refreshing state...
data.template_file.https_listener_ports_and_ssl_certs_keys_non_empty: Refreshing state...
data.template_file.http_listener_ports_keys_non_empty: Refreshing state...
data.template_file.https_listener_ports_and_acm_ssl_certs_keys_non_empty: Refreshing state...
data.aws_elb_service_account.main: Refreshing state...
data.aws_iam_policy_document.access_logs_bucket_policy: Refreshing state...
aws_s3_bucket.access_logs_with_logs_archived_and_deleted: Refreshing state... (ID: alb-auto-01-alb-internal-access-logs)
aws_alb_target_group.blackhole: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...lb-internal-blackhole/5ecee72ea527fed6)
aws_security_group.alb: Refreshing state... (ID: sg-6f572e1c)
data.aws_acm_certificate.certs: Refreshing state...
aws_security_group_rule.allow_all_outbound: Refreshing state... (ID: sgrule-2860570085)
aws_alb.alb_with_logs: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:.../auto-01-alb-internal/ded186e4fe4d1e63)
aws_security_group_rule.https_listeners_acm_certs: Refreshing state... (ID: sgrule-3566942265)
data.template_file.alb_arn: Refreshing state...
aws_route53_record.dns_record: Refreshing state... (ID: Z7NRLOD1YEUBY_services-auto-01.dev-bind.com_A)
aws_alb_listener.https_acm_certs: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...rnal/ded186e4fe4d1e63/389a2d12543540b8)
module.alb_access_logs_bucket.aws_s3_bucket.access_logs_with_logs_archived_and_deleted: Modifying... (ID: alb-auto-01-alb-internal-access-logs)
  lifecycle_rule.0.expiration.122674990.date:                         "" => ""
  lifecycle_rule.0.expiration.122674990.days:                         "" => "60"
  lifecycle_rule.0.expiration.122674990.expired_object_delete_marker: "" => "true"
  lifecycle_rule.0.expiration.793296368.date:                         "" => ""
  lifecycle_rule.0.expiration.793296368.days:                         "60" => "0"
  lifecycle_rule.0.expiration.793296368.expired_object_delete_marker: "false" => "false"
module.alb_access_logs_bucket.aws_s3_bucket.access_logs_with_logs_archived_and_deleted: Modifications complete (ID: alb-auto-01-alb-internal-access-logs)

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path:
Releasing state lock. This may take a few moments...

Outputs:

alb_arn = arn:aws:elasticloadbalancing:us-east-1:938991199149:loadbalancer/app/auto-01-alb-internal/ded186e4fe4d1e63
alb_dns_name = services-auto-01.dev-bind.com
alb_hosted_zone_id = Z35SXDOTRQ7X7K
alb_name = auto-01-alb-internal
alb_security_group_id = sg-6f572e1c
http_listener_arns = {}
https_listener_acm_cert_arns = {
  443 = arn:aws:elasticloadbalancing:us-east-1:938991199149:listener/app/auto-01-alb-internal/ded186e4fe4d1e63/389a2d12543540b8
}
https_listener_non_acm_cert_arns = {}
listener_arns = {
  443 = arn:aws:elasticloadbalancing:us-east-1:938991199149:listener/app/auto-01-alb-internal/ded186e4fe4d1e63/389a2d12543540b8
}

Wwhen I terragrunt plan a 2nd time after the apply, there is still a modification that says needs to happen. Have you seen this before?

➜  alb-internal git:(master) ✗ terragrunt plan
[terragrunt] [/Users/chad.small/Documents/workspace/infrastructure-live/dev/us-east-1/auto-01/networking/alb-internal] 2017/10/11 17:28:10 Running command: terraform --version
[terragrunt] 2017/10/11 17:28:11 Reading Terragrunt config file at /Users/chad.small/Documents/workspace/infrastructure-live/dev/us-east-1/auto-01/networking/alb-internal/terraform.tfvars
[terragrunt] 2017/10/11 17:28:11 Terraform files in /var/folders/n4/80hgcshd5_5990h8ssbn22gh0000gp/T/terragrunt/k5NzkryHKYusZH-I8VPSGsZt2kg/x2QcOf4pnGwF8_hmHxpUqhKfPeE/networking/alb are up to date. Will not download again.
[terragrunt] 2017/10/11 17:28:11 Copying files from /Users/chad.small/Documents/workspace/infrastructure-live/dev/us-east-1/auto-01/networking/alb-internal into /var/folders/n4/80hgcshd5_5990h8ssbn22gh0000gp/T/terragrunt/k5NzkryHKYusZH-I8VPSGsZt2kg/x2QcOf4pnGwF8_hmHxpUqhKfPeE/networking/alb
[terragrunt] 2017/10/11 17:28:11 Setting working directory to /var/folders/n4/80hgcshd5_5990h8ssbn22gh0000gp/T/terragrunt/k5NzkryHKYusZH-I8VPSGsZt2kg/x2QcOf4pnGwF8_hmHxpUqhKfPeE/networking/alb
[terragrunt] 2017/10/11 17:28:11 Remote state is already configured for backend s3
[terragrunt] 2017/10/11 17:28:11 Running command: terraform plan -lock-timeout=20m
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.terraform_remote_state.vpc: Refreshing state...
data.template_file.http_listener_ports_keys_non_empty: Refreshing state...
data.template_file.https_listener_ports_and_acm_ssl_certs_keys_non_empty: Refreshing state...
data.template_file.https_listener_ports_and_ssl_certs_keys_non_empty: Refreshing state...
aws_security_group.alb: Refreshing state... (ID: sg-6f572e1c)
aws_alb_target_group.blackhole: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...lb-internal-blackhole/5ecee72ea527fed6)
data.aws_acm_certificate.certs: Refreshing state...
data.aws_elb_service_account.main: Refreshing state...
data.aws_iam_policy_document.access_logs_bucket_policy: Refreshing state...
aws_s3_bucket.access_logs_with_logs_archived_and_deleted: Refreshing state... (ID: alb-auto-01-alb-internal-access-logs)
aws_security_group_rule.https_listeners_acm_certs: Refreshing state... (ID: sgrule-3566942265)
aws_security_group_rule.allow_all_outbound: Refreshing state... (ID: sgrule-2860570085)
aws_alb.alb_with_logs: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:.../auto-01-alb-internal/ded186e4fe4d1e63)
data.template_file.alb_arn: Refreshing state...
aws_route53_record.dns_record: Refreshing state... (ID: Z7NRLOD1YEUBY_services-auto-01.dev-bind.com_A)
aws_alb_listener.https_acm_certs: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...rnal/ded186e4fe4d1e63/389a2d12543540b8)
The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed. Cyan entries are data sources to be read.

Note: You didn't specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can't guarantee this is what will execute.

~ module.alb_access_logs_bucket.aws_s3_bucket.access_logs_with_logs_archived_and_deleted
    lifecycle_rule.0.expiration.122674990.date:                         "" => ""
    lifecycle_rule.0.expiration.122674990.days:                         "" => "60"
    lifecycle_rule.0.expiration.122674990.expired_object_delete_marker: "" => "true"
    lifecycle_rule.0.expiration.793296368.date:                         "" => ""
    lifecycle_rule.0.expiration.793296368.days:                         "60" => "0"
    lifecycle_rule.0.expiration.793296368.expired_object_delete_marker: "false" => "false"


Plan: 0 to add, 1 to change, 0 to destroy.
Releasing state lock. This may take a few moments...
➜  alb-internal git:(master) ✗

Unfortunately, this is a Terraform bug: https://github.com/terraform-providers/terraform-provider-aws/issues/291. Josh originally opened the bug more than a year ago, but there’s still no fix… For now, it’s best just to ignore that diff. It’s harmless, and running apply will basically be a no-op.