S3 state backend policy is not updated/created

[cebidhem]

Describe the bug

The s3 state backend policy does not get updated as expected. Given the current state bucket policy is empty, so I would expect terragrunt to detect it and prompt me to add EnforcedTLS and RootAccess.

Maybe this feature has been removed or flagged, but I haven't found anything in that sense searching in issues and documentation.

Steps To Reproduce

1. Ensure remote_state block doesn't include any skip_ nor disable_bucket_update property:

remote_state {
  backend = "s3"

  config = {
    bucket         = "tg-state-store"
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = local.merged.tf_state_bucket_region
    encrypt        = true
    dynamodb_table = "tg-state-lock"
  }

  generate = {
    path      = "backend.tf"
    if_exists = "overwrite_terragrunt"
  }
}

2. Ensure the state bucket policy is empty:

aws s3api get-bucket-policy --bucket tg-state-store                                                                                                                                

An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist

4. Create whatever unit (I used a policy because it is small)

5. Run terragrunt:

terragrunt run --log-level trace -- apply  

Find the logs below:

12:30:21.257 DEBUG  Terragrunt Version: 0.88.1
12:30:21.258 DEBUG  Skipping stack generation in .
12:30:21.262 DEBUG  Found locals block: evaluating the expressions.
12:30:21.264 DEBUG  Found locals block: evaluating the expressions.
12:30:21.272 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.272 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.272 DEBUG  Found locals block: evaluating the expressions.
12:30:21.275 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.275 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.275 DEBUG  Exposing include block 'root'
12:30:21.275 DEBUG  Evaluated 1 locals (remaining 0): name
12:30:21.276 DEBUG  Found locals block: evaluating the expressions.
12:30:21.280 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.280 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.280 DEBUG  Exposing include block 'root'
12:30:21.281 DEBUG  Found locals block: evaluating the expressions.
12:30:21.283 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.283 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.284 DEBUG  [Partial] Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
12:30:21.285 DEBUG  using cache key for version files: r01AJjVD7VSXCQk1ORuh_no_NRY
12:30:21.285 DEBUG  Running command: terraform -version
12:30:21.285 DEBUG  Engine is not enabled, running command directly in .
12:30:21.483 DEBUG  terraform version: 1.7.5
12:30:21.483 DEBUG  Reading Terragrunt config file at ./terragrunt.hcl
12:30:21.485 DEBUG  Found locals block: evaluating the expressions.
12:30:21.487 DEBUG  Found locals block: evaluating the expressions.
12:30:21.490 DEBUG  Evaluated 2 locals (remaining 1): custom_tags, merged
12:30:21.491 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.492 DEBUG  Found locals block: evaluating the expressions.
12:30:21.501 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.501 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.501 DEBUG  Exposing include block 'root'
12:30:21.501 DEBUG  Evaluated 1 locals (remaining 0): name
12:30:21.502 DEBUG  Found locals block: evaluating the expressions.
12:30:21.506 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.506 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.507 DEBUG  Exposing include block 'root'
12:30:21.507 DEBUG  Found locals block: evaluating the expressions.
12:30:21.515 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.516 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.516 DEBUG  [Partial] Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
12:30:21.516 DEBUG  Found locals block: evaluating the expressions.
12:30:21.517 DEBUG  Found locals block: evaluating the expressions.
12:30:21.521 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.521 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.521 DEBUG  Found locals block: evaluating the expressions.
12:30:21.524 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.524 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.525 DEBUG  Exposing include block 'root'
12:30:21.525 DEBUG  Evaluated 1 locals (remaining 0): name
12:30:21.529 DEBUG  Found locals block: evaluating the expressions.
12:30:21.532 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.532 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.532 DEBUG  Exposing include block 'root'
12:30:21.532 DEBUG  Found locals block: evaluating the expressions.
12:30:21.536 DEBUG  Evaluated 2 locals (remaining 1): custom_tags, merged
12:30:21.536 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.536 DEBUG  Included config ../../../../root.hcl has strategy deep merge: merging config in (deep) for dependency.
12:30:21.537 DEBUG  Found locals block: evaluating the expressions.
12:30:21.540 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.540 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.541 DEBUG  Exposing include block 'root'
12:30:21.542 DEBUG  Found locals block: evaluating the expressions.
12:30:21.546 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
12:30:21.546 DEBUG  Evaluated 1 locals (remaining 0): full_name
12:30:21.547 DEBUG  Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
12:30:21.547 DEBUG  Detected 1 Hooks
12:30:21.547 INFO   Downloading Terraform configurations from git::https://github.com/terraform-aws-modules/terraform-aws-iam.git?ref=v5.38.0 into ./.terragrunt-cache/uE2FDFUEmDsr_lfiOTEB7_a9wmE/3EnVVUCmsbqTbM1RfixO5_z_Z2w
12:30:24.069 DEBUG  Copying files from . into ./.terragrunt-cache/uE2FDFUEmDsr_lfiOTEB7_a9wmE/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
12:30:24.070 DEBUG  Setting working directory to ./.terragrunt-cache/uE2FDFUEmDsr_lfiOTEB7_a9wmE/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
12:30:24.070 DEBUG  Generated file ./.terragrunt-cache/uE2FDFUEmDsr_lfiOTEB7_a9wmE/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/versions_override.tf.
12:30:24.070 DEBUG  Generated file ./.terragrunt-cache/uE2FDFUEmDsr_lfiOTEB7_a9wmE/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/provider-aws.tf.
12:30:24.070 DEBUG  No encryption block in remote_state config
12:30:24.071 DEBUG  Generated file ./.terragrunt-cache/uE2FDFUEmDsr_lfiOTEB7_a9wmE/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/backend.tf.
12:30:24.071 DEBUG  Detected 1 Hooks
12:30:24.071 DEBUG  Running command: terraform init
12:30:24.071 DEBUG  Engine is not enabled, running command directly in ./.terragrunt-cache/uE2FDFUEmDsr_lfiOTEB7_a9wmE/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
12:30:24.158 INFO   terraform: Initializing the backend...
12:30:25.979 INFO   terraform: 
12:30:25.980 INFO   terraform: Successfully configured the backend "s3"! Terraform will automatically
12:30:25.980 INFO   terraform: use this backend unless the backend configuration changes.
12:30:26.517 INFO   terraform: Initializing provider plugins...
12:30:26.517 INFO   terraform: - Finding hashicorp/aws versions matching "~> 5.83"...
12:30:26.825 INFO   terraform: - Installing hashicorp/aws v5.100.0...
12:30:31.651 INFO   terraform: - Installed hashicorp/aws v5.100.0 (signed by HashiCorp)
12:30:31.652 INFO   terraform: Terraform has created a lock file .terraform.lock.hcl to record the provider
12:30:31.652 INFO   terraform: selections it made above. Include this file in your version control repository
12:30:31.652 INFO   terraform: so that Terraform can guarantee to make the same selections by default when
12:30:31.652 INFO   terraform: you run "terraform init" in the future.
12:30:31.652 INFO   terraform: Terraform has been successfully initialized!
12:30:31.653 INFO   terraform: 
12:30:31.653 INFO   terraform: You may now begin working with Terraform. Try running "terraform plan" to see
12:30:31.653 INFO   terraform: any changes that are required for your infrastructure. All Terraform commands
12:30:31.653 INFO   terraform: should now work.
12:30:31.653 INFO   terraform: If you ever set or change modules or backend configuration for Terraform,
12:30:31.653 INFO   terraform: rerun this command to reinitialize your working directory. If you forget, other
12:30:31.653 INFO   terraform: commands will detect it and remind you to do so if necessary.
12:30:31.654 DEBUG  Copying lock file from ./.terragrunt-cache/uE2FDFUEmDsr_lfiOTEB7_a9wmE/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/.terraform.lock.hcl to .
12:30:31.656 DEBUG  Detected 1 Hooks
12:30:31.656 DEBUG  Running command: terraform apply
12:30:31.656 DEBUG  Engine is not enabled, running command directly in ./.terragrunt-cache/uE2FDFUEmDsr_lfiOTEB7_a9wmE/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
12:30:43.460 STDOUT terraform: Terraform used the selected providers to generate the following execution
12:30:43.460 STDOUT terraform: plan. Resource actions are indicated with the following symbols:
12:30:43.460 STDOUT terraform:   + create
12:30:43.460 STDOUT terraform: Terraform will perform the following actions:
12:30:43.460 STDOUT terraform:   # aws_iam_policy.policy[0] will be created
12:30:43.460 STDOUT terraform:   + resource "aws_iam_policy" "policy" {
12:30:43.460 STDOUT terraform:       + arn              = (known after apply)
12:30:43.460 STDOUT terraform:       + attachment_count = (known after apply)
12:30:43.460 STDOUT terraform:       + description      = "IAM Policy"
12:30:43.460 STDOUT terraform:       + id               = (known after apply)
12:30:43.460 STDOUT terraform:       + name             = "test"
12:30:43.460 STDOUT terraform:       + name_prefix      = (known after apply)
12:30:43.460 STDOUT terraform:       + path             = "/"
12:30:43.460 STDOUT terraform:       + policy           = jsonencode(
12:30:43.460 STDOUT terraform:             {
12:30:43.460 STDOUT terraform:               + Statement = [
12:30:43.460 STDOUT terraform:                   + {
12:30:43.460 STDOUT terraform:                       + Action   = [
12:30:43.460 STDOUT terraform:                           + "secretsmanager:GetSecretValue",
12:30:43.460 STDOUT terraform:                         ]
12:30:43.460 STDOUT terraform:                       + Effect   = "Deny"
12:30:43.460 STDOUT terraform:                       + Resource = "*"
12:30:43.460 STDOUT terraform:                       + Sid      = "DenySecretsManagerDataAccess"
12:30:43.460 STDOUT terraform:                     },
12:30:43.460 STDOUT terraform:                 ]
12:30:43.460 STDOUT terraform:               + Version   = "2012-10-17"
12:30:43.460 STDOUT terraform:             }
12:30:43.460 STDOUT terraform:         )
12:30:43.460 STDOUT terraform:       + policy_id        = (known after apply)
12:30:43.460 STDOUT terraform:       + tags_all         = (known after apply)
12:30:43.460 STDOUT terraform:     }
12:30:43.460 STDOUT terraform: Plan: 1 to add, 0 to change, 0 to destroy.
12:30:43.460 STDOUT terraform: 
12:30:43.460 STDOUT terraform: Changes to Outputs:
12:30:43.460 STDOUT terraform:   + arn         = (known after apply)
12:30:43.460 STDOUT terraform:   + description = "IAM Policy"
12:30:43.460 STDOUT terraform:   + id          = (known after apply)
12:30:43.461 STDOUT terraform:   + name        = "test"
12:30:43.461 STDOUT terraform:   + path        = "/"
12:30:43.461 STDOUT terraform:   + policy      = jsonencode(
12:30:43.461 STDOUT terraform:         {
12:30:43.461 STDOUT terraform:           + Statement = [
12:30:43.461 STDOUT terraform:               + {
12:30:43.461 STDOUT terraform:                   + Action   = [
12:30:43.461 STDOUT terraform:                       + "secretsmanager:GetSecretValue",
12:30:43.461 STDOUT terraform:                     ]
12:30:43.461 STDOUT terraform:                   + Effect   = "Deny"
12:30:43.461 STDOUT terraform:                   + Resource = "*"
12:30:43.461 STDOUT terraform:                   + Sid      = "DenySecretsManagerDataAccess"
12:30:43.461 STDOUT terraform:                 },
12:30:43.461 STDOUT terraform:             ]
12:30:43.461 STDOUT terraform:           + Version   = "2012-10-17"
12:30:43.461 STDOUT terraform:         }
12:30:43.461 STDOUT terraform:     )
12:30:43.461 STDOUT terraform: 
12:30:43.461 STDOUT terraform: Do you want to perform these actions?
12:30:43.461 STDOUT terraform:   Terraform will perform the actions described above.
12:30:43.461 STDOUT terraform:   Only 'yes' will be accepted to approve.
12:30:43.461 STDOUT terraform:   Enter a value: 
yes
12:30:58.785 STDOUT terraform: aws_iam_policy.policy[0]: Creating...
12:30:59.465 STDOUT terraform: aws_iam_policy.policy[0]: Creation complete after 0s [id=arn:aws:iam::242454459143:policy/test]
12:31:00.284 STDOUT terraform: 
12:31:00.284 STDOUT terraform: Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
12:31:00.284 STDOUT terraform: 
12:31:00.284 STDOUT terraform: Outputs:
12:31:00.284 STDOUT terraform: 
12:31:00.284 STDOUT terraform: arn = "arn:aws:iam::0123456789012:policy/test"
12:31:00.284 STDOUT terraform: description = "IAM Policy"
12:31:00.284 STDOUT terraform: id = "arn:aws:iam::0123456789012:policy/test"
12:31:00.284 STDOUT terraform: name = "test"
12:31:00.284 STDOUT terraform: path = "/"
12:31:00.284 STDOUT terraform: policy = "{\"Statement\":[{\"Action\":[\"secretsmanager:GetSecretValue\"],\"Effect\":\"Deny\",\"Resource\":\"*\",\"Sid\":\"DenySecretsManagerDataAccess\"}],\"Version\":\"2012-10-17\"}"

Expected behavior

I would expect terragrunt to prompt me with this:

Remote state S3 bucket tg-state-store is out of date. Would you like Terragrunt to update it?

or at least, since I'm running in --log-level trace, to display in the logs S3 bucket is already up to date.

Must haves

• Steps for reproduction provided.

Nice to haves

• Terminal output
• Screenshots

Versions

• Terragrunt version: 0.88.1
• OpenTofu/Terraform version: 1.7.5
• Environment details (Ubuntu 20.04, Windows 10, etc.): MacOS 26.0.1

Additional context

I tested with terragrunt run -- init, terragrunt run --backend-bootstrap -- init, terragrunt run -- plan and terragrunt run -- apply with the same result.

[cebidhem]

I did run some more tests, notably with a fresh "environment".

Running terragrunt --log-level trace run --backend-bootstrap -- init when the state bucket does not exist at all makes Terragrunt ask for its creation and applies the policies correctly:

15:51:57.942 DEBUG  Terragrunt Version: 0.88.1
15:51:57.942 DEBUG  Skipping stack generation in .
15:51:57.943 DEBUG  Found locals block: evaluating the expressions.
15:51:57.944 DEBUG  Found locals block: evaluating the expressions.
15:51:57.947 DEBUG  Evaluated 2 locals (remaining 1): custom_tags, merged
15:51:57.947 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:57.947 DEBUG  Found locals block: evaluating the expressions.
15:51:57.951 DEBUG  Evaluated 2 locals (remaining 1): custom_tags, merged
15:51:57.951 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:57.951 DEBUG  Exposing include block 'root'
15:51:57.951 DEBUG  Evaluated 1 locals (remaining 0): name
15:51:57.952 DEBUG  Found locals block: evaluating the expressions.
15:51:57.955 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:57.955 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:57.956 DEBUG  Exposing include block 'root'
15:51:57.956 DEBUG  Found locals block: evaluating the expressions.
15:51:57.959 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:57.959 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:57.960 DEBUG  [Partial] Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
15:51:57.960 DEBUG  using cache key for version files: r01AJjVD7VSXCQk1ORuh_no_NRY
15:51:57.960 DEBUG  Running command: terraform -version
15:51:57.960 DEBUG  Engine is not enabled, running command directly in .
15:51:58.046 DEBUG  terraform version: 1.7.5
15:51:58.046 DEBUG  Reading Terragrunt config file at ./terragrunt.hcl
15:51:58.047 DEBUG  Found locals block: evaluating the expressions.
15:51:58.048 DEBUG  Found locals block: evaluating the expressions.
15:51:58.052 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:58.052 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.052 DEBUG  Found locals block: evaluating the expressions.
15:51:58.056 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:58.056 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.058 DEBUG  Exposing include block 'root'
15:51:58.058 DEBUG  Evaluated 1 locals (remaining 0): name
15:51:58.059 DEBUG  Found locals block: evaluating the expressions.
15:51:58.062 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:58.062 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.063 DEBUG  Exposing include block 'root'
15:51:58.063 DEBUG  Found locals block: evaluating the expressions.
15:51:58.067 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:58.067 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.067 DEBUG  [Partial] Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
15:51:58.068 DEBUG  Found locals block: evaluating the expressions.
15:51:58.069 DEBUG  Found locals block: evaluating the expressions.
15:51:58.072 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:58.072 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.072 DEBUG  Found locals block: evaluating the expressions.
15:51:58.075 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:58.076 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.076 DEBUG  Exposing include block 'root'
15:51:58.076 DEBUG  Evaluated 1 locals (remaining 0): name
15:51:58.077 DEBUG  Found locals block: evaluating the expressions.
15:51:58.080 DEBUG  Evaluated 2 locals (remaining 1): custom_tags, merged
15:51:58.081 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.081 DEBUG  Exposing include block 'root'
15:51:58.081 DEBUG  Found locals block: evaluating the expressions.
15:51:58.084 DEBUG  Evaluated 2 locals (remaining 1): custom_tags, merged
15:51:58.084 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.084 DEBUG  Included config ../../../../root.hcl has strategy deep merge: merging config in (deep) for dependency.
15:51:58.085 DEBUG  Found locals block: evaluating the expressions.
15:51:58.089 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:58.089 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.089 DEBUG  Exposing include block 'root'
15:51:58.091 DEBUG  Found locals block: evaluating the expressions.
15:51:58.093 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:51:58.093 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:51:58.094 DEBUG  Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
15:51:58.094 DEBUG  terraform files in ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy are up to date. Will not download again.
15:51:58.094 DEBUG  Copying files from . into ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
15:51:58.096 DEBUG  Setting working directory to ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
15:51:58.096 DEBUG  The file path ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/versions_override.tf already exists, but was a previously generated file by terragrunt. Since if_exists for code generation is set to "overwrite_terragrunt", regenerating file.
15:51:58.097 DEBUG  Generated file ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/versions_override.tf.
15:51:58.097 DEBUG  The file path ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/provider-aws.tf already exists, but was a previously generated file by terragrunt. Since if_exists for code generation is set to "overwrite_terragrunt", regenerating file.
15:51:58.097 DEBUG  Generated file ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/provider-aws.tf.
15:51:58.097 DEBUG  No encryption block in remote_state config
15:51:58.098 DEBUG  The file path ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/backend.tf already exists, but was a previously generated file by terragrunt. Since if_exists for code generation is set to "overwrite_terragrunt", regenerating file.
15:51:58.098 DEBUG  Generated file ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/backend.tf.
15:51:58.098 DEBUG  Checking if remote state bootstrap is necessary for the s3 backend
15:52:00.302 DEBUG  Bootstrapping remote state for the s3 backend
15:52:01.409 DEBUG  Checking if bucket test-tg-state-store exists
15:52:02.526 DEBUG  Remote state S3 bucket test-tg-state-store does not exist or you don't have permissions to access it.
Remote state S3 bucket test-tg-state-store does not exist or you don't have permissions to access it. Would you like Terragrunt to create it? (y/n) y
15:52:12.428 DEBUG  Create S3 bucket with retry test-tg-state-store
15:52:12.429 DEBUG  Create S3 bucket test-tg-state-store with versioning, SSE encryption, and access logging.
15:52:12.429 DEBUG  Creating S3 bucket test-tg-state-store
15:52:12.868 DEBUG  Created S3 bucket test-tg-state-store
15:52:12.868 DEBUG  Waiting for bucket test-tg-state-store to be created
15:52:12.868 DEBUG  Checking if bucket test-tg-state-store exists
15:52:12.954 DEBUG  S3 bucket test-tg-state-store created.
15:52:12.954 DEBUG  Enabling root access to S3 bucket test-tg-state-store
15:52:13.772 DEBUG  Policy not exists for bucket test-tg-state-store
15:52:13.960 DEBUG  Enabled root access to bucket test-tg-state-store
15:52:14.566 DEBUG  Enabled enforced TLS access to bucket test-tg-state-store
15:52:14.773 DEBUG  Enabled public access blocking for S3 bucket test-tg-state-store
15:52:14.773 DEBUG  No tags to apply to S3 bucket test-tg-state-store
15:52:14.773 DEBUG  Enabling versioning for S3 bucket test-tg-state-store
15:52:14.959 DEBUG  Enabled versioning for S3 bucket test-tg-state-store
15:52:14.959 DEBUG  Enabling server-side encryption for S3 bucket test-tg-state-store
15:52:15.874 DEBUG  Enabled server-side encryption for S3 bucket test-tg-state-store
15:52:15.874 DEBUG  Access Logging is disabled for the remote state AWS S3 bucket test-tg-state-store
15:52:15.874 DEBUG  No tags specified for bucket .
15:52:15.874 DEBUG  Checking if bucket test-tg-state-store exists
15:52:16.049 DEBUG  Verifying AWS S3 bucket versioning test-tg-state-store
15:52:16.233 DEBUG  Checking if bucket test-tg-state-store has root access
15:52:16.321 DEBUG  Checking if bucket test-tg-state-store has enforced TLS
15:52:16.500 DEBUG  S3 bucket is already up to date
15:52:16.583 DEBUG  Verifying AWS S3 bucket versioning test-tg-state-store
15:52:17.013 DEBUG  Lock table test-tg-state-lock does not exist in DynamoDB. Will need to create it just this first time.
15:52:17.013 DEBUG  Creating table test-tg-state-lock in DynamoDB
15:52:17.218 DEBUG  Table test-tg-state-lock is not yet in active state. Will check again after 10s.
15:52:27.310 DEBUG  Success! Table test-tg-state-lock is now in active state.
15:52:27.310 DEBUG  No tags for lock table given.
15:52:27.310 DEBUG  Detected 1 Hooks
15:52:27.311 DEBUG  Running command: terraform init
15:52:27.311 DEBUG  Engine is not enabled, running command directly in ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
15:52:27.408 STDOUT terraform: Initializing the backend...
15:52:28.972 STDOUT terraform: 
15:52:28.972 STDOUT terraform: Successfully configured the backend "s3"! Terraform will automatically
15:52:28.972 STDOUT terraform: use this backend unless the backend configuration changes.
15:52:29.496 STDOUT terraform: Initializing provider plugins...
15:52:29.496 STDOUT terraform: - Finding hashicorp/aws versions matching "~> 5.83"...
15:52:29.799 STDOUT terraform: - Installing hashicorp/aws v5.100.0...
15:52:34.371 STDOUT terraform: - Installed hashicorp/aws v5.100.0 (signed by HashiCorp)
15:52:34.372 STDOUT terraform: Terraform has created a lock file .terraform.lock.hcl to record the provider
15:52:34.372 STDOUT terraform: selections it made above. Include this file in your version control repository
15:52:34.372 STDOUT terraform: so that Terraform can guarantee to make the same selections by default when
15:52:34.372 STDOUT terraform: you run "terraform init" in the future.
15:52:34.373 STDOUT terraform: Terraform has been successfully initialized!
15:52:34.373 STDOUT terraform: 
15:52:34.373 STDOUT terraform: You may now begin working with Terraform. Try running "terraform plan" to see
15:52:34.373 STDOUT terraform: any changes that are required for your infrastructure. All Terraform commands
15:52:34.373 STDOUT terraform: should now work.
15:52:34.373 STDOUT terraform: If you ever set or change modules or backend configuration for Terraform,
15:52:34.373 STDOUT terraform: rerun this command to reinitialize your working directory. If you forget, other
15:52:34.373 STDOUT terraform: commands will detect it and remind you to do so if necessary.
15:52:34.374 DEBUG  Copying lock file from ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/.terraform.lock.hcl to .

Running aws s3api get-bucket-policy --bucket test-tg-state-store | jq to confirm:

{
  "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"EnforcedTLS\",\"Effect\":\"Deny\",\"Principal\":\"*\",\"Action\":\"s3:*\",\"Resource\":[\"arn:aws:s3:::test-tg-state-store\",\"arn:aws:s3:::test-tg-state-store/*\"],\"Condition\":{\"Bool\":{\"aws:SecureTransport\":\"false\"}}},{\"Sid\":\"RootAccess\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::957190007780:root\"},\"Action\":\"s3:*\",\"Resource\":[\"arn:aws:s3:::test-tg-state-store\",\"arn:aws:s3:::test-tg-state-store/*\"]}]}"
}

After editing the policy manually to remove one of the statements, and running terragrunt --log-level trace run --backend-bootstrap -- init again, terragrunt does not detect it and act as described in the issue:

15:56:57.109 DEBUG  Terragrunt Version: 0.88.1
15:56:57.110 DEBUG  Skipping stack generation in .
15:56:57.115 DEBUG  Found locals block: evaluating the expressions.
15:56:57.117 DEBUG  Found locals block: evaluating the expressions.
15:56:57.123 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.123 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.123 DEBUG  Found locals block: evaluating the expressions.
15:56:57.126 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.126 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.127 DEBUG  Exposing include block 'root'
15:56:57.127 DEBUG  Evaluated 1 locals (remaining 0): name
15:56:57.128 DEBUG  Found locals block: evaluating the expressions.
15:56:57.131 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.131 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.131 DEBUG  Exposing include block 'root'
15:56:57.132 DEBUG  Found locals block: evaluating the expressions.
15:56:57.134 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.134 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.134 DEBUG  [Partial] Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
15:56:57.135 DEBUG  using cache key for version files: r01AJjVD7VSXCQk1ORuh_no_NRY
15:56:57.135 DEBUG  Running command: terraform -version
15:56:57.135 DEBUG  Engine is not enabled, running command directly in .
15:56:57.317 DEBUG  terraform version: 1.7.5
15:56:57.317 DEBUG  Reading Terragrunt config file at ./terragrunt.hcl
15:56:57.317 DEBUG  Found locals block: evaluating the expressions.
15:56:57.318 DEBUG  Found locals block: evaluating the expressions.
15:56:57.321 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.321 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.321 DEBUG  Found locals block: evaluating the expressions.
15:56:57.323 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.323 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.324 DEBUG  Exposing include block 'root'
15:56:57.324 DEBUG  Evaluated 1 locals (remaining 0): name
15:56:57.324 DEBUG  Found locals block: evaluating the expressions.
15:56:57.327 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.327 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.329 DEBUG  Exposing include block 'root'
15:56:57.329 DEBUG  Found locals block: evaluating the expressions.
15:56:57.331 DEBUG  Evaluated 2 locals (remaining 1): custom_tags, merged
15:56:57.331 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.331 DEBUG  [Partial] Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
15:56:57.331 DEBUG  Found locals block: evaluating the expressions.
15:56:57.333 DEBUG  Found locals block: evaluating the expressions.
15:56:57.334 DEBUG  Evaluated 2 locals (remaining 1): custom_tags, merged
15:56:57.335 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.335 DEBUG  Found locals block: evaluating the expressions.
15:56:57.337 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.337 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.338 DEBUG  Exposing include block 'root'
15:56:57.338 DEBUG  Evaluated 1 locals (remaining 0): name
15:56:57.338 DEBUG  Found locals block: evaluating the expressions.
15:56:57.341 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.341 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.342 DEBUG  Exposing include block 'root'
15:56:57.342 DEBUG  Found locals block: evaluating the expressions.
15:56:57.344 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.344 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.344 DEBUG  Included config ../../../../root.hcl has strategy deep merge: merging config in (deep) for dependency.
15:56:57.345 DEBUG  Found locals block: evaluating the expressions.
15:56:57.348 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.348 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.348 DEBUG  Exposing include block 'root'
15:56:57.349 DEBUG  Found locals block: evaluating the expressions.
15:56:57.352 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
15:56:57.352 DEBUG  Evaluated 1 locals (remaining 0): full_name
15:56:57.352 DEBUG  Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
15:56:57.352 DEBUG  Detected 1 Hooks
15:56:57.352 INFO   Downloading Terraform configurations from git::https://github.com/terraform-aws-modules/terraform-aws-iam.git?ref=v5.38.0 into ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w
15:56:59.693 DEBUG  Copying files from . into ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
15:56:59.693 DEBUG  Setting working directory to ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
15:56:59.694 DEBUG  Generated file ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/provider-aws.tf.
15:56:59.694 DEBUG  Generated file ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/versions_override.tf.
15:56:59.694 DEBUG  No encryption block in remote_state config
15:56:59.694 DEBUG  Generated file ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/backend.tf.
15:56:59.694 DEBUG  Checking if remote state bootstrap is necessary for the s3 backend
15:57:02.524 DEBUG  Detected 1 Hooks
15:57:02.525 DEBUG  Running command: terraform init
15:57:02.525 DEBUG  Engine is not enabled, running command directly in ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy
15:57:02.648 STDOUT terraform: Initializing the backend...
15:57:04.222 STDOUT terraform: 
15:57:04.222 STDOUT terraform: Successfully configured the backend "s3"! Terraform will automatically
15:57:04.223 STDOUT terraform: use this backend unless the backend configuration changes.
15:57:04.745 STDOUT terraform: Initializing provider plugins...
15:57:04.745 STDOUT terraform: - Finding hashicorp/aws versions matching "~> 5.83"...
15:57:05.029 STDOUT terraform: - Installing hashicorp/aws v5.100.0...
15:57:09.662 STDOUT terraform: - Installed hashicorp/aws v5.100.0 (signed by HashiCorp)
15:57:09.663 STDOUT terraform: Terraform has created a lock file .terraform.lock.hcl to record the provider
15:57:09.663 STDOUT terraform: selections it made above. Include this file in your version control repository
15:57:09.663 STDOUT terraform: so that Terraform can guarantee to make the same selections by default when
15:57:09.663 STDOUT terraform: you run "terraform init" in the future.
15:57:09.664 STDOUT terraform: Terraform has been successfully initialized!
15:57:09.664 STDOUT terraform: 
15:57:09.664 STDOUT terraform: You may now begin working with Terraform. Try running "terraform plan" to see
15:57:09.664 STDOUT terraform: any changes that are required for your infrastructure. All Terraform commands
15:57:09.664 STDOUT terraform: should now work.
15:57:09.664 STDOUT terraform: If you ever set or change modules or backend configuration for Terraform,
15:57:09.664 STDOUT terraform: rerun this command to reinitialize your working directory. If you forget, other
15:57:09.664 STDOUT terraform: commands will detect it and remind you to do so if necessary.
15:57:09.665 DEBUG  Copying lock file from ./.terragrunt-cache/HoFHj5ZkXOzDiwuHYVf0risgmqw/3EnVVUCmsbqTbM1RfixO5_z_Z2w/modules/iam-policy/.terraform.lock.hcl to .

Which is also the case when deleting the policy completely.

[cebidhem]

I'm not a fully capable backend engineer but I think it might be because of the work on adding the bootstrap commands:

I went down that road and try to use backend bootstrap command, which effectively checked on the backend, and set the policy back.
terragrunt --log-level trace backend bootstrap

17:38:31.216 DEBUG  Terragrunt Version: 0.88.1
17:38:31.218 DEBUG  Reading Terragrunt config file at ./terragrunt.hcl
17:38:31.221 DEBUG  Found locals block: evaluating the expressions.
17:38:31.224 DEBUG  Found locals block: evaluating the expressions.
17:38:31.231 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
17:38:31.231 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.231 DEBUG  Found locals block: evaluating the expressions.
17:38:31.233 DEBUG  Evaluated 2 locals (remaining 1): custom_tags, merged
17:38:31.233 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.234 DEBUG  Exposing include block 'root'
17:38:31.234 DEBUG  Evaluated 1 locals (remaining 0): name
17:38:31.235 DEBUG  Found locals block: evaluating the expressions.
17:38:31.237 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
17:38:31.238 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.238 DEBUG  Exposing include block 'root'
17:38:31.239 DEBUG  Found locals block: evaluating the expressions.
17:38:31.241 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
17:38:31.241 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.241 DEBUG  [Partial] Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
17:38:31.241 DEBUG  Found locals block: evaluating the expressions.
17:38:31.242 DEBUG  Found locals block: evaluating the expressions.
17:38:31.246 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
17:38:31.246 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.246 DEBUG  Found locals block: evaluating the expressions.
17:38:31.249 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
17:38:31.249 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.249 DEBUG  Exposing include block 'root'
17:38:31.249 DEBUG  Evaluated 1 locals (remaining 0): name
17:38:31.250 DEBUG  Found locals block: evaluating the expressions.
17:38:31.253 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
17:38:31.253 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.254 DEBUG  Exposing include block 'root'
17:38:31.254 DEBUG  Found locals block: evaluating the expressions.
17:38:31.256 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
17:38:31.256 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.257 DEBUG  Included config ../../../../root.hcl has strategy deep merge: merging config in (deep) for dependency.
17:38:31.257 DEBUG  Found locals block: evaluating the expressions.
17:38:31.260 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
17:38:31.260 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.260 DEBUG  Exposing include block 'root'
17:38:31.261 DEBUG  Found locals block: evaluating the expressions.
17:38:31.264 DEBUG  Evaluated 2 locals (remaining 1): merged, custom_tags
17:38:31.264 DEBUG  Evaluated 1 locals (remaining 0): full_name
17:38:31.264 DEBUG  Included config ../../../../root.hcl has strategy deep merge: merging config in (deep).
17:38:31.264 DEBUG  Bootstrapping remote state for the s3 backend
17:38:32.611 DEBUG  Checking if bucket test-tg-state-store exists
17:38:33.803 DEBUG  Checking if bucket test-tg-state-store exists
17:38:33.975 DEBUG  Verifying AWS S3 bucket versioning test-tg-state-store
17:38:34.169 DEBUG  Checking if bucket test-tg-state-store has root access
17:38:34.261 DEBUG  Checking if bucket test-tg-state-store has enforced TLS
17:38:34.438 WARN   The remote state S3 bucket test-tg-state-store needs to be updated:
17:38:34.438 WARN     - Bucket Root Access
17:38:34.439 WARN     - Bucket Enforced TLS
Remote state S3 bucket test-tg-state-store is out of date. Would you like Terragrunt to update it? (y/n) y
17:38:43.413 DEBUG  Enabling root access to S3 bucket test-tg-state-store
17:38:44.432 DEBUG  Policy not exists for bucket test-tg-state-store
17:38:44.633 DEBUG  Enabled root access to bucket test-tg-state-store
17:38:45.257 DEBUG  Enabled enforced TLS access to bucket test-tg-state-store
17:38:45.351 DEBUG  Verifying AWS S3 bucket versioning test-tg-state-store

Running aws s3api get-bucket-policy --bucket test-tg-state-store | jq again:

{
  "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"EnforcedTLS\",\"Effect\":\"Deny\",\"Principal\":\"*\",\"Action\":\"s3:*\",\"Resource\":[\"arn:aws:s3:::test-tg-state-store\",\"arn:aws:s3:::test-tg-state-store/*\"],\"Condition\":{\"Bool\":{\"aws:SecureTransport\":\"false\"}}},{\"Sid\":\"RootAccess\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::957190007780:root\"},\"Action\":\"s3:*\",\"Resource\":[\"arn:aws:s3:::test-tg-state-store\",\"arn:aws:s3:::test-tg-state-store/*\"]}]}"                                                                                                          
}

Was the behaviour changed on purpose when introducing the backend commands?
If yes, what would be the best way to always ensure our state backend is compliant ?
If no, it seems to be a regression.

[yhakbar]

I imagine this is a bug related to our logic for short circuiting further API calls to AWS when we confirm that the bucket exists, not wanting to slow down backend bootstrapping for users. If someone from the community is interested in investigating this, we would welcome a PR addressing it.