Skip to content

OpenTofu owns identities and access; only secret values stay out of band

Status: accepted Date: 2026-09-29 Decider: Chris Issue: #894 Amends: the out-of-band identity and grant bootstrap of #138, #575 and #892

Context

Until now infra/set-secrets.sh created the runtime service accounts, the Secret Manager secrets, every secretAccessor grant, and the web identity's iam.serviceAccountUser on itself. OpenTofu only read these as data sources and checked the grants with preconditions (#138). The CD identity's run.admin and cloudtasks.admin were granted by hand.

This split had two faults:

  • A new environment could not reach its first working apply. OpenTofu created cvr-ingester-<env>, but its precondition needed the grants that the script could only make after the account existed (#892).
  • Access was not in code. The grants were not reviewed in pull requests, drift was not visible in a plan, and prod needed several manual gcloud steps that nobody recorded.

The established practice for GCP with Terraform or OpenTofu is that code owns service accounts, secret containers, and IAM bindings, and only secret values stay out of state.

Decision

  • infra/access.tf owns the runtime identities, the secret containers, every secret grant, the web identity's self iam.serviceAccountUser, and the CD identity's run.admin, cloudtasks.admin, iam.serviceAccountAdmin and secretmanager.admin. The CD identity is deployer_service_account in the tfvars.
  • infra/set-secrets.sh <project-id> only adds secret versions. Values come from the environment through stdin and never enter code or state.
  • Every job and service depends on its grants and on a google_secret_manager_secret_version data source with fetch_secret_data = false, so no workload is created before its secrets have a value (the #138 guarantee), and the value is not read.
  • Test adopts its existing identities and secrets through import blocks that apply only to the test environment.

Out of scope: the CD identity's base roles (editor, iam.serviceAccountUser, resourcemanager.projectIamAdmin) and Workload Identity Federation. They come from the GitHub bootstrap, because the identity that applies the code cannot grant itself the right to apply it.

Consequences

  • A new environment takes two applies: the first creates identities, secret containers and grants and stops at the missing versions; after set-secrets.sh, the second creates the workloads.
  • The first apply in a project grants the CD identity its extra roles and uses them in the same run. If IAM propagation is slow, the dependent grants can fail once; a second delivery completes them.
  • The CD identity can create service accounts and change their IAM policy in the projects it deploys to.
  • Secret rotation is set-secrets.sh again; no apply is needed, because workloads mount the latest version.