From ede1112f9b04b5973e88bc72b56da6a2591944ea Mon Sep 17 00:00:00 2001 From: GitHub Action Date: Mon, 14 Oct 2024 05:00:47 +0000 Subject: [PATCH] Publish docs from 9d30779f3647d6fa127cdbcfdc17d809891fd482 --- References/terraform/index.html | 72 ++++++++++++++++++++++++++++++++ search/search_index.json | 2 +- sitemap.xml.gz | Bin 127 -> 127 bytes 3 files changed, 73 insertions(+), 1 deletion(-) diff --git a/References/terraform/index.html b/References/terraform/index.html index 658696bd..61fd65fd 100644 --- a/References/terraform/index.html +++ b/References/terraform/index.html @@ -1282,6 +1282,12 @@

ResourceRef +

RetryStrategyEnum +(string alias)

+

+(Appears on: +TerraformSpec) +

RunnerPodMetadata

@@ -2056,6 +2062,39 @@

Terraform +retryStrategy
+ + +RetryStrategyEnum + + + + +(Optional) +

The strategy to use when retrying a previously failed reconciliation. +The default strategy is StaticInterval and the retry interval is based on the RetryInterval value. +The ExponentialBackoff strategy uses the formula: 2^reconciliationFailures * RetryInterval with a +maximum requeue duration of MaxRetryInterval.

+ + + + +maxRetryInterval
+ + +Kubernetes meta/v1.Duration + + + + +(Optional) +

The maximum requeue duration after a previously failed reconciliation. +Only applicable when RetryStrategy is set to ExponentialBackoff. +The default value is 24 hours when not specified.

+ + + + path
string @@ -2619,6 +2658,39 @@

TerraformSpec +retryStrategy
+ + +RetryStrategyEnum + + + + +(Optional) +

The strategy to use when retrying a previously failed reconciliation. +The default strategy is StaticInterval and the retry interval is based on the RetryInterval value. +The ExponentialBackoff strategy uses the formula: 2^reconciliationFailures * RetryInterval with a +maximum requeue duration of MaxRetryInterval.

+ + + + +maxRetryInterval
+ + +Kubernetes meta/v1.Duration + + + + +(Optional) +

The maximum requeue duration after a previously failed reconciliation. +Only applicable when RetryStrategy is set to ExponentialBackoff. +The default value is 24 hours when not specified.

+ + + + path
string diff --git a/search/search_index.json b/search/search_index.json index 8c5dcd29..7d619fa6 100644 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Overview","text":"

~> BREAKING NEWS: The Technology Preview of the Branch Planner is now available! Learn more or Get started with the Branch Planner.

TF-controller is a reliable controller for Flux to reconcile Terraform resources in the GitOps way. With the power of Flux together with Terraform, TF-controller allows you to GitOps-ify infrastructure, and application resources, in the Kubernetes and Terraform universe, at your own pace.

\"At your own pace\" means you don't need to GitOps-ify everything at once.

A high-level diagram of how TF-controller works is shown below:

TF-controller offers many GitOps models:

  1. GitOps Automation Model: GitOps your Terraform resources from the provision steps to the enforcement steps, like a whole EKS cluster.
  2. Hybrid GitOps Automation Model: GitOps parts of your existing infrastructure resources. For example, you have an existing EKS cluster. You can choose to GitOps only its nodegroup, or its security group.
  3. State Enforcement Model: You have a TFSTATE file, and you'd like to use GitOps enforce it, without changing anything else.
  4. Drift Detection Model: You have a TFSTATE file, and you'd like to use GitOps just for drift detection, so you can decide to do things later when a drift occurs.

To get started, follow the getting started guide.

"},{"location":"#features","title":"Features","text":"
  • Multi-Tenancy: TF-controller supports multi-tenancy by running Terraform plan and apply inside Runner Pods. When specifying .metadata.namespace and .spec.serviceAccountName, the Runner Pod uses the specified ServiceAccount and runs inside the specified Namespace. These settings enable the soft multi-tenancy model, which can be used within the Flux multi-tenancy setup. This feature is available since v0.9.0.
  • GitOps Automation for Terraform: With setting .spec.approvePlan=auto, it allows a Terraform object to be reconciled and act as the representation of your Terraform resources. The TF-controller uses the spec of the Terraform object to perform plan, apply its associated Terraform resources. It then stores the TFSTATE of the applied resources as a Secret inside the Kubernetes cluster. After .spec.interval passes, the controller performs drift detection to check if there is a drift occurred between your live system, and your Terraform resources. If a drift occurs, the plan to fix that drift will be generated and applied automatically. This feature is available since v0.3.0.
  • Drift detection: This feature is a part of the GitOps automation feature. The controller detects and fixes drift for your infrastructures, based on the Terraform resources and their TFSTATE. This feature is available since v0.5.0.
    • Drift detection is enabled by default. You can use the field .spec.disableDriftDetection to disable this behaviour. This feature is available since v0.7.0.
    • The Drift detection only mode, without plan or apply steps, allows you to perform read-only drift detection. This feature is available since v0.8.0.
  • Plan and Manual Approve: This feature allows you to separate the plan, out of the apply step, just like the Terraform workflow you are familiar with. A good thing about this is that it is done in a GitOps way. When a plan is generated, the controller shows you a message like 'set approvePlan: \"plan-main-123\" to apply this plan.'. You make change to the field .spec.approvePlan, commit and push to tell the TF-controller to apply the plan for you. With this GitOps workflow, you can optionally create and push this change to a new branch for your team member to review and approve too. This feature is available since v0.6.0.
  • First-class YAML-based Terraform: The Terraform object in v0.13.0+ allows you to better configure your Terraform resources via YAMLs, but without introducing any extra CRDs to your cluster. Together with a new generator called Tofu-Jet, we'll now be able to ship pre-generated primitive Terraform modules for all major cloud providers. A primitive Terraform module is a module that only contains a single primitive resource, like aws_iam_role, or aws_iam_policy. With this concept, we would be able to use Terraform without writing Terraform codes, and make it more GitOps-friendly at the same time. This feature is available since v0.13.0.
  • GitOps Dependency for Terraform: The Terraform object in v0.13.0+ allows you to specify a list of Terraform objects that it depends on. The controller will wait for the dependencies to be ready before it starts to reconcile the Terraform object. This allows you to create a dependency graph of your Terraform modules, and make sure the modules are applied in the correct order. Please use .spec.retryInterval (a small value like 20s) to control the retry interval when using this feature. This feature is available since v0.13.0.
"},{"location":"#support-matrix","title":"Support Matrix","text":"Version Terraform Source Controller Flux v2 v0.15 v1.3.9 v1.0.x v2.0.x v0.14 v1.3.9 v0.31.0 v0.41.x v0.13 v1.3.1 v0.31.0 v0.36.x v0.12 v1.1.9 v0.26.1 v0.32.x"},{"location":"getting_started/","title":"Getting Started","text":""},{"location":"getting_started/#preflight-checks","title":"Preflight Checks","text":"

Here are the requirements you need to set up before you start:

  1. For Terraform Controller v0.15+, it requires Flux v2.0 or later (not only the CLI, but also the controllers on the cluster). If you are not sure about the Flux version on your cluster, please re-bootstrap your cluster.
  2. For Terraform Controller v0.13 and v0.14, Flux 2 v0.32 - v0.41 (of course, not only the CLI, but also the controllers on the cluster).
  3. TF-controller uses the Controller/Runner architecture. The Controller acts as a client, and talks to each Runner's Pod via gRPC. Please make sure
    1. Each Runner's Pod in each Namespace is allowed to open, and serve at port 30000 (the gRPC port of a Runner), and the Controller can connect to it.
    2. The Controller needs to download tar.gz BLOBs from the Source controller via port 80.
    3. The Controller needs to post the events to the Notification controller via port 80.
"},{"location":"getting_started/#installation","title":"Installation","text":"

Before using TF-controller, you have to install Flux by using either flux install or flux bootstrap command. Please note that TF-controller now requires Flux v2.0 or later, so please make sure you have the latest version of Flux. After that you can install TF-controller with Flux HelmRelease by:

kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/release.yaml\n

For the most recent release candidate of TF-controller, please use rc.yaml.

kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/rc.yaml\n
"},{"location":"getting_started/#installation-on-gke","title":"Installation on GKE","text":"

As of September 2023, GKE Autopilot clusters will use Cloud DNS for internal DNS resolution. This means that the default DNS resolution method used by TF-controller will not work. To use TF-controller on GKE Autopilot, you must set flag --use-pod-subdomain-resolution=true on the TF-controller deployment. This flag can be set by adding the following to the TF-controller HelmRelease:

spec:\n  values:\n    usePodSubdomainResolution: true\n    runner:\n      allowedNamespaces:\n      - flux-system\n      - dev-team\n

Enabling this value will cause TF-controller to use the Pod's subdomain for DNS resolution instead of the default Pod resolution method. Pod's subdomain resolution requires a Service to be created for the Pod. The HelmRelease above will create a Service named tf-runner in each namespace specified by the runner.allowedNamespaces value.

We have provided a HelmRelease to install TF-controller on GKE Autopilot with Pod's subdomain resolution enabled here.

kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/rc-gke.yaml\n

Tested with GKE Autopilot v1.27.3-gke.100.

"},{"location":"getting_started/#with-branch-planner","title":"With Branch Planner","text":"
kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/branch-planner/release.yaml\n

For the most recent release candidate of TF-controller with Branch Planner, please use branch-planner/rc.yaml.

kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/branch-planner/rc.yaml\n

For more details about the Branch Planner, please visit the Branch Planner documentation.

"},{"location":"getting_started/#manual-installation","title":"Manual installation","text":"

With Helm by:

# Add tf-controller helm repository\nhelm repo add tf-controller https://flux-iac.github.io/tofu-controller\n\n# Install tf-controller\nhelm upgrade -i tf-controller tf-controller/tf-controller \\\n    --namespace flux-system\n

For details on configurable parameters of the TF-controller chart, please see chart readme.

Alternatively, you can install TF-controller via kubectl:

export TF_CON_VER=v0.15.1\nkubectl apply -f https://github.com/flux-iac/tofu-controller/releases/download/${TF_CON_VER}/tf-controller.crds.yaml\nkubectl apply -f https://github.com/flux-iac/tofu-controller/releases/download/${TF_CON_VER}/tf-controller.rbac.yaml\nkubectl apply -f https://github.com/flux-iac/tofu-controller/releases/download/${TF_CON_VER}/tf-controller.deployment.yaml\n
"},{"location":"getting_started/#quick-start","title":"Quick start","text":"

Here's a simple example of how to GitOps your Terraform resources with TF-controller and Flux.

"},{"location":"getting_started/#define-source","title":"Define source","text":"

First, we need to define a Source controller's source (GitRepository, Bucket, OCIRepository), for example:

apiVersion: source.toolkit.fluxcd.io/v1\nkind: GitRepository\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: https://github.com/tf-controller/helloworld\n  ref:\n    branch: main\n
"},{"location":"getting_started/#the-gitops-automation-mode","title":"The GitOps Automation mode","text":"

The GitOps automation mode could be enabled by setting .spec.approvePlan=auto. In this mode, Terraform resources will be planned, and automatically applied for you.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  interval: 1m\n  approvePlan: auto\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n

For a full list of features and how to use them, please follow the Use TF-controller guide.

"},{"location":"getting_started/#other-examples","title":"Other Examples","text":"
  • A Terraform GitOps with Flux to automatically reconcile your AWS IAM Policies.
  • GitOps an existing EKS cluster, by partially import its nodegroup and manage it with TF-controller: An EKS scaling example.
"},{"location":"tfctl/","title":"tfctl","text":"

tfctl is a command-line utility to help with tf-controller operations.

"},{"location":"tfctl/#installation","title":"Installation","text":"

To install tfctl via Homebrew, run the following command:

brew install weaveworks/tap/tfctl\n

You can also download the tfctl binary via the GitHub releases page: https://github.com/flux-iac/tofu-controller/releases.

Usage:\n  tfctl [command]\n\nAvailable Commands:\n  completion  Generate the autocompletion script for the specified shell\n  create      Create a Terraform resource\n  delete      Delete a Terraform resource\n  get         Get Terraform resources\n  help        Help about any command\n  install     Install the tf-controller\n  plan        Plan a Terraform configuration\n  reconcile   Trigger a reconcile of the provided resource\n  resume      Resume reconciliation for the provided resource\n  suspend     Suspend reconciliation for the provided resource\n  uninstall   Uninstall the tf-controller\n  version     Prints tf-controller and tfctl version information\n\nFlags:\n  -h, --help                help for tfctl\n      --kubeconfig string   Path to the kubeconfig file to use for CLI requests.\n  -n, --namespace string    The kubernetes namespace to use for CLI requests. (default \"flux-system\")\n      --terraform string    The location of the terraform binary. (default \"/usr/bin/terraform\")\n\nUse \"tfctl [command] --help\" for more information about a command.\n
"},{"location":"tfctl/#shell-completion","title":"Shell completion","text":"

It works the same way as flux CLI:

With bash:

# ~/.bashrc or ~/.profile\ncommand -v tfctl >/dev/null && . <(tfctl completion bash)\n

With fish:

tfctl completion fish > ~/.config/fish/completions/tfctl.fish\n

With powershell:

# Windows\n\ncd \"$env:USERPROFILE\\Documents\\WindowsPowerShell\\Modules\"\ntfctl completion powershell >> tfctl-completion.ps1\n\n# Linux\n\ncd \"${XDG_CONFIG_HOME:-\"$HOME/.config/\"}/powershell/modules\"\ntfctl completion powershell >> tfctl-completions.ps1\n

With zsh:

# ~/.zshrc or ~/.profile\ncommand -v tfctl >/dev/null && . <(tfctl completion zsh)\n
"},{"location":"References/terraform/","title":"API References","text":"Terraform API reference

Packages:

  • infra.contrib.fluxcd.io/v1alpha2
"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2","title":"infra.contrib.fluxcd.io/v1alpha2","text":"

Resource Types:

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.BackendConfigSpec","title":"BackendConfigSpec","text":"

(Appears on: TerraformSpec)

BackendConfigSpec is for specifying configuration for Terraform\u2019s Kubernetes backend

Field Description disable bool (Optional)

Disable is to completely disable the backend configuration.

secretSuffix string (Optional) inClusterConfig bool (Optional) customConfiguration string (Optional) configPath string (Optional) labels map[string]string (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.BackendConfigsReference","title":"BackendConfigsReference","text":"

(Appears on: TerraformSpec)

Field Description kind string

Kind of the values referent, valid values are (\u2018Secret\u2019, \u2018ConfigMap\u2019).

name string

Name of the configs referent. Should reside in the same namespace as the referring resource.

keys []string (Optional)

Keys is the data key where a specific value can be found at. Defaults to all keys.

optional bool (Optional)

Optional marks this BackendConfigsReference as optional. When set, a not found error for the values reference is ignored, but any Key or transient error will still result in a reconciliation failure.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.BranchPlanner","title":"BranchPlanner","text":"

(Appears on: TerraformSpec)

Field Description enablePathScope bool (Optional)

EnablePathScope specifies if the Branch Planner should or shouldn\u2019t check if a Pull Request has changes under .spec.path. If enabled extra resources will be created only if there are any changes in terraform files.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.CloudSpec","title":"CloudSpec","text":"

(Appears on: TerraformSpec)

Field Description organization string workspaces CloudWorkspacesSpec hostname string (Optional) token string (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.CloudWorkspacesSpec","title":"CloudWorkspacesSpec","text":"

(Appears on: CloudSpec)

Field Description name string (Optional) tags []string (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.CrossNamespaceSourceReference","title":"CrossNamespaceSourceReference","text":"

(Appears on: TerraformSpec)

CrossNamespaceSourceReference contains enough information to let you locate the typed Kubernetes resource object at cluster level.

Field Description apiVersion string (Optional)

API version of the referent.

kind string

Kind of the referent.

name string

Name of the referent.

namespace string (Optional)

Namespace of the referent, defaults to the namespace of the Kubernetes resource object that contains the reference.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.FileMapping","title":"FileMapping","text":"

(Appears on: TerraformSpec)

Field Description secretRef github.com/fluxcd/pkg/apis/meta.SecretKeyReference

Reference to a Secret that contains the file content

location string

Location can be either user\u2019s home directory or the Terraform workspace

path string

Path of the file - relative to the \u201clocation\u201d

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.ForceUnlockEnum","title":"ForceUnlockEnum (string alias)","text":"

(Appears on: TFStateSpec)

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.HealthCheck","title":"HealthCheck","text":"

(Appears on: TerraformSpec)

HealthCheck contains configuration needed to perform a health check after terraform is applied.

Field Description name string

Name of the health check.

type string

Type of the health check, valid values are (\u2018tcp\u2019, \u2018http\u2019). If tcp is specified, address is required. If http is specified, url is required.

url string (Optional)

URL to perform http health check on. Required when http type is specified. Go template can be used to reference values from the terraform output (e.g. https://example.org, {{.output_url}}).

address string (Optional)

Address to perform tcp health check on. Required when tcp type is specified. Go template can be used to reference values from the terraform output (e.g. 127.0.0.1:8080, {{.address}}:{{.port}}).

timeout Kubernetes meta/v1.Duration (Optional)

The timeout period at which the connection should timeout if unable to complete the request. When not specified, default 20s timeout is used.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.LockStatus","title":"LockStatus","text":"

(Appears on: TerraformStatus)

LockStatus defines the observed state of a Terraform State Lock

Field Description lastApplied string (Optional) pending string (Optional)

Pending holds the identifier of the Lock Holder to be used with Force Unlock

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.PlanStatus","title":"PlanStatus","text":"

(Appears on: TerraformStatus)

Field Description lastApplied string (Optional) pending string (Optional) isDestroyPlan bool (Optional) isDriftDetectionPlan bool (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.ReadInputsFromSecretSpec","title":"ReadInputsFromSecretSpec","text":"

(Appears on: TerraformSpec)

Field Description name string as string"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.Remediation","title":"Remediation","text":"

(Appears on: TerraformSpec)

Field Description retries int64 (Optional)

Retries is the number of retries that should be attempted on failures before bailing. Defaults to \u20180\u2019, a negative integer denotes unlimited retries.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.ResourceInventory","title":"ResourceInventory","text":"

(Appears on: TerraformStatus)

ResourceInventory contains a list of Kubernetes resource object references that have been applied by a Kustomization.

Field Description entries []ResourceRef

Entries of Kubernetes resource object references.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.ResourceRef","title":"ResourceRef","text":"

(Appears on: ResourceInventory)

ResourceRef contains the information necessary to locate a resource within a cluster.

Field Description n string

Terraform resource\u2019s name.

t string

Type is Terraform resource\u2019s type

id string

ID is the resource identifier. This is cloud-specific. For example, ARN is an ID on AWS.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RunnerPodMetadata","title":"RunnerPodMetadata","text":"

(Appears on: RunnerPodTemplate)

Field Description labels map[string]string (Optional)

Labels to add to the runner pod

annotations map[string]string (Optional)

Annotations to add to the runner pod

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RunnerPodSpec","title":"RunnerPodSpec","text":"

(Appears on: RunnerPodTemplate)

Field Description image string (Optional)

Runner pod image to use other than default

envFrom []Kubernetes core/v1.EnvFromSource (Optional)

List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.

env []Kubernetes core/v1.EnvVar (Optional)

List of environment variables to set in the container. Cannot be updated.

nodeSelector map[string]string (Optional)

Set the NodeSelector for the Runner Pod

affinity Kubernetes core/v1.Affinity (Optional)

Set the Affinity for the Runner Pod

tolerations []Kubernetes core/v1.Toleration (Optional)

Set the Tolerations for the Runner Pod

volumeMounts []Kubernetes core/v1.VolumeMount (Optional)

Set Volume Mounts for the Runner Pod

volumes []Kubernetes core/v1.Volume (Optional)

Set Volumes for the Runner Pod

initContainers []Kubernetes core/v1.Container (Optional)

Set up Init Containers for the Runner

hostAliases []Kubernetes core/v1.HostAlias (Optional)

Set host aliases for the Runner Pod

priorityClassName string (Optional)

Set PriorityClassName for the Runner Pod container

securityContext Kubernetes core/v1.SecurityContext (Optional)

Set SecurityContext for the Runner Pod container

resources Kubernetes core/v1.ResourceRequirements (Optional)

Set Resources for the Runner Pod container

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RunnerPodTemplate","title":"RunnerPodTemplate","text":"

(Appears on: TerraformSpec)

Field Description metadata RunnerPodMetadata (Optional) spec RunnerPodSpec (Optional) image string (Optional)

Runner pod image to use other than default

envFrom []Kubernetes core/v1.EnvFromSource (Optional)

List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.

env []Kubernetes core/v1.EnvVar (Optional)

List of environment variables to set in the container. Cannot be updated.

nodeSelector map[string]string (Optional)

Set the NodeSelector for the Runner Pod

affinity Kubernetes core/v1.Affinity (Optional)

Set the Affinity for the Runner Pod

tolerations []Kubernetes core/v1.Toleration (Optional)

Set the Tolerations for the Runner Pod

volumeMounts []Kubernetes core/v1.VolumeMount (Optional)

Set Volume Mounts for the Runner Pod

volumes []Kubernetes core/v1.Volume (Optional)

Set Volumes for the Runner Pod

initContainers []Kubernetes core/v1.Container (Optional)

Set up Init Containers for the Runner

hostAliases []Kubernetes core/v1.HostAlias (Optional)

Set host aliases for the Runner Pod

priorityClassName string (Optional)

Set PriorityClassName for the Runner Pod container

securityContext Kubernetes core/v1.SecurityContext (Optional)

Set SecurityContext for the Runner Pod container

resources Kubernetes core/v1.ResourceRequirements (Optional)

Set Resources for the Runner Pod container

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.TFStateSpec","title":"TFStateSpec","text":"

(Appears on: TerraformSpec)

TFStateSpec allows the user to set ForceUnlock

Field Description forceUnlock ForceUnlockEnum (Optional)

ForceUnlock a Terraform state if it has become locked for any reason. Defaults to no.

This is an Enum and has the expected values of:

  • auto
  • yes
  • no

WARNING: Only use auto in the cases where you are absolutely certain that no other system is using this state, you could otherwise end up in a bad place See https://www.terraform.io/language/state/locking#force-unlock for more information on the terraform state lock and force unlock.

lockIdentifier string (Optional)

LockIdentifier holds the Identifier required by Terraform to unlock the state if it ever gets into a locked state.

You\u2019ll need to put the Lock Identifier in here while setting ForceUnlock to either yes or auto.

Leave this empty to do nothing, set this to the value of the Lock Info: ID: [value], e.g. f2ab685b-f84d-ac0b-a125-378a22877e8d, to force unlock the state.

lockTimeout Kubernetes meta/v1.Duration (Optional)

LockTimeout is a Duration string that instructs Terraform to retry acquiring a lock for the specified period of time before returning an error. The duration syntax is a number followed by a time unit letter, such as 3s for three seconds.

Defaults to 0s which will behave as though LockTimeout was not set

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.Terraform","title":"Terraform","text":"

Terraform is the Schema for the terraforms API

Field Description metadata Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field. spec TerraformSpec approvePlan string (Optional)

ApprovePlan specifies name of a plan wanted to approve. If its value is \u201cauto\u201d, the controller will automatically approve every plan.

destroy bool (Optional)

Destroy produces a destroy plan. Applying the plan will destroy all resources.

backendConfig BackendConfigSpec (Optional) backendConfigsFrom []BackendConfigsReference (Optional) cloud CloudSpec (Optional) workspace string (Optional) vars []Variable (Optional)

List of input variables to set for the Terraform program.

varsFrom []VarsReference (Optional)

List of references to a Secret or a ConfigMap to generate variables for Terraform resources based on its data, selectively by varsKey. Values of the later Secret / ConfigMap with the same keys will override those of the former.

values Kubernetes pkg/apis/apiextensions/v1.JSON (Optional)

Values map to the Terraform variable \u201cvalues\u201d, which is an object of arbitrary values. It is a convenient way to pass values to Terraform resources without having to define a variable for each value. To use this feature, your Terraform file must define the variable \u201cvalues\u201d.

tfVarsFiles []string (Optional)

TfVarsFiles loads all given .tfvars files. It copycats the -var-file functionality.

fileMappings []FileMapping (Optional)

List of all configuration files to be created in initialization.

interval Kubernetes meta/v1.Duration

The interval at which to reconcile the Terraform.

retryInterval Kubernetes meta/v1.Duration (Optional)

The interval at which to retry a previously failed reconciliation. The default value is 15 when not specified.

path string (Optional)

Path to the directory containing Terraform (.tf) files. Defaults to \u2018None\u2019, which translates to the root path of the SourceRef.

sourceRef CrossNamespaceSourceReference

SourceRef is the reference of the source where the Terraform files are stored.

suspend bool (Optional)

Suspend is to tell the controller to suspend subsequent TF executions, it does not apply to already started executions. Defaults to false.

force bool (Optional)

Force instructs the controller to unconditionally re-plan and re-apply TF resources. Defaults to false.

readInputsFromSecrets []ReadInputsFromSecretSpec (Optional) writeOutputsToSecret WriteOutputsToSecretSpec (Optional)

A list of target secrets for the outputs to be written as.

disableDriftDetection bool (Optional)

Disable automatic drift detection. Drift detection may be resource intensive in the context of a large cluster or complex Terraform statefile. Defaults to false.

cliConfigSecretRef Kubernetes core/v1.SecretReference (Optional) healthChecks []HealthCheck (Optional)

List of health checks to be performed.

destroyResourcesOnDeletion bool (Optional)

Create destroy plan and apply it to destroy terraform resources upon deletion of this object. Defaults to false.

serviceAccountName string (Optional)

Name of a ServiceAccount for the runner Pod to provision Terraform resources. Default to tf-runner.

alwaysCleanupRunnerPod bool (Optional)

Clean the runner pod up after each reconciliation cycle

runnerTerminationGracePeriodSeconds int64 (Optional)

Configure the termination grace period for the runner pod. Use this parameter to allow the Terraform process to gracefully shutdown. Consider increasing for large, complex or slow-moving Terraform managed resources.

refreshBeforeApply bool (Optional)

RefreshBeforeApply forces refreshing of the state before the apply step.

runnerPodTemplate RunnerPodTemplate (Optional) enableInventory bool (Optional)

EnableInventory enables the object to store resource entries as the inventory for external use.

tfstate TFStateSpec (Optional) targets []string (Optional)

Targets specify the resource, module or collection of resources to target.

parallelism int32 (Optional)

Parallelism limits the number of concurrent operations of Terraform apply step. Zero (0) means using the default value.

storeReadablePlan string (Optional)

StoreReadablePlan enables storing the plan in a readable format.

webhooks []Webhook (Optional) dependsOn []github.com/fluxcd/pkg/apis/meta.NamespacedObjectReference (Optional) enterprise Kubernetes pkg/apis/apiextensions/v1.JSON (Optional)

Enterprise is the enterprise configuration placeholder.

planOnly bool (Optional)

PlanOnly specifies if the reconciliation should or should not stop at plan phase.

breakTheGlass bool (Optional)

BreakTheGlass specifies if the reconciliation should stop and allow interactive shell in case of emergency.

branchPlanner BranchPlanner (Optional)

BranchPlanner configuration.

remediation Remediation (Optional)

Remediation specifies what the controller should do when reconciliation fails. The default is to not perform any action.

status TerraformStatus"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.TerraformSpec","title":"TerraformSpec","text":"

(Appears on: Terraform)

TerraformSpec defines the desired state of Terraform

Field Description approvePlan string (Optional)

ApprovePlan specifies name of a plan wanted to approve. If its value is \u201cauto\u201d, the controller will automatically approve every plan.

destroy bool (Optional)

Destroy produces a destroy plan. Applying the plan will destroy all resources.

backendConfig BackendConfigSpec (Optional) backendConfigsFrom []BackendConfigsReference (Optional) cloud CloudSpec (Optional) workspace string (Optional) vars []Variable (Optional)

List of input variables to set for the Terraform program.

varsFrom []VarsReference (Optional)

List of references to a Secret or a ConfigMap to generate variables for Terraform resources based on its data, selectively by varsKey. Values of the later Secret / ConfigMap with the same keys will override those of the former.

values Kubernetes pkg/apis/apiextensions/v1.JSON (Optional)

Values map to the Terraform variable \u201cvalues\u201d, which is an object of arbitrary values. It is a convenient way to pass values to Terraform resources without having to define a variable for each value. To use this feature, your Terraform file must define the variable \u201cvalues\u201d.

tfVarsFiles []string (Optional)

TfVarsFiles loads all given .tfvars files. It copycats the -var-file functionality.

fileMappings []FileMapping (Optional)

List of all configuration files to be created in initialization.

interval Kubernetes meta/v1.Duration

The interval at which to reconcile the Terraform.

retryInterval Kubernetes meta/v1.Duration (Optional)

The interval at which to retry a previously failed reconciliation. The default value is 15 when not specified.

path string (Optional)

Path to the directory containing Terraform (.tf) files. Defaults to \u2018None\u2019, which translates to the root path of the SourceRef.

sourceRef CrossNamespaceSourceReference

SourceRef is the reference of the source where the Terraform files are stored.

suspend bool (Optional)

Suspend is to tell the controller to suspend subsequent TF executions, it does not apply to already started executions. Defaults to false.

force bool (Optional)

Force instructs the controller to unconditionally re-plan and re-apply TF resources. Defaults to false.

readInputsFromSecrets []ReadInputsFromSecretSpec (Optional) writeOutputsToSecret WriteOutputsToSecretSpec (Optional)

A list of target secrets for the outputs to be written as.

disableDriftDetection bool (Optional)

Disable automatic drift detection. Drift detection may be resource intensive in the context of a large cluster or complex Terraform statefile. Defaults to false.

cliConfigSecretRef Kubernetes core/v1.SecretReference (Optional) healthChecks []HealthCheck (Optional)

List of health checks to be performed.

destroyResourcesOnDeletion bool (Optional)

Create destroy plan and apply it to destroy terraform resources upon deletion of this object. Defaults to false.

serviceAccountName string (Optional)

Name of a ServiceAccount for the runner Pod to provision Terraform resources. Default to tf-runner.

alwaysCleanupRunnerPod bool (Optional)

Clean the runner pod up after each reconciliation cycle

runnerTerminationGracePeriodSeconds int64 (Optional)

Configure the termination grace period for the runner pod. Use this parameter to allow the Terraform process to gracefully shutdown. Consider increasing for large, complex or slow-moving Terraform managed resources.

refreshBeforeApply bool (Optional)

RefreshBeforeApply forces refreshing of the state before the apply step.

runnerPodTemplate RunnerPodTemplate (Optional) enableInventory bool (Optional)

EnableInventory enables the object to store resource entries as the inventory for external use.

tfstate TFStateSpec (Optional) targets []string (Optional)

Targets specify the resource, module or collection of resources to target.

parallelism int32 (Optional)

Parallelism limits the number of concurrent operations of Terraform apply step. Zero (0) means using the default value.

storeReadablePlan string (Optional)

StoreReadablePlan enables storing the plan in a readable format.

webhooks []Webhook (Optional) dependsOn []github.com/fluxcd/pkg/apis/meta.NamespacedObjectReference (Optional) enterprise Kubernetes pkg/apis/apiextensions/v1.JSON (Optional)

Enterprise is the enterprise configuration placeholder.

planOnly bool (Optional)

PlanOnly specifies if the reconciliation should or should not stop at plan phase.

breakTheGlass bool (Optional)

BreakTheGlass specifies if the reconciliation should stop and allow interactive shell in case of emergency.

branchPlanner BranchPlanner (Optional)

BranchPlanner configuration.

remediation Remediation (Optional)

Remediation specifies what the controller should do when reconciliation fails. The default is to not perform any action.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.TerraformStatus","title":"TerraformStatus","text":"

(Appears on: Terraform)

TerraformStatus defines the observed state of Terraform

Field Description ReconcileRequestStatus github.com/fluxcd/pkg/apis/meta.ReconcileRequestStatus

(Members of ReconcileRequestStatus are embedded into this type.)

observedGeneration int64 (Optional)

ObservedGeneration is the last reconciled generation.

conditions []Kubernetes meta/v1.Condition (Optional) lastAppliedRevision string (Optional)

The last successfully applied revision. The revision format for Git sources is /. lastAttemptedRevision string (Optional)

LastAttemptedRevision is the revision of the last reconciliation attempt.

lastPlannedRevision string (Optional)

LastPlannedRevision is the revision used by the last planning process. The result could be either no plan change or a new plan generated.

lastPlanAt Kubernetes meta/v1.Time (Optional)

LastPlanAt is the time when the last terraform plan was performed

lastDriftDetectedAt Kubernetes meta/v1.Time (Optional)

LastDriftDetectedAt is the time when the last drift was detected

lastAppliedByDriftDetectionAt Kubernetes meta/v1.Time (Optional)

LastAppliedByDriftDetectionAt is the time when the last drift was detected and terraform apply was performed as a result

availableOutputs []string (Optional) plan PlanStatus (Optional) inventory ResourceInventory (Optional)

Inventory contains the list of Terraform resource object references that have been successfully applied.

lock LockStatus (Optional) reconciliationFailures int64 (Optional)

ReconciliationFailures is the number of reconciliation failures since the last success or update.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.Variable","title":"Variable","text":"

(Appears on: TerraformSpec)

Field Description name string

Name is the name of the variable

value Kubernetes pkg/apis/apiextensions/v1.JSON (Optional) valueFrom Kubernetes core/v1.EnvVarSource (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.VarsReference","title":"VarsReference","text":"

(Appears on: TerraformSpec)

VarsReference contain a reference of a Secret or a ConfigMap to generate variables for Terraform resources based on its data, selectively by varsKey.

Field Description kind string

Kind of the values referent, valid values are (\u2018Secret\u2019, \u2018ConfigMap\u2019).

name string

Name of the values referent. Should reside in the same namespace as the referring resource.

varsKeys []string (Optional)

VarsKeys is the data key at which a specific value can be found. Defaults to all keys.

optional bool (Optional)

Optional marks this VarsReference as optional. When set, a not found error for the values reference is ignored, but any VarsKey or transient error will still result in a reconciliation failure.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.Webhook","title":"Webhook","text":"

(Appears on: TerraformSpec)

Field Description stage string enabled bool (Optional) url string payloadType string (Optional) errorMessageTemplate string (Optional) testExpression string"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.WriteOutputsToSecretSpec","title":"WriteOutputsToSecretSpec","text":"

(Appears on: TerraformSpec)

WriteOutputsToSecretSpec defines where to store outputs, and which outputs to be stored.

Field Description name string

Name is the name of the Secret to be written

labels map[string]string (Optional)

Labels to add to the outputted secret

annotations map[string]string (Optional)

Annotations to add to the outputted secret

outputs []string (Optional)

Outputs contain the selected names of outputs to be written to the secret. Empty array means writing all outputs, which is default.

This page was automatically generated with gen-crd-api-reference-docs

"},{"location":"adr/0000-template/","title":"0. Title","text":"
  • Status: [proposed | rejected | accepted | deprecated | \u2026 | superseded by ADR-0005]
  • Date: 2020-10-29 [YYY-MM-DD - date of the decision]
  • Authors: [list of GitHub handles for the authors]
  • Deciders: [list of GitHub handles for those that made the decision]
"},{"location":"adr/0000-template/#context","title":"Context","text":""},{"location":"adr/0000-template/#decision","title":"Decision","text":""},{"location":"adr/0000-template/#consequences","title":"Consequences","text":""},{"location":"adr/0000-use-adrs-for-decisions/","title":"1. Use ADRs to record decisions","text":"
  • Status: proposed
  • Date: 2023-06-20
  • Authors: @yitsushi
  • Deciders: @yitsushi @chanwit @yiannistri
"},{"location":"adr/0000-use-adrs-for-decisions/#context","title":"Context","text":"

Decisions that affect the development of Terraform Controller that are not captured via a proposal need to be captured in some way. We need a method that is lightweight and easy to discover the decision that have been made. The record of decisions will help future contributors to the project to understand why something has been implemented or is done a certain way.

"},{"location":"adr/0000-use-adrs-for-decisions/#decision","title":"Decision","text":"

The project will use Architectural Decision Records (ADR) to record decisions that are made outside of a proposal.

A template has been created based on prior work:

  • https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions
  • https://adr.github.io/madr/
"},{"location":"adr/0000-use-adrs-for-decisions/#consequences","title":"Consequences","text":"

When decisions are made that affect the entire project then a new ADR needs to be created. Likewise, if a decision has been superseded then we need to capture this as a new ADR and mark the previous ADR as superseded. Maintainers and contributors will need to decide when an ADR is to be created.

"},{"location":"adr/0001-pr-polling-workflow/","title":"1. Pull Request Polling","text":"
  • Status: proposed
  • Date: 2023-06-20
  • Authors: @yitsushi
  • Deciders: @yitsushi @chanwit @yiannistri
"},{"location":"adr/0001-pr-polling-workflow/#context","title":"Context","text":"

To detect pull request changes, we can use webhooks or polling using GitHub's API.

"},{"location":"adr/0001-pr-polling-workflow/#decision","title":"Decision","text":"

We decided to start with polling for security reasons. Using webhooks would require users to open an ingress to the cluster. Because of this requirement, we think security conscious folks may refuse to roll this out on production clusters especially in an air-gapped environment. This does not mean that we will never consider using webhooks for this, but that initially, polling is what we have chosen to implement.

The Branch-Based Planner has two components:

  1. Polling Server: Detect Pull Request changes and manage Teraform resource state.
  2. Informer: Make a comment when new plan output is available.
"},{"location":"adr/0001-pr-polling-workflow/#consequences","title":"Consequences","text":"

The list Pull Requests endpoint returns all required fields to detect new and closed pull requests. It's one request per repository, but listing comments has to use an API request per pull request. So we have to add a mechanism to avoid hitting API rate limits.

"},{"location":"adr/0002-deny-cross-ns-by-default/","title":"2. Deny cross-namespace refs by default","text":"
  • Status: [ proposed | rejected | accepted | deprecated ]
  • Date: 2023-07-18
  • Authors: @squaremo
  • Deciders: [list of GitHub handles for those that made the decision]
"},{"location":"adr/0002-deny-cross-ns-by-default/#context","title":"Context","text":"

Like Flux, the tf-controller API has a handful of places where it accepts cross-namespace references.

  • Terraform.spec.sourceRef -- refers to the Flux source object with the Terraform program
  • Terraform.spec.dependsOn[] -- refers to other objects that must be ready before this object can be run
  • .data.resources[] -- in the config struct used by the branch planner

In general in Kubernetes, references to objects in other namespaces are frowned upon, because

  • they break namespace isolation assurances; and,
  • they encourage the proliferation of permissions.

Both of these effects make a system less secure.

However: removing cross-namespace refs entirely would break some installations in a way that would be difficult to fix, because Flux deployments often rely on defining sources away from objects that use them.

"},{"location":"adr/0002-deny-cross-ns-by-default/#decision","title":"Decision","text":"

Deny cross-namespace references by default, but allow them to be enabled with a flag.

So that the default value means the right thing, the flag name must be enable-cross-namespace-refs, and the default false. To avoid confusion when people try to use the Flux version of this flag --disable-cross-namespace-refs, it should be supported too, but only respected if supplied.

"},{"location":"adr/0002-deny-cross-ns-by-default/#consequences","title":"Consequences","text":"

The changed default will break deployments that rely on cross-namespace refs, but they are easily fixed with the flag.

New deployments will be more secure, by default.

"},{"location":"adr/0003-workspace-blob-caching/","title":"3. Workspace BLOB Caching","text":"
  • Status: [ proposed | rejected | accepted | deprecated ]
  • Date: 2023-09-20
  • Authors: @chanwit
  • Deciders: TBD
"},{"location":"adr/0003-workspace-blob-caching/#context","title":"Context","text":"

The TF-Controller currently faces challenges related to the deletion of Terraform resources. These problems span across three categories:

  1. Single object deletion,
  2. Resources with dependencies deletion, and
  3. Namespace deletion.

These problems must be fixed in the above order as (2) and (3) require single object deletion to be resolved first.

Deleting a single TF object can sometimes be obstructed because it's tied to other resources like Source objects, Secrets, and ConfigMaps. If we try to remove it without deleting these resources, the TF object gets stuck in an inconsistent state, making it harder for users to manage their infrastructure smoothly. Therefore, the TF-Controller is being enhanced to address this problem more efficiently, using the contents of generated Workspace BLOBs. Each BLOB contains all necessary information from the associated Source, Secrets, and ConfigMaps to ensure that TF-Controller finalization procedures can delete objects correctly.

Currently, the TF-Controller downloads a Source BLOB and pushes it to a tf-runner. The tf-runner processes this BLOB to create a Workspace file system. It generates a backend configuration file, variable files, and other necessary files for the Workspace file system, using data from associated Secrets and ConfigMaps. This newly created Workspace file system is then compressed, sent back to the TF-Controller, and stored as a Workspace BLOB in the controller's storage. A caching mechanism for these BLOBs is essential to fixing the single TF object deletion process.

"},{"location":"adr/0003-workspace-blob-caching/#decision","title":"Decision","text":"
  1. BLOB Creation and Storage
  2. A gRPC function named CreateWorkspaceBlob will be invoked by the TF-Controller to tell tf-runner to compress the Workspace file system into a tar.gz BLOB, which is then retrieved back to the controller.
  3. The caching mechanism will be executed right before the Terraform Initialization step, ensuring that the latest and most relevant data is used.
  4. Each Workspace Blob will be cached on the TF-Controller's local disk, using the UUID of the Terraform object as the filename,${uuid}.tar.gz.
  5. To reduce the risk of unauthorized access to the cache entries, and cache collisions, the cache file will be deleted after the finalization process is complete.
  6. Persistence
  7. The persistence mechanism used by the Source Controller will be adopted for the TF-Controller's persistence volume.
  8. BLOB Encryption
  9. The encryption and decryption of the BLOBs will be tasked to the runner, with the controller solely responsible for storing encrypted BLOBs.
  10. Each namespace will require a service account, preferably named \"tf-runner\".
  11. The token of this service account, which is natively supported by Kubernetes, will serve as the most appropriate encryption key because it's stored in a Secret, access to which can be controlled by RBAC. Storing it in a Secret also allows the key to be rotated.
  12. Security Measures (Based on STRIDE Analysis)
  13. Spoofing: Implement Kubernetes RBAC for access restrictions and use mutual authentication for gRPC communications.
  14. Tampering: Use checksums for integrity verification and 0600 permissions to write-protect local disk storage.
  15. Repudiation: Ensure strong logging and auditing mechanisms for tracking activities.
  16. Information Disclosure: Utilize robust encryption algorithms, rotate encryption keys periodically, and secure service account tokens.
  17. Denial of Service: Monitor storage space and automate cleanup processes.
  18. Elevation of Privilege: Minimize permissions associated with service account tokens.
  19. First MVP & Future Planning
  20. For the initial MVP, the default pod local volume will be used.
  21. Since a controller restart will erase the BLOB cache, consideration for using persistent volumes should be made for subsequent versions.
"},{"location":"adr/0003-workspace-blob-caching/#consequence","title":"Consequence","text":"
  1. With the implementation of this architecture:
  2. Single object deletions will succeed in circumstances in which they previously got stuck.
  3. Security measures will ensure the safety of the new Workspace BLOB storage mechanics, minimizing potential risks.
  4. Using the default pod local volume might limit storage capabilities and risk data loss upon controller restart. This warrants the need for considering persistent volumes in future versions.
  5. Encryption and security measures will demand regular maintenance and monitoring, especially concerning key rotations and integrity checks.
"},{"location":"branch-planner/","title":"Branch Planner Overview","text":"

The GitOps methodology streamlines infrastructure provisioning and management, using Git as the source of truth. The Branch Planner, a component of TF-Controller, aims to take this a step further by allowing developers and operations teams to plan Terraform configurations on a branch that's separate from the main branch. This makes it easier to review and understand the potential impact of your changes before you run terraform apply.

The Branch Planner's most important feature is its seamless integration with the PR (Pull Request) user interface. When enabled through Helm values, it watches repositories that contain Terraform resources at regular intervals\u2014checking their referenced Source, and polling for Pull Requests using GitHub's API and the provided token. When changes are proposed on a new branch, Branch Planner runs a plan in the cluster and displays the results directly as comments on your PR. Once you're satisfied with the results, you can merge your branch into the main branch to trigger the TF-Controller to reconcile the updated code.

"},{"location":"branch-planner/#replan-commands","title":"Replan commands","text":"

The Branch Planner also allows users to manually trigger the replan process. By simply commenting !replan under the PR, the Branch Planner will be instructed to generate a new plan and post it under the PR as a new comment.

Now that you know what Branch Planner can do for you, follow the guide to get started.

"},{"location":"branch-planner/branch-planner-getting-started/","title":"Getting Started With Branch Planner","text":""},{"location":"branch-planner/branch-planner-getting-started/#prerequisites","title":"Prerequisites","text":"
  1. Flux is installed on the cluster.
  2. A GitHub API token. For public repositories, it's sufficient to enable Public Repositories without any additional permissions. For private repositories, you need the following permissions:
  3. Pull requests with Read-Write access. This is required to check Pull Request changes, list comments, and create or update comments.
  4. Metadata with Read-only access. This is automatically marked as \"mandatory\" because of the permissions listed above.
  5. General knowledge about Tofu-Controller (see docs).
"},{"location":"branch-planner/branch-planner-getting-started/#quick-start","title":"Quick Start","text":"

This section describes how to install Branch Planner using a HelmRelease object in the flux-system namespace with minimum configuration on a KinD cluster.

  1. Create a KinD cluster. 

    kind create cluster\n

  2. Install Flux. Make sure you have the latest version of Flux (v2 GA).

flux install\n
  1. Create a secret that contains a GitHub API token. If you do not use the gh CLI, copy and paste the token from GitHub's website.
export GITHUB_TOKEN=$(gh auth token)\n\nkubectl create secret generic branch-planner-token \\\n    --namespace=flux-system \\\n    --from-literal=\"token=${GITHUB_TOKEN}\"\n
  1. Install Branch Planner from a HelmRelease provided by the TF-Controller repository. Use TF-Controller v0.16.0-rc.2 or later.
kubectl apply -f https://raw.githubusercontent.com/weaveworks/tf-controller/fa4b3b85d316340d897fda4fed757265ba2cd30e/docs/branch_planner/release.yaml\n
  1. Create a Terraform object with a Source pointing to a repository. Your repository must contain a Terraform file\u2014for example, main.tf. Check out this demo for an example.
export GITHUB_USER=<your user>\nexport GITHUB_REPO=<your repo>\n\ncat <<EOF | kubectl apply -f -\n---\napiVersion: source.toolkit.fluxcd.io/v1\nkind: GitRepository\nmetadata:\n  name: branch-planner-demo\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: https://github.com/${GITHUB_USER}/${GITHUB_REPO}\n  ref:\n    branch: main\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: branch-planner-demo\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  path: ./\n  interval: 1m\n  sourceRef:\n    kind: GitRepository\n    name: branch-planner-demo\n    namespace: flux-system\nEOF\n
  1. Now you can create a pull request on your GitHub repo. The Branch Planner will create a new Terraform object with the plan-only mode enabled and will generate a new plan for you. It will post the plan as a new comment in the pull request.
"},{"location":"branch-planner/branch-planner-getting-started/#configure-branch-planner","title":"Configure Branch Planner","text":"

Branch Planner uses a ConfigMap as configuration. The ConfigMap is optional but useful for fine-tuning Branch Planner.

"},{"location":"branch-planner/branch-planner-getting-started/#configuration","title":"Configuration","text":"

By default, Branch Planner will look for the branch-planner ConfigMap in the same namespace as where the TF-Controller is installed. That ConfigMap allows users to specify which Terraform resources in a cluster the Brach Planner should monitor.

The ConfigMap has two fields:

  1. secretName, which contains the API token to access GitHub.
  2. resources, which defines a list of resources to watch.
---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  namespace: flux-system\n  name: branch-planner\ndata:\n  secretName: branch-planner-token\n  resources: |-\n    - namespace: terraform\n    - namespace: flux-system\n
"},{"location":"branch-planner/branch-planner-getting-started/#secret","title":"Secret","text":"

Branch Planner uses the referenced Secret with a token field that acquires the API token to fetch pull request information.

kubectl create secret generic branch-planner-token \\\n    --namespace=flux-system \\\n    --from-literal=\"token=${GITHUB_TOKEN}\"\n
"},{"location":"branch-planner/branch-planner-getting-started/#resources","title":"Resources","text":"

If the resources list is empty, nothing will be watched. The resource definition can be exact or namespace-wide.

With the following configuration file, the Branch Planner will watch all Terraform objects in the terraform namespace, and the exact-terraform-object Terraform object in default namespace.

data:\n  resources:\n    - namespace: default\n      name: exact-terraform-object\n    - namespace: terraform\n
"},{"location":"branch-planner/branch-planner-getting-started/#default-configuration","title":"Default Configuration","text":"

If no ConfigMap is found, the Branch Planner will not watch any namespaces for Terraform resources and look for a GitHub token in a secret named branch-planner-token in the flux-system namespace. Supplying a secret with a token is a necessary task, otherwise Branch Planner will not be able to interact with the GitHub API.

"},{"location":"branch-planner/branch-planner-getting-started/#enable-branch-planner","title":"Enable Branch Planner","text":"

To enable branch planner, set the branchPlanner.enabled to true in the Helm values files.

---\nbranchPlanner:\n  enabled: true\n
"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/","title":"Branch Planner and Terraform Cloud Integration","text":""},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#branch-planner-and-terraform-cloud-integration-getting-started","title":"Branch Planner and Terraform Cloud Integration: Getting Started","text":"

With Branch Planner, you can provision the main branch directly on Terraform Cloud. TF-Controller communicates with Terraform Cloud to run the necessary plans and apply your approved code. The state is securely stored on Terraform Cloud.

Note: For now, Branch Planner only supports GitHub as the Git provider. We plan to add other Git providers in time.

"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-by-step-guide","title":"Step-by-step Guide","text":""},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-1-create-a-terraform-cloud-token","title":"Step 1: Create a Terraform Cloud Token","text":"

Use the terraform login command to obtain a Terraform Cloud token. Then use the token to create a Kubernetes Secret.

kubectl create secret generic \\\n  tfc-cli-config \\\n  --namespace=flux-system \\\n  --from-file=terraform.tfrc=/dev/stdin << EOF\ncredentials \"app.terraform.io\" {\n  token = \"xxxxxxxxxxxxxx.atlasv1.zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz\"           \n}\nEOF\n
"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-2-create-a-terraform-cr","title":"Step 2: Create a Terraform CR","text":"

Create a Terraform CR to automatically plan and apply Terraform configurations on Terraform Cloud. In this example, the Branch Planner reads the Terraform configurations from a Git repository to plan, apply, and store the state in a Terraform Cloud workspace.

The token from Step 1 is specified as the value of spec.cliConfigSecretRef and is used to authenticate with Terraform Cloud.

---\napiVersion: source.toolkit.fluxcd.io/v1\nkind: GitRepository\nmetadata:\n  name: branch-planner-demo\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: https://github.com/tf-controller/branch-planner-demo\n  ref:\n    branch: main\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: branch-planner-tfc\n  namespace: flux-system\nspec:\n  interval: 2m\n  approvePlan: auto\n  cloud:\n    organization: weaveworks\n    workspaces:\n      name: branch-planner-tfc\n  cliConfigSecretRef:\n    name: tfc-cli-config\n    namespace: flux-system\n  vars:\n  - name: subject\n    value: \"world\"\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: branch-planner-demo\n    namespace: flux-system\n
"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-3-edit-file-create-a-branch-and-open-a-pull-request","title":"Step 3: Edit File, Create a Branch, and Open a Pull Request","text":"

  1. Navigate to Your Repository: Open a web browser and visit your GitHub repository. For our example, navigate here.

  2. Locate the File to Edit: Browse through the repository's file structure and click on the Terraform configuration file you wish to edit.

  3. Edit the File: Click on the pencil icon (edit) located on the top right of the file content. Make your desired changes to the Terraform configurations. For instance, you might change the \"Hello World\" content in the main.tf file.

Once you've made your edits, scroll down to prepare to commit the changes.

  1. Commit the Changes to a New Branch: Instead of committing directly to the main branch, choose the option to \"Create a new branch\" for this commit and start a pull request. Name the branch something descriptive\u2014for example, change-hello-world-message.

Click on the \"Propose Changes\" button.

  1. Open a Pull Request (PR): After proposing your changes, you'll be led to the \"Open a pull request\" page. Fill in the details of your PR, explaining the changes you made, their purpose, and any other pertinent information.

Click on the \"Create Pull Request\" button.

  1. Review Terraform Plan in PR Comments: Once the PR is created, the Branch Planner will trigger a Terraform plan. After the plan is completed, the results will be posted as a comment on the PR. This enables you and your team to review the expected changes before they're applied.
"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-4-review-approve-and-merge-the-pull-request","title":"Step 4: Review, Approve and Merge the Pull Request","text":"

  1. Review the Changes:

    • Navigate to the Pull Requests tab in your GitHub repository.
    • Click on the title of your pull request to see the details.
    • Examine the Files changed section to see the exact modifications made to the Terraform configurations.
    • Check the comments for the Terraform plan output generated by Branch Planner. Ensure the plan matches your expectations.

  2. Iterate on Changes if Necessary:

    • If you spot any discrepancies or wish to make further adjustments, click on the file in the Files changed section.
    • After making the desired edits, commit the changes to the same branch. This will automatically prompt TF-Controller and Branch Planner to generate a new plan.
    • If, for any reason, the automatic replan doesn't occur or you believe there might be an inconsistency, you can manually trigger a new plan by commenting !replan on the PR. Branch Planner will then process the request and display the new plan results.

  3. Approve the Changes:

    • If you're content with the changes and the associated Terraform plan, move to the Review changes button on the PR page.
    • Select the Approve option from the dropdown and optionally add any final comments.
    • Click Submit review to finalize your approval.

  4. Merge the Pull Request:

    • With the changes approved, click on the Merge pull request button.
    • Choose your desired merge strategy from the options provided, such as \"Squash and merge\" or \"Rebase and merge\".
    • Click Confirm merge.
    • Following the merge, TF-Controller will take over. It will send the updated Terraform configuration to Terraform Cloud, where the changes will be planned and then applied. The resulting infrastructure state will be securely stored within your Terraform Cloud workspace.
"},{"location":"use-tf-controller/","title":"Use TF-controller","text":"
  • Use TF-controller to provision resources and auto approve
  • Use TF-controller to plan and manually apply Terraform resources
  • Use TF-controller to provision resources and obtain outputs
  • Use TF-controller to detect drifts only without plan or apply
  • Use TF-controller with drift detection disabled
  • Use TF-controller with AWS EKS IRSA
  • Use TF-controller to set variables for Terraform resources
  • Use TF-controller with a custom backend
  • Use TF-controller with an OCI Artifact as Source
  • Use TF-controller to provision Terraform resources that are required health checks
  • Use TF-controller to provision resources and destroy them when the Terraform object gets deleted
  • Use TF-controller to force unlock Terraform states
  • Use TF-controller with Terraform Runners enabled via Env Variables
  • Use TF-controller to provision resources with customized Runner Pods
  • Use TF-controller with Terraform Enterprise
  • Use TF-controller with Terraform Private Registries
  • Use TF-controller with primitive modules
  • Use TF-controller with GitOps dependency management
  • Use TF-controller with the ready-to-use AWS package
  • User TF-controller with plan-only mode
  • Use TF-controller with external webhooks
  • Use TF-controller with Terraform Runners exposed via hostname/subdomain
  • How to backup and restore a Terraform state
  • How to build and use a custom runner image
  • How to integrate with Flux Receivers and Alerts?
  • How does the interval and retryInterval work?
  • How does the resource deletion work?
  • How to troubleshoot with Break the Glass mode
  • How to enable cross-namespace references
  • How to run TF-controller in Azure Kubernetes Service
  • How to upgrade TF-controller to a newer version
"},{"location":"use-tf-controller/backup-and-restore-a-Terraform-state/","title":"Backup and restore a Terraform state","text":""},{"location":"use-tf-controller/backup-and-restore-a-Terraform-state/#backup-the-tfstate","title":"Backup the tfstate","text":"

Assume that we have the my-stack Terraform object with its .spec.workspace set to \"default\".

kubectl get terraform\n\nNAME       READY     STATUS         AGE\nmy-stack   Unknown   Initializing   28s\n

We can backup its tfstate out of the cluster, like this:

WORKSPACE=default\nNAME=my-stack\n\nkubectl get secret tfstate-${WORKSPACE}-${NAME} \\\n  -ojsonpath='{.data.tfstate}' \\\n  | base64 -d | gzip -d > terraform.tfstate\n
"},{"location":"use-tf-controller/backup-and-restore-a-Terraform-state/#restore-the-tfstate","title":"Restore the tfstate","text":"

To restore the tfstate file or import an existing tfstate file to the cluster, we can use the following operation:

gzip terraform.tfstate\n\nWORKSPACE=default\nNAME=my-stack\n\nkubectl create secret \\\n  generic tfstate-${WORKSPACE}-${NAME} \\\n  --from-file=tfstate=terraform.tfstate.gz \\\n  --dry-run=client -o=yaml \\\n  | yq e '.metadata.annotations[\"encoding\"]=\"gzip\"' - \\\n  > tfstate-${WORKSPACE}-${NAME}.yaml\n\nkubectl apply -f tfstate-${WORKSPACE}-${NAME}.yaml\n
"},{"location":"use-tf-controller/build-and-use-a-custom-runner-image/","title":"Build and Use a Custom Runner Image","text":"

To build a custom runner image, you need a Dockerfile that extends the base image and that adds Terraform, plus any additional required tooling. The repository that contains the base images is here. All base image tags follow the following format: ${TF_CONTROLLER_VERSION}-base.

"},{"location":"use-tf-controller/build-and-use-a-custom-runner-image/#prerequisites","title":"Prerequisites","text":"

You need Docker and Git to build the image.

"},{"location":"use-tf-controller/build-and-use-a-custom-runner-image/#build-the-image","title":"Build the Image","text":"
  1. Create a Dockerfile that extends the base image and that adds Terraform, plus any additional required tooling. For example:
ARG BASE_IMAGE\nFROM $BASE_IMAGE\n\nARG TARGETARCH\nARG TF_VERSION=1.5.7\n\n# Switch to root to have permissions for operations\nUSER root\n\nADD https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_${TARGETARCH}.zip /terraform_${TF_VERSION}_linux_${TARGETARCH}.zip\nRUN unzip -q /terraform_${TF_VERSION}_linux_${TARGETARCH}.zip -d /usr/local/bin/ && \\\n    rm /terraform_${TF_VERSION}_linux_${TARGETARCH}.zip && \\\n    chmod +x /usr/local/bin/terraform\n\n# Switch back to the non-root user after operations\nUSER 65532:65532\n

Find the original Dockerfile for the runner here.

  1. Build the image from the directory containing the Dockerfile you created above:
export TF_CONTROLLER_VERSION=v0.16.0-rc.3\nexport TF_VERSION=1.5.7\nexport BASE_IMAGE=ghcr.io/flux-iac/tf-runner:${TF_CONTROLLER_VERSION}-base\nexport TARGETARCH=amd64\nexport REMOTE_REPO=ghcr.io/my-org/custom-runnner\ndocker build \\\n    --build-arg BASE_IMAGE=${BASE_IMAGE} \\\n    --build-arg TARGETARCH=${TARGETARCH} \\\n    --tag my-custom-runner:${TF_CONTROLLER_VERSION} .\ndocker tag my-custom-runner:${TF_CONTROLLER_VERSION} $REMOTE_REPO:${TF_CONTROLLER_VERSION}\ndocker push $REMOTE_REPO:${TF_CONTROLLER_VERSION}\n

Replace the relevant values above with the corresponding values in your organisation/implementation.

  1. Update the values.runner.image values in the TF-Controller Helm chart values to point to the new image:
values:\n  runner:\n    image:\n      repository: ghcr.io/my-org/custom-runnner\n      tag: v0.16.0-rc.3\n
  1. Commit and push the changes to Git. Confirm that the HelmRelease has been updated:
kubectl get deployments.apps -n flux-system tf-controller -o jsonpath='{.spec.template.spec.containers[*]}' | jq '.env[] | select(.name == \"RUNNER_POD_IMAGE\")'\n{\n  \"name\": \"RUNNER_POD_IMAGE\",\n  \"value\": \"ghcr.io/my-org/custom-runner:v0.16.0-rc3\"\n}\n
"},{"location":"use-tf-controller/build-and-use-a-custom-runner-image/#references","title":"References","text":"

A set of GitHub actions in the TF-Controller community repo facilitates a process similar to the above, but uses GitHub Actions to build and push the image.

"},{"location":"use-tf-controller/detect-drifts-only-without-plan-or-apply/","title":"Detect drifts only without plan or apply","text":""},{"location":"use-tf-controller/detect-drifts-only-without-plan-or-apply/#use-tf-controller-to-detect-drifts-only-without-plan-or-apply","title":"Use TF-Controller to detect drifts only without plan or apply","text":"

We can set .spec.approvePlan to disable to tell the controller to detect drifts of your Terraform resources only. Doing so will skip the plan and apply stages.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: hello-world\n  namespace: flux-system\nspec:\n  approvePlan: disable\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/detect-drifts-only-without-plan-or-apply/#troubleshooting","title":"Troubleshooting","text":""},{"location":"use-tf-controller/detect-drifts-only-without-plan-or-apply/#when-terraform-resource-detects-drift-but-no-plan-is-generated-for-approval","title":"When Terraform resource detects drift, but no plan is generated for approval","text":"

In this situation, you may not have spec.approvePlan set to disable. Try setting spec.approvePlan: auto and using tfctl replan to trigger a replan. After the drift disappears, you can set the spec.approvePlan: \"\" to get into the manual mode again.

"},{"location":"use-tf-controller/flux-receiver-and-alert/","title":"Integrate with Flux Receivers and Alerts","text":"

You can customize your Flux installation to use Flux API resources like Receivers and Alerts with third-party custom resource definitions such as the Terraform API CRD.

You will need to add a patch to the kustomization.yaml in your Flux cluster installation's bootstrap manifests. Find it under the flux-system directory.

"},{"location":"use-tf-controller/flux-receiver-and-alert/#enable-notifications-for-third-party-controllers","title":"Enable Notifications for Third-Party Controllers","text":"

Enable notifications for 3rd party Flux controllers such as tf-controller:

apiVersion: kustomize.config.k8s.io/v1beta1\nkind: Kustomization\nresources:\n  - gotk-components.yaml\n  - gotk-sync.yaml\npatches:\n  - patch: |\n      - op: add\n        path: /spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/eventSources/items/properties/kind/enum/-\n        value: Terraform\n      - op: add\n        path: /spec/versions/1/schema/openAPIV3Schema/properties/spec/properties/eventSources/items/properties/kind/enum/-\n        value: Terraform\n    target:\n      kind: CustomResourceDefinition\n      name:  alerts.notification.toolkit.fluxcd.io\n  - patch: |\n      - op: add\n        path: /spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/resources/items/properties/kind/enum/-\n        value: Terraform\n      - op: add\n        path: /spec/versions/1/schema/openAPIV3Schema/properties/spec/properties/resources/items/properties/kind/enum/-\n        value: Terraform\n    target:\n      kind: CustomResourceDefinition\n      name:  receivers.notification.toolkit.fluxcd.io\n  - patch: |\n      - op: add\n        path: /rules/-\n        value:\n          apiGroups: [ 'infra.contrib.fluxcd.io' ]\n          resources: [ '*' ]\n          verbs: [ '*' ]\n    target:\n      kind: ClusterRole\n      name:  crd-controller-flux-system\n
"},{"location":"use-tf-controller/force-unlock-terraform-states/","title":"Use TF-Controller to force unlock Terraform states","text":"

In some situations, you may need to perform the Terraform force-unlock operation on the tfstate inside the cluster.

There are three possible values of .spec.tfstate.forceUnlock, which are yes, no, and auto. The default value is no, which means that you disable this behaviour.

The auto force-unlock mode will automatically use the lock identifier produced by the associated state file instead of the specified lock identifier.

The recommended way is to do manual force unlock. To manually force-unlock, you need to:

  1. set forceUnlock to yes, and
  2. specify a lock identifier to unlock a specific locked state,

as the following example:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  tfstate:\n    forceUnlock: \"yes\"\n    lockIdentifier: f2ab685b-f84d-ac0b-a125-378a22877e8d\n
"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/","title":"Terraform Enterprise and Terraform Cloud Integration","text":"

Terraform is a secure and robust platform designed to store the Terraform states for your production systems. When working with Infrastructure as Code, managing and ensuring the state is both secure and consistent is critical.

TF-Controller supports both Terraform Cloud and Terraform Enterprise. The spec.cloud in the Terraform CRD enables users to integrate their Kubernetes configurations with Terraform workflows.

To get started, simply place your Terraform Cloud token in a Kubernetes Secret and specify it in the spec.cliConfigSecretRef field of the Terraform CR. The spec.cloud field specifies the organization and workspace name.

"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-enterprise","title":"Terraform Enterprise","text":"

Here are the steps to set up TF-Controller for your TFE instance.

"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-login","title":"Terraform Login","text":"

First, you need to obtain an API token from your TFE. You can use terraform login command to do so.

terraform login tfe.dev.example.com\n

Then you can find your API token inside $HOME/.terraform.d/credentials.tfrc.json. Content of the file will look like this:

{\n  \"credentials\": {\n    \"tfe.dev.example.com\": {\n      \"token\": \"mXXXXXXXXX.atlasv1.ixXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n    }\n  }\n}\n
"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#prepare-an-tfrc-file","title":"Prepare an TFRC file","text":"

TF-Controller accepts an TFRC file in the HCL format. So you have to prepare terraform.tfrc file using contents from above. 

credentials \"tfe.dev.example.com\" {\n  token = \"mXXXXXXXXX.atlasv1.ixXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n}\n

"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#create-a-secret","title":"Create a Secret","text":"

We will now create a Kubernetes Secret from yourterraform.tfrc file, name it tfe-cli-config and put it inside the flux-system namespace.

kubectl create secret generic \\\n  tfe-cli-config \\\n  --namespace=flux-system \\\n  --from-file=terraform.tfrc=./terraform.tfrc\n
"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-object","title":"Terraform Object","text":"

In your Terraform object, you'll have to 1. disable the backend by setting spec.backendConfig.disable: true, and 2. point spec.cliConfigSecretRef: to the Secret created in the previous step, like this:

---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: tfe-demo\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 2m\n  path: ./terraform/tfe-demo\n  backendConfig:\n    disable: true\n  cliConfigSecretRef:\n    name: tfe-cli-config\n    namespace: flux-system\n  vars:\n  - name: subject\n    value: World\n  sourceRef:\n    kind: GitRepository\n    name: flux-system\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: tfe-helloworld-output\n    outputs:\n    - greeting\n
"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-module","title":"Terraform Module","text":"

Don't forget that you need to tell your Terraform model to use your enterprise instance as well. Here's an example, 

terraform {\n  required_version = \">= 1.1.0\"\n  cloud {\n    hostname = \"tfe.dev.example.com\"\n    organization = \"weaveworks\"\n\n    workspaces {\n      name = \"dev\"\n    }\n  }\n}\n\nvariable \"subject\" {\n   type = string\n   default = \"World\"\n   description = \"Subject to hello\"\n}\n\noutput \"greeting\" {\n  value = \"Hello ${var.subject} from Terraform Enterprise\"\n}\n

"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-cloud","title":"Terraform Cloud","text":"

TF-Controller can send your Terraform resources to be planned and applied via Terraform Cloud. States are automatically stored in your Terraform Cloud's workspace. To use TF-Controller with Terraform Cloud, replace your hostname to app.terraform.io. Also, set spec.approvalPlan to auto.

Here's how the configuration looks:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: branch-planner-tfc\n  namespace: flux-system\nspec:\n  interval: 2m\n  approvePlan: auto\n  cloud:\n    organization: weaveworks\n    workspaces:\n      name: branch-planner-tfc\n  cliConfigSecretRef:\n    name: tfc-cli-config\n    namespace: flux-system\n
"},{"location":"use-tf-controller/integration-with-terraform-private-registries/","title":"Terraform Private Registries Integration","text":"

Using Terraform private registries with the tofu-controller is exactly as you would use them directly via Terraform. For example, you would like to use the tofu-controller to deploy code that contains the following module: 

module \"vpc\" {\n  source  = \"my.private.server/terraform-modules/path/to/module\"\n  version = \"1.2.3\"\n\n  ...\n  ...\n}\n
without configuring the terraform login process, deploying the module with the controller will result in the error: 
Failed to retrieve available versions for module \"vpc\" (main.tf:1) from\nmy.private.server: error looking up module versions: 401 Unauthorized.\n

"},{"location":"use-tf-controller/integration-with-terraform-private-registries/#terraform-login","title":"Terraform Login","text":"

As a human you would normally execute terraform login my.private.server to obtain a token from the registry, with the tofu-controller use the native terraform credentials configs instead.

Obtain a token from your private registry, then follow one of the below options:

"},{"location":"use-tf-controller/integration-with-terraform-private-registries/#using-credentials-file","title":"Using credentials file","text":"

content of credentials.tfrc should look like: 

{\n  \"credentials\": {\n    \"my.private.server\": {\n      \"token\": \"TOP_SECRET_TOKEN\"\n    }\n  }\n}\n

K8S secret example: 

apiVersion: \"v1\"\nkind: \"Secret\"\nmetadata:\n  name: tf-private-config\ntype: \"Opaque\"\nstringData:\n  credentials.tfrc: |-\n    {\n      \"credentials\": {\n        \"my.private.server\": {\n          \"token\": \"TOP_SECRET_TOKEN\"\n        }\n      }\n    }\n
Then deploy the Terraform object, while referencing the above tf-private-config secret 
apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: tf-private-demo\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 2m\n  path: ./terraform/tf-private-demo\n  cliConfigSecretRef:\n    name: tf-private-config\n    namespace: flux-system\n  sourceRef:\n    kind: GitRepository\n    name: flux-system\n    namespace: flux-system\n

"},{"location":"use-tf-controller/integration-with-terraform-private-registries/#using-environment-variables","title":"Using environment variables","text":"

Another option is to use environment variable credentials, Terraform object should look like: 

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: tf-private-demo\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 2m\n  path: ./terraform/tf-private-demo\n  sourceRef:\n    kind: GitRepository\n    name: flux-system\n    namespace: flux-system\n  # api referance https://flux-iac.github.io/tofu-controller/References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RunnerPodTemplate\n  runnerPodTemplate:\n    spec:\n      env:\n        - name: \"TF_TOKEN_my_private_server\"\n          value: \"TOP_SECRET_TOKEN\"\n      # or use get ENV from existing secret\n      envFrom:\n        - secretRef:\n            name: tf-private-token\n

"},{"location":"use-tf-controller/interval-and-retryInterval/","title":"Troubleshooting with Interval and retryInterval","text":""},{"location":"use-tf-controller/interval-and-retryInterval/#overview","title":"Overview","text":"

This document describes the requeue behavior of the Reconcile method in the TerraformReconciler struct in the code base. Understanding these behaviors can be crucial for troubleshooting, as well as for future development and refinement of the system.

"},{"location":"use-tf-controller/interval-and-retryInterval/#requeue-behaviors","title":"Requeue Behaviors","text":"

The Reconcile method has several requeue behaviors based on different conditions and errors. We will group them into four categories based on their requeue behavior:

"},{"location":"use-tf-controller/interval-and-retryInterval/#1-immediate-requeue-not-using-specified-interval-retryinterval","title":"1. Immediate Requeue (Not using specified interval / retryInterval)","text":"

In these scenarios, the Reconcile method returns an error which leads to an immediate requeue orchestrated by the Controller Runtime. The interval is based on the controller's configuration and not specified in the method itself:

  • When there's an error retrieving the Terraform object from the Kubernetes API.
  • After adding the finalizer, if there's an error in patching the Terraform object.
  • If there's a non-access-denied error in retrieving the source object.
  • When the ready condition is unknown or the status of the ready condition isn't unknown, and there's an error in patching the Terraform object.
  • In multiple situations where there's an error in patching the Terraform object's status.
  • If there's an error in creating or looking up the runner.
  • If there's an error while attempting to finalize the Terraform object.
"},{"location":"use-tf-controller/interval-and-retryInterval/#2-requeue-after-a-specific-interval-specretryinterval","title":"2. Requeue After a Specific Interval (spec.retryInterval)","text":"

In these scenarios, the method specifically asks for a requeue after a certain interval specified by spec.retryInterval (default to 15s).

  • The Terraform object is being deleted but there are still dependent resources that haven't been deleted.
  • The source object specified by spec.sourceRef is not found.
  • The source object doesn't have an associated artifact.
  • The dependencies do not meet the ready condition.
  • There's an error during the main reconciliation process.
  • Drift is detected during the reconciliation process.
"},{"location":"use-tf-controller/interval-and-retryInterval/#3-requeue-after-a-specific-interval-specinterval","title":"3. Requeue After a Specific Interval (spec.interval)","text":"

In this scenario, the method specifically asks for a requeue after a successful reconciliation:

The interval for the requeue is spec.interval.

"},{"location":"use-tf-controller/interval-and-retryInterval/#4-no-requeue-wait-for-manual-intervention","title":"4. No Requeue, wait for manual intervention","text":"

In these scenarios, the method returns without asking for a requeue, and the Controller Runtime will stop the reconciliation process until there is a manual intervention:

  • Access is denied when retrieving the source object.
  • The status of the plan is pending, and it's not set to force or auto-apply.
"},{"location":"use-tf-controller/plan-and-manually-apply-terraform-resources/","title":"Use TF-controller to plan and manually apply Terraform resources","text":"

Assume that you have a GitRepository object named helloworld pointing to a Git repository, and you want to plan and apply the Terraform resources under ./ of that Git repo. Let's walk through the steps of using TF-Controller to plan and manually apply Terraform resources.

  • Create a Terraform object and set the necessary fields in the spec:
  • approvePlan, which sets the mode. For plan and manual approval mode, either keep this field blank or omit it entirely.
  • interval, which determines how often TF-Controller will run the Terraform configuration
  • path, which specifies the location of the configuration files, in this case ./
  • sourceRef, which points to the helloworld GitRepository object
  • Once this object is created, use kubectl to obtain the approvePlan value and set it in the Terraform object.
  • After making our changes and pushing them to the Git repository, TF-Controller will apply the plan and create the real resources.

Here is an example:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: \"\" # or you can omit this field\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/plan-and-manually-apply-terraform-resources/#view-the-approval-message","title":"View the approval message","text":"

After a reconciliation loop, TF-Controller will generate a plan. Run this command to receive the .spec.approvePlan value from TF-Controller, which you'll need to approve the plan:

kubectl -n flux-system get tf/helloworld\n

This value enables you to edit the Terraform object file and set the spec.approvePlan field to the value obtained from the message.

After making your changes and pushing them to the Git repository, TF-Controller will apply the plan and create the real resources.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: hello-world\n  namespace: flux-system\nspec:\n  approvePlan: plan-main-b8e362c206 # first 8 digits of a commit hash is enough\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/provision-Terraform-resources-that-are-required-health-checks/","title":"Use TF-Controller to provision Terraform resources that are required health checks","text":"

For some Terraform resources, it may be useful to perform health checks on them to verify that they are ready to accept connection before the terraform goes into Ready state:

For example, our Terraform file is provisioned and contains the following outputs.

# main.tf\n\noutput \"rdsAddress\" {\n  value = \"mydb.xyz.us-east-1.rds.amazonaws.com\"\n}\n\noutput \"rdsPort\" {\n  value = \"3306\"\n}\n\noutput \"myappURL\" {\n  value = \"https://example.com/\"\n}\n

We can use standard Go template expressions, like ${{ .rdsAddress }}, to refer to those output values and use them to verify that the resources are up and running.

We support two types of health checks, tcp amd http. The tcp type allows us to verify a TCP connection, while the http type is for verify an HTTP URL. The default timeout of each health check is 20 seconds.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  healthChecks:\n    - name: rds\n      type: tcp\n      address: ${{ .rdsAddress }}:${{ .rdsPort }} \n      timeout: 10s # optional, defaults to 20s\n    - name: myapp\n      type: http\n      url: ${{ .myappURL }}\n      timeout: 5s\n    - name: url_not_from_output\n      type: http\n      url: \"https://example.org\"\n
"},{"location":"use-tf-controller/provision-resources-and-auto-approve/","title":"Use TF-Controller to provision resources and auto approve","text":"

To provision resources with TF-Controller, you need to create a Terraform object and a Flux source object, such as a GitRepository or OCIRepository object.

"},{"location":"use-tf-controller/provision-resources-and-auto-approve/#create-a-terraform-object","title":"Create a Terraform object","text":"

The Terraform object is a Kubernetes custom resource definition (CRD) object. It is the core object of TF-Controller and defines the Terraform module, backend configuration, and GitOps automation mode.

The Terraform module is a Terraform configuration that you can use to provision resources. It can either be placed inside a Git repository, or packaged as an OCI image in an OCI registry.

The backend configuration is the configuration for the Terraform backend to be used to store the Terraform state. It is optional. If not specified, the Kubernetes backend will be used by default.

"},{"location":"use-tf-controller/provision-resources-and-auto-approve/#gitops-automation-mode","title":"GitOps Automation mode","text":"

Use the GitOps automation mode to run the Terraform module. It determines how Terraform runs and manages your infrastructure. It is optional. If not specified, the \"plan-and-manually-apply\" mode is used by default. In the \"plan-and-manually-apply\" mode, TF-Controller will run a Terraform plan and output the proposed changes to a Git repository. A human must then review and manually apply the changes.

In the \"auto-apply\" mode, TF-Controller will automatically apply the changes after a Terraform plan is run. This can be useful for environments where changes can be made automatically, but it is important to ensure that the proper controls, like policies, are in place to prevent unintended changes from being applied.

To specify the GitOps automation mode in a Terraform object, set the spec.approvePlan field to the desired value. For example, to use the \"auto-apply\" mode, set it to spec.approvePlan: auto.

It is important to carefully consider which GitOps automation mode is appropriate for your use case to ensure that your infrastructure is properly managed and controlled.

The following is an example of a Terraform object; we use the \"auto-apply\" mode:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\nspec:\n  path: ./helloworld\n  interval: 10m\n  approvePlan: auto\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n

This code is defining a Terraform object in Kubernetes. The apiVersion field specifies the version of the Kubernetes API being used, and the kind field specifies that it is a Terraform object. The metadata block contains information about the object, including its name.

The spec field contains the specification for the Terraform object. The path field specifies the path to the Terraform configuration files, in this case a directory named \"helloworld\". The interval field specifies the frequency at which TF-Controller should run the Terraform configuration, in this case every 10 minutes. The approvePlan field specifies whether or not to automatically approve the changes proposed by a Terraform plan. In this case, it is set to auto, meaning that changes will be automatically approved.

The sourceRef field specifies the Flux source object to be used. In this case, it is a GitRepository object with the name \"helloworld\". This indicates that the Terraform configuration is stored in a Git repository object with the name helloworld.

"},{"location":"use-tf-controller/provision-resources-and-destroy-them-when-terraform-object-gets-deleted/","title":"Use TF-Controller to provision resources and destroy them when the Terraform object gets deleted","text":"

The resources provisioned by a Terraform object are not destroyed by default, and the tfstate of that Terraform object still remains in the cluster.

It means that you are safe to delete the Terraform object in the cluster and recreate it. If you recreate a new Terraform object with the same name, namespace, and workspace, it will continue to use the tfstate inside the cluster as the starting point to reconcile.

However, you may want to destroy provisioned resources when deleting the Terraform object. To enable destroy resources on object deletion, set .spec.destroyResourcesOnDeletion to true.

~> WARNING: This feature will destroy your resources on the cloud if the Terraform object gets deleted. Use it with caution.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  destroyResourcesOnDeletion: true\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/provision-resources-obtain-outputs/","title":"Use TF-Controller to provision resources and obtain outputs","text":"

Outputs created by Terraform can be written to a secret using .spec.writeOutputsToSecret.

"},{"location":"use-tf-controller/provision-resources-obtain-outputs/#write-all-outputs","title":"Write all outputs","text":"

We can specify a target secret in .spec.writeOutputsToSecret.name, and the controller will write all outputs to the secret by default.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: helloworld-output\n
"},{"location":"use-tf-controller/provision-resources-obtain-outputs/#write-outputs-selectively","title":"Write outputs selectively","text":"

Choose only a subset of outputs by specifying output names you'd like to write in the .spec.writeOutputsToSecret.outputs array.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: helloworld-output\n    outputs:\n    - hello_world\n    - my_sensitive_data\n
"},{"location":"use-tf-controller/provision-resources-obtain-outputs/#rename-outputs","title":"Rename outputs","text":"

Some time we'd like to use rename an output, so that it can be consumed by other Kubernetes controllers. For example, we might retrieve a key from a Secret manager, and it's an AGE key, which must be ending with \".agekey\" in the secret. In this case, we need to rename the output.

TF-controller supports mapping output names using the old_name:new_name format.

In the following example, we renamed age_key output as age.agekey entry for the helloworld-output secret's data, so that other components in the GitOps pipeline could consume it.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: helloworld-output\n    outputs:\n    - age_key:age.agekey\n
"},{"location":"use-tf-controller/provision-resources-obtain-outputs/#customize-metadata-of-the-outputted-secret","title":"Customize metadata of the outputted secret","text":"

Some situations require adding custom labels and annotations to the outputted secret. As an example, operators such as kubernetes-replicator allow replicating secrets from one namespace to another but use annotations to do so.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: helloworld-output\n    labels:\n      my-label: true\n    annotations:\n      my-annotation: \"very long string\"\n
"},{"location":"use-tf-controller/provision-resources-with-customized-runner-pods/","title":"Use TF-Controller to provision resources with customized Runner Pods","text":""},{"location":"use-tf-controller/provision-resources-with-customized-runner-pods/#customize-runner-pod-metadata","title":"Customize Runner Pod metadata","text":"

Sometimes you need to add custom labels and annotations to the runner pod used to reconcile Terraform. For example, for Azure AKS to grant pod active directory permissions using Azure Active Directory (AAD) Pod Identity, a label like aadpodidbinding: myIdentity on the pod is required.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  runnerPodTemplate:\n    metadata:\n      labels:\n        aadpodidbinding: myIdentity\n      annotations:\n        company.com/abc: xyz\n
"},{"location":"use-tf-controller/provision-resources-with-customized-runner-pods/#customize-runner-pod-image","title":"Customize Runner Pod Image","text":"

By default, the Terraform controller uses RUNNER_POD_IMAGE environment variable to identify the Runner Pod's image to use. You can customize the image on the global level by updating the value of the environment variable or, you can specify an image to use per Terraform object for its reconciliation.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  runnerPodTemplate:\n    spec:\n      image: registry.io/tf-runner:xyz\n

You can use runner.Dockerfile as a basis of customizing runner pod image.

"},{"location":"use-tf-controller/provision-resources-with-customized-runner-pods/#customize-runner-pod-specifications","title":"Customize Runner Pod Specifications","text":"

You can also customize various Runner Pod spec fields to control and configure how the Runner Pod runs. For example, you can configure Runner Pod spec affinity and tolerations if you need to run in on a specific set of nodes. Please see RunnerPodSpec for a list of the configurable Runner Pod spec fields.

"},{"location":"use-tf-controller/resource-deletion/","title":"Resource Deletion Dependencies in Terraform Controller","text":"

This document discusses potential difficulties you may encounter when deleting Terraform resources through the Terraform Controller and the necessary components to facilitate a smooth deletion process.

"},{"location":"use-tf-controller/resource-deletion/#source-object","title":"Source Object","text":"

The source object (e.g., GitRepository or OCIRepository) is a critical component of the Terraform resource deletion process. This object houses the Terraform source files (.tf files) that describe the configuration of the infrastructure resources.

During the deletion process, the Terraform Controller uses these source files to conduct a re-planning operation. This operation is instrumental to deleting the Terraform Custom Resource (CR).

However, if the source object is unavailable or has been deleted, the re-planning operation fails. As a result, the Terraform Controller cannot locate the resource state, leading to an infinite deletion attempt cycle, commonly known as a looping process.

If the source object was deleted and the deletion of a Terraform resource is stuck, clearing the finalizers let the system delete the resource skipping the finalizer of the Terraform Controller.

  1. Suspend the kustomization: 
    flux suspend kustomization tf-stk\n
  2. Patch the Terraform resource: 
    kubectl patch terraforms.infra.contrib.fluxcd.io \\\n  -n stk helloworld \\\n  -p '{\"metadata\":{\"finalizers\":null}}' \\\n  --type=merge\n

As it skips the finalizer of the Terraform controller, any cleanup the controller would do will be skipped too.

"},{"location":"use-tf-controller/resource-deletion/#role-bindings","title":"Role Bindings","text":"

Role bindings assign permissions to Terraform runners, allowing them to execute operations within the Kubernetes cluster. These bindings define the actions that the Terraform runners are authorized to carry out.

If role bindings are missing or misconfigured, the Terraform runners may lack the necessary permissions to execute the deletion process, causing the process to fail.

"},{"location":"use-tf-controller/resource-deletion/#secrets-and-configmaps","title":"Secrets and ConfigMaps","text":"

Before initiating the resource deletion process, the Terraform Controller leverages Secrets and ConfigMaps to generate a complete source before planning. Secrets store confidential data like API keys or passwords, while ConfigMaps hold configuration data in a key-value format.

Should any of these components be missing or misconfigured, the Terraform Controller may fail to generate an accurate deletion plan, which could impede the resource deletion process.

"},{"location":"use-tf-controller/resource-deletion/#troubleshooting","title":"Troubleshooting","text":"

To prevent the aforementioned issues, ensure the availability and proper configuration of the source object, role bindings, and Secrets and ConfigMaps during the deletion process.

As of now, we are actively working to address these limitations in the Terraform Controller. We appreciate your patience and welcome any feedback to help enhance the Terraform Controller's performance.

"},{"location":"use-tf-controller/set-variables-for-terraform-resources/","title":"Use TF-Controller to Set Variables for Terraform Resources","text":"

~> BREAKING CHANGE: This is a breaking change of the v1alpha1 API.

Users who are upgrading from TF-Controller <= 0.7.0 must update varsFrom from a single object to become an array of objects:

  varsFrom:\n    kind: ConfigMap\n    name: cluster-config\n

changes to

  varsFrom:\n  - kind: ConfigMap\n    name: cluster-config\n
"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#vars-and-varsfrom","title":"vars and varsFrom","text":"

You can pass variables to Terraform using the vars and varsFrom fields.

Inline variables can be set using vars. The varsFrom field accepts a list of ConfigMaps / Secrets. You may use the varsKeys property of varsFrom to select specific keys from the input or omit this field to select all keys from the input source.

Note that in the case of the same variable key being passed multiple times, the controller will use the lattermost instance of the key passed to varsFrom.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  vars:\n  - name: region\n    value: us-east-1\n  - name: env\n    value: dev\n  - name: instanceType\n    value: t3-small\n  varsFrom:\n  - kind: ConfigMap\n    name: cluster-config\n    varsKeys:\n    - nodeCount\n    - instanceType\n  - kind: Secret\n    name: cluster-creds\n
"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#variable-value-as-hcl","title":"Variable Value as HCL","text":"

The vars field supports HCL string, number, bool, object and list types. For example, the following variable can be populated using the accompanying Terraform spec:

variable \"cluster_spec\" {\n  type = object({\n      region     = string\n      env        = string\n      node_count = number\n      public     = bool\n  })\n}\n
apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  vars:\n  - name: cluster_spec\n    value:\n      region: us-east-1\n      env: dev\n      node_count: 10\n      public: false\n
"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#rename-variables-in-varsfrom","title":"Rename Variables in varsFrom","text":"

To rename a variable, you can use the varsKeys key within the varsFrom field. Here's the basic structure:

spec:\n  varsFrom:\n  - kind: Secret\n    name: <secret_name>\n    varsKeys:\n    - <original_variable_name>:<new_variable_name>\n
original_variable_name corresponds to the initial name of the variable in the referenced secret, while new_variable_name represents the alias you want to use within the Terraform code.

Consider this example below, where we rename nodeCount to node_count and instanceType to instance_type:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  varsFrom:\n  - kind: Secret\n    name: cluster-config\n    varsKeys:\n    - nodeCount:node_count\n    - instanceType:instance_type\n
"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#rename-output-variables","title":"Rename output variables","text":"

See Rename outputs for more details.

"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#rename-input-secrets","title":"Rename Input Secrets","text":"

See Rename input secrets for more details.

"},{"location":"use-tf-controller/terraform-init-steps/","title":"Terraform init steps","text":""},{"location":"use-tf-controller/terraform-init-steps/#aligning-tf-controller-with-terraforms-init-workflow-stage","title":"Aligning TF-Controller with Terraform's Init Workflow Stage","text":"

This page covers required and optional steps you should take in alignment with Terraform's \"init\" workflow stage. We cover \"plan,\" \"apply,\" and \"destroy\" steps in subsequent pages.

"},{"location":"use-tf-controller/terraform-init-steps/#define-source","title":"Define Source","text":"

First, we need to define the Source controller's source (GitRepository, Bucket, OCIRepository). For example:

apiVersion: source.toolkit.fluxcd.io/v1\nkind: GitRepository\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: https://github.com/tf-controller/helloworld\n  ref:\n    branch: main\n

Here's guidance for when your source is an OCI artifact.

"},{"location":"use-tf-controller/terraform-init-steps/#optional-steps","title":"Optional Steps","text":"

At this point you have options to enhance your use of TF-Controller: - Optional: Use TF-Controller with GitOps Dependency Management - This is to avoid the Kustomization controller's variable substitution - Optional: Using TF-Controller with Primitive Modules for an optional way to write Terraform code.

"},{"location":"use-tf-controller/terraform-init-steps/#resource-provisioning","title":"Resource Provisioning","text":"

Related resources, with optional steps noted:

  • Use TF-Controller to Provision Resources and Auto-Approve
  • Optional: Provision Resources and Destroy Them When the Terraform Object Gets Deleted
  • Optional: Provision Terraform Resources That Are Required Health Checks
    • You would check these during the \"apply\" workflow stage
  • Optional, operations-related: Using a Custom Backend
    • TF-Controller uses the Kubernetes backend by default

Be mindful of locking mechanism when pursuing these steps.

"},{"location":"use-tf-controller/terraform-init-steps/#optional-working-with-integrations","title":"Optional: Working with Integrations","text":"
  • Working with Terraform Cloud and Terraform Enterprise; see also: Terraform Cloud and Branch Planner
"},{"location":"use-tf-controller/terraform-init-steps/#context-related-steps","title":"Context-Related Steps","text":"
  • Optional: Use TF-Controller with Terraform Runners enabled via Env Variables
  • Optional: Set variables for Terraform resources
  • Optional: Provision resources and obtain outputs
  • Provision resources with customized Runner Pods
  • Optional: Use with external webhooks
"},{"location":"use-tf-controller/troubleshooting-with-break-the-glass-mode/","title":"Break the glass","text":""},{"location":"use-tf-controller/troubleshooting-with-break-the-glass-mode/#what-is-break-the-glass","title":"What is break the glass?","text":"

\"Break the glass\" refers to a troubleshooting mode specifically designed to provide a manual solution when TF-Controller is not performing as expected. This feature is available in the Terraform controller v0.15.0 and above.

~> WARNING: Please note that you cannot use this feature to fix the Terraform resources with v1alpha1 version of the Terraform CRD. It works only with v1alpha2 version of the Terraform CRD.

~> WARNING: Please also make sure that you have enough privileges to exec pods in your namespaces. Otherwise, you will not be able to use this feature.

There are two primary methods of initiating this mode:

  1. Using the tfctl command-line tool.
  2. Setting the spec.breakTheGlass field to true in the Terraform object.
"},{"location":"use-tf-controller/troubleshooting-with-break-the-glass-mode/#using-tfctl-to-break-the-glass","title":"Using tfctl to Break the Glass","text":"

In order to use this functionality, it needs to be enabled at the controller level; in order to do that, you can set the following Helm chart value to true:

allowBreakTheGlass: true\n

After the feature is enabled, to start a one-time troubleshooting session, you can use the tfctl break-glass command. For instance:

tfctl break-glass hello-world\n

This command initiates a session that allows you to execute any Terraform command to rectify the issues with your Terraform resources. It is noteworthy that this command does not require setting the spec.breakTheGlass field to true in the Terraform object.

After resolving the issues, you can simply exit the shell. GitOps will then continue to reconcile the Terraform object.

"},{"location":"use-tf-controller/troubleshooting-with-break-the-glass-mode/#break-the-glass-with-specbreaktheglass-field","title":"Break the glass with spec.breakTheGlass field","text":"

This feature is particularly useful for troubleshooting Terraform objects at their initialization stage or in situations with unexpected errors. It is generally not recommended to use this mode routinely for fixing Terraform resources.

You can enable the 'Break the Glass' feature for every reconciliation by setting the breakTheGlass field to true in the spec of the Terraform object.

Here is a sample example:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: hello-world\n  namespace: flux-system\nspec:\n  breakTheGlass: true\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/upgrade-tf-controller/","title":"Upgrading TF-Controller","text":"

Please follow these steps to upgrade TF-Controller:

  1. Read the latest release changelogs.
  2. Check your API versions.
  3. To make sure you don't get new state changes, suspend Terraform resources (tfctl suspend --all) to minimize the impact on live systems.
  4. Back up Terraform tfstates to avoid losing data. If you're using the default backend with secrets in Kubernetes, use your backup toolset (i.e., Velero) to back up the state data.
  5. Upgrade Flux first, following the Flux documentation.
  6. Disable auto-approval by either removing the approvePlan value or setting it to \"\".
  7. To prevent unintentional resource deletions, set the spec.destroyResourcesOnDeletion flag to false for critical or production systems (the default value is false)
  8. If the Flux upgrade goes well, proceed to upgrade the TF-controller via its image tag. Adjust the values in the HelmRelease to match the new version to which you are upgrading.
  9. Check the pod logs for the TF-Controller deployment and any runner logs in order to identify potential issues. If you check the warnings in the logs, you can also identify any required API changes. For example: v1alpha1 Terraform is deprecated, upgrade to v1alpha2.
  10. Push the changes you made.
  11. Resume your Terraform resources\u2014either one-by-one for critical resources, or all of them with tfctl resume --all
  12. Ensure no changes are planned for deletion. If you changed the value in step 6 from spec.destroyResourcesOnDeletion to false, resources will not be automatically removed.
  13. Revert back to auto-approval mode after ensuring stability.
  14. Resume any suspended Kustomization objects to restore GitOps automation.
  15. Restore spec.destroyResourcesOnDeletion, if this has been disabled for any resources in critical or production systems.

TF-Controller supports v1alpha1 for backward compatibility. This means that you need v1alpha2 for newer (as of September 2023) features such as: - the branch planner - pod sub-domain DNS resolutions - new PodSpec fields like PriorityClass, SecurityContext, and ResourceRequirements (Limits / Requests)

"},{"location":"use-tf-controller/use-cross-namespace-refs/","title":"Using Cross-Namespace References","text":"

The Terraform CRD in the API for TF-Controller includes fields which are references to other objects:

Name Purpose .spec.sourceRef Refers to a Flux source .spec.dependsOn[*] Each entry refers to a dependency .spec.cliConfigSecretRef Secret with tf config to use

Branch Planner configuration can also have cross-namespace references:

Name Purpose .secretNamespace Namespace of secret containing a GitHub token .resources[*] Each entry refers to a Terraform object to include in branch planning

All of these can refer to an object in a namespace different to that of the Terraform object. However, giving access to objects in other namespaces is generally considered a security risk, so this is disallowed by default. Only references that mention the same namespace, or that omit the namespace field, will be accepted. References using a different namespace will cause TF-Controller to stop processing the Terraform object and put it in a non-Ready state.

To allow cross-namespace references, use the flag --allow-cross-namespace-refs with TF-Controller and the Branch Planner. When using the Helm chart to install or update TF-Controller and Branch Planner, the value allowCrossNamespaceRefs will allow cross-namespace references for both.

"},{"location":"use-tf-controller/with-a-custom-backend/","title":"Option: Use TF-Controller with a Custom Backend","text":"

By default, TF-Controller uses the Kubernetes backend to store the Terraform state file (tfstate) in clusters.

The tfstate is stored in a secret named: tfstate-${workspace}-${secretSuffix}. The default suffix will be the name of the Terraform resource, however you may override this setting using .spec.backendConfig.secretSuffix. The default workspace name is \"default\", you can also override the workspace by setting .spec.workspace to another value.

If you wish to use a custom backend, you can configure it by defining the .spec.backendConfig.customConfiguration with one of the backends such as GCS or S3, for example:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  backendConfig:\n    customConfiguration: |\n      backend \"s3\" {\n        bucket                      = \"s3-terraform-state1\"\n        key                         = \"dev/terraform.tfstate\"\n        region                      = \"us-east-1\"\n        endpoint                    = \"http://localhost:4566\"\n        skip_credentials_validation = true\n        skip_metadata_api_check     = true\n        force_path_style            = true\n        dynamodb_table              = \"terraformlock\"\n        dynamodb_endpoint           = \"http://localhost:4566\"\n        encrypt                     = true\n      }\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  runnerPodTemplate:\n    spec:\n      image: registry.io/tf-runner:xyz\n

To add fields from secrets or configMaps, use backendConfigsFrom, for example to add access_key and secret_key from a secret:

apiVersion: v1\nkind: Secret\nmetadata:\n  name: terraform-s3-backend\n  namespace: flux-system\ntype: Opaque\ndata:\n  access_key: <base64 encoded key>\n  secret_key: <base64 encoded key>\n\n---\n\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  backendConfig:\n    customConfiguration: |\n      backend \"s3\" {\n        bucket                      = \"s3-terraform-state1\"\n        key                         = \"dev/terraform.tfstate\"\n        region                      = \"us-east-1\"\n        endpoint                    = \"http://localhost:4566\"\n        skip_credentials_validation = true\n        skip_metadata_api_check     = true\n        force_path_style            = true\n        dynamodb_table              = \"terraformlock\"\n        dynamodb_endpoint           = \"http://localhost:4566\"\n        encrypt                     = true\n      }\n  backendConfigsFrom:\n    - kind: Secret\n      name: terraform-s3-backend\n      keys:\n      - access_key\n      - secret_key\n      optional: false\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  runnerPodTemplate:\n    spec:\n      image: registry.io/tf-runner:xyz\n
"},{"location":"use-tf-controller/with-an-oci-artifact-as-source/","title":"Use TF-Controller with an OCI Artifact as Source","text":"

To use OCI artifacts as the source of Terraform objects, you need Flux 2 version v0.32.0 or higher.

Assuming that you have Terraform files (your root module may contain sub-modules) under ./modules, you can use Flux CLI to create an OCI artifact for your Terraform modules by running the following commands:

flux push artifact oci://ghcr.io/tf-controller/helloworld:$(git rev-parse --short HEAD) \\\n    --path=\"./modules\" \\\n    --source=\"$(git config --get remote.origin.url)\" \\\n    --revision=\"$(git branch --show-current)/$(git rev-parse HEAD)\"\n\nflux tag artifact oci://ghcr.io/tf-controller/helloworld:$(git rev-parse --short HEAD) \\\n    --tag main\n

Then you define a source (OCIRepository), and use it as the sourceRef of your Terraform object.

---\napiVersion: source.toolkit.fluxcd.io/v1beta2\nkind: OCIRepository\nmetadata:\n  name: helloworld-oci\nspec:\n  interval: 1m\n  url: oci://ghcr.io/tf-controller/helloworld\n  ref:\n    tag: main\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld-tf-oci\nspec:\n  path: ./\n  approvePlan: auto\n  interval: 1m\n  sourceRef:\n    kind: OCIRepository\n    name: helloworld-oci\n  writeOutputsToSecret:\n    name: helloworld-outputs\n
"},{"location":"use-tf-controller/with-aws-eks-irsa/","title":"Use TF-Controller with AWS EKS IRSA","text":"

AWS Elastic Kubernetes Service (EKS) offers IAM Roles for Service Accounts (IRSA) as a mechanism by which to provide credentials to Kubernetes pods. This can be used to provide the required AWS credentials to Terraform runners for performing plans and applies.

You can use eksctl to associate an OIDC provider with your EKS cluster. For example:

eksctl utils associate-iam-oidc-provider --cluster CLUSTER_NAME --approve\n

Then follow the instructions here to add a trust policy to the IAM role which grants the necessary permissions for Terraform. If you have installed TF-Controller following the README, then the namespace:serviceaccountname will be flux-system:tf-runner. You'll obtain a Role ARN to use in the next step.

Finally, annotate the ServiceAccount for the tf-runner with the obtained Role ARN in your cluster:

kubectl annotate -n flux-system serviceaccount tf-runner eks.amazonaws.com/role-arn=ROLE_ARN\n

If deploying the tf-controller via Helm, do this as follows:

values:\n  runner:\n    serviceAccount:\n      annotations:\n        eks.amazonaws.com/role-arn: ROLE_ARN\n
"},{"location":"use-tf-controller/with-azure/","title":"With azure","text":""},{"location":"use-tf-controller/with-azure/#use-tf-controller-with-azure","title":"Use TF-Controller with Azure","text":"

This content was provided by users @mingmingshiliyu and @maciekdude.

Use the OIDC flag and explicitly point to the token. Due to a bug in AzureRM 3.44.x, use version 3.47.x or later.

Set env variables on the runner pod:

        - name: ARM_USE_OIDC\n          value: \"true\"\n        - name: ARM_OIDC_TOKEN_FILE_PATH\n          value: \"/var/run/secrets/azure/tokens/azure-identity-token\"\n

Example yaml:

apiVersion: infra.contrib.fluxcd.io/v1alpha1\nkind: Terraform\nmetadata:\n  name: terraformhello\n  namespace: default\nspec:\n  tfstate:\n    forceUnlock: auto\n  backendConfig:\n    customConfiguration: |\n      backend \"azurerm\" {\n        resource_group_name  = \"l\"\n        storage_account_name = \"\"\n        container_name       = \"tfstate\"\n        key                  = \"helloworld.tfstate\"\n        use_oidc             = true\n      }\n  interval: 1m\n  serviceAccountName: service_account_registered_in_aad\n  approvePlan: auto\n  destroy: true\n  path: ./tests/fixture\n  sourceRef:\n    kind: GitRepository\n    name: terraformhello\n    namespace: flux-system\n  runnerPodTemplate:\n    spec:\n      image: azure_cli_runner.xxx\n      env:\n        - name: ARM_USE_OIDC\n          value: \"true\"\n        - name: ARM_SUBSCRIPTION_ID\n          value: \"\"\n        - name: ARM_TENANT_ID\n          value: \"\"\n        - name: ARM_CLIENT_ID\n          value: \"\"\n        - name: ARM_OIDC_TOKEN_FILE_PATH\n          value: \"/var/run/secrets/azure/tokens/azure-identity-token\"\n

Import existing resources to a tfstate file stored on a storage account.

"},{"location":"use-tf-controller/with-drift-detection-disabled/","title":"Use TF-Controller with drift detection disabled","text":"

Drift detection is enabled by default. You can set .spec.disableDriftDetection: true to disable it.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  disableDriftDetection: true\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/with-external-webhooks/","title":"Use TF-Controller with External Webhooks","text":"

The TF-Controller provides a way to integrate with webhooks to further validate Terraform plans and manage the Terraform execution process. With the webhook feature, you can implement custom policy checks, validations, and other logic to determine if the Terraform process should proceed.

"},{"location":"use-tf-controller/with-external-webhooks/#setting-up-the-webhook","title":"Setting up the Webhook","text":"
  1. Webhook URL: Specify the URL of your webhook, ensuring it points to a valid HTTPS endpoint.
  2. Expected Return: The webhook should return a valid JSON object. For instance: 
    {\"passed\": true}\n
  3. Accepted True Values: The true values can be true, \"true\", and \"yes\".
  4. Accepted False Values: The false values can be flse, \"false\", and \"no\".

Below is a breakdown of the relevant parts of the configuration:

  1. webhooks: This is the section where you specify all webhook related configurations.
  2. stage: Define at which stage the webhook will be triggered. Currenly, we support only the post-planning stage.
  3. url: The URL pointing to your webhook endpoint.
  4. testExpression: This expression is used to evaluate the response from the webhook. If it evaluates to true, the controller proceeds with the operation. In the example, the expression checks for the passed value from the webhook's JSON response.
  5. errorMessageTemplate: If testExpression evaluates to false, this template is used to extract the error message from the webhook's JSON response. This message will be displayed to the user.
"},{"location":"use-tf-controller/with-external-webhooks/#configuration-example","title":"Configuration Example","text":"

Here's a configuration example on how to use the webhook feature to integrate with Weave Policy Engine. 

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld-tf\nspec:\n  path: ./terraform\n  approvePlan: \"auto\"\n  interval: 1m\n  storeReadablePlan: human\n  sourceRef:\n    kind: GitRepository\n    name: helloworld-tf\n  webhooks:\n  - stage: post-planning\n    url: https://policy-agent.policy-system.svc/terraform/admission\n    testExpression: \"${{ .passed }}\"\n    errorMessageTemplate: \"Violation: ${{ (index (index .violations 0).occurrences 0).message }}\"\n  writeOutputsToSecret:\n    name: helloworld-outputs\n

Important Considerations:

  • Ensure that your webhook endpoint is secure, as the TF-Controller will be sending potentially sensitive Terraform plan data to it.
  • Test your webhook implementation thoroughly before deploying to production, as any issues could interrupt or halt your Terraform process.

With the webhook feature, you can create a more robust and flexible GitOps Terraform pipeline that respects custom organizational policies and other requirements.

"},{"location":"use-tf-controller/with-gitops-dependency-management/","title":"Use TF-controller with GitOps dependency management","text":"

TF-controller supports GitOps dependency management. The GitOps dependency management feature is based on the similar technique implemented in the Kustomization controller of Flux.

This means that you can use TF-controller to provision resources that depend on other resources at the GitOps level. For example, you can use TF-controller to provision an S3 bucket, and then use TF-controller to provision another resource to configure ACL for that bucket.

GitOps dependency management is different from Terraform's HCL dependency management in the way that it is not based on Terraform's mechanism, which is controlled by the Terraform binary. Instead, it is implemented at the controller level, which means that each Terraform module is reconciled and can be managed independently, while still being able to depend on other modules.

"},{"location":"use-tf-controller/with-gitops-dependency-management/#create-a-terraform-object","title":"Create a Terraform object","text":"

Similar to the same feature in the Kustomization controller, the dependency management feature is enabled by setting the dependsOn field in the Terraform object. The dependsOn field is a list of Terraform objects.

When the dependency is not satisfied, the Terraform object will be in the Unknown state, and it will be retry again every spec.retryInterval. The retry interval is same as the spec.interval by default, and it can be configured separately by setting the spec.retryInterval field.

First, create a Terraform object to provision the S3 bucket, name it aws-s3-bucket. The S3 bucket is provisioned by the Terraform module aws_s3_bucket in the OCI image aws-package. It is configured to use the auto-apply mode, and write outputs to the secret aws-s3-bucket-outputs.

---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: aws-s3-bucket\n  namespace: flux-system\nspec:\n  path: aws_s3_bucket\n  values:\n    bucket: my-tf-controller-test-bucket\n    tags:\n      Environment: Dev\n      Name: My bucket\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  interval: 2m\n  destroyResourcesOnDeletion: true\n  writeOutputsToSecret:\n    name: aws-s3-bucket-outputs\n    outputs:\n    - arn\n    - bucket\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n

Second, create a Terraform object to configure ACL for the S3 bucket, name it aws-s3-bucket-acl. The ACL is provisioned by the Terraform module aws_s3_bucket_acl, also from the OCI image aws-package-v4.33.0.

In the dependsOn field, specify the Terraform object that provisions the S3 bucket. This means that the ACL will be configured only after the S3 bucket is provisioned, and has its outputs Secret written. We can read the outputs of the S3 bucket from the Secret aws-s3-bucket-outputs, by specifying the spec.readInputsFromSecrets field. The spec.readInputsFromSecrets field is a list of Secret objects. Its name field is the name of the Secret, and its as field is the name of variable that can be used in the spec.values block.

For example, the spec.values.bucket field in the aws-s3-bucket-acl Terraform object is set to ${{ .aws_s3_bucket.bucket }}.

Please note that we use ${{ and }} as the delimiters for the variable name, instead of the Helm default ones, {{ and }}.

---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: aws-s3-bucket-acl\n  namespace: flux-system\nspec:\n  path: aws_s3_bucket_acl\n  values:\n    acl: private\n    bucket: ${{ .aws_s3_bucket.bucket }}\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  interval: 3m\n  dependsOn:\n  - name: aws-s3-bucket\n  readInputsFromSecrets:\n  - name: aws-s3-bucket-outputs\n    as: aws_s3_bucket\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n
"},{"location":"use-tf-controller/with-gitops-dependency-management/#avoid-kustomization-controllers-variable-substitution","title":"Avoid Kustomization controller's variable substitution","text":"

The Kustomization controller will substitute variables in the Terraform object, which will cause conflicts with the variable substitution in the GitOps dependency management feature. To avoid this, we need to add the kustomize.toolkit.fluxcd.io/substitute: disabled annotation to the Terraform object.

---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: aws-s3-bucket-acl\n  namespace: flux-system\n  annotations:\n    kustomize.toolkit.fluxcd.io/substitute: disabled\nspec:\n  path: aws_s3_bucket_acl\n  values:\n    acl: private\n    bucket: ${{ .aws_s3_bucket.bucket }}\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  interval: 3m\n  dependsOn:\n  - name: aws-s3-bucket\n  readInputsFromSecrets:\n  - name: aws-s3-bucket-outputs\n    as: aws_s3_bucket\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n
"},{"location":"use-tf-controller/with-ipv6/","title":"Use TF-Controller with IPv6 addresses.","text":"

TF-Controller uses pod IPv4 address to generate the in-cluster hostname to communicate with the runner instance. This logic fails when the runner pod has IPv6 address instead of IPv4.

The TF-Controller has a flag to use pod subdomain resolution instead of an IP address, with that enabled the controller will use cluster subdomains, and it works with IPv6 addresses as the resolution is happening at cluster level.

To enable this feature, you can set usePodSubdomainResolution to true in the Helm values file:

usePodSubdomainResolution: true\n
"},{"location":"use-tf-controller/with-plan-only-mode/","title":"Use TF-Controller with a plan-only mode","text":"

This plan-only mode is designed to be used in conjunction with the Branch Planner. But you can also use it whenever you want to run terraform plan only.

If planOnly is set to true, TF-Controller will skip the apply step, run terraform plan, and save the output.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  interval: 1m\n  planOnly: true\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/with-primitive-modules/","title":"Use TF-Controller with primitive modules","text":"

This document describes how to use TF-Controller with a primitive module. It requires TF-Controller v0.13+ to run the example.

"},{"location":"use-tf-controller/with-primitive-modules/#what-is-a-primitive-module","title":"What is a primitive module?","text":"

It's a Terraform module that contains only a single resource.

  • A Terraform primitive module must contains the \"values\" variable.
  • The \"values\" variable must be an object with fields of optional types.
  • The module must be placed under a directory, which is named after the resource.
  • The directory can optionally contain other files, for example the .terraform.lock.hcl.
  • We call a set of primitive modules bundled into an OCI image, a package.
"},{"location":"use-tf-controller/with-primitive-modules/#hello-world-primitive-module","title":"Hello World Primitive Module","text":"

Here is an example of how a primitive module can be defined in YAML. Assume that we have a ready-to-use OCI image with a primitive module for the imaginary resource aws_hello_world, and the image is tagged as ghcr.io/tf-controller/hello-primitive-modules/v4.32.0:v1.

We'll use the following Terraform object definition to provision the resource.

First, we need to create a Terraform object with the spec.sourceRef.kind field set to OCIRepository and the spec.sourceRef.name field set to the name of the OCIRepository object.

Second, we need to set the spec.path field to the name of the resource, in this case aws_hello_world.

Third, we need to set the spec.values field to the values of the resource. This is a YAML object that will be converted to an HCL variable, and passed to the Terraform module.

Finally, we need to set the spec.approvePlan field to auto to automatically approve the plan.

---\napiVersion: source.toolkit.fluxcd.io/v1beta2\nkind: OCIRepository\nmetadata:\n  name: hello-package-v4.32.0\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: oci://ghcr.io/tf-controller/hello-primitive-modules/v4.32.0\n  ref:\n    tag: v1\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: hello-world\n  namespace: flux-system\nspec:\n  path: aws_hello_world\n  values:\n    greeting: Hi\n    subject: my world\n  sourceRef:\n    kind: OCIRepository\n    name: hello-package-v4.32.0\n  interval: 1h0m\n  approvePlan: auto\n
"},{"location":"use-tf-controller/with-tf-runner-exposed-using-hostname-subdomain/","title":"Use TF-controller with Terraform Runners exposed via hostname/subdomain","text":"

TF-controller uses the Controller/Runner architecture. The Controller acts as a client, and talks to each Runner's Pod via gRPC over port 30000.

TF-controller must thus be able to reliably connect to each Runner's pod regardless of the cluster network topology.

"},{"location":"use-tf-controller/with-tf-runner-exposed-using-hostname-subdomain/#the-default-runner-dns-resolution","title":"The Default Runner DNS resolution","text":"

By default, TF-controller fetches the Runner's pod IP address after it is instantiated (e.g. 1.2.3.4).

It then transforms the IP address into its IP-based pod DNS A record (e.g. 1.2.3.4.<namespace>.pod.<cluster-domain>) which is used to connect to the Runner pod using gRPC protocol.

In standard Kubernetes cluster deployment, IP-based pod DNS resolution is usually provided by Coredns and especially the pods option of the Kubernetes plugin.

cluster.local {\n    kubernetes {\n        pods verified\n    }\n}\n

IMPORTANT: The gRPC communication between TF-controller and Runner's pod is secured with mTLS. TF-controller generates a valid wildcard TLS certificate for *.<namespace>.pod.<cluster-domain> hosts on the Runner's namespace. The Runner's pod present this certificate during TLS handshake with TF-controller.

"},{"location":"use-tf-controller/with-tf-runner-exposed-using-hostname-subdomain/#hostnamesubdomain-runner-dns-resolution","title":"Hostname/Subdomain Runner DNS resolution","text":"

The default configuration described above works for standard Kubernetes deployments. It does not work however when the cluster DNS provider do not support IP-based pod DNS resolution. This is the case for GCP Cloud DNS for example.

For such setup, you can switch the DNS resolution mode to Hostname/Subdomain. Enabling this option will :

  • Create a Headless service named tf-runner in each allowed namespace

```yaml hl_lines=\"4-5,8-10\" apiVersion: v1 kind: Service metadata: name: tf-runner namespace: hello-world spec: clusterIP: None ports: - name: grpc port: 30000 selector: app.kubernetes.io/created-by: tf-controller app.kubernetes.io/name: tf-runner 

- Set Runner's pod spec with `hostname: <terraform_object_name>` and `subdomain: tf-runner`\n\n```yaml hl_lines=\"12-13\"\napiVersion: v1\nkind: Pod\n  labels:\n    app.kubernetes.io/created-by: tf-controller\n    app.kubernetes.io/instance: tf-runner-3ac83e0f\n    app.kubernetes.io/name: tf-runner\n    infra.contrib.fluxcd.io/terraform: hello-world\n    tf.weave.works/tls-secret-name: terraform-runner.tls-1693866794\n  name: helloworld-tf-runner\n  namespace: hello-world\nspec:\n  hostname: helloworld\n  subdomain: tf-runner\n  containers:\n  - args:\n    - --grpc-port\n    - \"30000\"\n    - --tls-secret-name\n    - terraform-runner.tls-1693866794\n    - --grpc-max-message-size\n    - \"4\"\n    image: ghcr.io/weaveworks/tf-runner:v0.16.0-rc.2\n    name: tf-runner\n    ports:\n    - containerPort: 30000\n      name: grpc\n    resources:\n      limits:\n        cpu: 500m\n        ephemeral-storage: 1Gi\n        memory: 2Gi\n    securityContext:\n      allowPrivilegeEscalation: false\n      capabilities:\n        drop:\n        - ALL\n      readOnlyRootFilesystem: true\n      runAsNonRoot: true\n      runAsUser: 65532\n      seccompProfile:\n        type: RuntimeDefault\n  preemptionPolicy: PreemptLowerPriority\n  priority: 0\n  schedulerName: gke.io/optimize-utilization-scheduler\n  securityContext:\n    seccompProfile:\n      type: RuntimeDefault\n  serviceAccountName: tf-runner\n

The Runner's pod can then be targeted by TF-Controller using <terraform_object_name>.tf-runner.<namespace>.svc.<cluster-domain> (helloworld.tf-runner.hello-world.svc.cluster.local) as per Kubernetes specification instead of IP-based pod DNS resolution.

The switch is performed by setting the following Helm value usePodSubdomainResolution: true or running directly TF-controller with the option --use-pod-subdomain-resolution=true

IMPORTANT: The gRPC communication between TF-Controller and Runner's pod is secured with mTLS. TF-controller generates a valid wildcard TLS certificate for *.<namespace>.pod.<cluster-domain> and *.tf-runner.<namespace>.svc.<cluster-domain> hosts on the Runner's namespace. The Runner's pod present this certificate during TLS handshake with TF-Controller.

"},{"location":"use-tf-controller/with-tf-runner-logging/","title":"Control the logging behavior of Terraform Runner","text":"

A Terraform Runner uses two environment variables, DISABLE_TF_LOGS and ENABLE_SENSITIVE_TF_LOGS, to control the logging behavior of the Terraform execution.

To use these environment variables, they need to be set on each Terraform Runner pod where the Terraform code is being executed. This can typically be done by adding them to the pod's environment variables in the Terraform Runner deployment configuration.

  • The DISABLE_TF_LOGS variable, when set to \"1\", will disable all Terraform output logs to stdout and stderr.
  • The ENABLE_SENSITIVE_TF_LOGS variable, when set to \"1\", will enable logging of sensitive Terraform data, such as secret variables, to the local log. However, it is important to note that for the ENABLE_SENSITIVE_TF_LOGS to take effect, the DISABLE_TF_LOGS variable must also be set to \"1\".
"},{"location":"use-tf-controller/with-tf-runner-logging/#the-default-logging-behavior","title":"The Default Logging Behavior","text":"
  • By default, the logging level for the tf-runner is configured at the info level.
  • The DISABLE_TF_LOGS variable is not activated as part of the default settings.
  • The ENABLE_SENSITIVE_TF_LOGS variable remains inactive in the default configuration.
  • Calls to ShowPlan and ShowPlanRaw on the runner are not logged by default.
  • For Plan calls made on the runner, error messages are sanitized as a part of the default configuration.

For more information on configuring the Terraform Runner and its environment variables, please consult the documentation on customizing runners within the Weave TF-controller.

"},{"location":"use-tf-controller/with-tf-runner-logging/#logging-human-readable-plan","title":"Logging human-readable plan","text":"

The plan can be logged in a human-readable format just before the applying it in the tf-runner. To enable this, set the environment variable LOG_HUMAN_READABLE_PLAN to \"1\" on the runner.

"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/","title":"With the ready to use aws package","text":""},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#use-tf-controller-with-the-ready-to-use-aws-package","title":"Use TF-Controller with the ready-to-use AWS package","text":"

You need TF-Controller v0.13+ to running the example of TF-Controller with the ready-to-use AWS package.

"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#what-is-a-package","title":"What is a package?","text":"

A package is a collection of primitive Terraform modules that are bundled into an OCI image. You can think of a TF-controller's package as a thin wrapper around a Terraform module provider, and a TF-controller primitive module as a thin wrapper around a Terraform resource or a root module.

We will provide a set of ready-to-use packages for the most popular cloud providers. Currently, we ship the package for AWS only.

"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#aws-package","title":"AWS Package","text":"

To provide the out-of-the-box experience, the AWS Package is installed by default when you installed the TF-controller. Unlike other IaC implementation, our package model is designed to be very lightweight as a package is just a set of TF files in the form of OCI. Packages would not put any burden to your cluster. However, you can opt this package out by setting awsPackage.install: false in your Helm chart values.

If you run flux get sources oci you should see the AWS package installed in your cluster listed as aws-package.

flux get sources oci\nNAME          REVISION                    SUSPENDED   READY   MESSAGE                                                                                                         \naws-package   v4.38.0-v1alpha11/6033f3b   False       True    stored artifact for digest 'v4.38.0-v1alpha11/6033f3b'\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#a-step-by-step-tutorial","title":"A step-by-step tutorial","text":"

This section describes how to use the AWS package to provision an S3 bucket with ACL using the TF-controller.

"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#create-a-kind-local-cluster","title":"Create a KinD local cluster","text":"

If you don't have a Kubernetes cluster, you can create a KinD cluster with the following command:

kind create cluster\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#install-flux","title":"Install Flux","text":"

After you have a Kubernetes cluster, you can install Flux with the following command:

flux install\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#install-tf-controller","title":"Install TF-controller","text":"

Then, you can install the TF-controller with the following command:

kubectl apply -f https://raw.githubusercontent.com/weaveworks/tf-controller/main/docs/release.yaml\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#setup-aws-credentials","title":"Setup AWS credentials","text":"

To provision AWS resources, you need to provide the AWS credentials to your Terraform objects. You can do this by creating a secret with the AWS credentials and reference it in each of your Terraform objects.

```yaml\napiVersion: v1\nkind: Secret\nmetadata:\n  name: aws-credentials\n  namespace: flux-system\ntype: Opaque\nstringData:\n  AWS_ACCESS_KEY_ID: Axxxxxxxxxxxxxxxxxxx\n  AWS_SECRET_ACCESS_KEY: qxxxxxxxxxxxxxxxxxxxxxxxxx\n  AWS_REGION: us-east-1 # the region you want\n

To apply the secret, run the following command:

kubectl apply -f aws-credentials.yaml\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#setup-aws-bucket-and-acl","title":"Setup AWS Bucket and ACL","text":"

Now, you can create two Terraform objects, one for an S3 bucket, another one for ACL. Please note that we are using GitOps dependencies to make sure the ACL is created after the bucket is created. You can read more about the GitOps dependencies in the GitOps dependencies document.

```yaml\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: aws-s3-bucket\n  namespace: flux-system\n  labels:\n    tf.weave.works/composite: s3-bucket\nspec:\n  path: aws_s3_bucket\n  values:\n    bucket: my-tf-controller-test-bucket\n    tags:\n      Environment: Dev\n      Name: My bucket\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  retryInterval: 10s\n  interval: 2m\n  destroyResourcesOnDeletion: true\n  writeOutputsToSecret:\n    name: aws-s3-bucket-outputs\n    outputs:\n    - arn\n    - bucket\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: example-bucket-acl\n  namespace: flux-system\n  labels:\n    tf.weave.works/composite: s3-bucket\nspec:\n  path: aws_s3_bucket_acl\n  values:\n    acl: private\n    bucket: ${{ .aws_s3_bucket.bucket }}\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  retryInterval: 10s\n  interval: 3m\n  dependsOn:\n  - name: aws-s3-bucket\n  readInputsFromSecrets:\n  - name: aws-s3-bucket-outputs\n    as: aws_s3_bucket\n  destroyResourcesOnDeletion: true\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#rename-input-secrets","title":"Rename input secrets","text":"

The spec.readInputsFromSecrets field allows you to reference the Terraform outputs from other Terraform objects. In the context of this field, renaming makes it easier to reference the secrets in the spec.values field.

To rename a secret, you need to use the as key in the spec.readInputsFromSecrets field. The name key corresponds to the original name of the secret, while the as key represents the new name that you want to use to reference the secret.

In the example below, we can reference the bucket value from our aws_s3_bucket secret using ${{ .aws_s3_bucket.bucket }} instead of using the original secret name, which is aws-s3-bucket-outputs.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: example-bucket-acl\n  namespace: flux-system\nspec:\n  # ...\n  readInputsFromSecrets:\n  - name: aws-s3-bucket-outputs\n    as: aws_s3_bucket\n  values:\n    acl: private\n    bucket: ${{ .aws_s3_bucket.bucket }}\n  # ...\n
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Overview","text":"

~> BREAKING NEWS: The Technology Preview of the Branch Planner is now available! Learn more or Get started with the Branch Planner.

TF-controller is a reliable controller for Flux to reconcile Terraform resources in the GitOps way. With the power of Flux together with Terraform, TF-controller allows you to GitOps-ify infrastructure, and application resources, in the Kubernetes and Terraform universe, at your own pace.

\"At your own pace\" means you don't need to GitOps-ify everything at once.

A high-level diagram of how TF-controller works is shown below:

TF-controller offers many GitOps models:

  1. GitOps Automation Model: GitOps your Terraform resources from the provision steps to the enforcement steps, like a whole EKS cluster.
  2. Hybrid GitOps Automation Model: GitOps parts of your existing infrastructure resources. For example, you have an existing EKS cluster. You can choose to GitOps only its nodegroup, or its security group.
  3. State Enforcement Model: You have a TFSTATE file, and you'd like to use GitOps enforce it, without changing anything else.
  4. Drift Detection Model: You have a TFSTATE file, and you'd like to use GitOps just for drift detection, so you can decide to do things later when a drift occurs.

To get started, follow the getting started guide.

"},{"location":"#features","title":"Features","text":"
  • Multi-Tenancy: TF-controller supports multi-tenancy by running Terraform plan and apply inside Runner Pods. When specifying .metadata.namespace and .spec.serviceAccountName, the Runner Pod uses the specified ServiceAccount and runs inside the specified Namespace. These settings enable the soft multi-tenancy model, which can be used within the Flux multi-tenancy setup. This feature is available since v0.9.0.
  • GitOps Automation for Terraform: With setting .spec.approvePlan=auto, it allows a Terraform object to be reconciled and act as the representation of your Terraform resources. The TF-controller uses the spec of the Terraform object to perform plan, apply its associated Terraform resources. It then stores the TFSTATE of the applied resources as a Secret inside the Kubernetes cluster. After .spec.interval passes, the controller performs drift detection to check if there is a drift occurred between your live system, and your Terraform resources. If a drift occurs, the plan to fix that drift will be generated and applied automatically. This feature is available since v0.3.0.
  • Drift detection: This feature is a part of the GitOps automation feature. The controller detects and fixes drift for your infrastructures, based on the Terraform resources and their TFSTATE. This feature is available since v0.5.0.
    • Drift detection is enabled by default. You can use the field .spec.disableDriftDetection to disable this behaviour. This feature is available since v0.7.0.
    • The Drift detection only mode, without plan or apply steps, allows you to perform read-only drift detection. This feature is available since v0.8.0.
  • Plan and Manual Approve: This feature allows you to separate the plan, out of the apply step, just like the Terraform workflow you are familiar with. A good thing about this is that it is done in a GitOps way. When a plan is generated, the controller shows you a message like 'set approvePlan: \"plan-main-123\" to apply this plan.'. You make change to the field .spec.approvePlan, commit and push to tell the TF-controller to apply the plan for you. With this GitOps workflow, you can optionally create and push this change to a new branch for your team member to review and approve too. This feature is available since v0.6.0.
  • First-class YAML-based Terraform: The Terraform object in v0.13.0+ allows you to better configure your Terraform resources via YAMLs, but without introducing any extra CRDs to your cluster. Together with a new generator called Tofu-Jet, we'll now be able to ship pre-generated primitive Terraform modules for all major cloud providers. A primitive Terraform module is a module that only contains a single primitive resource, like aws_iam_role, or aws_iam_policy. With this concept, we would be able to use Terraform without writing Terraform codes, and make it more GitOps-friendly at the same time. This feature is available since v0.13.0.
  • GitOps Dependency for Terraform: The Terraform object in v0.13.0+ allows you to specify a list of Terraform objects that it depends on. The controller will wait for the dependencies to be ready before it starts to reconcile the Terraform object. This allows you to create a dependency graph of your Terraform modules, and make sure the modules are applied in the correct order. Please use .spec.retryInterval (a small value like 20s) to control the retry interval when using this feature. This feature is available since v0.13.0.
"},{"location":"#support-matrix","title":"Support Matrix","text":"Version Terraform Source Controller Flux v2 v0.15 v1.3.9 v1.0.x v2.0.x v0.14 v1.3.9 v0.31.0 v0.41.x v0.13 v1.3.1 v0.31.0 v0.36.x v0.12 v1.1.9 v0.26.1 v0.32.x"},{"location":"getting_started/","title":"Getting Started","text":""},{"location":"getting_started/#preflight-checks","title":"Preflight Checks","text":"

Here are the requirements you need to set up before you start:

  1. For Terraform Controller v0.15+, it requires Flux v2.0 or later (not only the CLI, but also the controllers on the cluster). If you are not sure about the Flux version on your cluster, please re-bootstrap your cluster.
  2. For Terraform Controller v0.13 and v0.14, Flux 2 v0.32 - v0.41 (of course, not only the CLI, but also the controllers on the cluster).
  3. TF-controller uses the Controller/Runner architecture. The Controller acts as a client, and talks to each Runner's Pod via gRPC. Please make sure
    1. Each Runner's Pod in each Namespace is allowed to open, and serve at port 30000 (the gRPC port of a Runner), and the Controller can connect to it.
    2. The Controller needs to download tar.gz BLOBs from the Source controller via port 80.
    3. The Controller needs to post the events to the Notification controller via port 80.
"},{"location":"getting_started/#installation","title":"Installation","text":"

Before using TF-controller, you have to install Flux by using either flux install or flux bootstrap command. Please note that TF-controller now requires Flux v2.0 or later, so please make sure you have the latest version of Flux. After that you can install TF-controller with Flux HelmRelease by:

kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/release.yaml\n

For the most recent release candidate of TF-controller, please use rc.yaml.

kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/rc.yaml\n
"},{"location":"getting_started/#installation-on-gke","title":"Installation on GKE","text":"

As of September 2023, GKE Autopilot clusters will use Cloud DNS for internal DNS resolution. This means that the default DNS resolution method used by TF-controller will not work. To use TF-controller on GKE Autopilot, you must set flag --use-pod-subdomain-resolution=true on the TF-controller deployment. This flag can be set by adding the following to the TF-controller HelmRelease:

spec:\n  values:\n    usePodSubdomainResolution: true\n    runner:\n      allowedNamespaces:\n      - flux-system\n      - dev-team\n

Enabling this value will cause TF-controller to use the Pod's subdomain for DNS resolution instead of the default Pod resolution method. Pod's subdomain resolution requires a Service to be created for the Pod. The HelmRelease above will create a Service named tf-runner in each namespace specified by the runner.allowedNamespaces value.

We have provided a HelmRelease to install TF-controller on GKE Autopilot with Pod's subdomain resolution enabled here.

kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/rc-gke.yaml\n

Tested with GKE Autopilot v1.27.3-gke.100.

"},{"location":"getting_started/#with-branch-planner","title":"With Branch Planner","text":"
kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/branch-planner/release.yaml\n

For the most recent release candidate of TF-controller with Branch Planner, please use branch-planner/rc.yaml.

kubectl apply -f https://raw.githubusercontent.com/flux-iac/tofu-controller/main/docs/branch-planner/rc.yaml\n

For more details about the Branch Planner, please visit the Branch Planner documentation.

"},{"location":"getting_started/#manual-installation","title":"Manual installation","text":"

With Helm by:

# Add tf-controller helm repository\nhelm repo add tf-controller https://flux-iac.github.io/tofu-controller\n\n# Install tf-controller\nhelm upgrade -i tf-controller tf-controller/tf-controller \\\n    --namespace flux-system\n

For details on configurable parameters of the TF-controller chart, please see chart readme.

Alternatively, you can install TF-controller via kubectl:

export TF_CON_VER=v0.15.1\nkubectl apply -f https://github.com/flux-iac/tofu-controller/releases/download/${TF_CON_VER}/tf-controller.crds.yaml\nkubectl apply -f https://github.com/flux-iac/tofu-controller/releases/download/${TF_CON_VER}/tf-controller.rbac.yaml\nkubectl apply -f https://github.com/flux-iac/tofu-controller/releases/download/${TF_CON_VER}/tf-controller.deployment.yaml\n
"},{"location":"getting_started/#quick-start","title":"Quick start","text":"

Here's a simple example of how to GitOps your Terraform resources with TF-controller and Flux.

"},{"location":"getting_started/#define-source","title":"Define source","text":"

First, we need to define a Source controller's source (GitRepository, Bucket, OCIRepository), for example:

apiVersion: source.toolkit.fluxcd.io/v1\nkind: GitRepository\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: https://github.com/tf-controller/helloworld\n  ref:\n    branch: main\n
"},{"location":"getting_started/#the-gitops-automation-mode","title":"The GitOps Automation mode","text":"

The GitOps automation mode could be enabled by setting .spec.approvePlan=auto. In this mode, Terraform resources will be planned, and automatically applied for you.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  interval: 1m\n  approvePlan: auto\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n

For a full list of features and how to use them, please follow the Use TF-controller guide.

"},{"location":"getting_started/#other-examples","title":"Other Examples","text":"
  • A Terraform GitOps with Flux to automatically reconcile your AWS IAM Policies.
  • GitOps an existing EKS cluster, by partially import its nodegroup and manage it with TF-controller: An EKS scaling example.
"},{"location":"tfctl/","title":"tfctl","text":"

tfctl is a command-line utility to help with tf-controller operations.

"},{"location":"tfctl/#installation","title":"Installation","text":"

To install tfctl via Homebrew, run the following command:

brew install weaveworks/tap/tfctl\n

You can also download the tfctl binary via the GitHub releases page: https://github.com/flux-iac/tofu-controller/releases.

Usage:\n  tfctl [command]\n\nAvailable Commands:\n  completion  Generate the autocompletion script for the specified shell\n  create      Create a Terraform resource\n  delete      Delete a Terraform resource\n  get         Get Terraform resources\n  help        Help about any command\n  install     Install the tf-controller\n  plan        Plan a Terraform configuration\n  reconcile   Trigger a reconcile of the provided resource\n  resume      Resume reconciliation for the provided resource\n  suspend     Suspend reconciliation for the provided resource\n  uninstall   Uninstall the tf-controller\n  version     Prints tf-controller and tfctl version information\n\nFlags:\n  -h, --help                help for tfctl\n      --kubeconfig string   Path to the kubeconfig file to use for CLI requests.\n  -n, --namespace string    The kubernetes namespace to use for CLI requests. (default \"flux-system\")\n      --terraform string    The location of the terraform binary. (default \"/usr/bin/terraform\")\n\nUse \"tfctl [command] --help\" for more information about a command.\n
"},{"location":"tfctl/#shell-completion","title":"Shell completion","text":"

It works the same way as flux CLI:

With bash:

# ~/.bashrc or ~/.profile\ncommand -v tfctl >/dev/null && . <(tfctl completion bash)\n

With fish:

tfctl completion fish > ~/.config/fish/completions/tfctl.fish\n

With powershell:

# Windows\n\ncd \"$env:USERPROFILE\\Documents\\WindowsPowerShell\\Modules\"\ntfctl completion powershell >> tfctl-completion.ps1\n\n# Linux\n\ncd \"${XDG_CONFIG_HOME:-\"$HOME/.config/\"}/powershell/modules\"\ntfctl completion powershell >> tfctl-completions.ps1\n

With zsh:

# ~/.zshrc or ~/.profile\ncommand -v tfctl >/dev/null && . <(tfctl completion zsh)\n
"},{"location":"References/terraform/","title":"API References","text":"Terraform API reference

Packages:

  • infra.contrib.fluxcd.io/v1alpha2
"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2","title":"infra.contrib.fluxcd.io/v1alpha2","text":"

Resource Types:

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.BackendConfigSpec","title":"BackendConfigSpec","text":"

(Appears on: TerraformSpec)

BackendConfigSpec is for specifying configuration for Terraform\u2019s Kubernetes backend

Field Description disable bool (Optional)

Disable is to completely disable the backend configuration.

secretSuffix string (Optional) inClusterConfig bool (Optional) customConfiguration string (Optional) configPath string (Optional) labels map[string]string (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.BackendConfigsReference","title":"BackendConfigsReference","text":"

(Appears on: TerraformSpec)

Field Description kind string

Kind of the values referent, valid values are (\u2018Secret\u2019, \u2018ConfigMap\u2019).

name string

Name of the configs referent. Should reside in the same namespace as the referring resource.

keys []string (Optional)

Keys is the data key where a specific value can be found at. Defaults to all keys.

optional bool (Optional)

Optional marks this BackendConfigsReference as optional. When set, a not found error for the values reference is ignored, but any Key or transient error will still result in a reconciliation failure.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.BranchPlanner","title":"BranchPlanner","text":"

(Appears on: TerraformSpec)

Field Description enablePathScope bool (Optional)

EnablePathScope specifies if the Branch Planner should or shouldn\u2019t check if a Pull Request has changes under .spec.path. If enabled extra resources will be created only if there are any changes in terraform files.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.CloudSpec","title":"CloudSpec","text":"

(Appears on: TerraformSpec)

Field Description organization string workspaces CloudWorkspacesSpec hostname string (Optional) token string (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.CloudWorkspacesSpec","title":"CloudWorkspacesSpec","text":"

(Appears on: CloudSpec)

Field Description name string (Optional) tags []string (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.CrossNamespaceSourceReference","title":"CrossNamespaceSourceReference","text":"

(Appears on: TerraformSpec)

CrossNamespaceSourceReference contains enough information to let you locate the typed Kubernetes resource object at cluster level.

Field Description apiVersion string (Optional)

API version of the referent.

kind string

Kind of the referent.

name string

Name of the referent.

namespace string (Optional)

Namespace of the referent, defaults to the namespace of the Kubernetes resource object that contains the reference.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.FileMapping","title":"FileMapping","text":"

(Appears on: TerraformSpec)

Field Description secretRef github.com/fluxcd/pkg/apis/meta.SecretKeyReference

Reference to a Secret that contains the file content

location string

Location can be either user\u2019s home directory or the Terraform workspace

path string

Path of the file - relative to the \u201clocation\u201d

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.ForceUnlockEnum","title":"ForceUnlockEnum (string alias)","text":"

(Appears on: TFStateSpec)

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.HealthCheck","title":"HealthCheck","text":"

(Appears on: TerraformSpec)

HealthCheck contains configuration needed to perform a health check after terraform is applied.

Field Description name string

Name of the health check.

type string

Type of the health check, valid values are (\u2018tcp\u2019, \u2018http\u2019). If tcp is specified, address is required. If http is specified, url is required.

url string (Optional)

URL to perform http health check on. Required when http type is specified. Go template can be used to reference values from the terraform output (e.g. https://example.org, {{.output_url}}).

address string (Optional)

Address to perform tcp health check on. Required when tcp type is specified. Go template can be used to reference values from the terraform output (e.g. 127.0.0.1:8080, {{.address}}:{{.port}}).

timeout Kubernetes meta/v1.Duration (Optional)

The timeout period at which the connection should timeout if unable to complete the request. When not specified, default 20s timeout is used.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.LockStatus","title":"LockStatus","text":"

(Appears on: TerraformStatus)

LockStatus defines the observed state of a Terraform State Lock

Field Description lastApplied string (Optional) pending string (Optional)

Pending holds the identifier of the Lock Holder to be used with Force Unlock

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.PlanStatus","title":"PlanStatus","text":"

(Appears on: TerraformStatus)

Field Description lastApplied string (Optional) pending string (Optional) isDestroyPlan bool (Optional) isDriftDetectionPlan bool (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.ReadInputsFromSecretSpec","title":"ReadInputsFromSecretSpec","text":"

(Appears on: TerraformSpec)

Field Description name string as string"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.Remediation","title":"Remediation","text":"

(Appears on: TerraformSpec)

Field Description retries int64 (Optional)

Retries is the number of retries that should be attempted on failures before bailing. Defaults to \u20180\u2019, a negative integer denotes unlimited retries.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.ResourceInventory","title":"ResourceInventory","text":"

(Appears on: TerraformStatus)

ResourceInventory contains a list of Kubernetes resource object references that have been applied by a Kustomization.

Field Description entries []ResourceRef

Entries of Kubernetes resource object references.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.ResourceRef","title":"ResourceRef","text":"

(Appears on: ResourceInventory)

ResourceRef contains the information necessary to locate a resource within a cluster.

Field Description n string

Terraform resource\u2019s name.

t string

Type is Terraform resource\u2019s type

id string

ID is the resource identifier. This is cloud-specific. For example, ARN is an ID on AWS.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RetryStrategyEnum","title":"RetryStrategyEnum (string alias)","text":"

(Appears on: TerraformSpec)

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RunnerPodMetadata","title":"RunnerPodMetadata","text":"

(Appears on: RunnerPodTemplate)

Field Description labels map[string]string (Optional)

Labels to add to the runner pod

annotations map[string]string (Optional)

Annotations to add to the runner pod

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RunnerPodSpec","title":"RunnerPodSpec","text":"

(Appears on: RunnerPodTemplate)

Field Description image string (Optional)

Runner pod image to use other than default

envFrom []Kubernetes core/v1.EnvFromSource (Optional)

List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.

env []Kubernetes core/v1.EnvVar (Optional)

List of environment variables to set in the container. Cannot be updated.

nodeSelector map[string]string (Optional)

Set the NodeSelector for the Runner Pod

affinity Kubernetes core/v1.Affinity (Optional)

Set the Affinity for the Runner Pod

tolerations []Kubernetes core/v1.Toleration (Optional)

Set the Tolerations for the Runner Pod

volumeMounts []Kubernetes core/v1.VolumeMount (Optional)

Set Volume Mounts for the Runner Pod

volumes []Kubernetes core/v1.Volume (Optional)

Set Volumes for the Runner Pod

initContainers []Kubernetes core/v1.Container (Optional)

Set up Init Containers for the Runner

hostAliases []Kubernetes core/v1.HostAlias (Optional)

Set host aliases for the Runner Pod

priorityClassName string (Optional)

Set PriorityClassName for the Runner Pod container

securityContext Kubernetes core/v1.SecurityContext (Optional)

Set SecurityContext for the Runner Pod container

resources Kubernetes core/v1.ResourceRequirements (Optional)

Set Resources for the Runner Pod container

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RunnerPodTemplate","title":"RunnerPodTemplate","text":"

(Appears on: TerraformSpec)

Field Description metadata RunnerPodMetadata (Optional) spec RunnerPodSpec (Optional) image string (Optional)

Runner pod image to use other than default

envFrom []Kubernetes core/v1.EnvFromSource (Optional)

List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.

env []Kubernetes core/v1.EnvVar (Optional)

List of environment variables to set in the container. Cannot be updated.

nodeSelector map[string]string (Optional)

Set the NodeSelector for the Runner Pod

affinity Kubernetes core/v1.Affinity (Optional)

Set the Affinity for the Runner Pod

tolerations []Kubernetes core/v1.Toleration (Optional)

Set the Tolerations for the Runner Pod

volumeMounts []Kubernetes core/v1.VolumeMount (Optional)

Set Volume Mounts for the Runner Pod

volumes []Kubernetes core/v1.Volume (Optional)

Set Volumes for the Runner Pod

initContainers []Kubernetes core/v1.Container (Optional)

Set up Init Containers for the Runner

hostAliases []Kubernetes core/v1.HostAlias (Optional)

Set host aliases for the Runner Pod

priorityClassName string (Optional)

Set PriorityClassName for the Runner Pod container

securityContext Kubernetes core/v1.SecurityContext (Optional)

Set SecurityContext for the Runner Pod container

resources Kubernetes core/v1.ResourceRequirements (Optional)

Set Resources for the Runner Pod container

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.TFStateSpec","title":"TFStateSpec","text":"

(Appears on: TerraformSpec)

TFStateSpec allows the user to set ForceUnlock

Field Description forceUnlock ForceUnlockEnum (Optional)

ForceUnlock a Terraform state if it has become locked for any reason. Defaults to no.

This is an Enum and has the expected values of:

  • auto
  • yes
  • no

WARNING: Only use auto in the cases where you are absolutely certain that no other system is using this state, you could otherwise end up in a bad place See https://www.terraform.io/language/state/locking#force-unlock for more information on the terraform state lock and force unlock.

lockIdentifier string (Optional)

LockIdentifier holds the Identifier required by Terraform to unlock the state if it ever gets into a locked state.

You\u2019ll need to put the Lock Identifier in here while setting ForceUnlock to either yes or auto.

Leave this empty to do nothing, set this to the value of the Lock Info: ID: [value], e.g. f2ab685b-f84d-ac0b-a125-378a22877e8d, to force unlock the state.

lockTimeout Kubernetes meta/v1.Duration (Optional)

LockTimeout is a Duration string that instructs Terraform to retry acquiring a lock for the specified period of time before returning an error. The duration syntax is a number followed by a time unit letter, such as 3s for three seconds.

Defaults to 0s which will behave as though LockTimeout was not set

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.Terraform","title":"Terraform","text":"

Terraform is the Schema for the terraforms API

Field Description metadata Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field. spec TerraformSpec approvePlan string (Optional)

ApprovePlan specifies name of a plan wanted to approve. If its value is \u201cauto\u201d, the controller will automatically approve every plan.

destroy bool (Optional)

Destroy produces a destroy plan. Applying the plan will destroy all resources.

backendConfig BackendConfigSpec (Optional) backendConfigsFrom []BackendConfigsReference (Optional) cloud CloudSpec (Optional) workspace string (Optional) vars []Variable (Optional)

List of input variables to set for the Terraform program.

varsFrom []VarsReference (Optional)

List of references to a Secret or a ConfigMap to generate variables for Terraform resources based on its data, selectively by varsKey. Values of the later Secret / ConfigMap with the same keys will override those of the former.

values Kubernetes pkg/apis/apiextensions/v1.JSON (Optional)

Values map to the Terraform variable \u201cvalues\u201d, which is an object of arbitrary values. It is a convenient way to pass values to Terraform resources without having to define a variable for each value. To use this feature, your Terraform file must define the variable \u201cvalues\u201d.

tfVarsFiles []string (Optional)

TfVarsFiles loads all given .tfvars files. It copycats the -var-file functionality.

fileMappings []FileMapping (Optional)

List of all configuration files to be created in initialization.

interval Kubernetes meta/v1.Duration

The interval at which to reconcile the Terraform.

retryInterval Kubernetes meta/v1.Duration (Optional)

The interval at which to retry a previously failed reconciliation. The default value is 15 when not specified.

retryStrategy RetryStrategyEnum (Optional)

The strategy to use when retrying a previously failed reconciliation. The default strategy is StaticInterval and the retry interval is based on the RetryInterval value. The ExponentialBackoff strategy uses the formula: 2^reconciliationFailures * RetryInterval with a maximum requeue duration of MaxRetryInterval.

maxRetryInterval Kubernetes meta/v1.Duration (Optional)

The maximum requeue duration after a previously failed reconciliation. Only applicable when RetryStrategy is set to ExponentialBackoff. The default value is 24 hours when not specified.

path string (Optional)

Path to the directory containing Terraform (.tf) files. Defaults to \u2018None\u2019, which translates to the root path of the SourceRef.

sourceRef CrossNamespaceSourceReference

SourceRef is the reference of the source where the Terraform files are stored.

suspend bool (Optional)

Suspend is to tell the controller to suspend subsequent TF executions, it does not apply to already started executions. Defaults to false.

force bool (Optional)

Force instructs the controller to unconditionally re-plan and re-apply TF resources. Defaults to false.

readInputsFromSecrets []ReadInputsFromSecretSpec (Optional) writeOutputsToSecret WriteOutputsToSecretSpec (Optional)

A list of target secrets for the outputs to be written as.

disableDriftDetection bool (Optional)

Disable automatic drift detection. Drift detection may be resource intensive in the context of a large cluster or complex Terraform statefile. Defaults to false.

cliConfigSecretRef Kubernetes core/v1.SecretReference (Optional) healthChecks []HealthCheck (Optional)

List of health checks to be performed.

destroyResourcesOnDeletion bool (Optional)

Create destroy plan and apply it to destroy terraform resources upon deletion of this object. Defaults to false.

serviceAccountName string (Optional)

Name of a ServiceAccount for the runner Pod to provision Terraform resources. Default to tf-runner.

alwaysCleanupRunnerPod bool (Optional)

Clean the runner pod up after each reconciliation cycle

runnerTerminationGracePeriodSeconds int64 (Optional)

Configure the termination grace period for the runner pod. Use this parameter to allow the Terraform process to gracefully shutdown. Consider increasing for large, complex or slow-moving Terraform managed resources.

refreshBeforeApply bool (Optional)

RefreshBeforeApply forces refreshing of the state before the apply step.

runnerPodTemplate RunnerPodTemplate (Optional) enableInventory bool (Optional)

EnableInventory enables the object to store resource entries as the inventory for external use.

tfstate TFStateSpec (Optional) targets []string (Optional)

Targets specify the resource, module or collection of resources to target.

parallelism int32 (Optional)

Parallelism limits the number of concurrent operations of Terraform apply step. Zero (0) means using the default value.

storeReadablePlan string (Optional)

StoreReadablePlan enables storing the plan in a readable format.

webhooks []Webhook (Optional) dependsOn []github.com/fluxcd/pkg/apis/meta.NamespacedObjectReference (Optional) enterprise Kubernetes pkg/apis/apiextensions/v1.JSON (Optional)

Enterprise is the enterprise configuration placeholder.

planOnly bool (Optional)

PlanOnly specifies if the reconciliation should or should not stop at plan phase.

breakTheGlass bool (Optional)

BreakTheGlass specifies if the reconciliation should stop and allow interactive shell in case of emergency.

branchPlanner BranchPlanner (Optional)

BranchPlanner configuration.

remediation Remediation (Optional)

Remediation specifies what the controller should do when reconciliation fails. The default is to not perform any action.

status TerraformStatus"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.TerraformSpec","title":"TerraformSpec","text":"

(Appears on: Terraform)

TerraformSpec defines the desired state of Terraform

Field Description approvePlan string (Optional)

ApprovePlan specifies name of a plan wanted to approve. If its value is \u201cauto\u201d, the controller will automatically approve every plan.

destroy bool (Optional)

Destroy produces a destroy plan. Applying the plan will destroy all resources.

backendConfig BackendConfigSpec (Optional) backendConfigsFrom []BackendConfigsReference (Optional) cloud CloudSpec (Optional) workspace string (Optional) vars []Variable (Optional)

List of input variables to set for the Terraform program.

varsFrom []VarsReference (Optional)

List of references to a Secret or a ConfigMap to generate variables for Terraform resources based on its data, selectively by varsKey. Values of the later Secret / ConfigMap with the same keys will override those of the former.

values Kubernetes pkg/apis/apiextensions/v1.JSON (Optional)

Values map to the Terraform variable \u201cvalues\u201d, which is an object of arbitrary values. It is a convenient way to pass values to Terraform resources without having to define a variable for each value. To use this feature, your Terraform file must define the variable \u201cvalues\u201d.

tfVarsFiles []string (Optional)

TfVarsFiles loads all given .tfvars files. It copycats the -var-file functionality.

fileMappings []FileMapping (Optional)

List of all configuration files to be created in initialization.

interval Kubernetes meta/v1.Duration

The interval at which to reconcile the Terraform.

retryInterval Kubernetes meta/v1.Duration (Optional)

The interval at which to retry a previously failed reconciliation. The default value is 15 when not specified.

retryStrategy RetryStrategyEnum (Optional)

The strategy to use when retrying a previously failed reconciliation. The default strategy is StaticInterval and the retry interval is based on the RetryInterval value. The ExponentialBackoff strategy uses the formula: 2^reconciliationFailures * RetryInterval with a maximum requeue duration of MaxRetryInterval.

maxRetryInterval Kubernetes meta/v1.Duration (Optional)

The maximum requeue duration after a previously failed reconciliation. Only applicable when RetryStrategy is set to ExponentialBackoff. The default value is 24 hours when not specified.

path string (Optional)

Path to the directory containing Terraform (.tf) files. Defaults to \u2018None\u2019, which translates to the root path of the SourceRef.

sourceRef CrossNamespaceSourceReference

SourceRef is the reference of the source where the Terraform files are stored.

suspend bool (Optional)

Suspend is to tell the controller to suspend subsequent TF executions, it does not apply to already started executions. Defaults to false.

force bool (Optional)

Force instructs the controller to unconditionally re-plan and re-apply TF resources. Defaults to false.

readInputsFromSecrets []ReadInputsFromSecretSpec (Optional) writeOutputsToSecret WriteOutputsToSecretSpec (Optional)

A list of target secrets for the outputs to be written as.

disableDriftDetection bool (Optional)

Disable automatic drift detection. Drift detection may be resource intensive in the context of a large cluster or complex Terraform statefile. Defaults to false.

cliConfigSecretRef Kubernetes core/v1.SecretReference (Optional) healthChecks []HealthCheck (Optional)

List of health checks to be performed.

destroyResourcesOnDeletion bool (Optional)

Create destroy plan and apply it to destroy terraform resources upon deletion of this object. Defaults to false.

serviceAccountName string (Optional)

Name of a ServiceAccount for the runner Pod to provision Terraform resources. Default to tf-runner.

alwaysCleanupRunnerPod bool (Optional)

Clean the runner pod up after each reconciliation cycle

runnerTerminationGracePeriodSeconds int64 (Optional)

Configure the termination grace period for the runner pod. Use this parameter to allow the Terraform process to gracefully shutdown. Consider increasing for large, complex or slow-moving Terraform managed resources.

refreshBeforeApply bool (Optional)

RefreshBeforeApply forces refreshing of the state before the apply step.

runnerPodTemplate RunnerPodTemplate (Optional) enableInventory bool (Optional)

EnableInventory enables the object to store resource entries as the inventory for external use.

tfstate TFStateSpec (Optional) targets []string (Optional)

Targets specify the resource, module or collection of resources to target.

parallelism int32 (Optional)

Parallelism limits the number of concurrent operations of Terraform apply step. Zero (0) means using the default value.

storeReadablePlan string (Optional)

StoreReadablePlan enables storing the plan in a readable format.

webhooks []Webhook (Optional) dependsOn []github.com/fluxcd/pkg/apis/meta.NamespacedObjectReference (Optional) enterprise Kubernetes pkg/apis/apiextensions/v1.JSON (Optional)

Enterprise is the enterprise configuration placeholder.

planOnly bool (Optional)

PlanOnly specifies if the reconciliation should or should not stop at plan phase.

breakTheGlass bool (Optional)

BreakTheGlass specifies if the reconciliation should stop and allow interactive shell in case of emergency.

branchPlanner BranchPlanner (Optional)

BranchPlanner configuration.

remediation Remediation (Optional)

Remediation specifies what the controller should do when reconciliation fails. The default is to not perform any action.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.TerraformStatus","title":"TerraformStatus","text":"

(Appears on: Terraform)

TerraformStatus defines the observed state of Terraform

Field Description ReconcileRequestStatus github.com/fluxcd/pkg/apis/meta.ReconcileRequestStatus

(Members of ReconcileRequestStatus are embedded into this type.)

observedGeneration int64 (Optional)

ObservedGeneration is the last reconciled generation.

conditions []Kubernetes meta/v1.Condition (Optional) lastAppliedRevision string (Optional)

The last successfully applied revision. The revision format for Git sources is /. lastAttemptedRevision string (Optional)

LastAttemptedRevision is the revision of the last reconciliation attempt.

lastPlannedRevision string (Optional)

LastPlannedRevision is the revision used by the last planning process. The result could be either no plan change or a new plan generated.

lastPlanAt Kubernetes meta/v1.Time (Optional)

LastPlanAt is the time when the last terraform plan was performed

lastDriftDetectedAt Kubernetes meta/v1.Time (Optional)

LastDriftDetectedAt is the time when the last drift was detected

lastAppliedByDriftDetectionAt Kubernetes meta/v1.Time (Optional)

LastAppliedByDriftDetectionAt is the time when the last drift was detected and terraform apply was performed as a result

availableOutputs []string (Optional) plan PlanStatus (Optional) inventory ResourceInventory (Optional)

Inventory contains the list of Terraform resource object references that have been successfully applied.

lock LockStatus (Optional) reconciliationFailures int64 (Optional)

ReconciliationFailures is the number of reconciliation failures since the last success or update.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.Variable","title":"Variable","text":"

(Appears on: TerraformSpec)

Field Description name string

Name is the name of the variable

value Kubernetes pkg/apis/apiextensions/v1.JSON (Optional) valueFrom Kubernetes core/v1.EnvVarSource (Optional)"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.VarsReference","title":"VarsReference","text":"

(Appears on: TerraformSpec)

VarsReference contain a reference of a Secret or a ConfigMap to generate variables for Terraform resources based on its data, selectively by varsKey.

Field Description kind string

Kind of the values referent, valid values are (\u2018Secret\u2019, \u2018ConfigMap\u2019).

name string

Name of the values referent. Should reside in the same namespace as the referring resource.

varsKeys []string (Optional)

VarsKeys is the data key at which a specific value can be found. Defaults to all keys.

optional bool (Optional)

Optional marks this VarsReference as optional. When set, a not found error for the values reference is ignored, but any VarsKey or transient error will still result in a reconciliation failure.

"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.Webhook","title":"Webhook","text":"

(Appears on: TerraformSpec)

Field Description stage string enabled bool (Optional) url string payloadType string (Optional) errorMessageTemplate string (Optional) testExpression string"},{"location":"References/terraform/#infra.contrib.fluxcd.io/v1alpha2.WriteOutputsToSecretSpec","title":"WriteOutputsToSecretSpec","text":"

(Appears on: TerraformSpec)

WriteOutputsToSecretSpec defines where to store outputs, and which outputs to be stored.

Field Description name string

Name is the name of the Secret to be written

labels map[string]string (Optional)

Labels to add to the outputted secret

annotations map[string]string (Optional)

Annotations to add to the outputted secret

outputs []string (Optional)

Outputs contain the selected names of outputs to be written to the secret. Empty array means writing all outputs, which is default.

This page was automatically generated with gen-crd-api-reference-docs

"},{"location":"adr/0000-template/","title":"0. Title","text":"
  • Status: [proposed | rejected | accepted | deprecated | \u2026 | superseded by ADR-0005]
  • Date: 2020-10-29 [YYY-MM-DD - date of the decision]
  • Authors: [list of GitHub handles for the authors]
  • Deciders: [list of GitHub handles for those that made the decision]
"},{"location":"adr/0000-template/#context","title":"Context","text":""},{"location":"adr/0000-template/#decision","title":"Decision","text":""},{"location":"adr/0000-template/#consequences","title":"Consequences","text":""},{"location":"adr/0000-use-adrs-for-decisions/","title":"1. Use ADRs to record decisions","text":"
  • Status: proposed
  • Date: 2023-06-20
  • Authors: @yitsushi
  • Deciders: @yitsushi @chanwit @yiannistri
"},{"location":"adr/0000-use-adrs-for-decisions/#context","title":"Context","text":"

Decisions that affect the development of Terraform Controller that are not captured via a proposal need to be captured in some way. We need a method that is lightweight and easy to discover the decision that have been made. The record of decisions will help future contributors to the project to understand why something has been implemented or is done a certain way.

"},{"location":"adr/0000-use-adrs-for-decisions/#decision","title":"Decision","text":"

The project will use Architectural Decision Records (ADR) to record decisions that are made outside of a proposal.

A template has been created based on prior work:

  • https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions
  • https://adr.github.io/madr/
"},{"location":"adr/0000-use-adrs-for-decisions/#consequences","title":"Consequences","text":"

When decisions are made that affect the entire project then a new ADR needs to be created. Likewise, if a decision has been superseded then we need to capture this as a new ADR and mark the previous ADR as superseded. Maintainers and contributors will need to decide when an ADR is to be created.

"},{"location":"adr/0001-pr-polling-workflow/","title":"1. Pull Request Polling","text":"
  • Status: proposed
  • Date: 2023-06-20
  • Authors: @yitsushi
  • Deciders: @yitsushi @chanwit @yiannistri
"},{"location":"adr/0001-pr-polling-workflow/#context","title":"Context","text":"

To detect pull request changes, we can use webhooks or polling using GitHub's API.

"},{"location":"adr/0001-pr-polling-workflow/#decision","title":"Decision","text":"

We decided to start with polling for security reasons. Using webhooks would require users to open an ingress to the cluster. Because of this requirement, we think security conscious folks may refuse to roll this out on production clusters especially in an air-gapped environment. This does not mean that we will never consider using webhooks for this, but that initially, polling is what we have chosen to implement.

The Branch-Based Planner has two components:

  1. Polling Server: Detect Pull Request changes and manage Teraform resource state.
  2. Informer: Make a comment when new plan output is available.
"},{"location":"adr/0001-pr-polling-workflow/#consequences","title":"Consequences","text":"

The list Pull Requests endpoint returns all required fields to detect new and closed pull requests. It's one request per repository, but listing comments has to use an API request per pull request. So we have to add a mechanism to avoid hitting API rate limits.

"},{"location":"adr/0002-deny-cross-ns-by-default/","title":"2. Deny cross-namespace refs by default","text":"
  • Status: [ proposed | rejected | accepted | deprecated ]
  • Date: 2023-07-18
  • Authors: @squaremo
  • Deciders: [list of GitHub handles for those that made the decision]
"},{"location":"adr/0002-deny-cross-ns-by-default/#context","title":"Context","text":"

Like Flux, the tf-controller API has a handful of places where it accepts cross-namespace references.

  • Terraform.spec.sourceRef -- refers to the Flux source object with the Terraform program
  • Terraform.spec.dependsOn[] -- refers to other objects that must be ready before this object can be run
  • .data.resources[] -- in the config struct used by the branch planner

In general in Kubernetes, references to objects in other namespaces are frowned upon, because

  • they break namespace isolation assurances; and,
  • they encourage the proliferation of permissions.

Both of these effects make a system less secure.

However: removing cross-namespace refs entirely would break some installations in a way that would be difficult to fix, because Flux deployments often rely on defining sources away from objects that use them.

"},{"location":"adr/0002-deny-cross-ns-by-default/#decision","title":"Decision","text":"

Deny cross-namespace references by default, but allow them to be enabled with a flag.

So that the default value means the right thing, the flag name must be enable-cross-namespace-refs, and the default false. To avoid confusion when people try to use the Flux version of this flag --disable-cross-namespace-refs, it should be supported too, but only respected if supplied.

"},{"location":"adr/0002-deny-cross-ns-by-default/#consequences","title":"Consequences","text":"

The changed default will break deployments that rely on cross-namespace refs, but they are easily fixed with the flag.

New deployments will be more secure, by default.

"},{"location":"adr/0003-workspace-blob-caching/","title":"3. Workspace BLOB Caching","text":"
  • Status: [ proposed | rejected | accepted | deprecated ]
  • Date: 2023-09-20
  • Authors: @chanwit
  • Deciders: TBD
"},{"location":"adr/0003-workspace-blob-caching/#context","title":"Context","text":"

The TF-Controller currently faces challenges related to the deletion of Terraform resources. These problems span across three categories:

  1. Single object deletion,
  2. Resources with dependencies deletion, and
  3. Namespace deletion.

These problems must be fixed in the above order as (2) and (3) require single object deletion to be resolved first.

Deleting a single TF object can sometimes be obstructed because it's tied to other resources like Source objects, Secrets, and ConfigMaps. If we try to remove it without deleting these resources, the TF object gets stuck in an inconsistent state, making it harder for users to manage their infrastructure smoothly. Therefore, the TF-Controller is being enhanced to address this problem more efficiently, using the contents of generated Workspace BLOBs. Each BLOB contains all necessary information from the associated Source, Secrets, and ConfigMaps to ensure that TF-Controller finalization procedures can delete objects correctly.

Currently, the TF-Controller downloads a Source BLOB and pushes it to a tf-runner. The tf-runner processes this BLOB to create a Workspace file system. It generates a backend configuration file, variable files, and other necessary files for the Workspace file system, using data from associated Secrets and ConfigMaps. This newly created Workspace file system is then compressed, sent back to the TF-Controller, and stored as a Workspace BLOB in the controller's storage. A caching mechanism for these BLOBs is essential to fixing the single TF object deletion process.

"},{"location":"adr/0003-workspace-blob-caching/#decision","title":"Decision","text":"
  1. BLOB Creation and Storage
  2. A gRPC function named CreateWorkspaceBlob will be invoked by the TF-Controller to tell tf-runner to compress the Workspace file system into a tar.gz BLOB, which is then retrieved back to the controller.
  3. The caching mechanism will be executed right before the Terraform Initialization step, ensuring that the latest and most relevant data is used.
  4. Each Workspace Blob will be cached on the TF-Controller's local disk, using the UUID of the Terraform object as the filename,${uuid}.tar.gz.
  5. To reduce the risk of unauthorized access to the cache entries, and cache collisions, the cache file will be deleted after the finalization process is complete.
  6. Persistence
  7. The persistence mechanism used by the Source Controller will be adopted for the TF-Controller's persistence volume.
  8. BLOB Encryption
  9. The encryption and decryption of the BLOBs will be tasked to the runner, with the controller solely responsible for storing encrypted BLOBs.
  10. Each namespace will require a service account, preferably named \"tf-runner\".
  11. The token of this service account, which is natively supported by Kubernetes, will serve as the most appropriate encryption key because it's stored in a Secret, access to which can be controlled by RBAC. Storing it in a Secret also allows the key to be rotated.
  12. Security Measures (Based on STRIDE Analysis)
  13. Spoofing: Implement Kubernetes RBAC for access restrictions and use mutual authentication for gRPC communications.
  14. Tampering: Use checksums for integrity verification and 0600 permissions to write-protect local disk storage.
  15. Repudiation: Ensure strong logging and auditing mechanisms for tracking activities.
  16. Information Disclosure: Utilize robust encryption algorithms, rotate encryption keys periodically, and secure service account tokens.
  17. Denial of Service: Monitor storage space and automate cleanup processes.
  18. Elevation of Privilege: Minimize permissions associated with service account tokens.
  19. First MVP & Future Planning
  20. For the initial MVP, the default pod local volume will be used.
  21. Since a controller restart will erase the BLOB cache, consideration for using persistent volumes should be made for subsequent versions.
"},{"location":"adr/0003-workspace-blob-caching/#consequence","title":"Consequence","text":"
  1. With the implementation of this architecture:
  2. Single object deletions will succeed in circumstances in which they previously got stuck.
  3. Security measures will ensure the safety of the new Workspace BLOB storage mechanics, minimizing potential risks.
  4. Using the default pod local volume might limit storage capabilities and risk data loss upon controller restart. This warrants the need for considering persistent volumes in future versions.
  5. Encryption and security measures will demand regular maintenance and monitoring, especially concerning key rotations and integrity checks.
"},{"location":"branch-planner/","title":"Branch Planner Overview","text":"

The GitOps methodology streamlines infrastructure provisioning and management, using Git as the source of truth. The Branch Planner, a component of TF-Controller, aims to take this a step further by allowing developers and operations teams to plan Terraform configurations on a branch that's separate from the main branch. This makes it easier to review and understand the potential impact of your changes before you run terraform apply.

The Branch Planner's most important feature is its seamless integration with the PR (Pull Request) user interface. When enabled through Helm values, it watches repositories that contain Terraform resources at regular intervals\u2014checking their referenced Source, and polling for Pull Requests using GitHub's API and the provided token. When changes are proposed on a new branch, Branch Planner runs a plan in the cluster and displays the results directly as comments on your PR. Once you're satisfied with the results, you can merge your branch into the main branch to trigger the TF-Controller to reconcile the updated code.

"},{"location":"branch-planner/#replan-commands","title":"Replan commands","text":"

The Branch Planner also allows users to manually trigger the replan process. By simply commenting !replan under the PR, the Branch Planner will be instructed to generate a new plan and post it under the PR as a new comment.

Now that you know what Branch Planner can do for you, follow the guide to get started.

"},{"location":"branch-planner/branch-planner-getting-started/","title":"Getting Started With Branch Planner","text":""},{"location":"branch-planner/branch-planner-getting-started/#prerequisites","title":"Prerequisites","text":"
  1. Flux is installed on the cluster.
  2. A GitHub API token. For public repositories, it's sufficient to enable Public Repositories without any additional permissions. For private repositories, you need the following permissions:
  3. Pull requests with Read-Write access. This is required to check Pull Request changes, list comments, and create or update comments.
  4. Metadata with Read-only access. This is automatically marked as \"mandatory\" because of the permissions listed above.
  5. General knowledge about Tofu-Controller (see docs).
"},{"location":"branch-planner/branch-planner-getting-started/#quick-start","title":"Quick Start","text":"

This section describes how to install Branch Planner using a HelmRelease object in the flux-system namespace with minimum configuration on a KinD cluster.

  1. Create a KinD cluster. 

    kind create cluster\n

  2. Install Flux. Make sure you have the latest version of Flux (v2 GA).

flux install\n
  1. Create a secret that contains a GitHub API token. If you do not use the gh CLI, copy and paste the token from GitHub's website.
export GITHUB_TOKEN=$(gh auth token)\n\nkubectl create secret generic branch-planner-token \\\n    --namespace=flux-system \\\n    --from-literal=\"token=${GITHUB_TOKEN}\"\n
  1. Install Branch Planner from a HelmRelease provided by the TF-Controller repository. Use TF-Controller v0.16.0-rc.2 or later.
kubectl apply -f https://raw.githubusercontent.com/weaveworks/tf-controller/fa4b3b85d316340d897fda4fed757265ba2cd30e/docs/branch_planner/release.yaml\n
  1. Create a Terraform object with a Source pointing to a repository. Your repository must contain a Terraform file\u2014for example, main.tf. Check out this demo for an example.
export GITHUB_USER=<your user>\nexport GITHUB_REPO=<your repo>\n\ncat <<EOF | kubectl apply -f -\n---\napiVersion: source.toolkit.fluxcd.io/v1\nkind: GitRepository\nmetadata:\n  name: branch-planner-demo\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: https://github.com/${GITHUB_USER}/${GITHUB_REPO}\n  ref:\n    branch: main\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: branch-planner-demo\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  path: ./\n  interval: 1m\n  sourceRef:\n    kind: GitRepository\n    name: branch-planner-demo\n    namespace: flux-system\nEOF\n
  1. Now you can create a pull request on your GitHub repo. The Branch Planner will create a new Terraform object with the plan-only mode enabled and will generate a new plan for you. It will post the plan as a new comment in the pull request.
"},{"location":"branch-planner/branch-planner-getting-started/#configure-branch-planner","title":"Configure Branch Planner","text":"

Branch Planner uses a ConfigMap as configuration. The ConfigMap is optional but useful for fine-tuning Branch Planner.

"},{"location":"branch-planner/branch-planner-getting-started/#configuration","title":"Configuration","text":"

By default, Branch Planner will look for the branch-planner ConfigMap in the same namespace as where the TF-Controller is installed. That ConfigMap allows users to specify which Terraform resources in a cluster the Brach Planner should monitor.

The ConfigMap has two fields:

  1. secretName, which contains the API token to access GitHub.
  2. resources, which defines a list of resources to watch.
---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  namespace: flux-system\n  name: branch-planner\ndata:\n  secretName: branch-planner-token\n  resources: |-\n    - namespace: terraform\n    - namespace: flux-system\n
"},{"location":"branch-planner/branch-planner-getting-started/#secret","title":"Secret","text":"

Branch Planner uses the referenced Secret with a token field that acquires the API token to fetch pull request information.

kubectl create secret generic branch-planner-token \\\n    --namespace=flux-system \\\n    --from-literal=\"token=${GITHUB_TOKEN}\"\n
"},{"location":"branch-planner/branch-planner-getting-started/#resources","title":"Resources","text":"

If the resources list is empty, nothing will be watched. The resource definition can be exact or namespace-wide.

With the following configuration file, the Branch Planner will watch all Terraform objects in the terraform namespace, and the exact-terraform-object Terraform object in default namespace.

data:\n  resources:\n    - namespace: default\n      name: exact-terraform-object\n    - namespace: terraform\n
"},{"location":"branch-planner/branch-planner-getting-started/#default-configuration","title":"Default Configuration","text":"

If no ConfigMap is found, the Branch Planner will not watch any namespaces for Terraform resources and look for a GitHub token in a secret named branch-planner-token in the flux-system namespace. Supplying a secret with a token is a necessary task, otherwise Branch Planner will not be able to interact with the GitHub API.

"},{"location":"branch-planner/branch-planner-getting-started/#enable-branch-planner","title":"Enable Branch Planner","text":"

To enable branch planner, set the branchPlanner.enabled to true in the Helm values files.

---\nbranchPlanner:\n  enabled: true\n
"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/","title":"Branch Planner and Terraform Cloud Integration","text":""},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#branch-planner-and-terraform-cloud-integration-getting-started","title":"Branch Planner and Terraform Cloud Integration: Getting Started","text":"

With Branch Planner, you can provision the main branch directly on Terraform Cloud. TF-Controller communicates with Terraform Cloud to run the necessary plans and apply your approved code. The state is securely stored on Terraform Cloud.

Note: For now, Branch Planner only supports GitHub as the Git provider. We plan to add other Git providers in time.

"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-by-step-guide","title":"Step-by-step Guide","text":""},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-1-create-a-terraform-cloud-token","title":"Step 1: Create a Terraform Cloud Token","text":"

Use the terraform login command to obtain a Terraform Cloud token. Then use the token to create a Kubernetes Secret.

kubectl create secret generic \\\n  tfc-cli-config \\\n  --namespace=flux-system \\\n  --from-file=terraform.tfrc=/dev/stdin << EOF\ncredentials \"app.terraform.io\" {\n  token = \"xxxxxxxxxxxxxx.atlasv1.zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz\"           \n}\nEOF\n
"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-2-create-a-terraform-cr","title":"Step 2: Create a Terraform CR","text":"

Create a Terraform CR to automatically plan and apply Terraform configurations on Terraform Cloud. In this example, the Branch Planner reads the Terraform configurations from a Git repository to plan, apply, and store the state in a Terraform Cloud workspace.

The token from Step 1 is specified as the value of spec.cliConfigSecretRef and is used to authenticate with Terraform Cloud.

---\napiVersion: source.toolkit.fluxcd.io/v1\nkind: GitRepository\nmetadata:\n  name: branch-planner-demo\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: https://github.com/tf-controller/branch-planner-demo\n  ref:\n    branch: main\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: branch-planner-tfc\n  namespace: flux-system\nspec:\n  interval: 2m\n  approvePlan: auto\n  cloud:\n    organization: weaveworks\n    workspaces:\n      name: branch-planner-tfc\n  cliConfigSecretRef:\n    name: tfc-cli-config\n    namespace: flux-system\n  vars:\n  - name: subject\n    value: \"world\"\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: branch-planner-demo\n    namespace: flux-system\n
"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-3-edit-file-create-a-branch-and-open-a-pull-request","title":"Step 3: Edit File, Create a Branch, and Open a Pull Request","text":"

  1. Navigate to Your Repository: Open a web browser and visit your GitHub repository. For our example, navigate here.

  2. Locate the File to Edit: Browse through the repository's file structure and click on the Terraform configuration file you wish to edit.

  3. Edit the File: Click on the pencil icon (edit) located on the top right of the file content. Make your desired changes to the Terraform configurations. For instance, you might change the \"Hello World\" content in the main.tf file.

Once you've made your edits, scroll down to prepare to commit the changes.

  1. Commit the Changes to a New Branch: Instead of committing directly to the main branch, choose the option to \"Create a new branch\" for this commit and start a pull request. Name the branch something descriptive\u2014for example, change-hello-world-message.

Click on the \"Propose Changes\" button.

  1. Open a Pull Request (PR): After proposing your changes, you'll be led to the \"Open a pull request\" page. Fill in the details of your PR, explaining the changes you made, their purpose, and any other pertinent information.

Click on the \"Create Pull Request\" button.

  1. Review Terraform Plan in PR Comments: Once the PR is created, the Branch Planner will trigger a Terraform plan. After the plan is completed, the results will be posted as a comment on the PR. This enables you and your team to review the expected changes before they're applied.
"},{"location":"branch-planner/branch-planner-tfc-integration-getting-started/#step-4-review-approve-and-merge-the-pull-request","title":"Step 4: Review, Approve and Merge the Pull Request","text":"

  1. Review the Changes:

    • Navigate to the Pull Requests tab in your GitHub repository.
    • Click on the title of your pull request to see the details.
    • Examine the Files changed section to see the exact modifications made to the Terraform configurations.
    • Check the comments for the Terraform plan output generated by Branch Planner. Ensure the plan matches your expectations.

  2. Iterate on Changes if Necessary:

    • If you spot any discrepancies or wish to make further adjustments, click on the file in the Files changed section.
    • After making the desired edits, commit the changes to the same branch. This will automatically prompt TF-Controller and Branch Planner to generate a new plan.
    • If, for any reason, the automatic replan doesn't occur or you believe there might be an inconsistency, you can manually trigger a new plan by commenting !replan on the PR. Branch Planner will then process the request and display the new plan results.

  3. Approve the Changes:

    • If you're content with the changes and the associated Terraform plan, move to the Review changes button on the PR page.
    • Select the Approve option from the dropdown and optionally add any final comments.
    • Click Submit review to finalize your approval.

  4. Merge the Pull Request:

    • With the changes approved, click on the Merge pull request button.
    • Choose your desired merge strategy from the options provided, such as \"Squash and merge\" or \"Rebase and merge\".
    • Click Confirm merge.
    • Following the merge, TF-Controller will take over. It will send the updated Terraform configuration to Terraform Cloud, where the changes will be planned and then applied. The resulting infrastructure state will be securely stored within your Terraform Cloud workspace.
"},{"location":"use-tf-controller/","title":"Use TF-controller","text":"
  • Use TF-controller to provision resources and auto approve
  • Use TF-controller to plan and manually apply Terraform resources
  • Use TF-controller to provision resources and obtain outputs
  • Use TF-controller to detect drifts only without plan or apply
  • Use TF-controller with drift detection disabled
  • Use TF-controller with AWS EKS IRSA
  • Use TF-controller to set variables for Terraform resources
  • Use TF-controller with a custom backend
  • Use TF-controller with an OCI Artifact as Source
  • Use TF-controller to provision Terraform resources that are required health checks
  • Use TF-controller to provision resources and destroy them when the Terraform object gets deleted
  • Use TF-controller to force unlock Terraform states
  • Use TF-controller with Terraform Runners enabled via Env Variables
  • Use TF-controller to provision resources with customized Runner Pods
  • Use TF-controller with Terraform Enterprise
  • Use TF-controller with Terraform Private Registries
  • Use TF-controller with primitive modules
  • Use TF-controller with GitOps dependency management
  • Use TF-controller with the ready-to-use AWS package
  • User TF-controller with plan-only mode
  • Use TF-controller with external webhooks
  • Use TF-controller with Terraform Runners exposed via hostname/subdomain
  • How to backup and restore a Terraform state
  • How to build and use a custom runner image
  • How to integrate with Flux Receivers and Alerts?
  • How does the interval and retryInterval work?
  • How does the resource deletion work?
  • How to troubleshoot with Break the Glass mode
  • How to enable cross-namespace references
  • How to run TF-controller in Azure Kubernetes Service
  • How to upgrade TF-controller to a newer version
"},{"location":"use-tf-controller/backup-and-restore-a-Terraform-state/","title":"Backup and restore a Terraform state","text":""},{"location":"use-tf-controller/backup-and-restore-a-Terraform-state/#backup-the-tfstate","title":"Backup the tfstate","text":"

Assume that we have the my-stack Terraform object with its .spec.workspace set to \"default\".

kubectl get terraform\n\nNAME       READY     STATUS         AGE\nmy-stack   Unknown   Initializing   28s\n

We can backup its tfstate out of the cluster, like this:

WORKSPACE=default\nNAME=my-stack\n\nkubectl get secret tfstate-${WORKSPACE}-${NAME} \\\n  -ojsonpath='{.data.tfstate}' \\\n  | base64 -d | gzip -d > terraform.tfstate\n
"},{"location":"use-tf-controller/backup-and-restore-a-Terraform-state/#restore-the-tfstate","title":"Restore the tfstate","text":"

To restore the tfstate file or import an existing tfstate file to the cluster, we can use the following operation:

gzip terraform.tfstate\n\nWORKSPACE=default\nNAME=my-stack\n\nkubectl create secret \\\n  generic tfstate-${WORKSPACE}-${NAME} \\\n  --from-file=tfstate=terraform.tfstate.gz \\\n  --dry-run=client -o=yaml \\\n  | yq e '.metadata.annotations[\"encoding\"]=\"gzip\"' - \\\n  > tfstate-${WORKSPACE}-${NAME}.yaml\n\nkubectl apply -f tfstate-${WORKSPACE}-${NAME}.yaml\n
"},{"location":"use-tf-controller/build-and-use-a-custom-runner-image/","title":"Build and Use a Custom Runner Image","text":"

To build a custom runner image, you need a Dockerfile that extends the base image and that adds Terraform, plus any additional required tooling. The repository that contains the base images is here. All base image tags follow the following format: ${TF_CONTROLLER_VERSION}-base.

"},{"location":"use-tf-controller/build-and-use-a-custom-runner-image/#prerequisites","title":"Prerequisites","text":"

You need Docker and Git to build the image.

"},{"location":"use-tf-controller/build-and-use-a-custom-runner-image/#build-the-image","title":"Build the Image","text":"
  1. Create a Dockerfile that extends the base image and that adds Terraform, plus any additional required tooling. For example:
ARG BASE_IMAGE\nFROM $BASE_IMAGE\n\nARG TARGETARCH\nARG TF_VERSION=1.5.7\n\n# Switch to root to have permissions for operations\nUSER root\n\nADD https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_${TARGETARCH}.zip /terraform_${TF_VERSION}_linux_${TARGETARCH}.zip\nRUN unzip -q /terraform_${TF_VERSION}_linux_${TARGETARCH}.zip -d /usr/local/bin/ && \\\n    rm /terraform_${TF_VERSION}_linux_${TARGETARCH}.zip && \\\n    chmod +x /usr/local/bin/terraform\n\n# Switch back to the non-root user after operations\nUSER 65532:65532\n

Find the original Dockerfile for the runner here.

  1. Build the image from the directory containing the Dockerfile you created above:
export TF_CONTROLLER_VERSION=v0.16.0-rc.3\nexport TF_VERSION=1.5.7\nexport BASE_IMAGE=ghcr.io/flux-iac/tf-runner:${TF_CONTROLLER_VERSION}-base\nexport TARGETARCH=amd64\nexport REMOTE_REPO=ghcr.io/my-org/custom-runnner\ndocker build \\\n    --build-arg BASE_IMAGE=${BASE_IMAGE} \\\n    --build-arg TARGETARCH=${TARGETARCH} \\\n    --tag my-custom-runner:${TF_CONTROLLER_VERSION} .\ndocker tag my-custom-runner:${TF_CONTROLLER_VERSION} $REMOTE_REPO:${TF_CONTROLLER_VERSION}\ndocker push $REMOTE_REPO:${TF_CONTROLLER_VERSION}\n

Replace the relevant values above with the corresponding values in your organisation/implementation.

  1. Update the values.runner.image values in the TF-Controller Helm chart values to point to the new image:
values:\n  runner:\n    image:\n      repository: ghcr.io/my-org/custom-runnner\n      tag: v0.16.0-rc.3\n
  1. Commit and push the changes to Git. Confirm that the HelmRelease has been updated:
kubectl get deployments.apps -n flux-system tf-controller -o jsonpath='{.spec.template.spec.containers[*]}' | jq '.env[] | select(.name == \"RUNNER_POD_IMAGE\")'\n{\n  \"name\": \"RUNNER_POD_IMAGE\",\n  \"value\": \"ghcr.io/my-org/custom-runner:v0.16.0-rc3\"\n}\n
"},{"location":"use-tf-controller/build-and-use-a-custom-runner-image/#references","title":"References","text":"

A set of GitHub actions in the TF-Controller community repo facilitates a process similar to the above, but uses GitHub Actions to build and push the image.

"},{"location":"use-tf-controller/detect-drifts-only-without-plan-or-apply/","title":"Detect drifts only without plan or apply","text":""},{"location":"use-tf-controller/detect-drifts-only-without-plan-or-apply/#use-tf-controller-to-detect-drifts-only-without-plan-or-apply","title":"Use TF-Controller to detect drifts only without plan or apply","text":"

We can set .spec.approvePlan to disable to tell the controller to detect drifts of your Terraform resources only. Doing so will skip the plan and apply stages.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: hello-world\n  namespace: flux-system\nspec:\n  approvePlan: disable\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/detect-drifts-only-without-plan-or-apply/#troubleshooting","title":"Troubleshooting","text":""},{"location":"use-tf-controller/detect-drifts-only-without-plan-or-apply/#when-terraform-resource-detects-drift-but-no-plan-is-generated-for-approval","title":"When Terraform resource detects drift, but no plan is generated for approval","text":"

In this situation, you may not have spec.approvePlan set to disable. Try setting spec.approvePlan: auto and using tfctl replan to trigger a replan. After the drift disappears, you can set the spec.approvePlan: \"\" to get into the manual mode again.

"},{"location":"use-tf-controller/flux-receiver-and-alert/","title":"Integrate with Flux Receivers and Alerts","text":"

You can customize your Flux installation to use Flux API resources like Receivers and Alerts with third-party custom resource definitions such as the Terraform API CRD.

You will need to add a patch to the kustomization.yaml in your Flux cluster installation's bootstrap manifests. Find it under the flux-system directory.

"},{"location":"use-tf-controller/flux-receiver-and-alert/#enable-notifications-for-third-party-controllers","title":"Enable Notifications for Third-Party Controllers","text":"

Enable notifications for 3rd party Flux controllers such as tf-controller:

apiVersion: kustomize.config.k8s.io/v1beta1\nkind: Kustomization\nresources:\n  - gotk-components.yaml\n  - gotk-sync.yaml\npatches:\n  - patch: |\n      - op: add\n        path: /spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/eventSources/items/properties/kind/enum/-\n        value: Terraform\n      - op: add\n        path: /spec/versions/1/schema/openAPIV3Schema/properties/spec/properties/eventSources/items/properties/kind/enum/-\n        value: Terraform\n    target:\n      kind: CustomResourceDefinition\n      name:  alerts.notification.toolkit.fluxcd.io\n  - patch: |\n      - op: add\n        path: /spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/resources/items/properties/kind/enum/-\n        value: Terraform\n      - op: add\n        path: /spec/versions/1/schema/openAPIV3Schema/properties/spec/properties/resources/items/properties/kind/enum/-\n        value: Terraform\n    target:\n      kind: CustomResourceDefinition\n      name:  receivers.notification.toolkit.fluxcd.io\n  - patch: |\n      - op: add\n        path: /rules/-\n        value:\n          apiGroups: [ 'infra.contrib.fluxcd.io' ]\n          resources: [ '*' ]\n          verbs: [ '*' ]\n    target:\n      kind: ClusterRole\n      name:  crd-controller-flux-system\n
"},{"location":"use-tf-controller/force-unlock-terraform-states/","title":"Use TF-Controller to force unlock Terraform states","text":"

In some situations, you may need to perform the Terraform force-unlock operation on the tfstate inside the cluster.

There are three possible values of .spec.tfstate.forceUnlock, which are yes, no, and auto. The default value is no, which means that you disable this behaviour.

The auto force-unlock mode will automatically use the lock identifier produced by the associated state file instead of the specified lock identifier.

The recommended way is to do manual force unlock. To manually force-unlock, you need to:

  1. set forceUnlock to yes, and
  2. specify a lock identifier to unlock a specific locked state,

as the following example:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  tfstate:\n    forceUnlock: \"yes\"\n    lockIdentifier: f2ab685b-f84d-ac0b-a125-378a22877e8d\n
"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/","title":"Terraform Enterprise and Terraform Cloud Integration","text":"

Terraform is a secure and robust platform designed to store the Terraform states for your production systems. When working with Infrastructure as Code, managing and ensuring the state is both secure and consistent is critical.

TF-Controller supports both Terraform Cloud and Terraform Enterprise. The spec.cloud in the Terraform CRD enables users to integrate their Kubernetes configurations with Terraform workflows.

To get started, simply place your Terraform Cloud token in a Kubernetes Secret and specify it in the spec.cliConfigSecretRef field of the Terraform CR. The spec.cloud field specifies the organization and workspace name.

"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-enterprise","title":"Terraform Enterprise","text":"

Here are the steps to set up TF-Controller for your TFE instance.

"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-login","title":"Terraform Login","text":"

First, you need to obtain an API token from your TFE. You can use terraform login command to do so.

terraform login tfe.dev.example.com\n

Then you can find your API token inside $HOME/.terraform.d/credentials.tfrc.json. Content of the file will look like this:

{\n  \"credentials\": {\n    \"tfe.dev.example.com\": {\n      \"token\": \"mXXXXXXXXX.atlasv1.ixXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n    }\n  }\n}\n
"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#prepare-an-tfrc-file","title":"Prepare an TFRC file","text":"

TF-Controller accepts an TFRC file in the HCL format. So you have to prepare terraform.tfrc file using contents from above. 

credentials \"tfe.dev.example.com\" {\n  token = \"mXXXXXXXXX.atlasv1.ixXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n}\n

"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#create-a-secret","title":"Create a Secret","text":"

We will now create a Kubernetes Secret from yourterraform.tfrc file, name it tfe-cli-config and put it inside the flux-system namespace.

kubectl create secret generic \\\n  tfe-cli-config \\\n  --namespace=flux-system \\\n  --from-file=terraform.tfrc=./terraform.tfrc\n
"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-object","title":"Terraform Object","text":"

In your Terraform object, you'll have to 1. disable the backend by setting spec.backendConfig.disable: true, and 2. point spec.cliConfigSecretRef: to the Secret created in the previous step, like this:

---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: tfe-demo\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 2m\n  path: ./terraform/tfe-demo\n  backendConfig:\n    disable: true\n  cliConfigSecretRef:\n    name: tfe-cli-config\n    namespace: flux-system\n  vars:\n  - name: subject\n    value: World\n  sourceRef:\n    kind: GitRepository\n    name: flux-system\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: tfe-helloworld-output\n    outputs:\n    - greeting\n
"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-module","title":"Terraform Module","text":"

Don't forget that you need to tell your Terraform model to use your enterprise instance as well. Here's an example, 

terraform {\n  required_version = \">= 1.1.0\"\n  cloud {\n    hostname = \"tfe.dev.example.com\"\n    organization = \"weaveworks\"\n\n    workspaces {\n      name = \"dev\"\n    }\n  }\n}\n\nvariable \"subject\" {\n   type = string\n   default = \"World\"\n   description = \"Subject to hello\"\n}\n\noutput \"greeting\" {\n  value = \"Hello ${var.subject} from Terraform Enterprise\"\n}\n

"},{"location":"use-tf-controller/integration-with-terraform-enterprise-or-cloud/#terraform-cloud","title":"Terraform Cloud","text":"

TF-Controller can send your Terraform resources to be planned and applied via Terraform Cloud. States are automatically stored in your Terraform Cloud's workspace. To use TF-Controller with Terraform Cloud, replace your hostname to app.terraform.io. Also, set spec.approvalPlan to auto.

Here's how the configuration looks:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: branch-planner-tfc\n  namespace: flux-system\nspec:\n  interval: 2m\n  approvePlan: auto\n  cloud:\n    organization: weaveworks\n    workspaces:\n      name: branch-planner-tfc\n  cliConfigSecretRef:\n    name: tfc-cli-config\n    namespace: flux-system\n
"},{"location":"use-tf-controller/integration-with-terraform-private-registries/","title":"Terraform Private Registries Integration","text":"

Using Terraform private registries with the tofu-controller is exactly as you would use them directly via Terraform. For example, you would like to use the tofu-controller to deploy code that contains the following module: 

module \"vpc\" {\n  source  = \"my.private.server/terraform-modules/path/to/module\"\n  version = \"1.2.3\"\n\n  ...\n  ...\n}\n
without configuring the terraform login process, deploying the module with the controller will result in the error: 
Failed to retrieve available versions for module \"vpc\" (main.tf:1) from\nmy.private.server: error looking up module versions: 401 Unauthorized.\n

"},{"location":"use-tf-controller/integration-with-terraform-private-registries/#terraform-login","title":"Terraform Login","text":"

As a human you would normally execute terraform login my.private.server to obtain a token from the registry, with the tofu-controller use the native terraform credentials configs instead.

Obtain a token from your private registry, then follow one of the below options:

"},{"location":"use-tf-controller/integration-with-terraform-private-registries/#using-credentials-file","title":"Using credentials file","text":"

content of credentials.tfrc should look like: 

{\n  \"credentials\": {\n    \"my.private.server\": {\n      \"token\": \"TOP_SECRET_TOKEN\"\n    }\n  }\n}\n

K8S secret example: 

apiVersion: \"v1\"\nkind: \"Secret\"\nmetadata:\n  name: tf-private-config\ntype: \"Opaque\"\nstringData:\n  credentials.tfrc: |-\n    {\n      \"credentials\": {\n        \"my.private.server\": {\n          \"token\": \"TOP_SECRET_TOKEN\"\n        }\n      }\n    }\n
Then deploy the Terraform object, while referencing the above tf-private-config secret 
apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: tf-private-demo\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 2m\n  path: ./terraform/tf-private-demo\n  cliConfigSecretRef:\n    name: tf-private-config\n    namespace: flux-system\n  sourceRef:\n    kind: GitRepository\n    name: flux-system\n    namespace: flux-system\n

"},{"location":"use-tf-controller/integration-with-terraform-private-registries/#using-environment-variables","title":"Using environment variables","text":"

Another option is to use environment variable credentials, Terraform object should look like: 

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: tf-private-demo\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 2m\n  path: ./terraform/tf-private-demo\n  sourceRef:\n    kind: GitRepository\n    name: flux-system\n    namespace: flux-system\n  # api referance https://flux-iac.github.io/tofu-controller/References/terraform/#infra.contrib.fluxcd.io/v1alpha2.RunnerPodTemplate\n  runnerPodTemplate:\n    spec:\n      env:\n        - name: \"TF_TOKEN_my_private_server\"\n          value: \"TOP_SECRET_TOKEN\"\n      # or use get ENV from existing secret\n      envFrom:\n        - secretRef:\n            name: tf-private-token\n

"},{"location":"use-tf-controller/interval-and-retryInterval/","title":"Troubleshooting with Interval and retryInterval","text":""},{"location":"use-tf-controller/interval-and-retryInterval/#overview","title":"Overview","text":"

This document describes the requeue behavior of the Reconcile method in the TerraformReconciler struct in the code base. Understanding these behaviors can be crucial for troubleshooting, as well as for future development and refinement of the system.

"},{"location":"use-tf-controller/interval-and-retryInterval/#requeue-behaviors","title":"Requeue Behaviors","text":"

The Reconcile method has several requeue behaviors based on different conditions and errors. We will group them into four categories based on their requeue behavior:

"},{"location":"use-tf-controller/interval-and-retryInterval/#1-immediate-requeue-not-using-specified-interval-retryinterval","title":"1. Immediate Requeue (Not using specified interval / retryInterval)","text":"

In these scenarios, the Reconcile method returns an error which leads to an immediate requeue orchestrated by the Controller Runtime. The interval is based on the controller's configuration and not specified in the method itself:

  • When there's an error retrieving the Terraform object from the Kubernetes API.
  • After adding the finalizer, if there's an error in patching the Terraform object.
  • If there's a non-access-denied error in retrieving the source object.
  • When the ready condition is unknown or the status of the ready condition isn't unknown, and there's an error in patching the Terraform object.
  • In multiple situations where there's an error in patching the Terraform object's status.
  • If there's an error in creating or looking up the runner.
  • If there's an error while attempting to finalize the Terraform object.
"},{"location":"use-tf-controller/interval-and-retryInterval/#2-requeue-after-a-specific-interval-specretryinterval","title":"2. Requeue After a Specific Interval (spec.retryInterval)","text":"

In these scenarios, the method specifically asks for a requeue after a certain interval specified by spec.retryInterval (default to 15s).

  • The Terraform object is being deleted but there are still dependent resources that haven't been deleted.
  • The source object specified by spec.sourceRef is not found.
  • The source object doesn't have an associated artifact.
  • The dependencies do not meet the ready condition.
  • There's an error during the main reconciliation process.
  • Drift is detected during the reconciliation process.
"},{"location":"use-tf-controller/interval-and-retryInterval/#3-requeue-after-a-specific-interval-specinterval","title":"3. Requeue After a Specific Interval (spec.interval)","text":"

In this scenario, the method specifically asks for a requeue after a successful reconciliation:

The interval for the requeue is spec.interval.

"},{"location":"use-tf-controller/interval-and-retryInterval/#4-no-requeue-wait-for-manual-intervention","title":"4. No Requeue, wait for manual intervention","text":"

In these scenarios, the method returns without asking for a requeue, and the Controller Runtime will stop the reconciliation process until there is a manual intervention:

  • Access is denied when retrieving the source object.
  • The status of the plan is pending, and it's not set to force or auto-apply.
"},{"location":"use-tf-controller/plan-and-manually-apply-terraform-resources/","title":"Use TF-controller to plan and manually apply Terraform resources","text":"

Assume that you have a GitRepository object named helloworld pointing to a Git repository, and you want to plan and apply the Terraform resources under ./ of that Git repo. Let's walk through the steps of using TF-Controller to plan and manually apply Terraform resources.

  • Create a Terraform object and set the necessary fields in the spec:
  • approvePlan, which sets the mode. For plan and manual approval mode, either keep this field blank or omit it entirely.
  • interval, which determines how often TF-Controller will run the Terraform configuration
  • path, which specifies the location of the configuration files, in this case ./
  • sourceRef, which points to the helloworld GitRepository object
  • Once this object is created, use kubectl to obtain the approvePlan value and set it in the Terraform object.
  • After making our changes and pushing them to the Git repository, TF-Controller will apply the plan and create the real resources.

Here is an example:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: \"\" # or you can omit this field\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/plan-and-manually-apply-terraform-resources/#view-the-approval-message","title":"View the approval message","text":"

After a reconciliation loop, TF-Controller will generate a plan. Run this command to receive the .spec.approvePlan value from TF-Controller, which you'll need to approve the plan:

kubectl -n flux-system get tf/helloworld\n

This value enables you to edit the Terraform object file and set the spec.approvePlan field to the value obtained from the message.

After making your changes and pushing them to the Git repository, TF-Controller will apply the plan and create the real resources.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: hello-world\n  namespace: flux-system\nspec:\n  approvePlan: plan-main-b8e362c206 # first 8 digits of a commit hash is enough\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/provision-Terraform-resources-that-are-required-health-checks/","title":"Use TF-Controller to provision Terraform resources that are required health checks","text":"

For some Terraform resources, it may be useful to perform health checks on them to verify that they are ready to accept connection before the terraform goes into Ready state:

For example, our Terraform file is provisioned and contains the following outputs.

# main.tf\n\noutput \"rdsAddress\" {\n  value = \"mydb.xyz.us-east-1.rds.amazonaws.com\"\n}\n\noutput \"rdsPort\" {\n  value = \"3306\"\n}\n\noutput \"myappURL\" {\n  value = \"https://example.com/\"\n}\n

We can use standard Go template expressions, like ${{ .rdsAddress }}, to refer to those output values and use them to verify that the resources are up and running.

We support two types of health checks, tcp amd http. The tcp type allows us to verify a TCP connection, while the http type is for verify an HTTP URL. The default timeout of each health check is 20 seconds.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  healthChecks:\n    - name: rds\n      type: tcp\n      address: ${{ .rdsAddress }}:${{ .rdsPort }} \n      timeout: 10s # optional, defaults to 20s\n    - name: myapp\n      type: http\n      url: ${{ .myappURL }}\n      timeout: 5s\n    - name: url_not_from_output\n      type: http\n      url: \"https://example.org\"\n
"},{"location":"use-tf-controller/provision-resources-and-auto-approve/","title":"Use TF-Controller to provision resources and auto approve","text":"

To provision resources with TF-Controller, you need to create a Terraform object and a Flux source object, such as a GitRepository or OCIRepository object.

"},{"location":"use-tf-controller/provision-resources-and-auto-approve/#create-a-terraform-object","title":"Create a Terraform object","text":"

The Terraform object is a Kubernetes custom resource definition (CRD) object. It is the core object of TF-Controller and defines the Terraform module, backend configuration, and GitOps automation mode.

The Terraform module is a Terraform configuration that you can use to provision resources. It can either be placed inside a Git repository, or packaged as an OCI image in an OCI registry.

The backend configuration is the configuration for the Terraform backend to be used to store the Terraform state. It is optional. If not specified, the Kubernetes backend will be used by default.

"},{"location":"use-tf-controller/provision-resources-and-auto-approve/#gitops-automation-mode","title":"GitOps Automation mode","text":"

Use the GitOps automation mode to run the Terraform module. It determines how Terraform runs and manages your infrastructure. It is optional. If not specified, the \"plan-and-manually-apply\" mode is used by default. In the \"plan-and-manually-apply\" mode, TF-Controller will run a Terraform plan and output the proposed changes to a Git repository. A human must then review and manually apply the changes.

In the \"auto-apply\" mode, TF-Controller will automatically apply the changes after a Terraform plan is run. This can be useful for environments where changes can be made automatically, but it is important to ensure that the proper controls, like policies, are in place to prevent unintended changes from being applied.

To specify the GitOps automation mode in a Terraform object, set the spec.approvePlan field to the desired value. For example, to use the \"auto-apply\" mode, set it to spec.approvePlan: auto.

It is important to carefully consider which GitOps automation mode is appropriate for your use case to ensure that your infrastructure is properly managed and controlled.

The following is an example of a Terraform object; we use the \"auto-apply\" mode:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\nspec:\n  path: ./helloworld\n  interval: 10m\n  approvePlan: auto\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n

This code is defining a Terraform object in Kubernetes. The apiVersion field specifies the version of the Kubernetes API being used, and the kind field specifies that it is a Terraform object. The metadata block contains information about the object, including its name.

The spec field contains the specification for the Terraform object. The path field specifies the path to the Terraform configuration files, in this case a directory named \"helloworld\". The interval field specifies the frequency at which TF-Controller should run the Terraform configuration, in this case every 10 minutes. The approvePlan field specifies whether or not to automatically approve the changes proposed by a Terraform plan. In this case, it is set to auto, meaning that changes will be automatically approved.

The sourceRef field specifies the Flux source object to be used. In this case, it is a GitRepository object with the name \"helloworld\". This indicates that the Terraform configuration is stored in a Git repository object with the name helloworld.

"},{"location":"use-tf-controller/provision-resources-and-destroy-them-when-terraform-object-gets-deleted/","title":"Use TF-Controller to provision resources and destroy them when the Terraform object gets deleted","text":"

The resources provisioned by a Terraform object are not destroyed by default, and the tfstate of that Terraform object still remains in the cluster.

It means that you are safe to delete the Terraform object in the cluster and recreate it. If you recreate a new Terraform object with the same name, namespace, and workspace, it will continue to use the tfstate inside the cluster as the starting point to reconcile.

However, you may want to destroy provisioned resources when deleting the Terraform object. To enable destroy resources on object deletion, set .spec.destroyResourcesOnDeletion to true.

~> WARNING: This feature will destroy your resources on the cloud if the Terraform object gets deleted. Use it with caution.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  destroyResourcesOnDeletion: true\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/provision-resources-obtain-outputs/","title":"Use TF-Controller to provision resources and obtain outputs","text":"

Outputs created by Terraform can be written to a secret using .spec.writeOutputsToSecret.

"},{"location":"use-tf-controller/provision-resources-obtain-outputs/#write-all-outputs","title":"Write all outputs","text":"

We can specify a target secret in .spec.writeOutputsToSecret.name, and the controller will write all outputs to the secret by default.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: helloworld-output\n
"},{"location":"use-tf-controller/provision-resources-obtain-outputs/#write-outputs-selectively","title":"Write outputs selectively","text":"

Choose only a subset of outputs by specifying output names you'd like to write in the .spec.writeOutputsToSecret.outputs array.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: helloworld-output\n    outputs:\n    - hello_world\n    - my_sensitive_data\n
"},{"location":"use-tf-controller/provision-resources-obtain-outputs/#rename-outputs","title":"Rename outputs","text":"

Some time we'd like to use rename an output, so that it can be consumed by other Kubernetes controllers. For example, we might retrieve a key from a Secret manager, and it's an AGE key, which must be ending with \".agekey\" in the secret. In this case, we need to rename the output.

TF-controller supports mapping output names using the old_name:new_name format.

In the following example, we renamed age_key output as age.agekey entry for the helloworld-output secret's data, so that other components in the GitOps pipeline could consume it.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: helloworld-output\n    outputs:\n    - age_key:age.agekey\n
"},{"location":"use-tf-controller/provision-resources-obtain-outputs/#customize-metadata-of-the-outputted-secret","title":"Customize metadata of the outputted secret","text":"

Some situations require adding custom labels and annotations to the outputted secret. As an example, operators such as kubernetes-replicator allow replicating secrets from one namespace to another but use annotations to do so.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  writeOutputsToSecret:\n    name: helloworld-output\n    labels:\n      my-label: true\n    annotations:\n      my-annotation: \"very long string\"\n
"},{"location":"use-tf-controller/provision-resources-with-customized-runner-pods/","title":"Use TF-Controller to provision resources with customized Runner Pods","text":""},{"location":"use-tf-controller/provision-resources-with-customized-runner-pods/#customize-runner-pod-metadata","title":"Customize Runner Pod metadata","text":"

Sometimes you need to add custom labels and annotations to the runner pod used to reconcile Terraform. For example, for Azure AKS to grant pod active directory permissions using Azure Active Directory (AAD) Pod Identity, a label like aadpodidbinding: myIdentity on the pod is required.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  runnerPodTemplate:\n    metadata:\n      labels:\n        aadpodidbinding: myIdentity\n      annotations:\n        company.com/abc: xyz\n
"},{"location":"use-tf-controller/provision-resources-with-customized-runner-pods/#customize-runner-pod-image","title":"Customize Runner Pod Image","text":"

By default, the Terraform controller uses RUNNER_POD_IMAGE environment variable to identify the Runner Pod's image to use. You can customize the image on the global level by updating the value of the environment variable or, you can specify an image to use per Terraform object for its reconciliation.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  runnerPodTemplate:\n    spec:\n      image: registry.io/tf-runner:xyz\n

You can use runner.Dockerfile as a basis of customizing runner pod image.

"},{"location":"use-tf-controller/provision-resources-with-customized-runner-pods/#customize-runner-pod-specifications","title":"Customize Runner Pod Specifications","text":"

You can also customize various Runner Pod spec fields to control and configure how the Runner Pod runs. For example, you can configure Runner Pod spec affinity and tolerations if you need to run in on a specific set of nodes. Please see RunnerPodSpec for a list of the configurable Runner Pod spec fields.

"},{"location":"use-tf-controller/resource-deletion/","title":"Resource Deletion Dependencies in Terraform Controller","text":"

This document discusses potential difficulties you may encounter when deleting Terraform resources through the Terraform Controller and the necessary components to facilitate a smooth deletion process.

"},{"location":"use-tf-controller/resource-deletion/#source-object","title":"Source Object","text":"

The source object (e.g., GitRepository or OCIRepository) is a critical component of the Terraform resource deletion process. This object houses the Terraform source files (.tf files) that describe the configuration of the infrastructure resources.

During the deletion process, the Terraform Controller uses these source files to conduct a re-planning operation. This operation is instrumental to deleting the Terraform Custom Resource (CR).

However, if the source object is unavailable or has been deleted, the re-planning operation fails. As a result, the Terraform Controller cannot locate the resource state, leading to an infinite deletion attempt cycle, commonly known as a looping process.

If the source object was deleted and the deletion of a Terraform resource is stuck, clearing the finalizers let the system delete the resource skipping the finalizer of the Terraform Controller.

  1. Suspend the kustomization: 
    flux suspend kustomization tf-stk\n
  2. Patch the Terraform resource: 
    kubectl patch terraforms.infra.contrib.fluxcd.io \\\n  -n stk helloworld \\\n  -p '{\"metadata\":{\"finalizers\":null}}' \\\n  --type=merge\n

As it skips the finalizer of the Terraform controller, any cleanup the controller would do will be skipped too.

"},{"location":"use-tf-controller/resource-deletion/#role-bindings","title":"Role Bindings","text":"

Role bindings assign permissions to Terraform runners, allowing them to execute operations within the Kubernetes cluster. These bindings define the actions that the Terraform runners are authorized to carry out.

If role bindings are missing or misconfigured, the Terraform runners may lack the necessary permissions to execute the deletion process, causing the process to fail.

"},{"location":"use-tf-controller/resource-deletion/#secrets-and-configmaps","title":"Secrets and ConfigMaps","text":"

Before initiating the resource deletion process, the Terraform Controller leverages Secrets and ConfigMaps to generate a complete source before planning. Secrets store confidential data like API keys or passwords, while ConfigMaps hold configuration data in a key-value format.

Should any of these components be missing or misconfigured, the Terraform Controller may fail to generate an accurate deletion plan, which could impede the resource deletion process.

"},{"location":"use-tf-controller/resource-deletion/#troubleshooting","title":"Troubleshooting","text":"

To prevent the aforementioned issues, ensure the availability and proper configuration of the source object, role bindings, and Secrets and ConfigMaps during the deletion process.

As of now, we are actively working to address these limitations in the Terraform Controller. We appreciate your patience and welcome any feedback to help enhance the Terraform Controller's performance.

"},{"location":"use-tf-controller/set-variables-for-terraform-resources/","title":"Use TF-Controller to Set Variables for Terraform Resources","text":"

~> BREAKING CHANGE: This is a breaking change of the v1alpha1 API.

Users who are upgrading from TF-Controller <= 0.7.0 must update varsFrom from a single object to become an array of objects:

  varsFrom:\n    kind: ConfigMap\n    name: cluster-config\n

changes to

  varsFrom:\n  - kind: ConfigMap\n    name: cluster-config\n
"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#vars-and-varsfrom","title":"vars and varsFrom","text":"

You can pass variables to Terraform using the vars and varsFrom fields.

Inline variables can be set using vars. The varsFrom field accepts a list of ConfigMaps / Secrets. You may use the varsKeys property of varsFrom to select specific keys from the input or omit this field to select all keys from the input source.

Note that in the case of the same variable key being passed multiple times, the controller will use the lattermost instance of the key passed to varsFrom.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  vars:\n  - name: region\n    value: us-east-1\n  - name: env\n    value: dev\n  - name: instanceType\n    value: t3-small\n  varsFrom:\n  - kind: ConfigMap\n    name: cluster-config\n    varsKeys:\n    - nodeCount\n    - instanceType\n  - kind: Secret\n    name: cluster-creds\n
"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#variable-value-as-hcl","title":"Variable Value as HCL","text":"

The vars field supports HCL string, number, bool, object and list types. For example, the following variable can be populated using the accompanying Terraform spec:

variable \"cluster_spec\" {\n  type = object({\n      region     = string\n      env        = string\n      node_count = number\n      public     = bool\n  })\n}\n
apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  vars:\n  - name: cluster_spec\n    value:\n      region: us-east-1\n      env: dev\n      node_count: 10\n      public: false\n
"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#rename-variables-in-varsfrom","title":"Rename Variables in varsFrom","text":"

To rename a variable, you can use the varsKeys key within the varsFrom field. Here's the basic structure:

spec:\n  varsFrom:\n  - kind: Secret\n    name: <secret_name>\n    varsKeys:\n    - <original_variable_name>:<new_variable_name>\n
original_variable_name corresponds to the initial name of the variable in the referenced secret, while new_variable_name represents the alias you want to use within the Terraform code.

Consider this example below, where we rename nodeCount to node_count and instanceType to instance_type:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  varsFrom:\n  - kind: Secret\n    name: cluster-config\n    varsKeys:\n    - nodeCount:node_count\n    - instanceType:instance_type\n
"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#rename-output-variables","title":"Rename output variables","text":"

See Rename outputs for more details.

"},{"location":"use-tf-controller/set-variables-for-terraform-resources/#rename-input-secrets","title":"Rename Input Secrets","text":"

See Rename input secrets for more details.

"},{"location":"use-tf-controller/terraform-init-steps/","title":"Terraform init steps","text":""},{"location":"use-tf-controller/terraform-init-steps/#aligning-tf-controller-with-terraforms-init-workflow-stage","title":"Aligning TF-Controller with Terraform's Init Workflow Stage","text":"

This page covers required and optional steps you should take in alignment with Terraform's \"init\" workflow stage. We cover \"plan,\" \"apply,\" and \"destroy\" steps in subsequent pages.

"},{"location":"use-tf-controller/terraform-init-steps/#define-source","title":"Define Source","text":"

First, we need to define the Source controller's source (GitRepository, Bucket, OCIRepository). For example:

apiVersion: source.toolkit.fluxcd.io/v1\nkind: GitRepository\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: https://github.com/tf-controller/helloworld\n  ref:\n    branch: main\n

Here's guidance for when your source is an OCI artifact.

"},{"location":"use-tf-controller/terraform-init-steps/#optional-steps","title":"Optional Steps","text":"

At this point you have options to enhance your use of TF-Controller: - Optional: Use TF-Controller with GitOps Dependency Management - This is to avoid the Kustomization controller's variable substitution - Optional: Using TF-Controller with Primitive Modules for an optional way to write Terraform code.

"},{"location":"use-tf-controller/terraform-init-steps/#resource-provisioning","title":"Resource Provisioning","text":"

Related resources, with optional steps noted:

  • Use TF-Controller to Provision Resources and Auto-Approve
  • Optional: Provision Resources and Destroy Them When the Terraform Object Gets Deleted
  • Optional: Provision Terraform Resources That Are Required Health Checks
    • You would check these during the \"apply\" workflow stage
  • Optional, operations-related: Using a Custom Backend
    • TF-Controller uses the Kubernetes backend by default

Be mindful of locking mechanism when pursuing these steps.

"},{"location":"use-tf-controller/terraform-init-steps/#optional-working-with-integrations","title":"Optional: Working with Integrations","text":"
  • Working with Terraform Cloud and Terraform Enterprise; see also: Terraform Cloud and Branch Planner
"},{"location":"use-tf-controller/terraform-init-steps/#context-related-steps","title":"Context-Related Steps","text":"
  • Optional: Use TF-Controller with Terraform Runners enabled via Env Variables
  • Optional: Set variables for Terraform resources
  • Optional: Provision resources and obtain outputs
  • Provision resources with customized Runner Pods
  • Optional: Use with external webhooks
"},{"location":"use-tf-controller/troubleshooting-with-break-the-glass-mode/","title":"Break the glass","text":""},{"location":"use-tf-controller/troubleshooting-with-break-the-glass-mode/#what-is-break-the-glass","title":"What is break the glass?","text":"

\"Break the glass\" refers to a troubleshooting mode specifically designed to provide a manual solution when TF-Controller is not performing as expected. This feature is available in the Terraform controller v0.15.0 and above.

~> WARNING: Please note that you cannot use this feature to fix the Terraform resources with v1alpha1 version of the Terraform CRD. It works only with v1alpha2 version of the Terraform CRD.

~> WARNING: Please also make sure that you have enough privileges to exec pods in your namespaces. Otherwise, you will not be able to use this feature.

There are two primary methods of initiating this mode:

  1. Using the tfctl command-line tool.
  2. Setting the spec.breakTheGlass field to true in the Terraform object.
"},{"location":"use-tf-controller/troubleshooting-with-break-the-glass-mode/#using-tfctl-to-break-the-glass","title":"Using tfctl to Break the Glass","text":"

In order to use this functionality, it needs to be enabled at the controller level; in order to do that, you can set the following Helm chart value to true:

allowBreakTheGlass: true\n

After the feature is enabled, to start a one-time troubleshooting session, you can use the tfctl break-glass command. For instance:

tfctl break-glass hello-world\n

This command initiates a session that allows you to execute any Terraform command to rectify the issues with your Terraform resources. It is noteworthy that this command does not require setting the spec.breakTheGlass field to true in the Terraform object.

After resolving the issues, you can simply exit the shell. GitOps will then continue to reconcile the Terraform object.

"},{"location":"use-tf-controller/troubleshooting-with-break-the-glass-mode/#break-the-glass-with-specbreaktheglass-field","title":"Break the glass with spec.breakTheGlass field","text":"

This feature is particularly useful for troubleshooting Terraform objects at their initialization stage or in situations with unexpected errors. It is generally not recommended to use this mode routinely for fixing Terraform resources.

You can enable the 'Break the Glass' feature for every reconciliation by setting the breakTheGlass field to true in the spec of the Terraform object.

Here is a sample example:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: hello-world\n  namespace: flux-system\nspec:\n  breakTheGlass: true\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/upgrade-tf-controller/","title":"Upgrading TF-Controller","text":"

Please follow these steps to upgrade TF-Controller:

  1. Read the latest release changelogs.
  2. Check your API versions.
  3. To make sure you don't get new state changes, suspend Terraform resources (tfctl suspend --all) to minimize the impact on live systems.
  4. Back up Terraform tfstates to avoid losing data. If you're using the default backend with secrets in Kubernetes, use your backup toolset (i.e., Velero) to back up the state data.
  5. Upgrade Flux first, following the Flux documentation.
  6. Disable auto-approval by either removing the approvePlan value or setting it to \"\".
  7. To prevent unintentional resource deletions, set the spec.destroyResourcesOnDeletion flag to false for critical or production systems (the default value is false)
  8. If the Flux upgrade goes well, proceed to upgrade the TF-controller via its image tag. Adjust the values in the HelmRelease to match the new version to which you are upgrading.
  9. Check the pod logs for the TF-Controller deployment and any runner logs in order to identify potential issues. If you check the warnings in the logs, you can also identify any required API changes. For example: v1alpha1 Terraform is deprecated, upgrade to v1alpha2.
  10. Push the changes you made.
  11. Resume your Terraform resources\u2014either one-by-one for critical resources, or all of them with tfctl resume --all
  12. Ensure no changes are planned for deletion. If you changed the value in step 6 from spec.destroyResourcesOnDeletion to false, resources will not be automatically removed.
  13. Revert back to auto-approval mode after ensuring stability.
  14. Resume any suspended Kustomization objects to restore GitOps automation.
  15. Restore spec.destroyResourcesOnDeletion, if this has been disabled for any resources in critical or production systems.

TF-Controller supports v1alpha1 for backward compatibility. This means that you need v1alpha2 for newer (as of September 2023) features such as: - the branch planner - pod sub-domain DNS resolutions - new PodSpec fields like PriorityClass, SecurityContext, and ResourceRequirements (Limits / Requests)

"},{"location":"use-tf-controller/use-cross-namespace-refs/","title":"Using Cross-Namespace References","text":"

The Terraform CRD in the API for TF-Controller includes fields which are references to other objects:

Name Purpose .spec.sourceRef Refers to a Flux source .spec.dependsOn[*] Each entry refers to a dependency .spec.cliConfigSecretRef Secret with tf config to use

Branch Planner configuration can also have cross-namespace references:

Name Purpose .secretNamespace Namespace of secret containing a GitHub token .resources[*] Each entry refers to a Terraform object to include in branch planning

All of these can refer to an object in a namespace different to that of the Terraform object. However, giving access to objects in other namespaces is generally considered a security risk, so this is disallowed by default. Only references that mention the same namespace, or that omit the namespace field, will be accepted. References using a different namespace will cause TF-Controller to stop processing the Terraform object and put it in a non-Ready state.

To allow cross-namespace references, use the flag --allow-cross-namespace-refs with TF-Controller and the Branch Planner. When using the Helm chart to install or update TF-Controller and Branch Planner, the value allowCrossNamespaceRefs will allow cross-namespace references for both.

"},{"location":"use-tf-controller/with-a-custom-backend/","title":"Option: Use TF-Controller with a Custom Backend","text":"

By default, TF-Controller uses the Kubernetes backend to store the Terraform state file (tfstate) in clusters.

The tfstate is stored in a secret named: tfstate-${workspace}-${secretSuffix}. The default suffix will be the name of the Terraform resource, however you may override this setting using .spec.backendConfig.secretSuffix. The default workspace name is \"default\", you can also override the workspace by setting .spec.workspace to another value.

If you wish to use a custom backend, you can configure it by defining the .spec.backendConfig.customConfiguration with one of the backends such as GCS or S3, for example:

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  backendConfig:\n    customConfiguration: |\n      backend \"s3\" {\n        bucket                      = \"s3-terraform-state1\"\n        key                         = \"dev/terraform.tfstate\"\n        region                      = \"us-east-1\"\n        endpoint                    = \"http://localhost:4566\"\n        skip_credentials_validation = true\n        skip_metadata_api_check     = true\n        force_path_style            = true\n        dynamodb_table              = \"terraformlock\"\n        dynamodb_endpoint           = \"http://localhost:4566\"\n        encrypt                     = true\n      }\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  runnerPodTemplate:\n    spec:\n      image: registry.io/tf-runner:xyz\n

To add fields from secrets or configMaps, use backendConfigsFrom, for example to add access_key and secret_key from a secret:

apiVersion: v1\nkind: Secret\nmetadata:\n  name: terraform-s3-backend\n  namespace: flux-system\ntype: Opaque\ndata:\n  access_key: <base64 encoded key>\n  secret_key: <base64 encoded key>\n\n---\n\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  backendConfig:\n    customConfiguration: |\n      backend \"s3\" {\n        bucket                      = \"s3-terraform-state1\"\n        key                         = \"dev/terraform.tfstate\"\n        region                      = \"us-east-1\"\n        endpoint                    = \"http://localhost:4566\"\n        skip_credentials_validation = true\n        skip_metadata_api_check     = true\n        force_path_style            = true\n        dynamodb_table              = \"terraformlock\"\n        dynamodb_endpoint           = \"http://localhost:4566\"\n        encrypt                     = true\n      }\n  backendConfigsFrom:\n    - kind: Secret\n      name: terraform-s3-backend\n      keys:\n      - access_key\n      - secret_key\n      optional: false\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n  runnerPodTemplate:\n    spec:\n      image: registry.io/tf-runner:xyz\n
"},{"location":"use-tf-controller/with-an-oci-artifact-as-source/","title":"Use TF-Controller with an OCI Artifact as Source","text":"

To use OCI artifacts as the source of Terraform objects, you need Flux 2 version v0.32.0 or higher.

Assuming that you have Terraform files (your root module may contain sub-modules) under ./modules, you can use Flux CLI to create an OCI artifact for your Terraform modules by running the following commands:

flux push artifact oci://ghcr.io/tf-controller/helloworld:$(git rev-parse --short HEAD) \\\n    --path=\"./modules\" \\\n    --source=\"$(git config --get remote.origin.url)\" \\\n    --revision=\"$(git branch --show-current)/$(git rev-parse HEAD)\"\n\nflux tag artifact oci://ghcr.io/tf-controller/helloworld:$(git rev-parse --short HEAD) \\\n    --tag main\n

Then you define a source (OCIRepository), and use it as the sourceRef of your Terraform object.

---\napiVersion: source.toolkit.fluxcd.io/v1beta2\nkind: OCIRepository\nmetadata:\n  name: helloworld-oci\nspec:\n  interval: 1m\n  url: oci://ghcr.io/tf-controller/helloworld\n  ref:\n    tag: main\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld-tf-oci\nspec:\n  path: ./\n  approvePlan: auto\n  interval: 1m\n  sourceRef:\n    kind: OCIRepository\n    name: helloworld-oci\n  writeOutputsToSecret:\n    name: helloworld-outputs\n
"},{"location":"use-tf-controller/with-aws-eks-irsa/","title":"Use TF-Controller with AWS EKS IRSA","text":"

AWS Elastic Kubernetes Service (EKS) offers IAM Roles for Service Accounts (IRSA) as a mechanism by which to provide credentials to Kubernetes pods. This can be used to provide the required AWS credentials to Terraform runners for performing plans and applies.

You can use eksctl to associate an OIDC provider with your EKS cluster. For example:

eksctl utils associate-iam-oidc-provider --cluster CLUSTER_NAME --approve\n

Then follow the instructions here to add a trust policy to the IAM role which grants the necessary permissions for Terraform. If you have installed TF-Controller following the README, then the namespace:serviceaccountname will be flux-system:tf-runner. You'll obtain a Role ARN to use in the next step.

Finally, annotate the ServiceAccount for the tf-runner with the obtained Role ARN in your cluster:

kubectl annotate -n flux-system serviceaccount tf-runner eks.amazonaws.com/role-arn=ROLE_ARN\n

If deploying the tf-controller via Helm, do this as follows:

values:\n  runner:\n    serviceAccount:\n      annotations:\n        eks.amazonaws.com/role-arn: ROLE_ARN\n
"},{"location":"use-tf-controller/with-azure/","title":"With azure","text":""},{"location":"use-tf-controller/with-azure/#use-tf-controller-with-azure","title":"Use TF-Controller with Azure","text":"

This content was provided by users @mingmingshiliyu and @maciekdude.

Use the OIDC flag and explicitly point to the token. Due to a bug in AzureRM 3.44.x, use version 3.47.x or later.

Set env variables on the runner pod:

        - name: ARM_USE_OIDC\n          value: \"true\"\n        - name: ARM_OIDC_TOKEN_FILE_PATH\n          value: \"/var/run/secrets/azure/tokens/azure-identity-token\"\n

Example yaml:

apiVersion: infra.contrib.fluxcd.io/v1alpha1\nkind: Terraform\nmetadata:\n  name: terraformhello\n  namespace: default\nspec:\n  tfstate:\n    forceUnlock: auto\n  backendConfig:\n    customConfiguration: |\n      backend \"azurerm\" {\n        resource_group_name  = \"l\"\n        storage_account_name = \"\"\n        container_name       = \"tfstate\"\n        key                  = \"helloworld.tfstate\"\n        use_oidc             = true\n      }\n  interval: 1m\n  serviceAccountName: service_account_registered_in_aad\n  approvePlan: auto\n  destroy: true\n  path: ./tests/fixture\n  sourceRef:\n    kind: GitRepository\n    name: terraformhello\n    namespace: flux-system\n  runnerPodTemplate:\n    spec:\n      image: azure_cli_runner.xxx\n      env:\n        - name: ARM_USE_OIDC\n          value: \"true\"\n        - name: ARM_SUBSCRIPTION_ID\n          value: \"\"\n        - name: ARM_TENANT_ID\n          value: \"\"\n        - name: ARM_CLIENT_ID\n          value: \"\"\n        - name: ARM_OIDC_TOKEN_FILE_PATH\n          value: \"/var/run/secrets/azure/tokens/azure-identity-token\"\n

Import existing resources to a tfstate file stored on a storage account.

"},{"location":"use-tf-controller/with-drift-detection-disabled/","title":"Use TF-Controller with drift detection disabled","text":"

Drift detection is enabled by default. You can set .spec.disableDriftDetection: true to disable it.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  approvePlan: auto\n  disableDriftDetection: true\n  interval: 1m\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/with-external-webhooks/","title":"Use TF-Controller with External Webhooks","text":"

The TF-Controller provides a way to integrate with webhooks to further validate Terraform plans and manage the Terraform execution process. With the webhook feature, you can implement custom policy checks, validations, and other logic to determine if the Terraform process should proceed.

"},{"location":"use-tf-controller/with-external-webhooks/#setting-up-the-webhook","title":"Setting up the Webhook","text":"
  1. Webhook URL: Specify the URL of your webhook, ensuring it points to a valid HTTPS endpoint.
  2. Expected Return: The webhook should return a valid JSON object. For instance: 
    {\"passed\": true}\n
  3. Accepted True Values: The true values can be true, \"true\", and \"yes\".
  4. Accepted False Values: The false values can be flse, \"false\", and \"no\".

Below is a breakdown of the relevant parts of the configuration:

  1. webhooks: This is the section where you specify all webhook related configurations.
  2. stage: Define at which stage the webhook will be triggered. Currenly, we support only the post-planning stage.
  3. url: The URL pointing to your webhook endpoint.
  4. testExpression: This expression is used to evaluate the response from the webhook. If it evaluates to true, the controller proceeds with the operation. In the example, the expression checks for the passed value from the webhook's JSON response.
  5. errorMessageTemplate: If testExpression evaluates to false, this template is used to extract the error message from the webhook's JSON response. This message will be displayed to the user.
"},{"location":"use-tf-controller/with-external-webhooks/#configuration-example","title":"Configuration Example","text":"

Here's a configuration example on how to use the webhook feature to integrate with Weave Policy Engine. 

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld-tf\nspec:\n  path: ./terraform\n  approvePlan: \"auto\"\n  interval: 1m\n  storeReadablePlan: human\n  sourceRef:\n    kind: GitRepository\n    name: helloworld-tf\n  webhooks:\n  - stage: post-planning\n    url: https://policy-agent.policy-system.svc/terraform/admission\n    testExpression: \"${{ .passed }}\"\n    errorMessageTemplate: \"Violation: ${{ (index (index .violations 0).occurrences 0).message }}\"\n  writeOutputsToSecret:\n    name: helloworld-outputs\n

Important Considerations:

  • Ensure that your webhook endpoint is secure, as the TF-Controller will be sending potentially sensitive Terraform plan data to it.
  • Test your webhook implementation thoroughly before deploying to production, as any issues could interrupt or halt your Terraform process.

With the webhook feature, you can create a more robust and flexible GitOps Terraform pipeline that respects custom organizational policies and other requirements.

"},{"location":"use-tf-controller/with-gitops-dependency-management/","title":"Use TF-controller with GitOps dependency management","text":"

TF-controller supports GitOps dependency management. The GitOps dependency management feature is based on the similar technique implemented in the Kustomization controller of Flux.

This means that you can use TF-controller to provision resources that depend on other resources at the GitOps level. For example, you can use TF-controller to provision an S3 bucket, and then use TF-controller to provision another resource to configure ACL for that bucket.

GitOps dependency management is different from Terraform's HCL dependency management in the way that it is not based on Terraform's mechanism, which is controlled by the Terraform binary. Instead, it is implemented at the controller level, which means that each Terraform module is reconciled and can be managed independently, while still being able to depend on other modules.

"},{"location":"use-tf-controller/with-gitops-dependency-management/#create-a-terraform-object","title":"Create a Terraform object","text":"

Similar to the same feature in the Kustomization controller, the dependency management feature is enabled by setting the dependsOn field in the Terraform object. The dependsOn field is a list of Terraform objects.

When the dependency is not satisfied, the Terraform object will be in the Unknown state, and it will be retry again every spec.retryInterval. The retry interval is same as the spec.interval by default, and it can be configured separately by setting the spec.retryInterval field.

First, create a Terraform object to provision the S3 bucket, name it aws-s3-bucket. The S3 bucket is provisioned by the Terraform module aws_s3_bucket in the OCI image aws-package. It is configured to use the auto-apply mode, and write outputs to the secret aws-s3-bucket-outputs.

---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: aws-s3-bucket\n  namespace: flux-system\nspec:\n  path: aws_s3_bucket\n  values:\n    bucket: my-tf-controller-test-bucket\n    tags:\n      Environment: Dev\n      Name: My bucket\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  interval: 2m\n  destroyResourcesOnDeletion: true\n  writeOutputsToSecret:\n    name: aws-s3-bucket-outputs\n    outputs:\n    - arn\n    - bucket\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n

Second, create a Terraform object to configure ACL for the S3 bucket, name it aws-s3-bucket-acl. The ACL is provisioned by the Terraform module aws_s3_bucket_acl, also from the OCI image aws-package-v4.33.0.

In the dependsOn field, specify the Terraform object that provisions the S3 bucket. This means that the ACL will be configured only after the S3 bucket is provisioned, and has its outputs Secret written. We can read the outputs of the S3 bucket from the Secret aws-s3-bucket-outputs, by specifying the spec.readInputsFromSecrets field. The spec.readInputsFromSecrets field is a list of Secret objects. Its name field is the name of the Secret, and its as field is the name of variable that can be used in the spec.values block.

For example, the spec.values.bucket field in the aws-s3-bucket-acl Terraform object is set to ${{ .aws_s3_bucket.bucket }}.

Please note that we use ${{ and }} as the delimiters for the variable name, instead of the Helm default ones, {{ and }}.

---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: aws-s3-bucket-acl\n  namespace: flux-system\nspec:\n  path: aws_s3_bucket_acl\n  values:\n    acl: private\n    bucket: ${{ .aws_s3_bucket.bucket }}\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  interval: 3m\n  dependsOn:\n  - name: aws-s3-bucket\n  readInputsFromSecrets:\n  - name: aws-s3-bucket-outputs\n    as: aws_s3_bucket\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n
"},{"location":"use-tf-controller/with-gitops-dependency-management/#avoid-kustomization-controllers-variable-substitution","title":"Avoid Kustomization controller's variable substitution","text":"

The Kustomization controller will substitute variables in the Terraform object, which will cause conflicts with the variable substitution in the GitOps dependency management feature. To avoid this, we need to add the kustomize.toolkit.fluxcd.io/substitute: disabled annotation to the Terraform object.

---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: aws-s3-bucket-acl\n  namespace: flux-system\n  annotations:\n    kustomize.toolkit.fluxcd.io/substitute: disabled\nspec:\n  path: aws_s3_bucket_acl\n  values:\n    acl: private\n    bucket: ${{ .aws_s3_bucket.bucket }}\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  interval: 3m\n  dependsOn:\n  - name: aws-s3-bucket\n  readInputsFromSecrets:\n  - name: aws-s3-bucket-outputs\n    as: aws_s3_bucket\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n
"},{"location":"use-tf-controller/with-ipv6/","title":"Use TF-Controller with IPv6 addresses.","text":"

TF-Controller uses pod IPv4 address to generate the in-cluster hostname to communicate with the runner instance. This logic fails when the runner pod has IPv6 address instead of IPv4.

The TF-Controller has a flag to use pod subdomain resolution instead of an IP address, with that enabled the controller will use cluster subdomains, and it works with IPv6 addresses as the resolution is happening at cluster level.

To enable this feature, you can set usePodSubdomainResolution to true in the Helm values file:

usePodSubdomainResolution: true\n
"},{"location":"use-tf-controller/with-plan-only-mode/","title":"Use TF-Controller with a plan-only mode","text":"

This plan-only mode is designed to be used in conjunction with the Branch Planner. But you can also use it whenever you want to run terraform plan only.

If planOnly is set to true, TF-Controller will skip the apply step, run terraform plan, and save the output.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: helloworld\n  namespace: flux-system\nspec:\n  interval: 1m\n  planOnly: true\n  path: ./\n  sourceRef:\n    kind: GitRepository\n    name: helloworld\n    namespace: flux-system\n
"},{"location":"use-tf-controller/with-primitive-modules/","title":"Use TF-Controller with primitive modules","text":"

This document describes how to use TF-Controller with a primitive module. It requires TF-Controller v0.13+ to run the example.

"},{"location":"use-tf-controller/with-primitive-modules/#what-is-a-primitive-module","title":"What is a primitive module?","text":"

It's a Terraform module that contains only a single resource.

  • A Terraform primitive module must contains the \"values\" variable.
  • The \"values\" variable must be an object with fields of optional types.
  • The module must be placed under a directory, which is named after the resource.
  • The directory can optionally contain other files, for example the .terraform.lock.hcl.
  • We call a set of primitive modules bundled into an OCI image, a package.
"},{"location":"use-tf-controller/with-primitive-modules/#hello-world-primitive-module","title":"Hello World Primitive Module","text":"

Here is an example of how a primitive module can be defined in YAML. Assume that we have a ready-to-use OCI image with a primitive module for the imaginary resource aws_hello_world, and the image is tagged as ghcr.io/tf-controller/hello-primitive-modules/v4.32.0:v1.

We'll use the following Terraform object definition to provision the resource.

First, we need to create a Terraform object with the spec.sourceRef.kind field set to OCIRepository and the spec.sourceRef.name field set to the name of the OCIRepository object.

Second, we need to set the spec.path field to the name of the resource, in this case aws_hello_world.

Third, we need to set the spec.values field to the values of the resource. This is a YAML object that will be converted to an HCL variable, and passed to the Terraform module.

Finally, we need to set the spec.approvePlan field to auto to automatically approve the plan.

---\napiVersion: source.toolkit.fluxcd.io/v1beta2\nkind: OCIRepository\nmetadata:\n  name: hello-package-v4.32.0\n  namespace: flux-system\nspec:\n  interval: 30s\n  url: oci://ghcr.io/tf-controller/hello-primitive-modules/v4.32.0\n  ref:\n    tag: v1\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: hello-world\n  namespace: flux-system\nspec:\n  path: aws_hello_world\n  values:\n    greeting: Hi\n    subject: my world\n  sourceRef:\n    kind: OCIRepository\n    name: hello-package-v4.32.0\n  interval: 1h0m\n  approvePlan: auto\n
"},{"location":"use-tf-controller/with-tf-runner-exposed-using-hostname-subdomain/","title":"Use TF-controller with Terraform Runners exposed via hostname/subdomain","text":"

TF-controller uses the Controller/Runner architecture. The Controller acts as a client, and talks to each Runner's Pod via gRPC over port 30000.

TF-controller must thus be able to reliably connect to each Runner's pod regardless of the cluster network topology.

"},{"location":"use-tf-controller/with-tf-runner-exposed-using-hostname-subdomain/#the-default-runner-dns-resolution","title":"The Default Runner DNS resolution","text":"

By default, TF-controller fetches the Runner's pod IP address after it is instantiated (e.g. 1.2.3.4).

It then transforms the IP address into its IP-based pod DNS A record (e.g. 1.2.3.4.<namespace>.pod.<cluster-domain>) which is used to connect to the Runner pod using gRPC protocol.

In standard Kubernetes cluster deployment, IP-based pod DNS resolution is usually provided by Coredns and especially the pods option of the Kubernetes plugin.

cluster.local {\n    kubernetes {\n        pods verified\n    }\n}\n

IMPORTANT: The gRPC communication between TF-controller and Runner's pod is secured with mTLS. TF-controller generates a valid wildcard TLS certificate for *.<namespace>.pod.<cluster-domain> hosts on the Runner's namespace. The Runner's pod present this certificate during TLS handshake with TF-controller.

"},{"location":"use-tf-controller/with-tf-runner-exposed-using-hostname-subdomain/#hostnamesubdomain-runner-dns-resolution","title":"Hostname/Subdomain Runner DNS resolution","text":"

The default configuration described above works for standard Kubernetes deployments. It does not work however when the cluster DNS provider do not support IP-based pod DNS resolution. This is the case for GCP Cloud DNS for example.

For such setup, you can switch the DNS resolution mode to Hostname/Subdomain. Enabling this option will :

  • Create a Headless service named tf-runner in each allowed namespace

```yaml hl_lines=\"4-5,8-10\" apiVersion: v1 kind: Service metadata: name: tf-runner namespace: hello-world spec: clusterIP: None ports: - name: grpc port: 30000 selector: app.kubernetes.io/created-by: tf-controller app.kubernetes.io/name: tf-runner 

- Set Runner's pod spec with `hostname: <terraform_object_name>` and `subdomain: tf-runner`\n\n```yaml hl_lines=\"12-13\"\napiVersion: v1\nkind: Pod\n  labels:\n    app.kubernetes.io/created-by: tf-controller\n    app.kubernetes.io/instance: tf-runner-3ac83e0f\n    app.kubernetes.io/name: tf-runner\n    infra.contrib.fluxcd.io/terraform: hello-world\n    tf.weave.works/tls-secret-name: terraform-runner.tls-1693866794\n  name: helloworld-tf-runner\n  namespace: hello-world\nspec:\n  hostname: helloworld\n  subdomain: tf-runner\n  containers:\n  - args:\n    - --grpc-port\n    - \"30000\"\n    - --tls-secret-name\n    - terraform-runner.tls-1693866794\n    - --grpc-max-message-size\n    - \"4\"\n    image: ghcr.io/weaveworks/tf-runner:v0.16.0-rc.2\n    name: tf-runner\n    ports:\n    - containerPort: 30000\n      name: grpc\n    resources:\n      limits:\n        cpu: 500m\n        ephemeral-storage: 1Gi\n        memory: 2Gi\n    securityContext:\n      allowPrivilegeEscalation: false\n      capabilities:\n        drop:\n        - ALL\n      readOnlyRootFilesystem: true\n      runAsNonRoot: true\n      runAsUser: 65532\n      seccompProfile:\n        type: RuntimeDefault\n  preemptionPolicy: PreemptLowerPriority\n  priority: 0\n  schedulerName: gke.io/optimize-utilization-scheduler\n  securityContext:\n    seccompProfile:\n      type: RuntimeDefault\n  serviceAccountName: tf-runner\n

The Runner's pod can then be targeted by TF-Controller using <terraform_object_name>.tf-runner.<namespace>.svc.<cluster-domain> (helloworld.tf-runner.hello-world.svc.cluster.local) as per Kubernetes specification instead of IP-based pod DNS resolution.

The switch is performed by setting the following Helm value usePodSubdomainResolution: true or running directly TF-controller with the option --use-pod-subdomain-resolution=true

IMPORTANT: The gRPC communication between TF-Controller and Runner's pod is secured with mTLS. TF-controller generates a valid wildcard TLS certificate for *.<namespace>.pod.<cluster-domain> and *.tf-runner.<namespace>.svc.<cluster-domain> hosts on the Runner's namespace. The Runner's pod present this certificate during TLS handshake with TF-Controller.

"},{"location":"use-tf-controller/with-tf-runner-logging/","title":"Control the logging behavior of Terraform Runner","text":"

A Terraform Runner uses two environment variables, DISABLE_TF_LOGS and ENABLE_SENSITIVE_TF_LOGS, to control the logging behavior of the Terraform execution.

To use these environment variables, they need to be set on each Terraform Runner pod where the Terraform code is being executed. This can typically be done by adding them to the pod's environment variables in the Terraform Runner deployment configuration.

  • The DISABLE_TF_LOGS variable, when set to \"1\", will disable all Terraform output logs to stdout and stderr.
  • The ENABLE_SENSITIVE_TF_LOGS variable, when set to \"1\", will enable logging of sensitive Terraform data, such as secret variables, to the local log. However, it is important to note that for the ENABLE_SENSITIVE_TF_LOGS to take effect, the DISABLE_TF_LOGS variable must also be set to \"1\".
"},{"location":"use-tf-controller/with-tf-runner-logging/#the-default-logging-behavior","title":"The Default Logging Behavior","text":"
  • By default, the logging level for the tf-runner is configured at the info level.
  • The DISABLE_TF_LOGS variable is not activated as part of the default settings.
  • The ENABLE_SENSITIVE_TF_LOGS variable remains inactive in the default configuration.
  • Calls to ShowPlan and ShowPlanRaw on the runner are not logged by default.
  • For Plan calls made on the runner, error messages are sanitized as a part of the default configuration.

For more information on configuring the Terraform Runner and its environment variables, please consult the documentation on customizing runners within the Weave TF-controller.

"},{"location":"use-tf-controller/with-tf-runner-logging/#logging-human-readable-plan","title":"Logging human-readable plan","text":"

The plan can be logged in a human-readable format just before the applying it in the tf-runner. To enable this, set the environment variable LOG_HUMAN_READABLE_PLAN to \"1\" on the runner.

"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/","title":"With the ready to use aws package","text":""},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#use-tf-controller-with-the-ready-to-use-aws-package","title":"Use TF-Controller with the ready-to-use AWS package","text":"

You need TF-Controller v0.13+ to running the example of TF-Controller with the ready-to-use AWS package.

"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#what-is-a-package","title":"What is a package?","text":"

A package is a collection of primitive Terraform modules that are bundled into an OCI image. You can think of a TF-controller's package as a thin wrapper around a Terraform module provider, and a TF-controller primitive module as a thin wrapper around a Terraform resource or a root module.

We will provide a set of ready-to-use packages for the most popular cloud providers. Currently, we ship the package for AWS only.

"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#aws-package","title":"AWS Package","text":"

To provide the out-of-the-box experience, the AWS Package is installed by default when you installed the TF-controller. Unlike other IaC implementation, our package model is designed to be very lightweight as a package is just a set of TF files in the form of OCI. Packages would not put any burden to your cluster. However, you can opt this package out by setting awsPackage.install: false in your Helm chart values.

If you run flux get sources oci you should see the AWS package installed in your cluster listed as aws-package.

flux get sources oci\nNAME          REVISION                    SUSPENDED   READY   MESSAGE                                                                                                         \naws-package   v4.38.0-v1alpha11/6033f3b   False       True    stored artifact for digest 'v4.38.0-v1alpha11/6033f3b'\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#a-step-by-step-tutorial","title":"A step-by-step tutorial","text":"

This section describes how to use the AWS package to provision an S3 bucket with ACL using the TF-controller.

"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#create-a-kind-local-cluster","title":"Create a KinD local cluster","text":"

If you don't have a Kubernetes cluster, you can create a KinD cluster with the following command:

kind create cluster\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#install-flux","title":"Install Flux","text":"

After you have a Kubernetes cluster, you can install Flux with the following command:

flux install\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#install-tf-controller","title":"Install TF-controller","text":"

Then, you can install the TF-controller with the following command:

kubectl apply -f https://raw.githubusercontent.com/weaveworks/tf-controller/main/docs/release.yaml\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#setup-aws-credentials","title":"Setup AWS credentials","text":"

To provision AWS resources, you need to provide the AWS credentials to your Terraform objects. You can do this by creating a secret with the AWS credentials and reference it in each of your Terraform objects.

```yaml\napiVersion: v1\nkind: Secret\nmetadata:\n  name: aws-credentials\n  namespace: flux-system\ntype: Opaque\nstringData:\n  AWS_ACCESS_KEY_ID: Axxxxxxxxxxxxxxxxxxx\n  AWS_SECRET_ACCESS_KEY: qxxxxxxxxxxxxxxxxxxxxxxxxx\n  AWS_REGION: us-east-1 # the region you want\n

To apply the secret, run the following command:

kubectl apply -f aws-credentials.yaml\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#setup-aws-bucket-and-acl","title":"Setup AWS Bucket and ACL","text":"

Now, you can create two Terraform objects, one for an S3 bucket, another one for ACL. Please note that we are using GitOps dependencies to make sure the ACL is created after the bucket is created. You can read more about the GitOps dependencies in the GitOps dependencies document.

```yaml\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: aws-s3-bucket\n  namespace: flux-system\n  labels:\n    tf.weave.works/composite: s3-bucket\nspec:\n  path: aws_s3_bucket\n  values:\n    bucket: my-tf-controller-test-bucket\n    tags:\n      Environment: Dev\n      Name: My bucket\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  retryInterval: 10s\n  interval: 2m\n  destroyResourcesOnDeletion: true\n  writeOutputsToSecret:\n    name: aws-s3-bucket-outputs\n    outputs:\n    - arn\n    - bucket\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n---\napiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: example-bucket-acl\n  namespace: flux-system\n  labels:\n    tf.weave.works/composite: s3-bucket\nspec:\n  path: aws_s3_bucket_acl\n  values:\n    acl: private\n    bucket: ${{ .aws_s3_bucket.bucket }}\n  sourceRef:\n    kind: OCIRepository\n    name: aws-package\n  approvePlan: auto\n  retryInterval: 10s\n  interval: 3m\n  dependsOn:\n  - name: aws-s3-bucket\n  readInputsFromSecrets:\n  - name: aws-s3-bucket-outputs\n    as: aws_s3_bucket\n  destroyResourcesOnDeletion: true\n  runnerPodTemplate:\n    spec:\n      envFrom:\n      - secretRef:\n          name: aws-credentials\n
"},{"location":"use-tf-controller/with-the-ready-to-use-aws-package/#rename-input-secrets","title":"Rename input secrets","text":"

The spec.readInputsFromSecrets field allows you to reference the Terraform outputs from other Terraform objects. In the context of this field, renaming makes it easier to reference the secrets in the spec.values field.

To rename a secret, you need to use the as key in the spec.readInputsFromSecrets field. The name key corresponds to the original name of the secret, while the as key represents the new name that you want to use to reference the secret.

In the example below, we can reference the bucket value from our aws_s3_bucket secret using ${{ .aws_s3_bucket.bucket }} instead of using the original secret name, which is aws-s3-bucket-outputs.

apiVersion: infra.contrib.fluxcd.io/v1alpha2\nkind: Terraform\nmetadata:\n  name: example-bucket-acl\n  namespace: flux-system\nspec:\n  # ...\n  readInputsFromSecrets:\n  - name: aws-s3-bucket-outputs\n    as: aws_s3_bucket\n  values:\n    acl: private\n    bucket: ${{ .aws_s3_bucket.bucket }}\n  # ...\n
"}]} \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz index 4f70100b3c53fc800fda69b82e9ec6a1d82eac97..84e85bf5814f20b2df61244f6db4caf3de36b1c7 100644 GIT binary patch delta 13 Ucmb=gXP58h;9!X7naExN02h`6I{*Lx delta 13 Ucmb=gXP58h;9%IpIgz~r02x~Yl>h($