Merge remote-tracking branch 'upstream/main' into fixed-rbac-for-reco…

…nciler

Merge from main

controllers/tc000243_remediation_retry_test.go

@@ -213,4 +213,39 @@ spec:
 		}
 		return nil
 	}, timeout, interval).Should(Equal(expected))
+
+	g.Expect(createdHelloWorldTF.Status.LastAttemptedRevision).To(Equal(testRepo.Status.Artifact.Revision))
+
+	It("should reset the retry count")
+	By("detecting a new artifact for the source GitRepository")
+	updatedTime = time.Now()
+	testRepo.Status = sourcev1.GitRepositoryStatus{
+		ObservedGeneration: int64(1),
+		Conditions: []metav1.Condition{
+			{
+				Type:               "Ready",
+				Status:             metav1.ConditionTrue,
+				LastTransitionTime: metav1.Time{Time: updatedTime},
+				Reason:             "GitOperationSucceed",
+				Message:            "Fetched revision: master/aaabbbccc6e3d0cbb7ed22ced771a0056455a2fb",
+			},
+		},
+		Artifact: &sourcev1.Artifact{
+			Path:           "gitrepository/flux-system/test-tf-controller/aaabbbccc6e3d0cbb7ed22ced771a0056455a2fb.tar.gz",
+			URL:            server.URL() + "/bad.tar.gz",
+			Revision:       "master/aaabbbccc6e3d0cbb7ed22ced771a0056455a2fb",
+			Digest:         "sha256:196d115c43583ccd10107d631d8a594be542a75911f9832a5ec2c1e22b65387b",
+			LastUpdateTime: metav1.Time{Time: updatedTime},
+		},
+	}
+	g.Expect(k8sClient.Status().Update(ctx, &testRepo)).Should(Succeed())
+
+	g.Eventually(func() interface{} {
+		err := k8sClient.Get(ctx, helloWorldTFKey, &createdHelloWorldTF)
+		if err != nil {
+			return nil
+		}
+
+		return createdHelloWorldTF.Status.LastAttemptedRevision
+	}, timeout, interval).Should(Equal(testRepo.Status.Artifact.Revision))
 }

docs/branch_planner/getting-started.md → docs/branch_planner/branch-planner-getting-started.md

@@ -1,33 +1,33 @@
 # Getting Started With Branch Planner
 
-When the Branch Planner is enabled through Helm values, it will watch all configured Terraform resources, check their referenced Source, and poll for Pull Requests using GitHub's API plus the provided token.
-
-When the Branch Planner detects an open Pull Request, it either creates a new Terraform object or updates an existing one, applying Plan Only mode based on the original
-Terraform object.
-
-When a Plan Output becomes available, the Branch Planner creates a new comment under the Pull Request with the content of the Plan Output included.
-
 ## Prerequisites
 
 1. Flux is installed on the cluster.
-2. A GitHub [API token](./least-required-permissions.md).
-3. Knowledge about GitOps Terraform Controller [(see docs)](https://weaveworks.github.io/tf-controller/).
+2. A GitHub API token. For public repositories, it's sufficient to enable `Public Repositories` without
+any additional permissions. For private repositories, you need the following permissions:
+  - `Pull requests` with Read-Write access. This is required to check Pull Request
+  changes, list comments, and create or update comments.
+  - `Metadata` with Read-only access. This is automatically marked as "mandatory"
+  because of the permissions listed above.
+3. General knowledge about TF-Controller [(see docs)](https://weaveworks.github.io/tf-controller/).
 
-## Quick Start Guide
+## Quick Start
 
-This section describe how to install Branch Planner using HelmRelease object in the `flux-system` namespace with minimum configuration on a KinD cluster.
+This section describes how to install Branch Planner using a HelmRelease object in the `flux-system` namespace with minimum configuration on a KinD cluster.
 
 1. Create a KinD cluster.
 ```
 kind create cluster
 ```
 
-2. Install Flux. Please make sure you have the latest version of Flux (v2 GA).
+2. Install Flux. Make sure you have the latest version of Flux (v2 GA).
+
 ```
 flux install
 ```
 
-3. Create a Secret that contains GitHub API token. If you do not use `gh` cli, please feel free to copy and paste the token from GitHub's website.
+3. Create a secret that contains a GitHub API token. If you do not use the `gh` CLI, copy and paste the token from GitHub's website.
+
 ```
 export GITHUB_TOKEN=$(gh auth token)
 
@@ -36,14 +36,14 @@ kubectl create secret generic branch-planner-token \
     --from-literal="token=${GITHUB_TOKEN}"
 ```
 
-4. Install Branch Planner from a HelmRelease provided by the TF-controller repository. Please make sure that you use TF Controller v0.16.0-rc.2 or later.
+4. Install Branch Planner from a HelmRelease provided by the TF-Controller repository. Use TF-Controller v0.16.0-rc.2 or later.
+
 ```
 kubectl apply -f https://raw.githubusercontent.com/weaveworks/tf-controller/fa4b3b85d316340d897fda4fed757265ba2cd30e/docs/branch_planner/release.yaml
 ```
 
-5. Create a Terraform object with a Source pointing to a repository.
-You repository must contain a Terraform file, for example `main.tf`.
-Please take a look at [https://github.com/tf-controller/branch-planner-demo](https://github.com/tf-controller/branch-planner-demo) for an example.
+5. Create a Terraform object with a Source pointing to a repository. Your repository must contain a Terraform file—for example, `main.tf`. Check out [this demo](https://github.com/tf-controller/branch-planner-demo) for an example.
+
 ```bash
 export GITHUB_USER=<your user>
 export GITHUB_REPO=<your repo>
@@ -76,15 +76,17 @@ spec:
     namespace: flux-system
 EOF
 ```
-6. Now you can go to your GitHub repo and create a Pull Request. The Branch Planner will create a new Terraform object with Plan Only mode enabled, and generate a new plan for you.
+
+6. Now you can create a pull request on your GitHub repo. The Branch Planner will create a new Terraform object with the plan-only mode enabled and will generate a new plan for you. It will post the plan as a new comment in the pull request.
 
 ## Configure Branch Planner
 
-Branch Planner uses a ConfigMap as configuration. That ConfigMap is optional to use but useful for fine-tuning Branch Planner.
+Branch Planner uses a ConfigMap as configuration. The ConfigMap is optional but useful for fine-tuning Branch Planner.
 
-### Custom Configuration
+### Configuration
 
-By default, Branch Planner will look for the `branch-planner` ConfigMap in the same namespace as where the `tf-controller` is installed. That ConfigMap allows users to precisely specify which Terraform resources in a cluster should be monitored by Branch Planner.
+By default, Branch Planner will look for the `branch-planner` ConfigMap in the same namespace as where the TF-Controller is installed.
+That ConfigMap allows users to specify which Terraform resources in a cluster the Brach Planner should monitor.
 
 The ConfigMap has two fields:
 
@@ -102,12 +104,13 @@ data:
   secretName: branch-planner-token
   resources: |-
     - namespace: terraform
+    - namespace: flux-system
 ```
 
 #### Secret
 
 Branch Planner uses the referenced Secret with a `token` field that acquires the
-API token to fetch Pull Request information.
+API token to fetch pull request information.
 
 ```bash
 kubectl create secret generic branch-planner-token \
@@ -134,7 +137,7 @@ data:
 
 ### Default Configuration
 
-If a ConfigMap is not found, it will watch the `flux-system` namespace for any Terraform resources and expect to find a GitHub token in a secret named `branch-planner-token` in the `flux-system` namespace. Note that supplying a secret with a token is a necessary task, otherwise Branch Planner will not be able to interact with the GitHub API. 
+If no ConfigMap is found, the Branch Planner will not watch any namespaces for Terraform resources and look for a GitHub token in a secret named `branch-planner-token` in the `flux-system` namespace. Supplying a secret with a token is a necessary task, otherwise Branch Planner will not be able to interact with the GitHub API.
 
 ## Enable Branch Planner
 

docs/branch_planner/tfc-integration.md → docs/branch_planner/branch-planner-tfc-integration-getting-started.md

@@ -1,75 +1,15 @@
-# Terraform Cloud Integration with Branch Planner
+## Branch Planner and Terraform Cloud Integration: Getting Started
 
-Terraform Cloud is a secure and robust platform designed to store the Terraform states 
-for your production systems. When working with Infrastructure as Code, 
-managing and ensuring the state is both secure and consistent is critical. 
+With Branch Planner, you can provision the `main` branch directly on Terraform Cloud. TF-Controller communicates with Terraform Cloud to run the necessary plans and apply your approved code. The state is securely stored on Terraform Cloud.
 
-The introduction of TF-Controller’s support for Terraform Cloud has further enhanced 
-the capabilities of managing Terraform operations through Kubernetes.
-
-## First-class Support Terraform Cloud
-
-TF-Controller is not just limited to supporting Terraform Cloud, 
-but it also extends its capabilities to Terraform Enterprise. 
-By utilizing the `spec.cloud` in Terraform CRD, users can seamlessly
-integrate their Kubernetes configurations with Terraform workflows 
-both with Terraform Cloud and Terraform Enterprise.
-
-To get started, all you need to do is putting your Terraform Cloud token in a Kubernetes Secret
-and specify it in the `spec.cliConfigSecretRef` field of the Terraform CR.
-Field `spec.cloud` is used to specify the organization and workspace name.
-
-After connecting your Terraform CR with Terraform Cloud,
-TF-Controller can now send your Terraform resources to be planned and applied via Terraform Cloud. 
-What’s more, states are automatically stored in your Terraform Cloud's workspace. 
-To use TF-Controller with Terraform Cloud `spec.approvalPlan` must be set to `auto`. 
-
-Here's a quick look at how the configuration looks:
-
-```yaml
-apiVersion: infra.contrib.fluxcd.io/v1alpha2
-kind: Terraform
-metadata:
-  name: branch-planner-tfc
-  namespace: flux-system
-spec:
-  interval: 2m
-  approvePlan: auto
-  cloud:
-    organization: weaveworks
-    workspaces:
-      name: branch-planner-tfc
-  cliConfigSecretRef:
-    name: tfc-cli-config
-    namespace: flux-system
-```
-
-## Enhancing the GitOps Workflow with Branch Planner
-
-The GitOps methodology aims to streamline the infrastructure provisioning and management using Git as the source of truth.
-The newly introduced Branch Planner is a component of TF-Controller that aims to take this a notch higher.
-
-Branch Planner allows developers and operations teams to plan Terraform configurations specifically on a separate branch.
-With this feature, the `main` branch can be provisioned directly on Terraform Cloud. 
-However, if you’re looking to test or review changes, you can simply create a new branch.
-
-The most important feature of Brach Planner is the seamless integration with the PR (Pull Request) user interface, 
-which is familiar territory for many developers. When changes are proposed on this new branch, 
-Branch Planner runs a plan in the cluster, and displays the results directly as comments under your PR.
-This makes it easier to review and understand the potential impact of your changes before they are applied.
-
-Once you're satisfied with the results, merging your branch into the `main` branch triggers the TF-Controller. 
-It communicates with Terraform Cloud to run the necessary plans and apply your approved code. 
-The state, as always, is securely stored on Terraform Cloud.
-
-**Note:** In its tech preview version, Branch Planner currently only supports GitHub as the Git provider.
+**Note:** For now, Branch Planner only supports GitHub as the Git provider. We plan to add other Git providers in time.
 
 ## Step-by-step Guide
 
 ### Step 1: Create a Terraform Cloud Token
 
-You can use `terraform login` command to obtain a Terraform Cloud token.
-Then use the token to create a Kubernetes Secret.
+Use the `terraform login` command to obtain a Terraform Cloud token. Then use the token to create a Kubernetes Secret.
+
 ```bash
 kubectl create secret generic \
   tfc-cli-config \
@@ -80,11 +20,12 @@ credentials "app.terraform.io" {
 }
 EOF
 ```
+
 ### Step 2: Create a Terraform CR
 
-The following example shows how to create a Terraform CR to automatically plan and apply Terraform configurations on Terraform Cloud.
-It reads the Terraform configurations from a Git repository, plan, apply and stores the state in a Terraform Cloud workspace.
-The token from Step 1 is specified as the value of `spec.cliConfigSecretRef` and used to authenticate with Terraform Cloud.
+Create a Terraform CR to automatically plan and apply Terraform configurations on Terraform Cloud. In this example, the Branch Planner reads the Terraform configurations from a Git repository to plan, apply, and store the state in a Terraform Cloud workspace.
+
+The token from Step 1 is specified as the value of `spec.cliConfigSecretRef` and is used to authenticate with Terraform Cloud.
 
 ```yaml
 ---
@@ -124,31 +65,34 @@ spec:
     namespace: flux-system
 ```
 
-### Step 3: Edit file, Create a Branch, and Open a Pull Request
+### Step 3: Edit File, Create a Branch, and Open a Pull Request
 
 1. **Navigate to Your Repository:** Open a web browser and visit your GitHub repository. 
-For our example, navigate to https://github.com/tf-controller/branch-planner-demo.
+For our example, navigate [here](https://github.com/tf-controller/branch-planner-demo).
 
 2. **Locate the File to Edit:** Browse through the repository's file structure and 
 click on the Terraform configuration file you wish to edit.
 
 3. **Edit the File:** Click on the pencil icon (edit) located on the top right of the file content.
-Make your desired changes to the Terraform configurations. For instance, you might change the Hello world content in the `main.tf` file.
+Make your desired changes to the Terraform configurations. For instance, you might change the "Hello World" content in the `main.tf` file.
+
 Once you've made your edits, scroll down to prepare to commit the changes.
 
 4. **Commit the Changes to a New Branch:** Instead of committing directly to the `main` branch, 
 choose the option to "Create a new branch" for this commit and start a pull request.
-Name the branch something descriptive, for example, `change-hello-world-message`.
-Click on the Propose Changes button.
+Name the branch something descriptive—for example, `change-hello-world-message`.
+
+Click on the "Propose Changes" button.
 
 5. **Open a Pull Request (PR):** After proposing your changes, you'll be led to the "Open a pull request" page.
 Fill in the details of your PR, explaining the changes you made, their purpose, and any other pertinent information.
-Click on the **[Create Pull Request]** button.
+
+Click on the "Create Pull Request" button.
 
 6. **Review Terraform Plan in PR Comments:** Once the PR is created,
 the Branch Planner will trigger a Terraform plan. After the plan is completed,
 the results will be posted as a comment on the PR.
-This provides an opportunity for you and your team to review the expected changes before they're applied.
+This enables you and your team to review the expected changes before they're applied.
 
 ### Step 4: Review, Approve and Merge the Pull Request
 

docs/branch_planner/index.md

@@ -1,15 +1,14 @@
-# Branch Planner User Guide
+# Branch Planner Overview
 
-## Overview
+The GitOps methodology streamlines infrastructure provisioning and management, using Git as the source of truth. The Branch Planner, a component of TF-Controller, aims to take this a step further by allowing developers and operations teams to plan Terraform configurations on a branch that's separate from the `main` branch. This makes it easier to review and understand the potential impact of your changes before you run `terraform apply`.
 
-The Branch Planner, a new component of the Terraform Controller, is specifically designed to enhance the flexibility and robustness of Terraform Controller planning operations. This feature, currently in its technology preview phase, facilitates Terraform planning across branches, creating a streamlined and familiar PR-based workflow for users.
+The Branch Planner's most important feature is its seamless integration with the PR (Pull Request) user interface. When enabled through Helm values, it watches repositories that contain Terraform resources at regular intervals—checking their referenced Source, and polling for Pull Requests using GitHub's API and the provided token. When changes are proposed on a new branch, Branch Planner runs a plan in the cluster and displays the results directly as comments on your PR. Once you're satisfied with the results, you can merge your branch into the `main` branch to trigger the TF-Controller to reconcile the updated code.
 
-### How does it work?
-
-When the Branch Planner starts, it polls repositories that contain Terraform resources at regular intervals, in order to detect Pull Requests (PR) that change those resources. Upon detecting that a PR exists, the Branch Planner initialises a Terraform object in Plan Only mode for the corresponding branch. In this mode, Terraform Controller generates Terraform plans but does not apply them. Once the plan is generated, Branch Planner posts the plan under the PR as a comment enabling users to review the plan. When the Terraform files of the corresponding branch get updated, Branch Planner posts the updated plan under the PR as new comment, keeping the PR up-to-date with the latest Terraform plan.
+![branch planner](branch-planner.png)
 
 ### Replan commands
 
 The Branch Planner also allows users to manually trigger the replan process. By simply commenting `!replan` under the PR, the Branch Planner will be instructed to generate a new plan and post it under the PR as a new comment.
 
-Now that you know what Branch Planner can do for you, follow the [guide to get started](./getting-started.md).
+Now that you know what Branch Planner can do for you, follow the [guide to get started](./branch-planner-getting-started.md).
+

docs/use_tf_controller/with-azure.md

@@ -0,0 +1,61 @@
+## Use TF-Controller with Azure
+
+This content was [provided](https://github.com/weaveworks/tf-controller/issues/561) by users [@mingmingshiliyu](https://github.com/mingmingshiliyu) and [@maciekdude](https://github.com/maciekdude).
+
+Use the OIDC flag and explicitly point to the token. Due to a bug in AzureRM 3.44.x, use version 3.47.x or later.
+
+Set env variables on the runner pod:
+
+```
+        - name: ARM_USE_OIDC
+          value: "true"
+        - name: ARM_OIDC_TOKEN_FILE_PATH
+          value: "/var/run/secrets/azure/tokens/azure-identity-token"
+```
+
+Example yaml:
+
+```yaml
+apiVersion: infra.contrib.fluxcd.io/v1alpha1
+kind: Terraform
+metadata:
+  name: terraformhello
+  namespace: default
+spec:
+  tfstate:
+    forceUnlock: auto
+  backendConfig:
+    customConfiguration: |
+      backend "azurerm" {
+        resource_group_name  = "l"
+        storage_account_name = ""
+        container_name       = "tfstate"
+        key                  = "helloworld.tfstate"
+        use_oidc             = true
+      }
+  interval: 1m
+  serviceAccountName: service_account_registered_in_aad
+  approvePlan: auto
+  destroy: true
+  path: ./tests/fixture
+  sourceRef:
+    kind: GitRepository
+    name: terraformhello
+    namespace: flux-system
+  runnerPodTemplate:
+    spec:
+      image: azure_cli_runner.xxx
+      env:
+        - name: ARM_USE_OIDC
+          value: "true"
+        - name: ARM_SUBSCRIPTION_ID
+          value: ""
+        - name: ARM_TENANT_ID
+          value: ""
+        - name: ARM_CLIENT_ID
+          value: ""
+        - name: ARM_OIDC_TOKEN_FILE_PATH
+          value: "/var/run/secrets/azure/tokens/azure-identity-token"
+```
+
+Import existing resources to a tfstate file stored on a storage account.