Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Kubernets docs #1220

Merged
merged 24 commits into from
Dec 27, 2023
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
d92fc43
doc: Add Azure kubernetes section
happyRip Nov 24, 2023
8dc7c3d
doc: Add troubleshooting
happyRip Dec 5, 2023
552908b
doc: Fix ECS deployment Database Operations CLI example
happyRip Dec 5, 2023
7d608f3
doc: Fix TTS shortcodes
happyRip Dec 7, 2023
998267f
doc: Remove emphasis from references
happyRip Dec 7, 2023
0b3786b
doc: Various fixes
happyRip Dec 7, 2023
2cf0738
doc: Add missing references
happyRip Dec 7, 2023
80a2f00
doc: Add config values
happyRip Dec 8, 2023
999a702
doc: Add a link to Azure Portal
happyRip Dec 8, 2023
6ea013f
doc: Add a warning about Bitnami Redis
happyRip Dec 8, 2023
89e5267
doc: Add deployment explanation
happyRip Dec 11, 2023
869a7d0
doc: Fix spelling
happyRip Dec 11, 2023
2817927
doc: Change `self-hosted` section name to `generic`
happyRip Dec 11, 2023
d3b3066
doc: Add Azure Architecture overview
happyRip Dec 15, 2023
c49020b
doc: Add Azure prerequisites
happyRip Dec 18, 2023
e1967b7
doc: Add an experimental warning to Azure K8s
happyRip Dec 18, 2023
851be69
doc: Restructure Kubernetes docs
happyRip Dec 18, 2023
b555eff
doc: Azure Architecture fixes
happyRip Dec 21, 2023
69c6ce8
doc: Add a note in Configuration section
happyRip Dec 21, 2023
996c51a
doc: Fix Configuration example values
happyRip Dec 21, 2023
e893af4
doc: List rights required to create resources
happyRip Dec 21, 2023
b23fca5
doc: Fix Kubernetes flavours
happyRip Dec 22, 2023
4d85076
doc: Add Azure Architecture section link
happyRip Dec 22, 2023
ccdb191
doc: Fix spelling
happyRip Dec 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Alternatively, you can run ops commands using the AWS CLI:
aws ecs run-task \
--task-definition TASK_DEFINITION \
--cluster CLUSTER_NAME \
--launch-type EC2 \
--launch-type FARGATE \
--network-configuration '{
"awsvpcConfiguration":{
"subnets":[
Expand Down
11 changes: 11 additions & 0 deletions doc/content/the-things-stack/host/kubernetes/azure/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
title: "Azure"
description: ""
weight: 1
---

{{% tts %}} can be deployed to Azure as a set of highly available services on Azure Kubernetes Service using Terraform and Helm.
happyRip marked this conversation as resolved.
Show resolved Hide resolved
KrishnaIyer marked this conversation as resolved.
Show resolved Hide resolved

This guide gives an overview of the architecture and shows how to deploy your own highly available {{% tts %}} cluster.
KrishnaIyer marked this conversation as resolved.
Show resolved Hide resolved

<!--more-->
KrishnaIyer marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
title: "Configuration"
description: ""
weight: 2
aliases: [/getting-started/kubernetes/azure/configuration]
---

{{% tts %}} requires a few configuration files to be prepared. In this section, we show how to create them and store them for further use.

<!--more-->

## Terraform Backend

Create a `config.azurerm.tfbackend` file and put it in `1-infrastructure`, `2-kubernetes` and one of the chosen DNS provider templates directories. We recommend `azure-dns` provider for this deployment.

```
storage_account_name = <storage_account_name>
container_name = <storage_container_name>
key = <terraform_state_file>
use_azuread_auth = true
happyRip marked this conversation as resolved.
Show resolved Hide resolved
```

## Deployment Configuration

Create a `deployment.auto.tfvars.json` file in `1-infrastructure` directory.

The following contains only the minimum mandatory fields for this configuration file. For a full list of possible values check the `variables.tf` file in this directory.

```
{
"azure_ad_admin_group_object_id": # Object ID of the AKS admin group.
"deployment_name": # Name of the deployment.
"environment": # 'prod', 'staging' or 'dev'.
"cluster": # Cluster identifier for multi-cluster deployments.
"location": # Azure location
"resource_group": {
"create": # If set to `true` a new Resource Group will be created on deployment.
# Otherwise a Resource Group is going to be imported based on "name" parameter.
"name": # Optional custom Azure Resource Group name. Mandatory when "create" is set to `false`.
},
"domain": {
"name": # Domain where The Things Stack is available.
"dns_zone": # Azure DNS zone.
}
}
happyRip marked this conversation as resolved.
Show resolved Hide resolved
```

## ACME Configuration

Create an `acme.auto.tfvars.json` file in the DNS templates directory.

It is only required to set the `acme_email` field.

```
{
"acme_email": # ACME email that will receive notifications about expiring Certificates.
}
```

## {{% tts %}} Values

Create a `tts.values.yaml` file in the `2-kubernetes` templates directory.

The following contains only the minimum mandatory fields for this values file. For a full list of possible values check the `values.yaml` file of {{% tts %}} Helm chart.

```yaml
license:
key: # TTS license key
global:
deployment:
initialTenant:
tenantID: # Initial tenant ID
adminEmail: # Admin email
adminUserID: # Admin ID
adminPassword: # Admin password
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
---
title: "Deployment"
description: ""
weight: 3
---

This page describes the steps for deploying {{% tts %}} on Azure Kubernetes Service.

<!--more-->

## Azure Infrastructure

Start with the `1-infrastructure` templates that create the foundation for your deployment.

If you haven't done this before create a Terraform backend configuration file `config.azurerm.tfbackend`. You can read more about it in the [Configuration]({{< ref "/the-things-stack/host/kubernetes/azure/configuration#terraform-backend" >}}) section.

Initialize the Terraform state.

```bash
$ make init
```

Plan the deployment and verify everything is going to be set up correctly.

```bash
$ make plan
```

Now apply the template.
KrishnaIyer marked this conversation as resolved.
Show resolved Hide resolved

```bash
$ make apply
```

Get the `kubeconfig` next.

```bash
# Set KUBECONFIG if you don't want to override the default kubeconfig.
$ export KUBECONFIG='local.kubeconfig'

# Set KUBE_CONFIG_PATH to the location of the kubeconfig.
# If you didn't set the KUBECONFIG variable use '~/.kube/config' instead.
# Ref https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs#file-config
$ export KUBE_CONFIG_PATH="$(pwd)/${KUBECONFIG}"

$ az aks get-credentials --resource-group $(terraform output -raw resource_group_name) --name $(terraform output -raw aks_cluster_name)
```

{{< note >}}
If `var.allow_cluster_admin_access` is disabled, then `kubelogin` must be installed.
happyRip marked this conversation as resolved.
Show resolved Hide resolved
{{</ note >}}

Follow the promp to authenticate the `kubectl`.
happyRip marked this conversation as resolved.
Show resolved Hide resolved

```bash
$ kubectl get pods
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code xxxxxx to authenticate.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the second line accidentally in bash block?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the first line starting with a $ is the command used and the following lines are the output of that command.

```

Export the DNS values used for [DNS Infrastructure]({{< relref "#dns-infrastructure" >}}). Use the provider chosen during the [Configuration]({{< ref "/the-things-stack/host/kubernetes/azure/configuration#acme-configuration" >}}).

```bash
$ DNS_PROVIDER='<provider>'
$ make dns.values | jq > "$(git rev-parse --show-toplevel)/dns/${DNS_PROVIDER}/dns.auto.tfvars.json"
```

Export the Infrastructural values used for [Kubernetes Templates]({{< relref "#kubernetes-templates" >}}).

```bash
$ make infra.values | jq > '../2-kubernetes/infra.auto.tfvars.json'
```

## DNS Infrastructure

Now we need to set up the DNS infrastructure. Head over to the directory of the provider you chose during the last steps of the [Azure Infrastructure]({{< relref "#azure-infrastructure" >}}) section.

If you haven't done this before create a Terraform backend configuration file `config.azurerm.tfbackend`. You can read more about it in [Terraform Backend Configuration]({{< relref "/the-things-stack/host/kubernetes/azure/configuration#terraform-backend" >}}).

Initialize the Terraform state.

{{< note >}}
`KUBE_CONFIG_PATH` should point to your `kubeconfig`.

```bash
# Example
$ export KUBE_CONFIG_PATH='~/.kube/config'
```

{{</ note >}}

```bash
$ make init
```

Plan the deployment and verify everything is going to be set up correctly.

```bash
$ make plan
```

Now apply the template.

```bash
$ make apply
```

Verify the certificate is in ready state. This may take a couple of minutes after the deployment for the certificate to reach a ready state.

```bash
$ kubectl -n tts get cert
```

In case the certificate won't reach a ready state head over to [Troubleshooting Problems with ACME / Let's Encrypt Certificates](https://cert-manager.io/docs/troubleshooting/acme/).

## Kubernetes Templates

Head over to the `2-kubernetes` directory next.

If you haven't done this before create a Terraform backend configuration file `config.azurerm.tfbackend`. You can read more about it in [Terraform Backend Configuration]({{< relref "/the-things-stack/host/kubernetes/azure/configuration#terraform-backend" >}}).

Initialize the Terraform state.

```bash
$ make init
```

Plan the deployment and verify everything is going to be set up correctly.

{{< note >}}
`KUBE_CONFIG_PATH` should point to your `kubeconfig`.

```bash
# Example
$ export KUBE_CONFIG_PATH='~/.kube/config'
```

{{</ note >}}

```bash
$ make plan
```

Now apply the template.

```bash
$ make apply
```

Once all the targets are healthy and if the configuration was correct, {{% tts %}} console will be accessible at `https://<tenantID>.<domain>`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
title: "Prerequisites"
description: ""
weight: 1
aliases: [/getting-started/kubernetes/azure/prerequisites]
---

This section contains prerequisites for deploying {{% tts %}} on Azure Kubernetes Service.

<!--more-->

## Tools

1. [Terraform](https://www.terraform.io/)
2. [`kubectl`](https://kubernetes.io/docs/reference/kubectl/)
3. [Helm version 3](https://helm.sh/docs/intro/). Check the version of Helm installed using `helm version`. Helm v3.8 or above is required.
happyRip marked this conversation as resolved.
Show resolved Hide resolved

## License

{{% tts %}} requires a license key to run. Please [contact our sales team](mailto:[email protected]) for your license key.

## (Optional) Packet Broker Access

{{% tts %}} contains the Packet Broker Agent component that can communicate with [Packet Broker](https://packetbroker.net/).

Packet Broker is disabled by default in the Helm charts. When enabled, it can operate either only a Forwarder or as both a Forwarder and a Home Network. Check the [Packet Broker section]({{< ref "/the-things-stack/packet-broker" >}}) for more details.

- If the cluster acts simply as a Forwarder that forwards traffic to Packet Broker, then all that is needed are access credentials.
- If the cluster also needs to work as a Packer Broker Home Network, in addition to the access credentials, the cluster either needs a NetID from the LoRa Alliance or The Things Industries can lease a DevAddr Block.

Please [contact our sales team](mailto:[email protected]) for access credentials and a Device Address Block (if necessary).
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: "Troubleshooting"
description: ""
weight: 9
aliases: [/getting-started/kubernetes/azure/troubleshooting]
---

This guide contains general troubleshooting information.

<!--more-->

## Error: Get "http://localhost/api/v1/namespaces/...": dial tcp [::1]:80: connect: connection refused
happyRip marked this conversation as resolved.
Show resolved Hide resolved

Set `KUBECONFIG` and `KUBE_CONFIG_PATH` variables to the location of the `kubeconfig` file. The default location is `~/.local/kubeconfig`.

## pkg/util/store:driver (driver error)

{{% tts %}} runs Kubernetes jobs to initialize and migrate Postgres. This error can occur if the {{% tts %}} is accessed either before these jobs are run or if the jobs failed to execute. Check the status of the jobs for more details on what went wrong.
39 changes: 0 additions & 39 deletions doc/content/the-things-stack/host/kubernetes/preparation/_index.md

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
title: "Self Managed"
description: ""
weight: 2
---

This guide is meant for users who want to deploy {{% tts %}} on Kubernetes with self-hosted infrastructure. It gives an overview of the Helm charts and the steps required to deploy them.

<!--more-->
happyRip marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ aliases: [/getting-started/kubernetes/configuration]

{{% tts %}} Helm chart is configured using `yaml` files. This guide assumes that these fields are saved in a `<deployment>.values.yaml` file and used for deployment/updates.

<!--more-->

The following is a list of mandatory minimum fields. For a full list of possible values, check the `values.yaml` file at the root of the {{% tts %}} Helm chart.

{{< warning >}} Some values in this file are secrets. Make sure to check this file into a secure repository.{{</ warning >}}
Expand All @@ -28,6 +30,7 @@ global:
region: # region
azure: # Set only if provider is "azure".
accountName: # account name
clientID: # client ID
gcp: # Set only if provider is "gcp".
# Base64 encoded GCP credentials.json file.
# One option is to run `$ cat <credentials>.json | base64`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ aliases: [/getting-started/kubernetes/install-charts]

Use the following steps to install the chart.

<!--more-->

1. Login to the registry. This step can be skipped if you've already logged in via Docker Client.

```bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ aliases: [/getting-started/kubernetes/monitoring]

{{% tts %}} exposes Prometheus metrics at the `/metrics` endpoint. This route is protected by Basic Auth with username `metrics` and password `global.http.metrics.password`.

<!--more-->

To scrape these metrics, you can install Prometheus in the cluster. The installation details are outside the scope of this document.

The following scrape config can be used to scrape metrics from each of the components of The Things Stack.
Expand Down
Loading
Loading