Merge branch 'TheThingsIndustries:master' into lilygo

doc/config/_default/config.toml

@@ -24,7 +24,7 @@ pygmentsUseClasses = true
   github_repository = "https://github.com/TheThingsIndustries/lorawan-stack-docs"
   github_repository_edit = "https://github.com/TheThingsIndustries/lorawan-stack-docs/blob/master/doc/content"
   tts_github_repository = "https://github.com/TheThingsNetwork/lorawan-stack"
-  version = "3.30.2"
+  version = "3.32.0"
 
 [markup]
   [markup.goldmark]

doc/content/api/concepts/auth/_index.md

@@ -58,10 +58,9 @@ In this section we explain how to create a _user API key_, which can be used to
 
 {{< tabs/tab "Console" >}}
 
-In the users dropdown on the right side, select the **Personal API Keys** option and select **+Add API Key**.
+In the **User settings** dropdown on the left-hand sidebar, select the **API Keys** option and select **+ Add API Key**.
 
 {{< figure src="personal-api-keys.png" alt="Personal API key" >}}
-{{< figure src="add-api-key-button.png" alt="Add API key" >}}
 
 Choose an optional expiry date and choose the required rights.
 {{< figure src="application-rights.png" alt="Selectable rights" >}}

doc/content/api/concepts/auth/oauth-access-tokens/_index.md

@@ -22,7 +22,9 @@ To register OAuth client you can use {{% tts %}} [Console]({{< ref "/the-things-
 
 {{< tabs/tab "Console" >}}
 
-To register an OAuth client using the Console, first navigate to `https://<HOSTNAME>/oauth` in your web browser to log into your Account App. When logged in, switch to the **OAuth Clients** tab on the top menu and click the **Add OAuth Client** button.
+To register an OAuth client using the Console, click the **User settings** dropdown in the left-hand sidebar and select **OAuth clients**. Then click **+ Add OAuth client** in the top-right.
+
+{{< figure src="oauth-client.png" alt="OAuth client" >}}
 
 Enter the **OAuth Client ID**, **Name** and **Description** for your OAuth client. You can also add the **Redirect URLs** against which authorization requests will be checked, and **Logout redirect URLs** agains which client initiated requests are checked.
 

doc/content/api/reference/grpc/end_device.md

@@ -355,6 +355,8 @@ See the [EndDevice message](#message:EndDevice) and its sub-messages for additio
 
 {{< proto/message message="ListEndDevicesRequest" >}}
 
+{{< proto/message message="ListEndDevicesRequest.Filter" >}}
+
 {{< proto/message message="Location" >}}
 
 {{< proto/message message="LoRaAllianceProfileIdentifiers" >}}

doc/content/api/reference/grpc/gateway.md

@@ -146,6 +146,8 @@ The Gateway Server exposes the list of available frequency plans with the `Confi
 
 {{< proto/message message="ListGatewaysRequest" >}}
 
+{{< proto/message message="ListGatewaysRequest.Filter" >}}
+
 {{< proto/message message="Location" >}}
 
 {{< proto/message message="OrganizationOrUserIdentifiers" >}}

doc/content/api/reference/grpc/gateway_server.md

@@ -14,10 +14,6 @@ List of Gateway Server APIs.
 
 {{< proto/method service="Gs" method="BatchGetGatewayConnectionStats" >}}
 
-## The `GatewayConfigurator` service
-
-{{< proto/method service="GatewayConfigurator" method="PullConfiguration" >}}
-
 ## The `GatewayConfigurationService` service
 
 {{< proto/method service="GatewayConfigurationService" method="GetGatewayConfiguration" >}}

doc/content/api/reference/grpc/tenant.md

@@ -110,6 +110,10 @@ Unlike the other services, the tenant services do **not** accept API keys or OAu
 
 {{< proto/message package="tti.lorawan.v3" message="Configuration.Cluster.ApplicationServer.Webhooks.Queue" >}}
 
+{{< proto/message package="tti.lorawan.v3" message="Configuration.Cluster.GatewayServer" >}}
+
+{{< proto/message package="tti.lorawan.v3" message="Configuration.Cluster.GatewayServer.MTLSAuthentication" >}}
+
 {{< proto/message package="tti.lorawan.v3" message="Configuration.Cluster.IdentityServer" >}}
 
 {{< proto/message package="tti.lorawan.v3" message="Configuration.Cluster.IdentityServer.UserRegistration" >}}

doc/content/devices/adding-devices/_index.md

@@ -21,7 +21,7 @@ This guide assumes that your device is in the [LoRaWAN® Device Repository](http
 
 To create a device, first open the application you wish to add the device in. Go to **End devices** in the left menu and click on **+Register end device** to reach the end device registration page.
 
-{{< figure src="application-overview.png" alt="Application overview" >}}
+{{< figure src="application-end-devices.png" alt="Application overview" >}}
 
 You will be presented with options to easily onboard your device using its QR code (if you have it), and to register your end device from the [LoRaWAN® Device Repository](https://github.com/TheThingsNetwork/lorawan-devices/) or manually.
 
@@ -83,7 +83,7 @@ For LoRaWAN version 1.0.x devices, you will see an **AppKey** field, and for LoR
 
 {{< /tabs/container >}}
 
-Finally, the console pre-fills **End device ID** in the format `eui-{Device EUI}` by default. You can edit this field and give the device a unique identifier. See [ID and EUI constraints]({{< ref "reference/id-eui-constraints" >}}) for guidelines about choosing a unique ID.
+Finally, enter an **End device ID**. Advisable is doing it in the format `eui-{Device EUI}`. You can edit this field and give the device a unique identifier. See [ID and EUI constraints]({{< ref "reference/id-eui-constraints" >}}) for guidelines about choosing a unique ID.
 
 Now verify that all the fields are filled and click the **Register end device** button to create the end device.
 

doc/content/devices/adding-devices/manual/_index.md

@@ -26,4 +26,4 @@ Choosing the incorrect LoRaWAN version can lead to complex errors. Activation ma
 
 {{< figure src="device-input-method.png" alt="ABP input method" >}}
 
-The above fields are common for both OTAA and ABP devices. Now, based on the device you are trying to add, choose the speficic guide for OTAA or ABP below.
+The above fields are common for both OTAA and ABP devices. Now, based on the device you are trying to add, choose the specific guide for OTAA or ABP below.

doc/content/devices/configuring-devices/device-location/_index.md

@@ -19,6 +19,8 @@ The end device location can be manually set in two ways.
 1. Pinning the location on the map widget.
 2. Entering the **Latitude**, **Longitude** and **Altitude** values.
 
+Make sure you select **Save Changes** at the bottom of the page.
+
 {{< figure src="device-location.png" alt="Gateway location" >}}
 
 {{< /tabs/tab >}}

doc/content/gateways/concepts/adding-gateways/_index.md

@@ -222,9 +222,9 @@ curl -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" \
 
 {{< tabs/tab "Console" >}}
 
-Once you have added your gateway to {{% tts %}}, you can also set its location to be displayed on a map widget by clicking **Change location settings**.
+Once you have added your gateway to {{% tts %}}, you can also set its location to be displayed on a map widget by clicking **Location** in the left-hand menu.
 
-If you do not mind your gateway's location to be publicly displayed, check the **Publish location** box.
+If you do not mind your gateway's location to be publicly displayed, check the **Share location within network** box.
 
 The gateway location can be manually set by entering the **Latitude**, **Longitude** and **Altitude** values.
 

doc/content/gateways/concepts/udp/_index.md

@@ -25,7 +25,7 @@ An example `global_conf.json` is available in the [{{% udp-pf %}} Github reposit
 
 ## Download Configuration in the Console {#download-configuration-in-the-console}
 
-To download a `global_conf.json` file for your gateway, open the Gateway overview page in the console. Click the **Download global_conf.json** button to download the file.
+To download a `global_conf.json` file for your gateway, open the Gateway overview page in the console. Click the **Hamburger menu** in the top-right and then click **Download global_conf.json** to download the file.
 
 {{< figure src="conf.png" alt="Download global_conf.json" >}}
 

doc/content/integrations/webhooks/retries.md

@@ -10,7 +10,8 @@ The webhook retries feature allows webhooks to be enqueued and be retried multip
 
 <!--more-->
 
-{{< note  "The webhook retries feature in The Things Stack is currently in beta. Please contact [email protected] to evaluate this feature on your {{% tts %}} Cloud tenant." />}}
+A [{{% tts %}} Cloud Plus subscription](https://www.thethingsindustries.com/stack/plans/) is required to use this feature.
+If you would like to evaluate this feature on your {{% tts %}} Cloud tenant without a {{% tts %}} Cloud Plus subscription, please contact `[email protected]`.
 
 By default, webhook HTTP requests are tried at most once for each application uplink. This means that if the endpoint is unavailable temporarily, the HTTP requests done in this period are lost, and so is the associated traffic.
 
@@ -20,11 +21,9 @@ Webhook queuing allows individual HTTP requests to be retried at later times, wh
 
 The webhook retries feature can be enabled on both new and existing webhooks via the Console, by ticking the **Retries** checkbox in the webhook settings:
 
-{{< figure src="../webhook-queue-before.png" alt="Enabling webhook retries" >}}
-
 After the checkbox has been ticked, and the webhook has been saved, future requests will be enqueued and retried on failure.
 
-{{< figure src="../webhook-queue-after.png" alt="Webhook queueing enabled" >}}
+{{< figure src="../webhook-retries.png" alt="Webhook queueing enabled" >}}
 
 ## What counts as a failed try ?
 

doc/content/the-things-stack/alerting/_index.md

@@ -10,4 +10,5 @@ Alerting in {{% tts %}} refers to notifications which are sent when certain cond
 
 <!--more-->
 
-{{< note  "Alerting in The Things Stack is currently in beta. Please contact [email protected] to evaluate this feature on your {{% tts %}} Cloud tenant." />}}
+A [{{% tts %}} Cloud Plus subscription](https://www.thethingsindustries.com/stack/plans/) is required to use this feature.
+If you would like to evaluate this feature on your {{% tts %}} Cloud tenant without a {{% tts %}} Cloud Plus subscription, please contact `[email protected]`.

doc/content/the-things-stack/alerting/receivers-and-profiles/_index.md

@@ -14,22 +14,18 @@ Alert notifications are dispatched by {{% tts %}} using _alert notification rece
 
 ## Alert Notification Receivers
 
-Alert notification receivers represent endpoints to which {{% tts %}} sends alert notifications. An endpoint in this case can be an email address, a phone number or an URL. The receivers can be managed via the Console, using the **Alerting** section of the **Admin Panel**.
-
-{{< figure src="receivers-landing.png" alt="Alert Notification Receivers overview" >}}
+Alert notification receivers represent endpoints to which {{% tts %}} sends alert notifications. An endpoint in this case can be an email address, a phone number or an URL. The receivers can be managed via the Console, using the **Alerting settings** section of the **Admin Panel**.
 
 By clicking on **Add an alert receiver** one can create a new alert notification receiver. Fill in the details and click on **Save changes** in order to save your newly created receiver.
 
-{{< figure src="add-receiver.png" alt="Add Alert Notification Receiver" >}}
+{{< figure src="receiver.png" alt="Add Alert Notification Receiver" >}}
 
 ## Alert Notification Profiles
 
 Once the alert notifications receivers are setup, we can create alert notification profiles. Alert notification profiles are bundles of alert notification receivers which can be attached to entities in order to dispatch alert notifications to multiple endpoints. It is also possible to set a default alert notification profile which is used for entities without an explicit profile set.
 
-The profiles can be managed also via the Console, by switching to the the **Alert Profiles** tab in the **Alerting** section of the **Admin Panel**.
-
-{{< figure src="profiles-landing.png" alt="Alert Notification Profiles overview" >}}
+The profiles can be managed also via the Console, by switching to the the **Alert Profiles** tab in the **Alerting settings** section of the **Admin Panel**.
 
-By clicking on *Add an alert profile** one can create a new alert notification profile. Fill in the details and click on **Save changes** in order to save your newly created profile. It is also possible to mark this profile as the default profile in this page.
+By clicking on **Add an alert profile** one can create a new alert notification profile. Fill in the details and click on **Save changes** in order to save your newly created profile. It is also possible to mark this profile as the default profile in this page.
 
-{{< figure src="add-profile.png" alt="Add Alert Notification Profile" >}}
+{{< figure src="profile.png" alt="Add Alert Notification Profile" >}}

doc/content/the-things-stack/host/aws/ecs/changelog/index.md

@@ -2,12 +2,32 @@
 title: "Template Changelog"
 aliases: [/getting-started/aws/ecs/changelog]
 ---
+
 # Upgrading
 
 All meaningful changes to templates are documented in this file.
 
 ## Unreleased
 
+## 3.32.0
+
+- Add support for managed gateways via The Things Gateway Controller. The Gateway Configuration Server and Device Claiming Server use TLS client authentication.
+  - When using AWS Private CA (`CertificateAuthorityARN` in `4-2a-configuration`), the client certificate can be issued automatically.
+  - To specify a custom TLS client certificate, enable `EnableTTGCCustomCertificate` in `4-1-secrets` and specify the certificate and key according to the format in the description.
+- Add support for gateways using The Things Industries Gateway Protocol. This requires TLS mutual authentication and TLS termination by the proxy. Make sure that `SupportProxyTLS` is enabled. This adds a new public listener (port `8889`) that is mapped to the proxy that forwards traffic to the Gateway Server (port `1889`).
+- Fixed the rate-limiting profile for the `ApplicationUpStorage` service in the Application Server.
+- Add default values for the default and maximum page sizes in the `ApplicationUpStorage` service in the Application Server.
+
+## 3.31.1
+
+### Proxy
+
+- Add `TenantAccess` grpc service and routes.
+
+## 3.31.0
+
+- Updated ECS AMIs to the latest versions.
+
 ## 3.30.2
 
 - TimescaleDB replicas are now split to a separate template. Previously replicas had an ephemeral disk that needed to be copied when the replica was re-deployed. This made some upgrades really long to complete. Now the replicas are standalone and have their own disk that can be reattached to a new instance. This change should make upgrades faster and more reliable.

doc/content/the-things-stack/interact/console/login/_index.md

@@ -13,12 +13,12 @@ On the Console landing page, click **Login with your {{% tts %}} Account** in or
 
 {{< figure src="login.png" alt="Login" >}}
 
-If you do not have an account yet, you can register one by clicking **Create an account**, filling your details and clicking **Register**.
+If you do not have an account yet, you can register one by clicking **Login using credentials** and then **Create an account**. Then fill in your details and click **Create account**.
 
 {{< figure src="register.png" alt="Create an account" >}}
 
 When you use a new account, you may not be able to continue until you have confirmed your email address or your account is approved by an admin user.
 
-After entering your credentials and logging in, you will reach the Console overview page.
+After entering your credentials and logging in, you will reach the Console Home page.
 
 {{< figure src="overview.png" alt="Overview" >}}

doc/content/the-things-stack/interact/console/personal-settings/_index.md

@@ -33,7 +33,7 @@ Click **Save changes** to save any modified info.
 
 ### Change password
 
-To set up a new password for your account, click the **Expand** button next to the **Change password** section.
+To set up a new password for your account, click **Password** in the left-hand sidebar.
 
 Type in your **Current password**, then create a **New password** and confirm it. We highly recommend you create a strong password to enhance your profile's security.
 
@@ -51,7 +51,7 @@ In this section, we explain how to create personal API keys that grant certain r
 
 {{< tabs/tab "Console" >}}
 
-To create a personal API key, click on your avatar in the upper right corner in the Console and select **Personal API keys**.
+To create a personal API key, click on **User settings** in the left-hand sidebar and select **API keys**.
 
 Enter a **Name** for your key, set the **Expiry date**, select rights that you want to grant and then press **Create API Key**.
 

doc/content/the-things-stack/management/user-management/org/_index.md

@@ -24,7 +24,7 @@ To manage organizations, click the **Organizations** tab in the top menu.
 
 {{< figure src="orgs.png" alt="Organizations" >}}
 
-To add an organization, click **Add organization**.
+To add an organization, click **Create organization**.
 
 {{< figure src="add-org.png" alt="Add organization" >}}
 
@@ -34,7 +34,9 @@ Use the left hand menu in your organization's settings to add or remove collabor
 
 ### Create Organization API Key
 
-When a user is a member of an organization which is a collaborator for an entity, the user's rights are the intersection of the user's rights in the organization and the organization's rights on the entity. To grant rights to your organization, navigate to **API Keys** on the left hand menu of your organization's settings and select **Add API Key**.
+When a user is a member of an organization which is a collaborator for an entity, the user's rights are the intersection of the user's rights in the organization and the organization's rights on the entity. To grant rights to your organization, navigate to **API Keys** on the top hand menu of your organization's settings and select **Add API Key**.
+
+{{< figure src="organization-api-keys.png" alt="Application API Key" >}}
 
 Enter a **Name** for your key, set the **Expiry date**, select rights that you want to grant and then press **Create API Key**.
 

doc/content/the-things-stack/migrating/migration-tool/export-from-firefly.md

@@ -11,6 +11,7 @@ This section contains instructions on how to configure migration tool and use it
 
 ### Before you begin
 
+- Using the actual JoinEUI of the end devices is mandatory when registering them on {{% tts %}}. Since Firefly does not store the JoinEUI of the device, users need to obtain this information separately, such as from the device manufacturer.
 - The export process will halt if any error occurs.
 - Use the `--invalidate-keys` option to invalidate the root and/or session keys of the devices on the Firefly server. This is necessary to prevent both networks from communicating with the same device. The last byte of the keys will be incremented by 0x01. This enables an easy rollback if necessary. Setting this flag to false (default) would result in a "dry run", where the devices are exported but they will still be able to communicate with the Firefly server.
 
@@ -20,13 +21,13 @@ Configure with environment variables, or command-line arguments.
 
 See `ttn-lw-migrate firefly {device|application} --help` for more details.
 
-The following example shows how to set options via environment variables.
+The following example shows how to set options via environment variables. These are example values. Please use actual ones based on your case.
 
 ```bash
 $ export FIREFLY_HOST=example.com       # Host of the Firefly API
 $ export FIREFLY_API_KEY=abcdefgh       # Firefly API Key
 $ export APP_ID=my-test-app             # Application ID for the exported devices
-$ export JOIN_EUI=1111111111111111      # JoinEUI for the exported devices
+$ export JOIN_EUI=1111111111111111      # JoinEUI for the exported devices.
 $ export FREQUENCY_PLAN_ID=EU_863_870   # Frequency Plan ID for the exported devices
 $ export MAC_VERSION=1.0.2b             # LoRaWAN MAC version for the exported devices
 ```
@@ -60,6 +61,8 @@ FF11111111111134
 ABCD111111111100
 ```
 
+Please note that all these devices should have the same JoinEUI.
+
 And then export with:
 
 ```bash

doc/content/the-things-stack/packet-broker/configure/_index.md

@@ -39,7 +39,7 @@ Packet Broker can be configured using either the Console or the CLI.
 
 {{< tabs/tab "Console" >}}
 
-To configure Packet Broker, click the **Packet Broker** button in the top right menu.
+To configure Packet Broker, click on **Admin panel** in the left-hand menu and select **Packet Broker**.
 
 {{< figure src="pb-menu.png" alt="Packet Broker Menu Button" >}}
 

doc/content/ttn-lw-cli/ttn-lw-cli_end-devices_list.md

@@ -24,6 +24,7 @@ ttn-lw-cli end-devices list [application-id] [flags]
       --claim-authentication-code.valid-to     select the claim_authentication_code.valid_to field
       --claim-authentication-code.value        select the claim_authentication_code.value field
       --description                            select the description field
+      --filter.updated-since updated_at        filter the updated_at field by this timestamp (YYYY-MM-DDTHH:MM:SSZ)
   -h, --help                                   help for list
       --join-server-address                    select the join_server_address field
       --last-seen-at                           select the last_seen_at field