Merge branch 'develop' into catalog-importer-schema-extensions

2gis · Dec 7, 2023 · 9f2eadd · 9f2eadd
2 parents ae86dae + de605a1
commit 9f2eadd
Show file tree

Hide file tree

Showing 82 changed files with 871 additions and 408 deletions.
diff --git a/.github/workflows/lint.yaml b/.github/workflows/lint.yaml
@@ -6,6 +6,7 @@ on:
   pull_request:
     branches:
       - master
+      - develop
 
 jobs:
   lint:

diff --git a/Breaking-Changes.md b/Breaking-Changes.md
@@ -6,12 +6,38 @@
 
 - Changes in data for catalog, if you have a version lower than 1.16.0, you need to update to version 1.16.0 to get the latest data
 
+## [1.15.0]
+
+### navi-router
+
+- `router.keyManagementService.apis.*` tokens renamed, `-api` suffix added
+
+## [1.14.0]
+
+### keys-api
+
+- Added new required dgctlStorage parameters
+
+```yaml
+dgctlStorage:
+  host: ''
+  bucket: keys
+  accessKey: ''
+  secretKey: ''
+  manifest: manifest.json
+```
+
 ## [1.13.0]
 
 ### tiles-api
 
 - Backward compatibility with `license` versions before `2.0.0` (on-premise version `1.8.0`) is broken.
 - License v2 over HTTPS is required.
+- Update migrations, when updating the service you need to update the data
+
+### pro-api
+
+- Add required topic `assetDataTopic`
 
 ## [1.12.0]
 

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,50 @@
 # 2GIS On-Premise Changelog
 
+## [1.15.0] (2023-11-30)
+#### [Breaking-Changes](Breaking-Changes.md#1500)
+#### Images
+```
+keys
+	- keys-ui:0.5.0
+	+ keys-ui:0.6.0
+navi-castle
+	- navi-castle:1.0.7
+	+ navi-castle:1.7.0
+navi-router
+	- navi-router:6.16.0
+	+ navi-router:6.17.0.8
+
+```
+
+## [1.14.0] (2023-11-15)
+#### [Breaking-Changes](Breaking-Changes.md#1400)
+#### Images
+```
+keys
+	- keys-backend:1.63.0
+	+ keys-backend:1.69.0
+pro-api
+	- pro-api:1.1.57
+	+ pro-api:1.1.60
+	- pro-importer:1.1.57
+	+ pro-importer:1.1.60
+	- pro-importer:1.1.57
+	+ pro-importer:1.1.60
+	- pro-permissions-api:1.1.57
+	+ pro-permissions-api:1.1.60
+pro-ui
+	- pro-ui:1.2.0
+	+ pro-ui:1.3.0
+tiles-api
+	- tiles-api-importer:4.49.0
+	+ tiles-api-importer:4.51.1
+	- tiles-api-proxy:4.49.0
+	+ tiles-api-proxy:4.51.1
+	- tiles-api:4.49.0
+	+ tiles-api:4.51.1
+
+```
+
 ## [1.13.0] (2023-10-24)
 #### [Breaking-Changes](Breaking-Changes.md#1300)
 #### Images

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,78 @@
+# Contributing to [on-premise-helm-charts](https://github.com/2gis/on-premise-helm-charts/tree/master)
+
+Thank you for your interest in contributing to our Helm charts repository. We welcome contributions from the community and appreciate your help in improving our charts.
+
+Before you start, please take a moment to review the following guidelines to ensure a smooth and productive contribution process.
+
+## Table of Contents
+
+- [How to Contribute](#how-to-contribute)
+  - [Contribution Guidelines](#contribution-guidelines)
+- [Requirements for the design of PR](#requirements-for-the-design-of-pr)
+- [Gitflow](#gitflow)
+  - [Basic guidelines](#basic-guidelines)
+  - [Urgent bugfix guidelines](#urgent-bugfix-guidelines)
+
+## How to Contribute
+
+- We are open to getting contributions from other people. Please note that DCO is not required.
+
+- We are only interested in accepting contributions that improve the quality of our software. Please make sure that your contributions are well-tested and documented.
+
+### Contribution Guidelines
+
+- Create a new branch for your changes. Choose a descriptive branch name.
+- Make your changes, ensuring they follow our Style Guide (see [Style Guide](styleguide.md)).
+- Make your changes and commit them with clear, concise messages to your branch.
+- Test your changes locally to verify they work as expected.
+- Create a pull request (PR) to the `develop` branch.
+- Provide a clear and detailed description of your changes in the PR (see [Requirements for the design of PR](#requirements-for-the-design-of-pr)).
+- Be responsive to any feedback or questions on your PR.
+
+Once your PR is approved, it will be merged. Thank you for your contribution!
+
+## Requirements for the design of PR
+
+- New appVersion (or tag) specified
+- The title of the PR begins with the services name, the addition feature/bugfix is welcome, and the short description of the feature follows. Good examples of PR names:
+  > LICENSE Feature: add super license;
+  
+  > [tiles-api] Upgraded version and added breaking changes
+  
+  > navi-back: upstream sync
+- The PRs description clearly explains the feature and how to check it.
+Good example of the description:
+
+  > A values.yml file contains a new value, value1. The purpose of this is as follows. Value1 could have "a" or "b" values. One may verify this new feature by utilizing the curl command ('curl example')
+
+## Gitflow
+
+This project uses the Gitflow branching model.
+
+### Basic Guidelines
+
+- The main branch is `master`.
+- All development should happen on feature branches.
+- Feature branches should be named with a descriptive name.
+- Once a feature branch is complete, it should be merged into the `develop` branch with a pull request.
+- The maintainers team will review PRs once a week (on Mondays).
+- The PR will be reviewed if:
+  - it is not in Draft
+  - it is not in WIP
+- In the event of questions about a PR, the need for improvements, the lack of an image provided to us, etc., we start a discussion. The next time this PR will be reviewed the following week (i.e., the team has a week to fix, improve, push, etc.). This way, everything will be predictable and work can be planned.
+- After the PR is reviewed, it is merged into develop.
+- If it is necessary to create a branch for a task, teams branch off from develop (this is where the latest changes will be).
+- When preparing for a release, nothing additional will be taken except for what has already been reviewed and merged into develop. We check all deployments and the operation from the develop branch. If everything is OK, we take it to master, set the tag of the new release with the minor bump.
+
+### Urgent bugfix Guidelines
+
+If it is necessary to release an urgent bug fix without going through the develop branch:
+
+- Create a branch from master.
+- Make the fix.
+- Create a PR to master.
+- The maintainers team checks the fix and all deployments (as in a standard release).
+- Merge the branch into master, set the tag of the new release with a bugfix bump.
+- Move this commit to develop
+
+![GitFlow](GitFlow.png)
diff --git a/GitFlow.png b/GitFlow.png
diff --git a/README.md b/README.md
@@ -3,3 +3,19 @@
 This repository contains [Helm charts](https://helm.sh/docs/topics/charts/) for deploying [2GIS products](https://dev.2gis.com/) on your own sites.
 
 To learn more about 2GIS On-Premise services, visit [docs.2gis.com](https://docs.2gis.com/en/on-premise/overview).
+
+## Contributing
+
+For information on how to contribute to the project, please see the [CONTRIBUTING.md](CONTRIBUTING.md) file.
+
+## Changelog
+
+For a complete changelog of the project, please see the [CHANGELOG.md](CHANGELOG.md) file.
+
+## Breaking Changes
+
+For information on any breaking changes in the project, please see the [Breaking-Changes.md](Breaking-Changes.md) file.
+
+## Style Guide
+
+For information on the project style guide, please see the [styleguide.md](styleguide.md) file.
diff --git a/charts/catalog-api/Chart.yaml b/charts/catalog-api/Chart.yaml
@@ -3,7 +3,7 @@ name: catalog-api
 type: application
 description: A Helm chart for Kubernetes to deploy Catalog APIs
 
-version: 1.13.0
+version: 1.15.0
 appVersion: 3.600.0
 
 maintainers:

diff --git a/charts/citylens/Chart.yaml b/charts/citylens/Chart.yaml
@@ -3,7 +3,7 @@ name: citylens
 type: application
 description: A Helm chart for Kubernetes to deploy Citylens service
 
-version: 1.13.0
+version: 1.15.0
 appVersion: 1.1.2
 
 maintainers:

diff --git a/charts/dgtt/Chart.yaml b/charts/dgtt/Chart.yaml
@@ -3,7 +3,7 @@ name: dgtt
 description: DGTT Helm chart for Kubernetes
 type: application
 
-version: 1.13.0
+version: 1.15.0
 appVersion: 0.2.27-b188673e09
 
 maintainers:

diff --git a/charts/floors-api/Chart.yaml b/charts/floors-api/Chart.yaml
@@ -3,7 +3,7 @@ name: floors-api
 description: Helm for floors service
 type: application
 
-version: 1.13.0
+version: 1.15.0
 appVersion: 1.0.4
 
 maintainers:

diff --git a/charts/gis-platform/Chart.yaml b/charts/gis-platform/Chart.yaml
@@ -4,7 +4,7 @@ description: GIS Platform
 
 type: application
 
-version: 1.13.0
+version: 1.15.0
 appVersion: 2023.8.3-0
 
 dependencies:

diff --git a/charts/keycloak/Chart.yaml b/charts/keycloak/Chart.yaml
@@ -5,7 +5,7 @@ description: Keycloak is a high performance Java-based identity and access manag
   solution. It lets developers add an authentication layer to their applications with
   minimum effort.
 
-version: 1.13.0
+version: 1.15.0
 appVersion: 21.1.1-debian-11-r4
 
 maintainers:

diff --git a/charts/keys/Chart.yaml b/charts/keys/Chart.yaml
@@ -3,8 +3,8 @@ name: keys
 type: application
 description: A Helm chart for Kubernetes to deploy API Keys service
 
-version: 1.13.0
-appVersion: 1.63.0
+version: 1.15.0
+appVersion: 1.69.0
 
 maintainers:
 - name: 2gis

diff --git a/charts/keys/README.md b/charts/keys/README.md
@@ -31,9 +31,9 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about
 | `imagePullSecrets`         | Kubernetes image pull secrets.    | `[]`                           |
 | `imagePullPolicy`          | Pull policy.                      | `IfNotPresent`                 |
 | `backend.image.repository` | Backend service image repository. | `2gis-on-premise/keys-backend` |
-| `backend.image.tag`        | Backend service image tag.        | `1.63.0`                       |
+| `backend.image.tag`        | Backend service image tag.        | `1.69.0`                       |
 | `admin.image.repository`   | Admin service image repository.   | `2gis-on-premise/keys-ui`      |
-| `admin.image.tag`          | Admin service image tag.          | `0.5.0`                        |
+| `admin.image.tag`          | Admin service image tag.          | `0.6.0`                        |
 | `redis.image.repository`   | Redis image repository.           | `2gis-on-premise/keys-redis`   |
 | `redis.image.tag`          | Redis image tag.                  | `6.2.6-alpine3.15`             |
 
@@ -78,6 +78,7 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about
 | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- |
 | `api.adminUsers`                            | Usernames and passwords of admin users. Format: `username1:password1,username2:password2`.                                                                                                                                 | `""`            |
 | `api.adminSessionTTL`                       | TTL of the admin users sessions. Duration string is a sequence of decimal numbers with optional fraction and unit suffix, like `100ms`, `2.3h` or `4h35m`. Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. | `336h`          |
+| `api.logLevel`                              | Log level for the service. Can be: `trace`, `debug`, `info`, `warning`, `error`, `fatal`.                                                                                                                                  | `warning`       |
 | `api.replicas`                              | A replica count for the pod.                                                                                                                                                                                               | `1`             |
 | `api.strategy.type`                         | Type of Kubernetes deployment. Can be `Recreate` or `RollingUpdate`.                                                                                                                                                       | `RollingUpdate` |
 | `api.strategy.rollingUpdate.maxUnavailable` | Maximum number of pods that can be created over the desired number of pods when doing [rolling update](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment).                   | `0`             |
@@ -115,21 +116,24 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about
 
 ### Import service settings
 
-| Name                  | Description                                                                                                         | Value |
-| --------------------- | ------------------------------------------------------------------------------------------------------------------- | ----- |
-| `import.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}`  |
+| Name                  | Description                                                                                                         | Value     |
+| --------------------- | ------------------------------------------------------------------------------------------------------------------- | --------- |
+| `import.logLevel`     | Log level for the service. Can be: `trace`, `debug`, `info`, `warning`, `error`, `fatal`.                           | `warning` |
+| `import.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}`      |
 
 ### Migrate service settings
 
-| Name                          | Description                                                                                                         | Value |
-| ----------------------------- | ------------------------------------------------------------------------------------------------------------------- | ----- |
-| `migrate.initialDelaySeconds` | Delay in seconds at the service startup.                                                                            | `0`   |
-| `migrate.nodeSelector`        | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}`  |
+| Name                          | Description                                                                                                         | Value     |
+| ----------------------------- | ------------------------------------------------------------------------------------------------------------------- | --------- |
+| `migrate.logLevel`            | Log level for the service. Can be: `trace`, `debug`, `info`, `warning`, `error`, `fatal`.                           | `warning` |
+| `migrate.initialDelaySeconds` | Delay in seconds at the service startup.                                                                            | `0`       |
+| `migrate.nodeSelector`        | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}`      |
 
 ### Tasker service settings
 
 | Name                                           | Description                                                                                                                                                                                              | Value           |
 | ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- |
+| `tasker.logLevel`                              | Log level for the service. Can be: `trace`, `debug`, `info`, `warning`, `error`, `fatal`.                                                                                                                | `warning`       |
 | `tasker.delay`                                 | Delay in seconds at the service startup.                                                                                                                                                                 | `30s`           |
 | `tasker.strategy.type`                         | Type of Kubernetes deployment. Can be `Recreate` or `RollingUpdate`.                                                                                                                                     | `RollingUpdate` |
 | `tasker.strategy.rollingUpdate.maxUnavailable` | Maximum number of pods that can be created over the desired number of pods when doing [rolling update](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#rolling-update-deployment). | `0`             |
@@ -209,6 +213,16 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about
 | `predefined.service.keys`    | Keys map as: service -> key.    | `{}`  |
 | `predefined.service.aliases` | Aliases map as: service -> key. | `{}`  |
 
+### Deployment Artifacts Storage settings
+
+| Name                     | Description                                                                                                                                                                                                                                              | Value           |
+| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- |
+| `dgctlStorage.host`      | S3 endpoint. Format: `host:port`. **Required**                                                                                                                                                                                                           | `""`            |
+| `dgctlStorage.bucket`    | S3 bucket name.                                                                                                                                                                                                                                          | `keys`          |
+| `dgctlStorage.accessKey` | S3 access key for accessing the bucket. **Required**                                                                                                                                                                                                     | `""`            |
+| `dgctlStorage.secretKey` | S3 secret key for accessing the bucket. **Required**                                                                                                                                                                                                     | `""`            |
+| `dgctlStorage.manifest`  | The path to the [manifest file](https://docs.2gis.com/en/on-premise/overview#nav-lvl2@paramCommon_deployment_steps). Format: `manifests/0000000000.json` <br> This file contains the description of pieces of data that the service requires to operate. | `manifest.json` |
+
 ### Limits
 
 | Name                                | Description                        | Value   |

diff --git a/charts/keys/templates/api/deployment.yaml b/charts/keys/templates/api/deployment.yaml
@@ -47,6 +47,7 @@ spec:
           resources:
             {{- toYaml .Values.api.resources | nindent 12 }}
           env:
+            {{- include "keys.env.api" . | nindent 12 }}
             {{- include "keys.env.db.deploys" . | nindent 12 }}
             {{- include "keys.env.redis" . | nindent 12 }}
             {{- include "keys.env.auth" . | nindent 12 }}

diff --git a/charts/keys/templates/helpers.tpl b/charts/keys/templates/helpers.tpl
@@ -96,8 +96,24 @@ app.kubernetes.io/instance: {{ .Release.Name }}
 app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
 {{- end }}
 
+{{- define "keys.env.api" -}}
+- name: KEYS_LOG_LEVEL
+  value: "{{ .Values.api.logLevel }}"
+{{- end }}
+
+{{- define "keys.env.import" -}}
+- name: KEYS_LOG_LEVEL
+  value: "{{ .Values.import.logLevel }}"
+{{- end }}
+
+{{- define "keys.env.migrate" -}}
+- name: KEYS_LOG_LEVEL
+  value: "{{ .Values.migrate.logLevel }}"
+{{- end }}
 
 {{- define "keys.env.tasker" -}}
+- name: KEYS_LOG_LEVEL
+  value: "{{ .Values.tasker.logLevel }}"
 - name: KEYS_TASKER_DELAY
   value: "{{ .Values.tasker.delay }}"
 {{- end }}
@@ -247,6 +263,25 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
 {{ end }}
 {{- end }}
 
+{{- define "keys.env.dgctlStorage" -}}
+- name: KEYS_S3_ENDPOINT
+  value: "{{ .Values.dgctlStorage.host }}"
+- name: KEYS_S3_BUCKET
+  value: "{{ .Values.dgctlStorage.bucket }}"
+- name: KEYS_S3_ACCESS_KEY
+  valueFrom:
+    secretKeyRef:
+      name: {{ include "keys.secret.jobs.name" . }}
+      key: dgctlStorageAccessKey
+- name: KEYS_S3_SECRET_KEY
+  valueFrom:
+    secretKeyRef:
+      name: {{ include "keys.secret.jobs.name" . }}
+      key: dgctlStorageSecretKey
+- name: KEYS_MANIFEST_PATH
+  value: "{{ required "A valid .Values.dgctlStorage.manifest entry required" .Values.dgctlStorage.manifest }}"
+{{- end }}
+
 {{/*
 Return the target Kubernetes version
 */}}
-Original file line number
+Diff line change
@@ Expand Up / @@ -6,6 +6,7 @@ on: @@
       pull_request:
         branches:
           - master
+          - develop
     jobs:
       lint:
@@ Expand Down @@