Skip to content

Commit

Permalink
GEFEST-856 Деплой counter в чарте Keys
Browse files Browse the repository at this point in the history
  • Loading branch information
@salkrr
salkrr committed Dec 11, 2024
1 parent 7a56257 commit 8856bec
Show file tree
Hide file tree
Showing 4 changed files with 267 additions and 0 deletions.
31 changes: 31 additions & 0 deletions charts/keys/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,33 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about
| `dispatcher.cleaner.cron.successfulJobsHistoryLimit` | Specifies the number of successful finished jobs to keep. See [jobs history limits](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#jobs-history-limits). | `3` |
| `dispatcher.cleaner.cron.suspend` | You can suspend execution of Jobs for a CronJob, by setting the field to true. See [schedule suspension](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#schedule-suspension). | `false` |
| `dispatcher.cleaner.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}` |
| `counter.enabled` | Counter worker is enabled. | `false` |
| `counter.replicas` | A replica count for the pod. | `1` |
| `counter.resources.requests.cpu` | A CPU request. | `20m` |
| `counter.resources.requests.memory` | A memory request. | `32Mi` |
| `counter.resources.limits.cpu` | A CPU limit. | `1000m` |
| `counter.resources.limits.memory` | A memory limit. | `512Mi` |
| `counter.logLevel` | Log level for the worker. Can be: `trace`, `debug`, `info`, `warning`, `error`, `fatal`. | `warning` |
| `counter.preloader.refreshTick` | Refresh interval for in-memory cache with keys limitations info. The smaller the interval, the faster the worker will know about changes in limitations. | `1m` |
| `counter.updateStatusQueryTimeout` | Timeout for database queries to update key status. | `1s` |
| `counter.buffer` | **Settings for in-memory buffer for statistics data.** | |
| `counter.buffer.size` | The maximum size of the buffer. When the limit is reached, the data from the buffer is transferred to Redis. | `1000` |
| `counter.buffer.delay` | The maximum interval between data transfer operations from the buffer to Redis. | `1s` |

### Redis settings

| Name | Description | Value |
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | ------- |
| `counter.redis.retries` | | `5` |
| `counter.redis.minRetryBackoff` | | `100ms` |
| `counter.redis.maxRetryBackoff` | | `3s` |
| `counter.annotations` | Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
| `counter.labels` | Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
| `counter.podAnnotations` | Kubernetes [pod annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/). | `{}` |
| `counter.podLabels` | Kubernetes [pod labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/). | `{}` |
| `counter.nodeSelector` | Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector). | `{}` |
| `counter.affinity` | Kubernetes pod [affinity settings](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity). | `{}` |
| `counter.tolerations` | Kubernetes [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings. | `{}` |

### Redis settings

Expand Down Expand Up @@ -237,6 +264,10 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about
| `kafka.tls.rootCert` | Root certificate. | `""` |
| `kafka.tls.cert` | Client certificate. | `""` |
| `kafka.tls.key` | Client key. | `""` |
| `kafka.stats` | **Settings for consuming key's usage statistics messages.** | |
| `kafka.stats.topic` | Topic to consume stat messages from. **Required** | `""` |
| `kafka.stats.groupId` | Name of the consumer group to use. **Required** | `""` |
| `kafka.stats.clientId` | Client id. If empty hostname will be used. | `""` |
| `kafka.audit` | **Settings for sending audit messages.** | |
| `kafka.audit.topic` | Topic to produce audit messages. **Required** | `""` |
| `kafka.audit.produce.retryCount` | Number of retries to produce a message. | `5` |
Expand Down
86 changes: 86 additions & 0 deletions charts/keys/templates/counter/statefulset.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
{{- if .Values.counter.enabled }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ include "keys.counter.name" . }}
labels:
{{- include "keys.counter.labels" . | nindent 4 }}
{{- with .Values.counter.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
serviceName: {{ include "keys.counter.name" . }}
replicas: {{ .Values.counter.replicas }}
selector:
matchLabels:
{{- include "keys.counter.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
{{- include "keys.counter.selectorLabels" . | nindent 8 }}
spec:
{{ $kafkaSslEnabled := include "kafka.ssl.enabled" . }}
{{- if $kafkaSslEnabled }}
initContainers:
- name: copy-kafka-certs
image: '{{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag | default .Chart.AppVersion }}'
imagePullPolicy: {{ .Values.imagePullPolicy }}
resources:
{{- toYaml .Values.counter.resources | nindent 12 }}
command:
- /bin/sh
- -ec
- |
cp /tmp/certs/* /etc/2gis/secret/kafka/
chmod 600 /etc/2gis/secret/kafka/client.key
volumeMounts:
- name: {{ include "keys.kafka-raw.name" . | quote }}
mountPath: /tmp/certs
- name: {{ include "keys.kafka.name" . | quote }}
mountPath: /etc/2gis/secret/kafka
{{- end }}
containers:
- name: counter
image: {{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag }}
imagePullPolicy: {{ .Values.imagePullPolicy }}
command: [ "keys-counter" ]
ports:
- name: http
containerPort: 8090
resources:
{{- toYaml .Values.counter.resources | nindent 12 }}
env:
{{- include "keys.env.db.deploys" . | nindent 12 }}
{{- include "keys.env.redis" . | nindent 12 }}
{{- include "keys.env.counter" . | nindent 12 }}
{{- if $kafkaSslEnabled }}
volumeMounts:
- name: {{ printf "%s" (include "keys.kafka.name" .) | quote }}
mountPath: /etc/2gis/secret/kafka
{{- end }}
{{- if or $kafkaSslEnabled }}
volumes:
- name: {{ include "keys.kafka-raw.name" . | quote }}
secret:
secretName: {{ include "keys.kafka.name" . | quote }}
- name: {{ include "keys.kafka.name" . | quote }}
emptyDir: {}
{{- end }}
{{- with .Values.counter.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.counter.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.counter.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
64 changes: 64 additions & 0 deletions charts/keys/templates/helpers.tpl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@
{{ include "keys.name" . }}-cleaner
{{- end }}

{{- define "keys.counter.name" -}}
{{ include "keys.name" . }}-counter
{{- end }}

{{- define "keys.migrate.name" -}}
{{ include "keys.name" . }}-migrate
{{- end }}
Expand Down Expand Up @@ -120,6 +124,16 @@ app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}

{{- define "keys.counter.selectorLabels" -}}
app.kubernetes.io/name: {{ .Chart.Name }}-counter
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{- define "keys.counter.labels" -}}
{{ include "keys.counter.selectorLabels" . }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}

{{- define "keys.import.labels" -}}
app.kubernetes.io/name: {{ .Chart.Name }}-import
app.kubernetes.io/instance: {{ .Release.Name }}
Expand Down Expand Up @@ -324,6 +338,56 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
value: "{{ .Values.admin.badge.backgroundColor }}"
{{- end }}

{{- define "keys.env.counter" -}}
- name: KEYS_LOG_LEVEL
value: "{{ .Values.counter.logLevel }}"
- name: KEYS_COUNTER_BUFFER_SIZE
value: "{{ .Values.counter.buffer.size }}"
- name: KEYS_COUNTER_BUFFER_DELAY
value: "{{ .Values.counter.buffer.delay }}"
- name: KEYS_COUNTER_PRELOADER_REFRESH_TICK
value: "{{ .Values.counter.preloader.refreshTick }}"
- name: KEYS_COUNTER_UPDATE_STATUS_QUERY_TIMEOUT
value: "{{ .Values.counter.updateStatusQueryTimeout }}"
- name: KEYS_KAFKA_MAIN_BROKERS
value: "{{ required "A valid .Values.kafka.bootstrapServers entry required" .Values.kafka.bootstrapServers }}"
- name: KEYS_KAFKA_MAIN_GROUP_ID
value: "{{ required "A valid .Values.kafka.stats.groupId entry required" .Values.kafka.stats.groupId }}"
- name: KEYS_KAFKA_MAIN_CLIENT_ID
value: "{{ .Values.kafka.stats.clientId }}"
- name: KEYS_KAFKA_MAIN_STATS_TOPIC
value: "{{ required "A valid .Values.kafka.stats.topic entry required" .Values.kafka.stats.topic }}"
- name: KEYS_KAFKA_MAIN_USERNAME
value: "{{ .Values.kafka.username }}"
{{- if .Values.kafka.password }}
- name: KEYS_KAFKA_MAIN_PASSWORD
valueFrom:
secretKeyRef:
name: {{ include "keys.kafka.name" . }}
key: password
{{- end }}
- name: KEYS_REDIS_RETRIES
value: "{{ .Values.counter.redis.retries }}"
- name: KEYS_REDIS_MIN_RETRY_BACKOFF
value: "{{ .Values.counter.redis.minRetryBackoff }}"
- name: KEYS_REDIS_MAX_RETRY_BACKOFF
value: "{{ .Values.counter.redis.maxRetryBackoff }}"
- name: KEYS_KAFKA_MAIN_SECURITY_PROTOCOL
value: "{{ .Values.kafka.securityProtocol }}"
- name: KEYS_KAFKA_MAIN_SASL_MECHANISM
value: "{{ .Values.kafka.saslMechanism }}"
{{- if (include "kafka.ssl.enabled" .) }}
- name: KEYS_KAFKA_MAIN_TLS_SKIP_SERVER_CERTIFICATE_VERIFY
value: "{{ .Values.kafka.tls.skipServerCertificateVerify }}"
- name: KEYS_KAFKA_MAIN_TLS_CLIENT_CERTIFICATE_PATH
value: "/etc/2gis/secret/kafka/client.crt"
- name: KEYS_KAFKA_MAIN_TLS_CLIENT_KEY_PATH
value: "/etc/2gis/secret/kafka/client.key"
- name: KEYS_KAFKA_MAIN_TLS_CA_CERT_PATH
value: "/etc/2gis/secret/kafka/ca.crt"
{{- end }}
{{- end }}

{{- define "keys.env.predef" -}}
{{ range $service, $key := .Values.predefined.service.keys }}
- name: KEYS_PREDEF_SERVICE_KEY_{{ $service | upper }}
Expand Down
86 changes: 86 additions & 0 deletions charts/keys/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -437,6 +437,81 @@ dispatcher:
nodeSelector: {}


counter:

# @param counter.enabled Counter worker is enabled.

enabled: false

# @param counter.replicas A replica count for the pod.

replicas: 1

# @param counter.resources.requests.cpu A CPU request.
# @param counter.resources.requests.memory A memory request.
# @param counter.resources.limits.cpu A CPU limit.
# @param counter.resources.limits.memory A memory limit.

resources:
requests:
cpu: 20m
memory: 32Mi
limits:
cpu: 1000m
memory: 512Mi

# @param counter.logLevel Log level for the worker. Can be: `trace`, `debug`, `info`, `warning`, `error`, `fatal`.
logLevel: warning

# @param counter.preloader.refreshTick Refresh interval for in-memory cache with keys limitations info. The smaller the interval, the faster the worker will know about changes in limitations.

preloader:
refreshTick: 1m

# @param counter.updateStatusQueryTimeout Timeout for database queries to update key status.

updateStatusQueryTimeout: 1s

# @extra counter.buffer **Settings for in-memory buffer for statistics data.**

# @param counter.buffer.size The maximum size of the buffer. When the limit is reached, the data from the buffer is transferred to Redis.
# @param counter.buffer.delay The maximum interval between data transfer operations from the buffer to Redis.

buffer:
size: 1000
delay: 1s

# @section Redis settings

# @param counter.redis.retries
# @param counter.redis.minRetryBackoff
# @param counter.redis.maxRetryBackoff

redis:
retries: 5
minRetryBackoff: 100ms
maxRetryBackoff: 3s

# @param counter.annotations Kubernetes [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/).
# @param counter.labels Kubernetes [labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).

annotations: {}
labels: {}

# @param counter.podAnnotations Kubernetes [pod annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/).
# @param counter.podLabels Kubernetes [pod labels](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/).

podAnnotations: {}
podLabels: {}

# @param counter.nodeSelector Kubernetes [node selectors](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector).
# @param counter.affinity Kubernetes pod [affinity settings](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity).
# @param counter.tolerations Kubernetes [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) settings.

nodeSelector: {}
affinity: {}
tolerations: {}

# @section Redis settings

redis:
Expand Down Expand Up @@ -559,6 +634,17 @@ kafka:
cert: ''
key: ''

# @extra kafka.stats **Settings for consuming key's usage statistics messages.**

# @param kafka.stats.topic Topic to consume stat messages from. **Required**
# @param kafka.stats.groupId Name of the consumer group to use. **Required**
# @param kafka.stats.clientId Client id. If empty hostname will be used.

stats:
topic: ''
groupId: ''
clientId: ''

# @extra kafka.audit **Settings for sending audit messages.**

# @param kafka.audit.topic Topic to produce audit messages. **Required**
Expand Down

0 comments on commit 8856bec

Please sign in to comment.