Skip to content

Commit

Permalink
Add in kafdrop helm subchart for alert-stream-broker and enable value…
Browse files Browse the repository at this point in the history
…s for usdf-dev-alert-stream-broker. Add kafdrop kafa user template. The usdf-dev-alert-stream-broker is connecting to the external bootstrap because TLS is disabled for that listener.
  • Loading branch information
dspeck1 committed Apr 25, 2024
1 parent 31f6183 commit 31175c1
Show file tree
Hide file tree
Showing 15 changed files with 414 additions and 0 deletions.
4 changes: 4 additions & 0 deletions applications/alert-stream-broker/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ dependencies:
- name: alert-database
version: 2.1.0

- name: kafdrop
condition: kafdrop.enabled
version: 1.0.0

- name: strimzi-registry-operator
version: 2.1.0
repository: https://lsst-sqre.github.io/charts/
Expand Down
26 changes: 26 additions & 0 deletions applications/alert-stream-broker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Alert transmission to community brokers

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| kafdrop.enabled | bool | `false` | Whether Kafdrop is enabled |
| strimzi-registry-operator.clusterName | string | `"alert-broker"` | |
| strimzi-registry-operator.clusterNamespace | string | `"alert-stream-broker"` | |
| strimzi-registry-operator.operatorNamespace | string | `"alert-stream-broker"` | |
Expand Down Expand Up @@ -82,6 +83,7 @@ Alert transmission to community brokers
| alert-stream-broker.maxMillisecondsRetained | string | `"604800000"` | Maximum amount of time to save simulated alerts in the replay topic, in milliseconds. Default is 7 days. |
| alert-stream-broker.nameOverride | string | `""` | |
| alert-stream-broker.schemaID | int | `1` | Integer ID to use in the prefix of alert data packets. This should be a valid Confluent Schema Registry ID associated with the schema used. |
| alert-stream-broker.serviceAccounts.kafdrop.enabled | bool | `true` | |
| alert-stream-broker.strimziAPIVersion | string | `"v1beta2"` | Version of the Strimzi Custom Resource API. The correct value depends on the deployed version of Strimzi. See [this blog post](https://strimzi.io/blog/2021/04/29/api-conversion/) for more. |
| alert-stream-broker.superusers | list | `["kafka-admin"]` | A list of usernames for users who should have global admin permissions. These users will be created, along with their credentials. |
| alert-stream-broker.testTopicName | string | `"alert-stream-test"` | Name of the topic which will be used to send test alerts. |
Expand Down Expand Up @@ -125,3 +127,27 @@ Alert transmission to community brokers
| alert-stream-simulator.schemaID | int | `1` | Integer ID to use in the prefix of alert data packets. This should be a valid Confluent Schema Registry ID associated with the schema used. |
| alert-stream-simulator.staticTopicName | string | `"alerts-static"` | Name of the topic which will hold a static single visit of sample data. |
| alert-stream-simulator.strimziAPIVersion | string | `"v1beta2"` | API version of the Strimzi installation's custom resource definitions |
| kafdrop.affinity | object | `{}` | Affinity configuration |
| kafdrop.cmdArgs | string | See `values.yaml` | Command line arguments to Kafdrop |
| kafdrop.existingSecret | string | Do not use a secret | Existing Kubernetes secrect use to set kafdrop environment variables. Set `SCHEMAREGISTRY_AUTH` for basic auth credentials in the form `<username>:<password>` |
| kafdrop.host | string | `"localhost"` | The hostname to report for the RMI registry (used for JMX) |
| kafdrop.image.pullPolicy | string | `"IfNotPresent"` | Image pull policy |
| kafdrop.image.repository | string | `"obsidiandynamics/kafdrop"` | Kafdrop Docker image repository |
| kafdrop.image.tag | string | `"4.0.1"` | Kafdrop image version |
| kafdrop.ingress.annotations | object | `{}` | Additional ingress annotations |
| kafdrop.ingress.enabled | bool | `false` | Whether to enable the ingress |
| kafdrop.ingress.hostname | string | None, must be set if ingress is enabled | Ingress hostname |
| kafdrop.ingress.path | string | `"/kafdrop"` | Ingress path |
| kafdrop.jmx.port | int | `8686` | Port to use for JMX. If unspecified, JMX will not be exposed. |
| kafdrop.jvm.opts | string | `""` | JVM options |
| kafdrop.kafka.broker | string | `""` | Bootstrap list of Kafka host/port pairs |
| kafdrop.nodeSelector | object | `{}` | Node selector configuration |
| kafdrop.podAnnotations | object | `{}` | Pod annotations |
| kafdrop.replicaCount | int | `1` | Number of kafdrop pods to run in the deployment. |
| kafdrop.resources | object | See `values.yaml` | Kubernetes requests and limits for Kafdrop |
| kafdrop.schemaregistry | string | `""` | The endpoint of Schema Registry |
| kafdrop.server.port | int | `9000` | The web server port to listen on |
| kafdrop.server.servlet.contextPath | string | `"/kafdrop"` | The context path to serve requests on |
| kafdrop.service.annotations | object | `{}` | Additional annotations to add to the service |
| kafdrop.service.port | int | `9000` | Service port |
| kafdrop.tolerations | list | `[]` | Tolerations configuration |
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Kafka broker cluster for distributing alerts
| maxMillisecondsRetained | string | `"604800000"` | Maximum amount of time to save simulated alerts in the replay topic, in milliseconds. Default is 7 days. |
| nameOverride | string | `""` | |
| schemaID | int | `1` | Integer ID to use in the prefix of alert data packets. This should be a valid Confluent Schema Registry ID associated with the schema used. |
| serviceAccounts.kafdrop.enabled | bool | `true` | |
| strimziAPIVersion | string | `"v1beta2"` | Version of the Strimzi Custom Resource API. The correct value depends on the deployed version of Strimzi. See [this blog post](https://strimzi.io/blog/2021/04/29/api-conversion/) for more. |
| superusers | list | `["kafka-admin"]` | A list of usernames for users who should have global admin permissions. These users will be created, along with their credentials. |
| testTopicName | string | `"alert-stream-test"` | Name of the topic which will be used to send test alerts. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,35 @@ spec:
operation: All
{{- end }}
{{- end }}
{{- if .Values.serviceAccounts.kafdrop.enabled }}
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
name: kafdrop
labels:
strimzi.io/cluster: {{ .Values.cluster.name }}
spec:
authentication:
type: scram-sha-512
password:
valueFrom:
secretKeyRef:
name: alert-stream-broker-secrets
key: kafdrop-password
authorization:
type: simple
acls:
- resource:
type: group
name: "*"
patternType: literal
operation: All
- resource:
type: topic
name: "*"
patternType: literal
type: allow
host: "*"
operation: All
{{- end }}
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@ users:
# matches.
groups: ["rubin-testing"]

# Service Accounts to enable
serviceAccounts:
kafdrop:
enabled: true

zookeeper:
# -- Number of Zookeeper replicas to run.
Expand Down
7 changes: 7 additions & 0 deletions applications/alert-stream-broker/charts/kafdrop/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: v2
name: kafdrop
version: 1.0.0
description: A subchart to deploy the Kafdrop UI for Sasquatch.
sources:
- https://github.com/obsidiandynamics/kafdrop
appVersion: 3.30.0
36 changes: 36 additions & 0 deletions applications/alert-stream-broker/charts/kafdrop/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# kafdrop

A subchart to deploy the Kafdrop UI for Sasquatch.

## Source Code

* <https://github.com/obsidiandynamics/kafdrop>

## Values

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| affinity | object | `{}` | Affinity configuration |
| cmdArgs | string | See `values.yaml` | Command line arguments to Kafdrop |
| existingSecret | string | Do not use a secret | Existing Kubernetes secrect use to set kafdrop environment variables. Set `SCHEMAREGISTRY_AUTH` for basic auth credentials in the form `<username>:<password>` |
| host | string | `"localhost"` | The hostname to report for the RMI registry (used for JMX) |
| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy |
| image.repository | string | `"obsidiandynamics/kafdrop"` | Kafdrop Docker image repository |
| image.tag | string | `"4.0.1"` | Kafdrop image version |
| ingress.annotations | object | `{}` | Additional ingress annotations |
| ingress.enabled | bool | `false` | Whether to enable the ingress |
| ingress.hostname | string | None, must be set if ingress is enabled | Ingress hostname |
| ingress.path | string | `"/kafdrop"` | Ingress path |
| jmx.port | int | `8686` | Port to use for JMX. If unspecified, JMX will not be exposed. |
| jvm.opts | string | `""` | JVM options |
| kafka.broker | string | `""` | Bootstrap list of Kafka host/port pairs |
| nodeSelector | object | `{}` | Node selector configuration |
| podAnnotations | object | `{}` | Pod annotations |
| replicaCount | int | `1` | Number of kafdrop pods to run in the deployment. |
| resources | object | See `values.yaml` | Kubernetes requests and limits for Kafdrop |
| schemaregistry | string | `""` | The endpoint of Schema Registry |
| server.port | int | `9000` | The web server port to listen on |
| server.servlet.contextPath | string | `"/kafdrop"` | The context path to serve requests on |
| service.annotations | object | `{}` | Additional annotations to add to the service |
| service.port | int | `9000` | Service port |
| tolerations | list | `[]` | Tolerations configuration |
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
1. Get the application URL by running these commands:
{{- if .Values.ingress.enabled }}
{{- range .Values.ingress.hosts }}
http{{ if $.Values.ingress.tls }}s{{ end }}://{{ . }}{{ $.Values.ingress.path }}
{{- end }}
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "chart.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl port-forward $POD_NAME 8080:80
{{- end }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "chart.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "chart.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "chart.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
Common labels
*/}}
{{- define "kafdrop.labels" -}}
helm.sh/chart: {{ include "chart.name" . }}
{{ include "kafdrop.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{/*
Selector labels
*/}}
{{- define "kafdrop.selectorLabels" -}}
app.kubernetes.io/name: {{ include "chart.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "chart.fullname" . }}
labels:
{{- include "kafdrop.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
{{- include "kafdrop.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
{{- include "kafdrop.selectorLabels" . | nindent 8 }}
annotations:
{{- with .Values.podAnnotations }}
{{ toYaml . | indent 8 }}
{{- end }}
spec:
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
{{- if .Values.existingSecret -}}
envFrom:
- secretRef:
name: {{ .Values.existingSecret }}
{{- end }}
env:
- name: KAFKA_BROKERCONNECT
value: {{ .Values.kafka.broker | quote }}
- name: JVM_OPTS
value: {{ .Values.jvm.opts | quote }}
- name: HOST
value: {{ .Values.host | quote }}
- name: JMX_PORT
value: {{ .Values.jmx.port | quote }}
- name: SERVER_SERVLET_CONTEXTPATH
value: {{ .Values.server.servlet.contextPath | trimSuffix "/" | quote }}
- name: SERVER_PORT
value: {{ .Values.server.port | quote }}
- name: CMD_ARGS
value: {{ .Values.cmdArgs | quote }}
- name: SCHEMAREGISTRY_CONNECT
value: {{ .Values.schemaregistry | quote }}
- name: KAFKA_PROPERTIES_FILE
value: "/tmp/kafka.properties"
- name: KAFKA_PROPERTIES
valueFrom:
secretKeyRef:
name: alert-stream-broker-secrets
key: kafdrop-kafka-properties
ports:
- name: http
containerPort: {{ .Values.server.port }}
protocol: TCP
livenessProbe:
httpGet:
path: "{{ .Values.server.servlet.contextPath | trimSuffix "/" }}/actuator/health"
port: http
initialDelaySeconds: 180
periodSeconds: 30
timeoutSeconds: 10
readinessProbe:
httpGet:
path: "{{ .Values.server.servlet.contextPath | trimSuffix "/" }}/actuator/health"
port: http
initialDelaySeconds: 20
periodSeconds: 5
timeoutSeconds: 10
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{ toYaml . | indent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{ toYaml . | indent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{ toYaml . | indent 8 }}
{{- end }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{{- if .Values.ingress.enabled -}}
{{- $fullName := include "chart.fullname" . -}}
{{- $ingressPath := .Values.ingress.path -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ $fullName }}
labels:
{{- include "kafdrop.labels" . | nindent 4 }}
annotations:
{{- with .Values.ingress.annotations }}
{{ toYaml . | indent 4 }}
{{- end }}
spec:
ingressClassName: "nginx"
rules:
- host: {{ .Values.ingress.hostname | quote }}
http:
paths:
- path: {{ $ingressPath }}
pathType: Prefix
backend:
service:
name: {{ $fullName }}
port:
number: {{ .Values.service.port }}
{{- end }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "chart.fullname" . }}
labels:
{{- include "kafdrop.labels" . | nindent 4 }}
spec:
type: ClusterIP
ports:
- port: {{ .Values.service.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "kafdrop.selectorLabels" . | nindent 4 }}
Loading

0 comments on commit 31175c1

Please sign in to comment.