diff --git a/.github/workflows/ci-release.yml b/.github/workflows/ci-release.yml new file mode 100644 index 0000000..35920e6 --- /dev/null +++ b/.github/workflows/ci-release.yml @@ -0,0 +1,26 @@ +name: CI Release Charts + +on: + push: + branches: + - main + +jobs: + release: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v2 + with: + fetch-depth: 0 + + - name: Configure Git + run: | + git config user.name "$GITHUB_ACTOR" + git config user.email "$GITHUB_ACTOR@users.noreply.github.com" + + - name: Run chart-releaser + uses: helm/chart-releaser-action@v1.1.0 + env: + CR_TOKEN: "${{ secrets.GITHUB_TOKEN }}" + CR_SKIP_EXISTING: true \ No newline at end of file diff --git a/.gitignore b/.gitignore index ccc9fd9..2d08f31 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,9 @@ -*.DS_Store \ No newline at end of file +**.DS_Store +_config.yml + +# Helm +**/Chart.lock +*/*/charts/ + +# Idea +.idea diff --git a/CODEOWNERS b/CODEOWNERS new file mode 100644 index 0000000..55b5119 --- /dev/null +++ b/CODEOWNERS @@ -0,0 +1 @@ +* @Typeform/data-engineering \ No newline at end of file diff --git a/README.md b/README.md index a6976b2..1a96ca4 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # What is RudderStack? -[RudderStack](https://rudderstack.com/) is a **customer data pipeline** tool for collecting, routing and processing data from your websites, apps, cloud tools, and data warehouse. +[RudderStack](https://rudderstack.com/) is a **customer data pipeline** tool for collecting, routing and processing data +from your websites, apps, cloud tools, and data warehouse. More information on RudderStack can be found [here](https://github.com/rudderlabs/rudder-server). @@ -8,20 +9,22 @@ More information on RudderStack can be found [here](https://github.com/rudderlab ```bash $ git clone git@github.com:rudderlabs/rudderstack-helm.git -$ cd rudderstack-helm/ +$ cd rudderstack-helm/charts/rudderstack +$ helm dependency build $ helm install my-release ./ --set rudderWorkspaceToken="" ``` ## Introduction -The RudderStack Helm chart creates a Rudderstack deployment on a [Kubernetes](http://kubernetes.io) cluster -using the [Helm](https://helm.sh) package manager. +The RudderStack Helm chart creates a Rudderstack deployment on a [Kubernetes](http://kubernetes.io) cluster using +the [Helm](https://helm.sh) package manager. ## Prerequisites - Kubectl installed and connected to your kubernetes cluster - Helm installed -- Workspace token from the [RudderStack dashboard](https://app.rudderstack.com). Set up your account and copy your workspace token from the top of the home page. +- Workspace token from the [RudderStack dashboard](https://app.rudderstack.com). Set up your account and copy your + workspace token from the top of the home page. ## Installing the Chart @@ -31,7 +34,9 @@ To install the chart with the release name `my-release`, from the root directory $ helm install my-release ./ --set rudderWorkspaceToken="" ``` -The command deploys Rudderstack on the default Kubernetes cluster configured with `kubectl`. The [configuration](#configuration) section lists the most significant parameters that can be configured during deployment. +The command deploys Rudderstack on the default Kubernetes cluster configured with `kubectl`. +The [configuration](#configuration) section lists the most significant parameters that can be configured during +deployment. ## Upgrading the Chart @@ -51,21 +56,104 @@ $ helm uninstall my-release This removes all the components created by this chart. +## Developing the Chart + +To run a dry-run to evaluate if the changes proposed would be applied properly we can execute: + +```bash +helm template ./ | kubectl apply --dry-run=client -f - +``` + +## Postgres dependency + +We contemplate three options on having Postgres as a dependency. + +- Deploying it as a **Sidecar** in the same stateful resource +- Deploying a new Statefulset with Postgres. +- Providing an external Postgres. + +### Sidecar mode + +To enable the sidecar mode, specify: + +```yaml +postgresql: + mode: sidecar + statefulset_enabled: false +``` + +### Stateful mode + +To enable the sidecar mode, specify: + +```yaml +postgresql: + mode: statefulset + statefulset_enabled: true +``` + +## HPA : Horizontal Pod Autoscaling + +> Only recommended with **postgresql sidecar mode enable**. + +> Currently, only supported for `backend.controlPlaneJSON:true` since the **[pre-stop hook](charts/rudderstack/pre-stop.sh)** +> reads from the local config guaranteeing that all the events reached the destination so no event is lost on +> the autoscaling down process. + +Horizontal Pod Autoscaling is available in case of resource efficiency requirement. + +```yaml +backend: + terminationGracePeriodSeconds: xx + lifecycleSleepTime: xx + hpa: + enabled: true +``` + +Also, make sure you define the `lifecycleSleepTime` & the `terminationGracePeriodSeconds` bigger +than `BatchRouter.uploadFreqInS` otherwise K8s will kill the pods before flushing the data into their destinations. + ## Open-source Control Plane -If you are using open-source config-generator UI, you need to set the parameter `controlPlaneJSON` to `true` in the `values.yaml` file. Export workspace-config from the config-generator and copy/paste the contents into the `workspaceConfig.json` file. +If you are using open-source config-generator UI, you need to set the parameter `controlPlaneJSON` to `true` in +the `values.yaml` file. Export workspace-config from the config-generator and copy/paste the contents into +the `workspaceConfig.json` file. ```bash $ helm install my-release ./ --set backend.controlPlaneJSON=true ``` +## Extending the Chart + +Since we are publishing the Chart under the {{ TBC by the RudderStack team }} page. It's possible to extend this Chart +by adding it as a dependency into your own Chart, so there is no need to git clone this repo for deploying RudderStack +open-source into your infrastructure. + +```yaml +apiVersion: v2 +name: rudderstack +description: Customer Data Pipeline tool for collecting, routing and processing data. +maintainers: + - name: Data Platform + email: xxxx@xxxx.com +version: 0.4.5 +appVersion: 1.16.0 +dependencies: + # https://github.com/rudderlabs/rudderstack-helm + - name: rudderstack + version: 0.4.5 + repository: https://TBC.github.io/rudderstack-helm # To Be Confirmed by the RudderStack team +``` + ## GCP -If you are using Google Cloud Storage or Google BigQuery for the following cases, you have to replace the contents of the file [rudder-google-application-credentials.json](rudder-google-application-credentials.json) with your service account: +If you are using Google Cloud Storage or Google BigQuery for the following cases, you have to replace the contents of +the file [rudder-google-application-credentials.json](charts/rudderstack/rudder-google-application-credentials.json) +with your service account: - - GCS as a destination - - GCS for dumping jobs - - BigQuery as a warehouse destination. +- GCS as a destination +- GCS for dumping jobs +- BigQuery as a warehouse destination. ## Configuration @@ -83,7 +171,8 @@ The following table lists the configurable parameters of the Rudderstack chart a | `backend.extraEnvVars` | Extra environments variables to be used by the backend in the deployments | `Refer values.yaml file` | | `backend.controlPlaneJSON` | If `true`, backend will read config from the workspaceConfig.json file | `false` | -Each of these parameters can be changed in `values.yaml`. Or specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example: +Each of these parameters can be changed in `values.yaml`. Or specify each parameter using +the `--set key=value[,key=value]` argument to `helm install`. For example: ```bash $ helm install --name my-release \ @@ -92,6 +181,7 @@ $ helm install --name my-release \ ``` **Note:** Configuration specific to: + - Backend can be edited in [rudder-config.yaml](https://docs.rudderlabs.com/administrators-guide/config-parameters). - PostgreSQL can be edited in `pg_hba.conf`, `postgresql.conf` @@ -100,15 +190,16 @@ $ helm install --name my-release \ Installing this Helm chart will deploy the following pods and containers in the configured cluster: #### POD - {Release name}-rudderstack-0 : + - rudderstack-backend - rudderstack-telegraf-sidecar - -#### POD - {Release name}-rudderstack-postgresql-0 : -- {Release name}-rudderstack-postgresql +- rudderstack-postgresql-sidecar #### POD - {Release name}-rudderstack-transformer-xxxxxxxxxx-xxxxx: + - transformer ## Contact Us -For any queries related to using the RudderStack Helm Chart, feel free to start a conversation on our [Slack](https://resources.rudderstack.com/join-rudderstack-slack) channel. +For any queries related to using the RudderStack Helm Chart, feel free to start a conversation on +our [Slack](https://resources.rudderstack.com/join-rudderstack-slack) channel. diff --git a/Chart.yaml b/charts/rudderstack/Chart.yaml similarity index 54% rename from Chart.yaml rename to charts/rudderstack/Chart.yaml index 58b4b93..9a6452a 100644 --- a/Chart.yaml +++ b/charts/rudderstack/Chart.yaml @@ -14,21 +14,19 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. -version: 0.3.0 +version: 0.4.9 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. appVersion: 1.16.0 -# WIP -#dependencies: -# - name: nginx-ingress -# version: ~1.6.0 -# repository: https://helm.nginx.com/stable -# condition: (optional) A yaml path that resolves to a boolean, used for enabling/disabling charts (e.g. subchart1.enabled ) -# tags: # (optional) -# - Tags can be used to group charts for enabling/disabling together -# enabled: (optional) Enabled bool determines if chart should be loaded -# import-values: # (optional) -# - ImportValues holds the mapping of source values to parent key to be imported. Each item can be a string or pair of child/parent sublist items. -# alias: (optional) Alias usable alias to be used for the chart. Useful when you have to add the same chart multiple times +dependencies: + - name: transformer + version: 0.1.0 + repository: file://../subcharts/transformer + condition: transformer.enabled + + - name: postgresql + version: 7.7.2 + repository: file://../subcharts/postgresql + condition: postgresql.statefulset_enabled diff --git a/pg_hba.conf b/charts/rudderstack/pg_hba.conf similarity index 100% rename from pg_hba.conf rename to charts/rudderstack/pg_hba.conf diff --git a/postgresql.conf b/charts/rudderstack/postgresql.conf similarity index 100% rename from postgresql.conf rename to charts/rudderstack/postgresql.conf diff --git a/charts/rudderstack/pre-stop.sh b/charts/rudderstack/pre-stop.sh new file mode 100755 index 0000000..7f79dc4 --- /dev/null +++ b/charts/rudderstack/pre-stop.sh @@ -0,0 +1,50 @@ +#!/bin/sh +START=$(date +%s.%N) + +# Install JQ +JQ=/usr/bin/jq +curl -sLo $JQ https://stedolan.github.io/jq/download/linux64/jq +chmod +x $JQ + +# Extract source IDs +SOURCE_IDS=$(more $RSERVER_BACKEND_CONFIG_CONFIG_JSONPATH | jq -r .sources[].id) +STATSD_PREFIX="rudder_server_pre_stop" + +# Emit Start datum to Telegraf +echo "${STATSD_PREFIX}_start:1|c" | nc -w 1 -u localhost 8125 + +# Check pending events per Source ID +HAS_PENDING_EVENTS=true +while $HAS_PENDING_EVENTS ; do + TOTAL_PENDING_EVENTS=0 + echo "1. Checking pending events...." + echo "${STATSD_PREFIX}_check:1|c" | nc -w 1 -u localhost 8125 + + for source_id in $SOURCE_IDS; do + PENDING_EVENTS=$(curl -s --request POST --url http://localhost:8080/v1/pending-events \ + -u ${source_id}:potato --header 'Content-Type: application/json' \ + --data "{\"source_id\": \"${source_id}\"}" | jq .pending_events) + echo "2. Pending $PENDING_EVENTS events for Source [$source_id]" + TOTAL_PENDING_EVENTS=$(( $TOTAL_PENDING_EVENTS + $PENDING_EVENTS )) + done + + echo "3. Checking result..." + echo "${STATSD_PREFIX}_pending_events:$TOTAL_PENDING_EVENTS|g" | nc -w 1 -u localhost 8125 + if [[ ${TOTAL_PENDING_EVENTS} -eq 0 ]]; then + echo "4. No pending events" + HAS_PENDING_EVENTS=false + else + echo "4. Total pending events: $TOTAL_PENDING_EVENTS" + fi + echo "5. Sleeping for 5 seconds..." + sleep 5s + +done + +echo "${STATSD_PREFIX}_done:1|c" | nc -w 1 -u localhost 8125 +END=$(date +%s.%N) +PRE_STOP_DURATION_SEC=$(echo "$END - $START" | bc) +echo "${STATSD_PREFIX}_time:${PRE_STOP_DURATION_SEC}|ms" | nc -w 1 -u localhost 8125 + +# Giving time to Telegraf to forward pre-stop metrics +sleep 60s diff --git a/rudder-config.yaml b/charts/rudderstack/rudder-config.yaml similarity index 99% rename from rudder-config.yaml rename to charts/rudderstack/rudder-config.yaml index b9af72a..b170298 100644 --- a/rudder-config.yaml +++ b/charts/rudderstack/rudder-config.yaml @@ -115,7 +115,7 @@ Router: BatchRouter: mainLoopSleep: 2s jobQueryBatchSize: 100000 - uploadFreq: 30s + uploadFreqInS: 30 warehouseServiceMaxRetryTime: 3h noOfWorkers: 8 maxFailedCountForJob: 128 diff --git a/rudder-google-application-credentials.json b/charts/rudderstack/rudder-google-application-credentials.json similarity index 100% rename from rudder-google-application-credentials.json rename to charts/rudderstack/rudder-google-application-credentials.json diff --git a/templates/NOTES.txt b/charts/rudderstack/templates/NOTES.txt similarity index 100% rename from templates/NOTES.txt rename to charts/rudderstack/templates/NOTES.txt diff --git a/templates/_helpers.tpl b/charts/rudderstack/templates/_helpers.tpl similarity index 72% rename from templates/_helpers.tpl rename to charts/rudderstack/templates/_helpers.tpl index 6e69832..9a80e55 100644 --- a/templates/_helpers.tpl +++ b/charts/rudderstack/templates/_helpers.tpl @@ -75,6 +75,14 @@ app.kubernetes.io/instance: {{ .Release.Name }} {{- printf "%s-%s" (include "rudderstack.fullname" .) "telegraf-sidecar" -}} {{- end -}} +{{- define "postgresql.name" -}} +{{- printf "%s-%s" (include "rudderstack.name" .) "postgresql-sidecar" -}} +{{- end -}} + +{{- define "postgresql.fullname" -}} +{{- printf "%s-%s" (include "rudderstack.fullname" .) "postgresql-sidecar" -}} +{{- end -}} + {{/* Return the appropriate apiVersion for statefulset. */}} @@ -87,6 +95,72 @@ Return the appropriate apiVersion for statefulset. {{- end -}} +{{/*postgresql helper functions */}} + +{{/* +Return the proper Storage Class +*/}} +{{- define "postgresql.storageClass" -}} +{{- if .Values.global -}} + {{- if .Values.global.storageClass -}} + {{- if (eq "-" .Values.global.storageClass) -}} + {{- printf "storageClassName: \"\"" -}} + {{- else }} + {{- printf "storageClassName: %s" .Values.global.storageClass -}} + {{- end -}} + {{- else -}} + {{- if .Values.postgresql.persistence.storageClass -}} + {{- if (eq "-" .Values.postgresql.persistence.storageClass) -}} + {{- printf "storageClassName: \"\"" -}} + {{- else }} + {{- printf "storageClassName: %s" .Values.postgresql.persistence.storageClass -}} + {{- end -}} + {{- end -}} + {{- end -}} +{{- else -}} + {{- if .Values.postgresql.persistence.storageClass -}} + {{- if (eq "-" .Values.postgresql.persistence.storageClass) -}} + {{- printf "storageClassName: \"\"" -}} + {{- else }} + {{- printf "storageClassName: %s" .Values.postgresql.persistence.storageClass -}} + {{- end -}} + {{- end -}} +{{- end -}} +{{- end -}} + +{{/* +Return PostgreSQL replication username +*/}} +{{- define "postgresql.username" -}} + {{- .Values.postgresql.username -}} +{{- end -}} + +{{/* +Return PostgreSQL port +*/}} +{{- define "postgresql.port" -}} + {{- .Values.postgresql.port -}} +{{- end -}} + +{{/* +Return PostgreSQL created database +*/}} +{{- define "postgresql.database" -}} + {{- .Values.postgresql.database -}} +{{- end -}} + + +{{- define "postgresql.computeDBHost" -}} + {{- if (eq .Values.postgresql.mode "sidecar") -}} + {{- printf "\"false\"" -}} + {{- else -}} + {{- if (eq .Values.postgresql.mode "statefulset") -}} + {{- printf "\"true\"" -}} + {{- else -}} + {{- printf "\"false\"" -}} + {{- end -}} + {{- end -}} +{{- end -}} {{/*telegraf helper functions */}} diff --git a/charts/rudderstack/templates/configmap-rudder-scripts.yaml b/charts/rudderstack/templates/configmap-rudder-scripts.yaml new file mode 100644 index 0000000..4a4fd3c --- /dev/null +++ b/charts/rudderstack/templates/configmap-rudder-scripts.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "backend.fullname" . }}-scripts + labels: + app: {{ include "backend.name" . }} + release: {{ .Release.Name | quote }} +data: + pre-stop.sh: |- +{{.Files.Get "pre-stop.sh" | indent 4}} diff --git a/templates/configmap-rudder-server.yaml b/charts/rudderstack/templates/configmap-rudder-server.yaml similarity index 100% rename from templates/configmap-rudder-server.yaml rename to charts/rudderstack/templates/configmap-rudder-server.yaml diff --git a/templates/configmap-telegraf.yaml b/charts/rudderstack/templates/configmap-telegraf.yaml similarity index 98% rename from templates/configmap-telegraf.yaml rename to charts/rudderstack/templates/configmap-telegraf.yaml index 1a3d36f..86d7bed 100644 --- a/templates/configmap-telegraf.yaml +++ b/charts/rudderstack/templates/configmap-telegraf.yaml @@ -12,6 +12,7 @@ data: telegraf.conf: |+ [global_tags] namespace = "${KUBE_NAMESPACE}" + env = "${ENV}" [agent] collection_jitter = "0s" debug = false diff --git a/charts/rudderstack/templates/hpa-statefulset.yaml b/charts/rudderstack/templates/hpa-statefulset.yaml new file mode 100644 index 0000000..b1038c2 --- /dev/null +++ b/charts/rudderstack/templates/hpa-statefulset.yaml @@ -0,0 +1,22 @@ +{{- if .Values.backend.hpa.enabled }} +apiVersion: autoscaling/v2beta2 +kind: HorizontalPodAutoscaler +metadata: + name: {{ include "rudderstack.fullname" . }}-hpa + labels: +{{- include "rudderstack.labels" . | nindent 4 }} +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: StatefulSet + name: {{ include "rudderstack.fullname" . }} + minReplicas: {{ .Values.backend.hpa.minReplicas }} + maxReplicas: {{ .Values.backend.hpa.maxReplicas }} + metrics: + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.backend.hpa.cpuTargetPercent }} +{{- end }} diff --git a/templates/ingress.yaml b/charts/rudderstack/templates/ingress.yaml similarity index 100% rename from templates/ingress.yaml rename to charts/rudderstack/templates/ingress.yaml diff --git a/templates/secret-gcp.yaml b/charts/rudderstack/templates/secret-gcp.yaml similarity index 100% rename from templates/secret-gcp.yaml rename to charts/rudderstack/templates/secret-gcp.yaml diff --git a/templates/service.yaml b/charts/rudderstack/templates/service.yaml similarity index 100% rename from templates/service.yaml rename to charts/rudderstack/templates/service.yaml diff --git a/templates/statefulset.yaml b/charts/rudderstack/templates/statefulset.yaml similarity index 50% rename from templates/statefulset.yaml rename to charts/rudderstack/templates/statefulset.yaml index 68466f7..74fe5d7 100644 --- a/templates/statefulset.yaml +++ b/charts/rudderstack/templates/statefulset.yaml @@ -7,6 +7,7 @@ metadata: spec: serviceName: {{ include "rudderstack.fullname" . }} replicas: {{ .Values.global.backendReplicaCount }} + revisionHistoryLimit: 1 selector: matchLabels: {{- include "rudderstack.selectorLabels" . | nindent 6 }} @@ -18,16 +19,28 @@ spec: checksum/rudder-config: {{ .Files.Get "rudder-config.yaml" | sha256sum }} checksum/rudder-bigquery-credentials: {{ .Files.Get "bigquery-credentials.json" | sha256sum }} spec: - {{- with .Values.global.imagePullSecrets }} + {{- if .Values.serviceAccount.enabled }} + serviceAccountName: {{ default "default" .Values.serviceAccount.name }} + {{- end }} + {{- if .Values.backend.terminationGracePeriodSeconds }} + terminationGracePeriodSeconds: {{ .Values.backend.terminationGracePeriodSeconds }} + {{- end }} + {{- with .Values.global.imagePullSecrets }} imagePullSecrets: {{- toYaml . | nindent 8 }} - {{- end }} + {{- end }} volumes: + {{- if .Values.backend.configMapName }} - configMap: defaultMode: 420 - name: {{ include "backend.fullname" . }}-config + name: {{ .Values.backend.configMapName }} name: backend-config-volume - {{- if .Values.telegraf_sidecar.enabled}} + {{- end }} + - configMap: + defaultMode: 0777 + name: {{ include "backend.fullname" . }}-scripts + name: backend-scripts-volume + {{- if .Values.telegraf_sidecar.enabled }} - name: telegraf configMap: defaultMode: 420 @@ -39,6 +52,10 @@ spec: defaultMode: 420 secretName: {{ include "rudderstack.fullname" . }}-gcp {{- end }} + {{- if eq .Values.postgresql.mode "sidecar" }} + - name: {{ include "backend.name" . }}-postgresql-data + emptyDir: {} + {{- end }} containers: - name: {{ include "backend.name" . }} image: "{{ .Values.backend.image.repository }}:{{ .Values.backend.image.version }}" @@ -46,6 +63,8 @@ spec: volumeMounts: - mountPath: {{ .Values.backend.config.mountPath }} name: backend-config-volume + - mountPath: {{ .Values.backend.scripts.mountPath }} + name: backend-scripts-volume - mountPath: {{ .Values.backend.persistence.mountPath }} name: {{ include "backend.name" . }}-data {{- if .Values.gcpCredentialSecret.enabled }} @@ -70,32 +89,30 @@ spec: {{- if .Values.backend.extraEnvVars }} {{- .Values.backend.extraEnvVars | toYaml | nindent 10 }} {{- end }} + {{- if .Values.rudderWorkspaceToken }} - name: CONFIG_BACKEND_TOKEN value: {{ .Values.rudderWorkspaceToken }} - {{- if .Values.backend.controlPlaneJSON }} + {{- end }} + {{- if .Values.backend.controlPlaneJSON }} - name: RSERVER_BACKEND_CONFIG_CONFIG_FROM_FILE value: "{{ .Values.backend.controlPlaneJSON }}" - name: RSERVER_BACKEND_CONFIG_CONFIG_JSONPATH value: "{{ .Values.backend.config.mountPath }}/workspaceConfig.json" - {{- end}} + {{- end}} + - name: COMPUTE_DB_HOST_IN_K8S + value: {{ include "postgresql.computeDBHost" . }} - name: DEST_TRANSFORM_URL value: "http://{{ include "transformer.fullname" . }}:{{ .Values.transformer.service.port}}" - - name: COMPUTE_DB_HOST_IN_K8S - value: "true" - - name: POSTGRES_POD_NAME - value: "{{ .Release.Name }}-{{ .Values.postgresql.nameOverride }}" - - name: POSTGRES_HEADLESS_SVC - value: "{{ .Release.Name }}-{{ .Values.postgresql.nameOverride }}-headless" - name: JOBS_DB_HOST - value: "This is expected to be set by docker entrypoint script" + value: "{{ .Values.postgresql.host }}" - name: JOBS_DB_USER - value: "{{ .Values.postgresql.postgresqlUsername }}" + value: "{{ .Values.postgresql.username }}" - name: JOBS_DB_PORT - value: "{{ .Values.postgresql.service.port }}" + value: "{{ .Values.postgresql.port }}" - name: JOBS_DB_DB_NAME - value: "{{ .Values.postgresql.postgresqlDatabase }}" + value: "{{ .Values.postgresql.database }}" - name: JOBS_DB_PASSWORD - value: "{{ .Values.postgresql.postgresqlPassword }}" + value: "{{ .Values.postgresql.password }}" - name: INSTANCE_ID valueFrom: fieldRef: @@ -104,15 +121,30 @@ spec: value: {{ .Release.Namespace }} command: ["/docker-entrypoint.sh"] args: ["/bin/sh","-c","/wait-for $JOBS_DB_HOST:$(JOBS_DB_PORT) -- /rudder-server"] - {{- if .Values.telegraf_sidecar.enabled}} + {{- if .Values.backend.controlPlaneJSON }} + lifecycle: + preStop: + exec: + command: ["/bin/sh", "-c", "{{ .Values.backend.scripts.mountPath }}/pre-stop.sh > /proc/1/fd/1"] + {{- end }} + {{- if .Values.telegraf_sidecar.enabled }} - name: {{ include "telegraf-sidecar.name" .}} image: "{{ .Values.telegraf_sidecar.image.repo }}:{{ .Values.telegraf_sidecar.image.tag }}" imagePullPolicy: {{ default "" .Values.telegraf_sidecar.image.pullPolicy | quote }} env: - name: KUBE_NAMESPACE value: {{ .Release.Namespace }} + - name: ENV + value: {{ .Values.global.env }} + {{- if .Values.telegraf_sidecar.extraEnvVars }} + {{- .Values.telegraf_sidecar.extraEnvVars | toYaml | nindent 10 }} + {{- end }} command: ["telegraf"] args: ["--config", "{{.Values.telegraf_sidecar.config.mountPath}}/telegraf.conf"] + lifecycle: + preStop: + exec: + command: ["/bin/sh", "-c", "sleep {{ .Values.backend.lifecycleSleepTime }}"] {{- if .Values.telegraf_sidecar.resources }} resources: {{- toYaml .Values.telegraf_sidecar.resources | nindent 10 }} @@ -120,8 +152,58 @@ spec: volumeMounts: - name: telegraf mountPath: {{.Values.telegraf_sidecar.config.mountPath}} - {{- end}} - + {{- end }} + {{- if eq .Values.postgresql.mode "sidecar" }} + - name: {{ include "postgresql.name" .}} + image: "{{ .Values.postgresql.image.repository }}:{{ .Values.postgresql.image.tag }}" + imagePullPolicy: {{ default "" .Values.postgresql.image.pullPolicy | quote }} + resources: + {{- toYaml .Values.postgresql.resources | nindent 10 }} + volumeMounts: + - name: {{ include "backend.name" . }}-postgresql-data + mountPath: {{ .Values.postgresql.persistence.mountPath }} + ports: + - name: postgresql + containerPort: {{ template "postgresql.port" . }} + env: + {{- if .Values.postgresql.extraEnvVars }} + {{- .Values.postgresql.extraEnvVars | toYaml | nindent 10 }} + {{- end }} + - name: PGDATA + value: "{{ .Values.postgresql.persistence.mountPath }}/pgdata" + - name: POSTGRES_DB + value: "{{ .Values.postgresql.database }}" + - name: POSTGRES_USER + value: "{{ .Values.postgresql.username }}" + - name: POSTGRES_PASSWORD + value: "{{ .Values.postgresql.password }}" + livenessProbe: + exec: + command: + - /bin/sh + - -c + - exec pg_isready -U {{ include "postgresql.username" . | quote }} -d {{ (include "postgresql.database" .) | quote }} -h 127.0.0.1 -p {{ template "postgresql.port" . }} + initialDelaySeconds: {{ .Values.postgresql.livenessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.postgresql.livenessProbe.periodSeconds }} + timeoutSeconds: {{ .Values.postgresql.livenessProbe.timeoutSeconds }} + successThreshold: {{ .Values.postgresql.livenessProbe.successThreshold }} + failureThreshold: {{ .Values.postgresql.livenessProbe.failureThreshold }} + readinessProbe: + exec: + command: + - /bin/sh + - -c + - exec pg_isready -U {{ include "postgresql.username" . | quote }} -d {{ (include "postgresql.database" .) | quote }} -h 127.0.0.1 -p {{ template "postgresql.port" . }} + initialDelaySeconds: {{ .Values.postgresql.readinessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.postgresql.readinessProbe.periodSeconds }} + timeoutSeconds: {{ .Values.postgresql.readinessProbe.timeoutSeconds }} + successThreshold: {{ .Values.postgresql.readinessProbe.successThreshold }} + failureThreshold: {{ .Values.postgresql.readinessProbe.failureThreshold }} + lifecycle: + preStop: + exec: + command: [ "/bin/sh", "-c", "sleep {{ .Values.backend.lifecycleSleepTime }}" ] + {{ end }} volumeClaimTemplates: - metadata: diff --git a/templates/tests/test-connection.yaml b/charts/rudderstack/templates/tests/test-connection.yaml similarity index 100% rename from templates/tests/test-connection.yaml rename to charts/rudderstack/templates/tests/test-connection.yaml diff --git a/values.yaml b/charts/rudderstack/values.yaml similarity index 72% rename from values.yaml rename to charts/rudderstack/values.yaml index da7fa59..92294ad 100644 --- a/values.yaml +++ b/charts/rudderstack/values.yaml @@ -9,6 +9,10 @@ # Please uncomment below lines and fill values accordingly. # Please enter api token obtained from rudder dashboard below # rudderWorkspaceToken: +serviceAccount: + create: false + enabled: false + name: 'rudderstack' gcpCredentialSecret: enabled: false @@ -26,13 +30,21 @@ global: ## GKE, AWS & OpenStack) ## storageClass: "" + env: "dev" backend: + # Use a bigger value than BatchRouter.uploadFreqInS allowing + # the server to flush all the pending data, usefull when HPA enabled. + terminationGracePeriodSeconds: 60 + # Important when autoscaling down, giving time to the pre-stop.sh script + # to interact with postgres and telegraf sidecars + lifecycleSleepTime: 60 image: repository: rudderlabs/rudder-server version: 1-alpine pullPolicy: Always controlPlaneJSON: false + configMapName: rudderstack-backend-config ingress: enabled: false tls: false @@ -55,7 +67,19 @@ backend: memory: 2560Mi limits: memory: 5120Mi - + hpa: + # if: + # hpa.enabled: true + # then: + # make sure to specify resources.limits.cpu & resources.requests.cpu + # so HPA can work properly. + # caution: + # enable it only if you are using the postgres sidecar approach + # since each backend needs one dedicated db + enabled: false + minReplicas: 3 + maxReplicas: 10 + cpuTargetPercent: 70 nodeSelector: {} persistence: @@ -68,6 +92,8 @@ backend: config: mountPath: "/etc/rudderstack" + scripts: + mountPath: "/etc/rudderstack/scripts" extraEnvVars: - name: CONFIG_BACKEND_URL @@ -95,6 +121,7 @@ backend: value: "INFO" # eg. DEBUG, ERROR transformer: + enabled: true replicaCount: 1 service: port: 9090 @@ -109,18 +136,22 @@ transformer: memory: 768Mi postgresql: - nameOverride: "rudderstack-postgresql" - postgresqlUsername: rudder - postgresqlPassword: password - postgresqlDatabase: jobsdb - postgresqlRunAsUser: 70 - imagePullPolicy: IfNotPresent + mode: sidecar # sidecar | statefulset | external + statefulset_enabled: false + username: rudder + password: password + database: jobsdb + host: localhost + port: 5432 image: repository: postgres tag: "11-alpine" pullPolicy: IfNotPresent persistence: + mountPath: /etc/postgresql size: 100Gi + accessModes: + - ReadWriteOnce replication: enabled: false service: @@ -129,8 +160,24 @@ postgresql: resources: requests: memory: 2048Mi + cpu: 250m + ephemeral-storage: "5Gi" limits: memory: 4096Mi + cpu: 1000m + ephemeral-storage: "20Gi" + livenessProbe: + initialDelaySeconds: 30 + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 6 + readinessProbe: + initialDelaySeconds: 5 + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 6 telegraf_sidecar: enabled: true @@ -155,7 +202,7 @@ telegraf_sidecar: outputs: - influxdb: urls: [] - # - "http://influxdb.monitoring.svc:8086" + # - "http://influxdb.monitoring.svc:8086" database: "telegraf" statsd_percentiles: - 50 diff --git a/workspaceConfig.json b/charts/rudderstack/workspaceConfig.json similarity index 100% rename from workspaceConfig.json rename to charts/rudderstack/workspaceConfig.json diff --git a/workspaceConfig.json.example b/charts/rudderstack/workspaceConfig.json.example similarity index 100% rename from workspaceConfig.json.example rename to charts/rudderstack/workspaceConfig.json.example diff --git a/charts/postgresql/.helmignore b/charts/subcharts/postgresql/.helmignore similarity index 100% rename from charts/postgresql/.helmignore rename to charts/subcharts/postgresql/.helmignore diff --git a/charts/postgresql/Chart.yaml b/charts/subcharts/postgresql/Chart.yaml similarity index 100% rename from charts/postgresql/Chart.yaml rename to charts/subcharts/postgresql/Chart.yaml diff --git a/charts/postgresql/OWNERS b/charts/subcharts/postgresql/OWNERS similarity index 100% rename from charts/postgresql/OWNERS rename to charts/subcharts/postgresql/OWNERS diff --git a/charts/postgresql/README.md b/charts/subcharts/postgresql/README.md similarity index 100% rename from charts/postgresql/README.md rename to charts/subcharts/postgresql/README.md diff --git a/charts/postgresql/ci/default-values.yaml b/charts/subcharts/postgresql/ci/default-values.yaml similarity index 100% rename from charts/postgresql/ci/default-values.yaml rename to charts/subcharts/postgresql/ci/default-values.yaml diff --git a/charts/postgresql/ci/shmvolume-disabled-values.yaml b/charts/subcharts/postgresql/ci/shmvolume-disabled-values.yaml similarity index 100% rename from charts/postgresql/ci/shmvolume-disabled-values.yaml rename to charts/subcharts/postgresql/ci/shmvolume-disabled-values.yaml diff --git a/charts/postgresql/files/README.md b/charts/subcharts/postgresql/files/README.md similarity index 100% rename from charts/postgresql/files/README.md rename to charts/subcharts/postgresql/files/README.md diff --git a/charts/postgresql/files/conf.d/README.md b/charts/subcharts/postgresql/files/conf.d/README.md similarity index 100% rename from charts/postgresql/files/conf.d/README.md rename to charts/subcharts/postgresql/files/conf.d/README.md diff --git a/charts/postgresql/files/docker-entrypoint-initdb.d/README.md b/charts/subcharts/postgresql/files/docker-entrypoint-initdb.d/README.md similarity index 100% rename from charts/postgresql/files/docker-entrypoint-initdb.d/README.md rename to charts/subcharts/postgresql/files/docker-entrypoint-initdb.d/README.md diff --git a/charts/postgresql/templates/NOTES.txt b/charts/subcharts/postgresql/templates/NOTES.txt similarity index 100% rename from charts/postgresql/templates/NOTES.txt rename to charts/subcharts/postgresql/templates/NOTES.txt diff --git a/charts/postgresql/templates/_helpers.tpl b/charts/subcharts/postgresql/templates/_helpers.tpl similarity index 100% rename from charts/postgresql/templates/_helpers.tpl rename to charts/subcharts/postgresql/templates/_helpers.tpl diff --git a/charts/postgresql/templates/configmap.yaml b/charts/subcharts/postgresql/templates/configmap.yaml similarity index 100% rename from charts/postgresql/templates/configmap.yaml rename to charts/subcharts/postgresql/templates/configmap.yaml diff --git a/charts/postgresql/templates/extended-config-configmap.yaml b/charts/subcharts/postgresql/templates/extended-config-configmap.yaml similarity index 100% rename from charts/postgresql/templates/extended-config-configmap.yaml rename to charts/subcharts/postgresql/templates/extended-config-configmap.yaml diff --git a/charts/postgresql/templates/initialization-configmap.yaml b/charts/subcharts/postgresql/templates/initialization-configmap.yaml similarity index 100% rename from charts/postgresql/templates/initialization-configmap.yaml rename to charts/subcharts/postgresql/templates/initialization-configmap.yaml diff --git a/charts/postgresql/templates/metrics-configmap.yaml b/charts/subcharts/postgresql/templates/metrics-configmap.yaml similarity index 100% rename from charts/postgresql/templates/metrics-configmap.yaml rename to charts/subcharts/postgresql/templates/metrics-configmap.yaml diff --git a/charts/postgresql/templates/metrics-svc.yaml b/charts/subcharts/postgresql/templates/metrics-svc.yaml similarity index 100% rename from charts/postgresql/templates/metrics-svc.yaml rename to charts/subcharts/postgresql/templates/metrics-svc.yaml diff --git a/charts/postgresql/templates/networkpolicy.yaml b/charts/subcharts/postgresql/templates/networkpolicy.yaml similarity index 100% rename from charts/postgresql/templates/networkpolicy.yaml rename to charts/subcharts/postgresql/templates/networkpolicy.yaml diff --git a/charts/postgresql/templates/prometheusrule.yaml b/charts/subcharts/postgresql/templates/prometheusrule.yaml similarity index 100% rename from charts/postgresql/templates/prometheusrule.yaml rename to charts/subcharts/postgresql/templates/prometheusrule.yaml diff --git a/charts/postgresql/templates/secrets.yaml b/charts/subcharts/postgresql/templates/secrets.yaml similarity index 100% rename from charts/postgresql/templates/secrets.yaml rename to charts/subcharts/postgresql/templates/secrets.yaml diff --git a/charts/postgresql/templates/serviceaccount.yaml b/charts/subcharts/postgresql/templates/serviceaccount.yaml similarity index 100% rename from charts/postgresql/templates/serviceaccount.yaml rename to charts/subcharts/postgresql/templates/serviceaccount.yaml diff --git a/charts/postgresql/templates/servicemonitor.yaml b/charts/subcharts/postgresql/templates/servicemonitor.yaml similarity index 100% rename from charts/postgresql/templates/servicemonitor.yaml rename to charts/subcharts/postgresql/templates/servicemonitor.yaml diff --git a/charts/postgresql/templates/statefulset-slaves.yaml b/charts/subcharts/postgresql/templates/statefulset-slaves.yaml similarity index 100% rename from charts/postgresql/templates/statefulset-slaves.yaml rename to charts/subcharts/postgresql/templates/statefulset-slaves.yaml diff --git a/charts/postgresql/templates/statefulset.yaml b/charts/subcharts/postgresql/templates/statefulset.yaml similarity index 100% rename from charts/postgresql/templates/statefulset.yaml rename to charts/subcharts/postgresql/templates/statefulset.yaml diff --git a/charts/postgresql/templates/svc-headless.yaml b/charts/subcharts/postgresql/templates/svc-headless.yaml similarity index 100% rename from charts/postgresql/templates/svc-headless.yaml rename to charts/subcharts/postgresql/templates/svc-headless.yaml diff --git a/charts/postgresql/templates/svc-read.yaml b/charts/subcharts/postgresql/templates/svc-read.yaml similarity index 100% rename from charts/postgresql/templates/svc-read.yaml rename to charts/subcharts/postgresql/templates/svc-read.yaml diff --git a/charts/postgresql/templates/svc.yaml b/charts/subcharts/postgresql/templates/svc.yaml similarity index 100% rename from charts/postgresql/templates/svc.yaml rename to charts/subcharts/postgresql/templates/svc.yaml diff --git a/charts/postgresql/values-production.yaml b/charts/subcharts/postgresql/values-production.yaml similarity index 100% rename from charts/postgresql/values-production.yaml rename to charts/subcharts/postgresql/values-production.yaml diff --git a/charts/postgresql/values.schema.json b/charts/subcharts/postgresql/values.schema.json similarity index 100% rename from charts/postgresql/values.schema.json rename to charts/subcharts/postgresql/values.schema.json diff --git a/charts/postgresql/values.yaml b/charts/subcharts/postgresql/values.yaml similarity index 100% rename from charts/postgresql/values.yaml rename to charts/subcharts/postgresql/values.yaml diff --git a/charts/transformer/.helmignore b/charts/subcharts/transformer/.helmignore similarity index 100% rename from charts/transformer/.helmignore rename to charts/subcharts/transformer/.helmignore diff --git a/charts/transformer/Chart.yaml b/charts/subcharts/transformer/Chart.yaml similarity index 100% rename from charts/transformer/Chart.yaml rename to charts/subcharts/transformer/Chart.yaml diff --git a/charts/transformer/templates/NOTES.txt b/charts/subcharts/transformer/templates/NOTES.txt similarity index 100% rename from charts/transformer/templates/NOTES.txt rename to charts/subcharts/transformer/templates/NOTES.txt diff --git a/charts/transformer/templates/_helpers.tpl b/charts/subcharts/transformer/templates/_helpers.tpl similarity index 100% rename from charts/transformer/templates/_helpers.tpl rename to charts/subcharts/transformer/templates/_helpers.tpl diff --git a/charts/transformer/templates/deployment.yaml b/charts/subcharts/transformer/templates/deployment.yaml similarity index 100% rename from charts/transformer/templates/deployment.yaml rename to charts/subcharts/transformer/templates/deployment.yaml diff --git a/charts/subcharts/transformer/templates/hpa-deployment.yaml b/charts/subcharts/transformer/templates/hpa-deployment.yaml new file mode 100644 index 0000000..f1ebf7a --- /dev/null +++ b/charts/subcharts/transformer/templates/hpa-deployment.yaml @@ -0,0 +1,22 @@ +{{- if .Values.hpa.enabled }} +apiVersion: autoscaling/v2beta2 +kind: HorizontalPodAutoscaler +metadata: + name: {{ include "transformer.fullname" . }}-hpa + labels: +{{- include "transformer.labels" . | nindent 4 }} +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: {{ include "transformer.fullname" . }} + minReplicas: {{ .Values.hpa.minReplicas }} + maxReplicas: {{ .Values.hpa.maxReplicas }} + metrics: + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.hpa.cpuTargetPercent }} +{{- end }} diff --git a/charts/transformer/templates/service.yaml b/charts/subcharts/transformer/templates/service.yaml similarity index 100% rename from charts/transformer/templates/service.yaml rename to charts/subcharts/transformer/templates/service.yaml diff --git a/charts/transformer/templates/tests/test-connection.yaml b/charts/subcharts/transformer/templates/tests/test-connection.yaml similarity index 100% rename from charts/transformer/templates/tests/test-connection.yaml rename to charts/subcharts/transformer/templates/tests/test-connection.yaml diff --git a/charts/transformer/values.yaml b/charts/subcharts/transformer/values.yaml similarity index 81% rename from charts/transformer/values.yaml rename to charts/subcharts/transformer/values.yaml index fa76f71..60856cf 100644 --- a/charts/transformer/values.yaml +++ b/charts/subcharts/transformer/values.yaml @@ -16,6 +16,17 @@ service: type: ClusterIP # port: 9090 +hpa: + # if: + # hpa.enabled: true + # then: + # make sure to specify resources.limits.cpu & resources.requests.cpu + # so HPA can work properly. + enabled: false + minReplicas: 1 + maxReplicas: 5 + cpuTargetPercent: 70 + resources: {} # We usually recommend not to specify default resources and to leave this as a conscious