Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat] Configurable templates helm chart variant #30

Merged
merged 45 commits into from
Sep 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
b0977c3
update cds to major version 8
anirudhprasad-sap Jul 15, 2024
a8f936d
build plugin import changed
anirudhprasad-sap Jul 15, 2024
2735c65
added init method to build plugin
anirudhprasad-sap Jul 15, 2024
2c15c93
Update @sap/cds-dk to 8.0.3
anirudhprasad-sap Jul 22, 2024
97cf962
Tests updated
anirudhprasad-sap Jul 22, 2024
fffa9f4
Update add.test.js
anirudhprasad-sap Jul 22, 2024
ca06987
test update required for cds 8
anirudhprasad-sap Jul 26, 2024
070692c
Update "@sap/cds-dk" to 8.1.0
anirudhprasad-sap Jul 29, 2024
6490ac9
Merge branch 'main' into cdsV8Update
anirudhprasad-sap Jul 29, 2024
9f9695e
package-lock update
anirudhprasad-sap Jul 29, 2024
a4caffd
update readme
anirudhprasad-sap Aug 9, 2024
b7fba4c
cap-operator-dynamic-templates first version
anirudhprasad-sap Aug 9, 2024
6d488bd
consumedBTPServices check changed
anirudhprasad-sap Aug 9, 2024
0c84b01
more fixes
anirudhprasad-sap Aug 9, 2024
70e044a
isDynamicTemplateChart fun call fix
anirudhprasad-sap Aug 12, 2024
124089b
reworked plugin
anirudhprasad-sap Aug 12, 2024
ac11a23
enhancement + added hanaInstanceId
anirudhprasad-sap Aug 13, 2024
73a4d7e
function renamed
anirudhprasad-sap Aug 13, 2024
15c05f6
remove app.version and added template injection for originalAppName
anirudhprasad-sap Aug 13, 2024
2b12d87
CRO template fix + enhancements
anirudhprasad-sap Aug 14, 2024
f2a42ec
Merge main
anirudhprasad-sap Aug 20, 2024
a4ae12f
Removed portal + minor fixes
anirudhprasad-sap Aug 20, 2024
c0d11eb
convert-to-flexible-template-chart
anirudhprasad-sap Aug 21, 2024
355b0f4
convert-to-flexible-template-chart enhanced
anirudhprasad-sap Aug 21, 2024
bf98925
more updates
anirudhprasad-sap Aug 22, 2024
18d3236
update
anirudhprasad-sap Aug 22, 2024
4e86be5
update cap-op-plugin
anirudhprasad-sap Aug 22, 2024
b0aa24b
cleaned up tests
anirudhprasad-sap Aug 22, 2024
9a0d472
added unit test for cap-operator-with-flexible-templates
anirudhprasad-sap Aug 22, 2024
e285550
addcheck test changed
anirudhprasad-sap Aug 22, 2024
bfd08a7
Merge branch 'main' into newHelmChart
anirudhprasad-sap Aug 22, 2024
c68059b
more unit tests + minor fixes
anirudhprasad-sap Aug 23, 2024
1aae450
more unit tests for cap-op-plugin
anirudhprasad-sap Aug 23, 2024
4ca6bf6
moved configurable template to flag
anirudhprasad-sap Aug 26, 2024
2d7f0b4
more checks and tests
anirudhprasad-sap Aug 27, 2024
d5ae516
more checks
anirudhprasad-sap Aug 27, 2024
93fcdf8
convert keys to camelcase
anirudhprasad-sap Aug 27, 2024
071f02c
Merge branch 'main' into newHelmChart
anirudhprasad-sap Aug 27, 2024
8bacb86
update
anirudhprasad-sap Aug 27, 2024
9954322
readme update
anirudhprasad-sap Aug 30, 2024
0d4872e
Merge branch 'main' into newHelmChart
anirudhprasad-sap Sep 5, 2024
34feaab
removed cap & cav prefix on cap-operator resources
anirudhprasad-sap Sep 10, 2024
d30a714
appName derivation change
anirudhprasad-sap Sep 13, 2024
cca2381
xs-security.json fix for mta
anirudhprasad-sap Sep 13, 2024
5a10bc4
refactor run method
anirudhprasad-sap Sep 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 40 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,17 @@ To integrate the CAP Operator Plugin into your project, follow these steps:
```
![](.images/cds-add-cap-operator-with-templates.gif)

* `--with-configurable-templates`

With this option, the plugin adds a chart with configurable templates. This is required when applications need to utilize template functions in the CAP Operator resources. In this version of the chart, all the CAP Operator resource configurations are defined in `templates/cap-operator-cros.yaml`. If you choose this option, you can skip the `cds build` step since the chart already contains the `templates` folder.

```sh
cds add cap-operator --with-configurable-templates
```
![](.images/cds-add-cap-operator-with-configurable-templates.gif)

> If you have already added your chart and want to switch to `--with-configurable-templates`, you can use the plugin to convert the existing chart. For more information, see the [Converting to Configurable Templates](#converting-to-configurable-templates-chart) section.

* `--force`

This option allows you to overwrite the existing chart folder.
Expand Down Expand Up @@ -73,18 +84,23 @@ To integrate the CAP Operator Plugin into your project, follow these steps:

3. Once you've executed the command above, the basic chart folder or chart folder with templates is added to your project directory, depending on your choice.

## Configure the Plugin
## Configure Your Chart

The generated `chart/values.yaml` contains two types of information:

* Design-time deployment
- [serviceInstances](https://github.com/SAP/sap-btp-service-operator?tab=readme-ov-file#service-instance)
- [serviceBindings](https://github.com/SAP/sap-btp-service-operator?tab=readme-ov-file#service-binding)
- workloads - There are two types of workloads:
- [Deployment definition](https://sap.github.io/cap-operator/docs/usage/resources/capapplicationversion/#workloads-with-deploymentdefinition)
- [Job definition](https://sap.github.io/cap-operator/docs/usage/resources/capapplicationversion/#workloads-with-jobdefinition)
- [tenantOperations](https://sap.github.io/cap-operator/docs/usage/resources/capapplicationversion/#sequencing-tenant-operations)
- [contentJobs](https://sap.github.io/cap-operator/docs/usage/resources/capapplicationversion/#sequencing-content-jobs)
* Without option `--with-configurable-templates`
- [serviceInstances](https://github.com/SAP/sap-btp-service-operator?tab=readme-ov-file#service-instance)
- [serviceBindings](https://github.com/SAP/sap-btp-service-operator?tab=readme-ov-file#service-binding)
- workloads - There are two types of workloads:
- [Deployment definition](https://sap.github.io/cap-operator/docs/usage/resources/capapplicationversion/#workloads-with-deploymentdefinition)
- [Job definition](https://sap.github.io/cap-operator/docs/usage/resources/capapplicationversion/#workloads-with-jobdefinition)
- [tenantOperations](https://sap.github.io/cap-operator/docs/usage/resources/capapplicationversion/#sequencing-tenant-operations)
- [contentJobs](https://sap.github.io/cap-operator/docs/usage/resources/capapplicationversion/#sequencing-content-jobs)
* With option `--with-configurable-templates`
- [serviceInstances](https://github.com/SAP/sap-btp-service-operator?tab=readme-ov-file#service-instance)
- [serviceBindings](https://github.com/SAP/sap-btp-service-operator?tab=readme-ov-file#service-binding)
- workloads - With this option all the workload configuations are maintained in `templates/cap-operator-cros.yaml` and in the `values.yaml` you can only define the images for the workloads.

* Runtime deployment
- app
Expand All @@ -111,7 +127,7 @@ The generated `chart/values.yaml` contains two types of information:

![](.images/cds-build.gif)

> If you've already added the `templates` folder during the initial plugin call using `--with-templates` option, you can skip this step as the Helm chart is already complete.
> If you've already added the `templates` folder during the initial plugin call using `--with-templates` or `--with-configurable-templates` option, you can skip this step as the Helm chart is already complete.

2. Up to this point, you've only filled in the design time information in the chart. But to deploy the application, you need to create a `runtime-values.yaml` file with all the runtime values, as mentioned in the section on configuration. You can generate the file using the plugin itself.

Expand Down Expand Up @@ -175,6 +191,21 @@ The generated `chart/values.yaml` contains two types of information:
helm upgrade -i -n <namespace> <release-name> <project-path>/gen/chart --set-file serviceInstances.xsuaa.jsonParameters=<project-path>/xs-security.json -f <project-path>/chart/runtime-values.yaml
```

## Converting to Configurable Templates Chart

If you've already added the basic chart folder and want to switch to configurable templates chart, you can use the plugin to convert the chart. To do so, run the following command:

```sh
npx cap-op-plugin convert-to-configurable-template-chart
```

If you want to convert the existing `runtime-values.yaml` as well to the new format, you can do so by passing the `runtime-values.yaml` path via the `--with-runtime-yaml` option:

```sh
npx cap-op-plugin convert-to-configurable-template-chart --with-runtime-yaml <project-path>/chart/runtime-values.yaml
```
![](.images/cap-op-plugin-convert-to-configurable-templates.gif)

## Example

As a reference, check out the [CAP Operator Helm chart](https://github.com/cap-js/incidents-app/tree/cap-operator-plugin/chart) in the sample incident app. Also, take a look at the corresponding [runtime-values.yaml](https://github.com/cap-js/incidents-app/blob/cap-operator-plugin/chart/runtime-values.yaml) file.
Expand Down
112 changes: 97 additions & 15 deletions bin/cap-op-plugin.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,35 @@ const yaml = require('@sap/cds-foss').yaml
const Mustache = require('mustache')
const { spawn } = require('child_process')

const { ask, mergeObj, isCAPOperatorChart } = require('../lib/util')
const { ask, mergeObj, isCAPOperatorChart, isConfigurableTemplateChart, transformValuesAndFillCapOpCroYaml } = require('../lib/util')

const SUPPORTED = { 'generate-runtime-values': ['--with-input-yaml'] }
const SUPPORTED = { 'generate-runtime-values': ['--with-input-yaml'], 'convert-to-configurable-template-chart': ['--with-runtime-yaml'] }

async function capOperatorPlugin(cmd, option, inputYamlPath) {
async function capOperatorPlugin(cmd, option, yamlPath) {
try {
if (!cmd) return _usage()
if (!Object.keys(SUPPORTED).includes(cmd)) return _usage(`Unknown command ${cmd}.`)
if (option && !SUPPORTED[cmd].includes(option)) return _usage(`Invalid option ${option}.`)
if (option === '--with-input-yaml' && !inputYamlPath) return _usage(`Input yaml path is missing.`)

if (cmd === 'generate-runtime-values') await generateRuntimeValues(option, inputYamlPath)
if (cmd === 'generate-runtime-values') {
if (option === '--with-input-yaml' && !yamlPath)
return _usage(`Input yaml path is missing.`)

if (option === '--with-input-yaml' && !yamlPath && cds.utils.exists(cds.utils.path.join(cds.root,yamlPath)))
return _usage(`Input yaml path ${yamlPath} does not exist.`)

await generateRuntimeValues(option, yamlPath)
}

if (cmd === 'convert-to-configurable-template-chart') {
if (option === '--with-runtime-yaml' && !yamlPath)
return _usage(`Input runtime yaml path is missing.`)

if (option === '--with-runtime-yaml' && !yamlPath && cds.utils.exists(cds.utils.path.join(cds.root,yamlPath)))
return _usage(`Input runtime yaml path ${yamlPath} does not exist.`)

await convertToconfigurableTemplateChart(option, yamlPath)
}
} catch (e) {
if (isCli) {
console.error(e.message)
Expand Down Expand Up @@ -46,21 +63,82 @@ COMMANDS

generate-runtime-values [--with-input-yaml <input-yaml-path>] Generate runtime-values.yaml file for the cap-operator chart

convert-to-configurable-template-chart [--with-runtime-yaml <runtime-yaml-path>] Convert existing chart to configurable template chart

EXAMPLES

cap-op-plugin generate-runtime-values
cap-op-plugin generate-runtime-values --with-input-yaml /path/to/input.yaml

cap-op-plugin convert-to-configurable-template-chart
cap-op-plugin convert-to-configurable-template-chart --with-runtime-yaml /path/to/runtime.yaml
`
)
}

async function transformRuntimeValues(runtimeYamlPath) {
console.log('Transforming runtime values file '+ cds.utils.path.join(cds.root,runtimeYamlPath) + ' to the configurable template chart format.')
let runtimeYaml = yaml.parse(await cds.utils.read(cds.utils.path.join(cds.root, runtimeYamlPath)))
if (runtimeYaml?.workloads?.server?.deploymentDefinition?.env) {
const index = runtimeYaml.workloads.server.deploymentDefinition.env.findIndex(e => e.name === 'CDS_CONFIG')
if (index > -1) {
const cdsConfigValueJson = JSON.parse(runtimeYaml.workloads.server.deploymentDefinition.env[index].value)
if (cdsConfigValueJson?.requires?.['cds.xt.DeploymentService']?.hdi?.create?.database_id){
runtimeYaml['hanaInstanceId'] = cdsConfigValueJson.requires['cds.xt.DeploymentService'].hdi.create.database_id
delete runtimeYaml['workloads']
await cds.utils.write(yaml.stringify(runtimeYaml)).to(cds.utils.path.join(cds.root, runtimeYamlPath))
}
}
}
}

async function isRuntimeValueAlreadyTransformed(runtimeYamlPath) {
let runtimeYaml = yaml.parse(await cds.utils.read(cds.utils.path.join(cds.root, runtimeYamlPath)))
return !!runtimeYaml['hanaInstanceId']
}

async function convertToconfigurableTemplateChart(option, runtimeYamlPath) {
if (!((cds.utils.exists('chart') && isCAPOperatorChart(cds.utils.path.join(cds.root,'chart')))))
throw new Error("No CAP Operator chart found in the project. Please run 'cds add cap-operator --force' to add the CAP Operator chart folder.")

if (isConfigurableTemplateChart(cds.utils.path.join(cds.root,'chart'))){
console.log("Exisiting chart is already a configurable template chart. No need for conversion.")
if (option === '--with-runtime-yaml' && runtimeYamlPath && !(await isRuntimeValueAlreadyTransformed(runtimeYamlPath)))
await transformRuntimeValues(runtimeYamlPath)
else
console.log('Runtime values file '+ cds.utils.path.join(cds.root,runtimeYamlPath) + ' already in the configurable template chart format.')
return
}

console.log('Converting chart '+cds.utils.path.join(cds.root,'chart')+' to configurable template chart.')

// Copy templates
await cds.utils.copy(cds.utils.path.join(__dirname, '../files/configurableTemplatesChart/templates')).to(cds.utils.path.join(cds.root,'chart','templates'))

// Copy values.schema.json
await cds.utils.copy(cds.utils.path.join(__dirname, '../files/configurableTemplatesChart/values.schema.json')).to(cds.utils.path.join(cds.root,'chart', 'values.schema.json'))

// Add annotation to chart.yaml
const chartYaml = yaml.parse(await cds.utils.read(cds.utils.path.join(cds.root, 'chart/Chart.yaml')))
chartYaml['annotations']['app.kubernetes.io/part-of'] = 'cap-operator-configurable-templates'
await cds.utils.write(yaml.stringify(chartYaml)).to(cds.utils.path.join(cds.root, 'chart/Chart.yaml'))

// Transform
await transformValuesAndFillCapOpCroYaml()

if (option === '--with-runtime-yaml' && runtimeYamlPath) {
await transformRuntimeValues(runtimeYamlPath)
}
}

async function generateRuntimeValues(option, inputYamlPath) {
if (!((cds.utils.exists('chart') && isCAPOperatorChart(cds.utils.path.join(cds.root,'chart'))))) {
throw new Error("No CAP Operator chart found in the project. Please run 'cds add cap-operator --force' to add the CAP Operator chart folder.")
}

let answerStruct = {}
const { appName, appDescription } = getAppDetails()
const isConfigurableTempChart = isConfigurableTemplateChart(cds.utils.path.join(cds.root,'chart'))

if (option === '--with-input-yaml' && inputYamlPath) {

Expand Down Expand Up @@ -104,6 +182,17 @@ async function generateRuntimeValues(option, inputYamlPath) {
if (!answerStruct['imagePullSecret'])
delete runtimeValuesYaml['imagePullSecrets']

if (isConfigurableTempChart && answerStruct['hanaInstanceId'])
runtimeValuesYaml['hanaInstanceId'] = answerStruct['hanaInstanceId']

if (!isConfigurableTempChart)
updateWorkloadEnv(runtimeValuesYaml, valuesYaml, answerStruct)

await cds.utils.write(yaml.stringify(runtimeValuesYaml)).to(cds.utils.path.join(cds.root, 'chart/runtime-values.yaml'))
console.log("Generated 'runtime-values.yaml' file in the 'chart' folder.")
}

function updateWorkloadEnv(runtimeValuesYaml, valuesYaml, answerStruct) {
runtimeValuesYaml['workloads'] = {}
for (const [workloadKey, workloadDetails] of Object.entries(valuesYaml.workloads)) {

Expand Down Expand Up @@ -131,17 +220,10 @@ async function generateRuntimeValues(option, inputYamlPath) {

// remove workload definition where env is empty
for (const [workloadKey, workloadDetails] of Object.entries(runtimeValuesYaml.workloads)) {
if (workloadDetails?.deploymentDefinition?.env.length === 0) {
delete runtimeValuesYaml['workloads'][workloadKey]
}

if (workloadDetails?.jobDefinition?.env.length === 0) {
if (workloadDetails?.deploymentDefinition?.env.length === 0 || workloadDetails?.jobDefinition?.env.length === 0) {
delete runtimeValuesYaml['workloads'][workloadKey]
}
}

await cds.utils.write(yaml.stringify(runtimeValuesYaml)).to(cds.utils.path.join(cds.root, 'chart/runtime-values.yaml'))
console.log("Generated 'runtime-values.yaml' file in the 'chart' folder.")
}

function getServiceInstanceKeyName(serviceInstances, offeringName) {
Expand Down Expand Up @@ -193,8 +275,8 @@ async function getShootDomain() {
}

if (isCli) {
const [, , cmd, option, inputYamlPath] = process.argv;
(async () => await capOperatorPlugin(cmd, option, inputYamlPath ?? undefined))()
const [, , cmd, option, yamlPath] = process.argv;
(async () => await capOperatorPlugin(cmd, option, yamlPath ?? undefined))()
}

module.exports = { capOperatorPlugin }
2 changes: 1 addition & 1 deletion files/approuter.yaml.hbs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
workloads:
app-router:
appRouter:
name: app-router
labels:
sme.sap.com/app-type: {{appName}}
Expand Down
10 changes: 7 additions & 3 deletions files/chart/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
{{- define "capApplicationVersionName" -}}
{{ printf "cav-%s-%d" (include "appName" $) (.Release.Revision) }}
{{ printf "%s-%d" (include "appName" $) (.Release.Revision) }}
{{- end -}}

{{- define "appName" -}}
{{- $xsuaa := index .Values.serviceInstances "xsuaa" -}}
{{ printf "%s" $xsuaa.parameters.xsappname }}
{{- range $sik, $siv := .Values.serviceInstances}}
{{- if and (eq (get $siv "serviceOfferingName") "xsuaa") (eq (get $siv "servicePlanName") "broker") -}}
{{ printf "%s" $siv.parameters.xsappname }}
{{- break -}}
{{- end -}}
{{- end -}}
{{- end -}}
12 changes: 6 additions & 6 deletions files/chart/templates/cap-operator-cros.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
apiVersion: sme.sap.com/v1alpha1
kind: CAPApplication
metadata:
name: cap-{{ include "appName" $ }}
name: {{ include "appName" $ }}
spec:
domains:
primary: {{.Values.app.domains.primary}}
Expand All @@ -18,10 +18,10 @@ spec:
value: {{ $v | default "invalidValue"}}
{{- end }}
btpAppName: {{ include "appName" $ }}
globalAccountId: "{{.Values.btp.globalAccountId}}"
globalAccountId: {{.Values.btp.globalAccountId}}
provider:
subDomain: "{{.Values.btp.provider.subdomain}}"
tenantId: "{{.Values.btp.provider.tenantId}}"
subDomain: {{.Values.btp.provider.subdomain}}
tenantId: {{.Values.btp.provider.tenantId}}
btp:
services:
{{- $serviceInstances := .Values.serviceInstances }}
Expand Down Expand Up @@ -51,8 +51,8 @@ metadata:
helm.sh/resource-policy: keep
name: {{ include "capApplicationVersionName" $ }}
spec:
capApplicationInstance: "cap-{{ include "appName" $ }}"
version: "{{ .Release.Revision }}"
capApplicationInstance: {{ include "appName" $ }}
version: {{ .Release.Revision }}
registrySecrets:
{{- range .Values.imagePullSecrets }}
- {{.}}
Expand Down
9 changes: 9 additions & 0 deletions files/configurableTemplatesChart/Chart.yaml.hbs
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: v2
name: {{appName}}
description: Helm chart to deploy {{appName}} using CAP Operator
type: application
version: 0.0.1
appVersion: {{appVersion}}
annotations:
app.kubernetes.io/managed-by: cap-operator-plugin
app.kubernetes.io/part-of: cap-operator-configurable-templates
41 changes: 41 additions & 0 deletions files/configurableTemplatesChart/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
{{- define "capApplicationVersionName" -}}
{{ printf "%s-%d" (include "appName" $) (.Release.Revision) }}
{{- end -}}

{{- define "appName" -}}
{{- range $sik, $siv := .Values.serviceInstances}}
{{- if and (eq (get $siv "serviceOfferingName") "xsuaa") (eq (get $siv "servicePlanName") "broker") -}}
{{ printf "%s" $siv.parameters.xsappname }}
{{- break -}}
{{- end -}}
{{- end -}}
{{- end -}}

{{- define "hasService" -}}
{{- $found := "false" -}}
{{- $offeringName := .offeringName -}}
{{- $planName := .planName -}}
{{- $si := .si -}}
{{- range $sik, $siv := $si}}
{{- if and (eq (get $siv "serviceOfferingName") $offeringName) (eq (get $siv "servicePlanName") $planName) -}}
{{- $found = "true" -}}
{{- end -}}
{{- end -}}
{{- $found -}}
{{- end -}}

{{- define "domainPatterns" -}}
{{- if .Values.app.domains.secondary -}}
{{- $doms := list .Values.app.domains.primary -}}
{{- range .Values.app.domains.secondary -}}
{{- $doms = append $doms . -}}
{{- end -}}
{{- if gt (len $doms) 1 -}}
{{- join "|" $doms | printf "(%s)" -}}
{{- else -}}
{{- first $doms -}}
{{- end -}}
{{- else -}}
{{- printf "%s" .Values.app.domains.primary -}}
{{- end -}}
{{- end -}}
Loading
Loading