Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/mustache bug #197

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
b8f190f
chore: Rename container & add hostname
mfd2007 Aug 15, 2024
1c7e02e
feat: add secvisogram container
mfd2007 Aug 15, 2024
b5c7570
fix: dependencies of containers fixed
mfd2007 Aug 15, 2024
f0883ee
feat: add reverse proxy
mfd2007 Aug 15, 2024
6c50cd2
chore: Rename container & add hostname
mfd2007 Aug 15, 2024
d87a406
feat: add secvisogram container
mfd2007 Aug 15, 2024
5d3c2bb
fix: dependencies of containers fixed
mfd2007 Aug 15, 2024
cf5ec72
feat: add reverse proxy
mfd2007 Aug 15, 2024
00335b0
fix: make /about - response valid json
mfd2007 Aug 15, 2024
16c8af5
feat: add reverse proxy
mfd2007 Aug 15, 2024
0621699
docs: Update readme start test environment
mfd2007 Aug 15, 2024
65faf71
fix: volume path of secvisogram container
mfd2007 Aug 16, 2024
f2a0e59
doc: add further information
mfd2007 Aug 16, 2024
8c9a75d
fix: remove volume definition
mfd2007 Aug 16, 2024
4660e5e
chore: remove unnecessary urls from oauth-proxy
mfd2007 Aug 16, 2024
466529c
fix: logout problems in combination of Proxy and keycloak
mfd2007 Aug 20, 2024
87b6528
Update README.md
mfd2007 Aug 21, 2024
40c9e61
Update README.md
mfd2007 Aug 21, 2024
f2017a1
docs: Clarification
mfd2007 Aug 21, 2024
24373f8
Merge branch 'fix/hostdockerinternal' of [email protected]:secvisogram/c…
mfd2007 Aug 21, 2024
c6a2eec
chore: Migrate mustache templates and scripts from secvisogram.
mfd2007 Aug 21, 2024
f3e1992
test: Update test to match changed template
mfd2007 Aug 21, 2024
83553de
Merge pull request #195 from secvisogram/fix/hostdockerinternal
tschmidtb51 Aug 21, 2024
faeefd2
fix: Add missing template-function for href
mfd2007 Aug 22, 2024
c9d65ce
chore: Migrate mustache templates and scripts from secvisogram.
mfd2007 Aug 21, 2024
7469094
test: Update test to match changed template
mfd2007 Aug 21, 2024
f4ed1cc
fix: Add missing template-function for href
mfd2007 Aug 22, 2024
9c33159
chore: merge fix_external entities
mfd2007 Aug 22, 2024
13f55dd
fix: problems with merge
mfd2007 Aug 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 68 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,31 +87,94 @@ If you want different passwords, database names or ports you can change them
in that file. Please note that the following setup is for development purposes
only and should not be used in production.

```mermaid
C4Component
title Component diagram for CSAF CMS Backend

Person(user,"User")
Container(reverseproxy, "Reverse-Proxy", "nginx")

Container_Boundary(c4, "Internal") {
Container(secvisogram, "Secvisogram", "nginx + javascript", "Provides secvisogramm via their web browser.")

Container_Boundary(c2, "Keycloak") {
Container(keycloak, "Keycloak", "keycloak")
ContainerDb(keycloak-db, "PostGreSQL", "Keycloak-Database")
}

Container_Boundary(c3, "Oauth") {
Container(oauth, "OAuth2-Proxy", "Authentication for REST-API")
Container(validator, "CSAF validator service", "node")

Container_Boundary(c1, "Backend") {
Container(backend, "CSAF-CMS-Backend", "Spring Boot")
ContainerDb(backend-db, "CouchDB", "CMS-Backend-Database")
}
}
}

Rel(user, reverseproxy,"","HTTPS")
Rel(reverseproxy, secvisogram,"/")
Rel(reverseproxy, oauth,"/api/*")
Rel(reverseproxy, keycloak,"/realm/csaf/")
Rel(oauth, validator, "/api/v1/test")
Rel(oauth, validator, "/api/v1/validate")
Rel(oauth, backend, "/api/v1/advisories/*")
Rel(backend, backend-db,"")
Rel(backend, keycloak,"")
Rel(keycloak, keycloak-db,"")


```

- run `docker compose up`
- After Keycloak is up, open a second terminal window and run
`docker compose up csaf-keycloak-cli` to import a realm with all the users
and roles already set up.
- To set up our CouchDB server open `http://127.0.0.1:5984/_utils/#/setup`
and run the [Single Node Setup](https://docs.couchdb.org/en/stable/setup/single-node.html). This creates databases like **_users** and
stops CouchDB from spamming our logs
and run the [Single Node Setup](https://docs.couchdb.org/en/stable/setup/single-node.html). This creates databases like **_users** and stops CouchDB from spamming our logs (Admin credentials from .env)
- Create a database in CouchDB with the name specified in `CSAF_COUCHDB_DBNAME`
- Open `http://localhost:9000/` and log in with the admin user.
- The port is defined in .env - CSAF_KEYCLOAK_PORT, default 9000
- Select `CSAF`-Realm
- On the left side, navigate to "Clients" and select the Secvisogram client.
- Select the **Credentials** tab and copy the Secret. This is our
`CSAF_CLIENT_SECRET` environment variable.
- [Generate a cookie secret](https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/overview/#generating-a-cookie-secret)
and paste it in `CSAF_COOKIE_SECRET`.
- Create a database in CouchDB with the name specified in `CSAF_COUCHDB_DBNAME`
- restart compose
- (required for exports) install [pandoc (tested with version 2.18)](https://pandoc.org/installing.html)
as well as [weasyprint (tested with version 56.0)](https://weasyprint.org/) and make sure both are in
your PATH
- (optional for exports) define the path to a company logo that should be used in the exports through the environment variable `CSAF_COMPANY_LOGO_PATH`. The path can either be relative to the project root or absolute. See .env.example file for an example.

You should now be able to start the spring boot application, navigate to
`localhost:4180/api/v1/about`, log in with one of the users and get a
`http://localhost/api/v1/about`, log in with one of the users and get a
response from the server.
The port is defined in .env - CSAF_APP_EXTERNAL_PORT, default 4180

You should now be able to access Secvisogram, navigate to `http://localhost/`.
There are the following default users:
|User |Password |Roles |
|----- |-------- |----- |
|registered |registered |**registered** |
|author |author |registered, editor, **author** |
|editor |editor |registered, **editor** |
|publisher |publisher |registered, editor, **publisher** |
|reviewer |reviewer |registered, **reviewer** |
|auditor |auditor |**auditor** |
|all |all |**auditor, reviewer, publisher, editor, author, registred** |
|none |none | |

### Login & Logout in combination with Secvisogram

Some explantion on the logoutUrl configured in `.well-known/appspecific/de.bsi.secvisogram.json` for Secvisogram

```
"logoutUrl": "/oauth2/sign_out?rd=http://localhost/realms/csaf/protocol/openid-connect/logout?post_logout_redirect_uri=http%3A%2F%2Flocalhost&client_id=secvisogram",
```

`/oauth2/sign_out` is the logout URI from the OAUTH-Proxy. This will invalidate the session on the proxy. Then, a redirect to Keycloak (`http://localhost/realms/csaf/protocol/openid-connect/logout?post_logout_redirect_uri=http%3A%2F%2Flocalhost&client_id=secvisogram`) is necessary to log out from the session on Keyloak. Subsequently, there is a redirect back to Secvisogram (`localhost`).
When hostnames are changed, this has to adapted.

### build and execute tests

Expand Down
94 changes: 74 additions & 20 deletions compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
###############################################################################

services:
csaf-couchdb:
cms-couchdb:
image: couchdb:3.3
container_name: csaf-couchdb
hostname: couchdb.csaf.internal
#container_name: cms-couchdb
restart: on-failure
env_file: .env
environment:
Expand All @@ -15,10 +16,15 @@ services:
- csaf-couchdb-data:/opt/couchdb/data
ports:
- "${CSAF_COUCHDB_PORT}:5984"
networks:
default:
aliases:
- "couchdb.csaf.internal"

csaf-keycloak-db:
keycloak-db:
image: postgres:14
container_name: csaf-keycloak-db
hostname: keycloak-db.csaf.internal
#container_name: keycloak-db
volumes:
- csaf-keycloak-db-data:/var/lib/postgresql/data
env_file: .env
Expand All @@ -29,17 +35,22 @@ services:
restart: on-failure
ports:
- "${CSAF_KEYCLOAK_DATABASE_PORT}:5432"
networks:
default:
aliases:
- "keycloak-db.csaf.internal"

csaf-keycloak:
keycloak:
image: quay.io/keycloak/keycloak:20.0
container_name: csaf-keycloak
hostname: keycloak.csaf.internal
#container_name: keycloak
env_file: .env
environment:
# https://www.keycloak.org/server/all-config
KC_HEALTH_ENABLED: "true"
KC_METRICS_ENABLED: "true"
KC_DB: postgres
KC_DB_URL_HOST: csaf-keycloak-db
KC_DB_URL_HOST: keycloak-db.csaf.internal
KC_DB_URL_PORT: 5432
KC_DB_URL_DATABASE: ${CSAF_KEYCLOAK_DATABASE_NAME}
KC_DB_USERNAME: ${CSAF_KEYCLOAK_DATABASE_USER}
Expand All @@ -48,36 +59,42 @@ services:
KEYCLOAK_ADMIN: ${CSAF_KEYCLOAK_ADMIN_USER}
KEYCLOAK_ADMIN_PASSWORD: ${CSAF_KEYCLOAK_ADMIN_PASSWORD}
depends_on:
- csaf-keycloak-db
- keycloak-db
restart: on-failure
ports:
- "${CSAF_KEYCLOAK_PORT}:8080"
command: ["start-dev"] # https://www.keycloak.org/server/configuration#_starting_keycloak_in_production_mode

networks:
default:
aliases:
- "keycloak.csaf.internal"

# Run this manually to import the default keycloak config since 'depends_on' is currently broken.
csaf-keycloak-cli:
keycloak-cli:
image: adorsys/keycloak-config-cli:latest-20.0.1
container_name: csaf-keycloak-cli
#container_name: keycloak-cli
profiles: [ "run_manually" ]
env_file: .env
environment:
KEYCLOAK_URL: "http://csaf-keycloak:8080/"
KEYCLOAK_URL: "http://keycloak.csaf.internal:8080/"
KEYCLOAK_USER: ${CSAF_KEYCLOAK_ADMIN_USER}
KEYCLOAK_PASSWORD: ${CSAF_KEYCLOAK_ADMIN_PASSWORD}
IMPORT_FILES_LOCATIONS: "/config/csaf-realm.json"
volumes:
- ./keycloak:/config:z
restart: on-failure
depends_on:
- keycloak

csaf-oauth2-proxy:
oauth2-proxy:
image: bitnami/oauth2-proxy:7.4.0
container_name: csaf-oauth2-proxy
hostname: oauth2.csaf.internal
#container_name: oauth2-proxy
command: [""]
env_file: .env
environment:
# listening address and proxy target
OAUTH2_PROXY_HTTP_ADDRESS: "0.0.0.0:4180"
OAUTH2_PROXY_UPSTREAMS: "http://host.docker.internal:${CSAF_VALIDATOR_PORT}/api/v1/validate,http://host.docker.internal:${CSAF_VALIDATOR_PORT}/api/v1/tests,http://host.docker.internal:${CSAF_CMS_BACKEND_PORT}/api/v1/"
OAUTH2_PROXY_UPSTREAMS: "http://host.docker.internal:${CSAF_CMS_BACKEND_PORT}/api/v1/"

# Security related config
OAUTH2_PROXY_COOKIE_SECURE: "false"
Expand All @@ -91,7 +108,7 @@ services:
OAUTH2_PROXY_PROVIDER_DISPLAY_NAME: "CSAF OIDC Provider"
# You need to set your keycloak "Frontend URL", in our case "http://localhost:9000/auth/"
# If you don't want to use autodiscovery, you have to set all urls by hand (login-url, oidc-jwks-url, redeem-url, ...)
OAUTH2_PROXY_OIDC_ISSUER_URL: "http://csaf-keycloak:8080/realms/${CSAF_REALM}"
OAUTH2_PROXY_OIDC_ISSUER_URL: "http://keycloak.csaf.internal:8080/realms/${CSAF_REALM}"
OAUTH2_PROXY_INSECURE_OIDC_SKIP_ISSUER_VERIFICATION: "true"
OAUTH2_PROXY_WHITELIST_DOMAINS: "localhost:4180,localhost:8080"

Expand All @@ -113,16 +130,53 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
restart: on-failure

csaf-validation-server:
depends_on:
- keycloak
networks:
default:
aliases:
- "oauth2.csaf.internal"

validator:
build:
context: https://github.com/secvisogram/csaf-validator-service.git#main
container_name: csaf-validation-server
#container_name: validator
hostname: validator.csaf.internal
env_file: .env
ports:
- "$CSAF_VALIDATOR_PORT:8082"
networks:
default:
aliases:
- "validator.csaf.internal"

secvisogram:
build:
context: ./docker/secvisogram
dockerfile: Dockerfile
hostname: secvisogram.csaf.internal
volumes:
- "./docker/secvisogram/appspecific:/usr/share/nginx/html/.well-known/appspecific"
networks:
default:
aliases:
- "secvisogram.csaf.internal"


reverse-proxy:
image: nginx:1.23-alpine
hostname: "reverseproxy.csaf.internal"
restart: on-failure
ports:
- "80:80"
volumes:
- "./docker/reverseproxy/nginx.conf:/etc/nginx/nginx.conf"
depends_on:
- secvisogram
- keycloak
- oauth2-proxy
- validator

volumes:
csaf-couchdb-data:
driver: local
Expand Down
62 changes: 62 additions & 0 deletions docker/reverseproxy/nginx.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
worker_processes 1;

events { worker_connections 1024; }

http {
sendfile on;

proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;


#https://www.getpagespeed.com/server-setup/nginx/tuning-proxy_buffer_size-in-nginx
proxy_buffer_size 16k; # should be enough for most PHP websites, or adjust as above
proxy_busy_buffers_size 24k; # essentially, proxy_buffer_size + 2 small buffers of 4k
proxy_buffers 64 4k; # should be enough for most PHP websites, adjust as above to get an accurate value

server {
listen 80;

location /realms {
proxy_pass http://keycloak.csaf.internal:8080/realms;
proxy_redirect off;
}

location /resources{
proxy_pass http://keycloak.csaf.internal:8080/resources;
proxy_redirect off;
}

location /validate/api/v1/tests {
proxy_pass http://validator.csaf.internal:8082/api/v1/tests;
proxy_redirect off;
}

location /validate/api/v1/validate {
proxy_pass http://validator.csaf.internal:8082/api/v1/validate;
proxy_redirect off;
}

location /api/ {
proxy_pass http://oauth2.csaf.internal:4180;
proxy_redirect off;
}

location /oauth2 {
proxy_pass http://oauth2.csaf.internal:4180/oauth2;
proxy_redirect off;
}

location /.well-known/appspecific/de.bsi.secvisogram.json {
proxy_pass http://secvisogram.csaf.internal/.well-known/appspecific/de.bsi.secvisogram.json;
proxy_redirect off;
}

location / {
proxy_pass http://secvisogram.csaf.internal/;
proxy_redirect off;
}
}
}
18 changes: 18 additions & 0 deletions docker/secvisogram/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Build Stage 1
# This build created a staging docker image
#
FROM node:20-alpine AS build
WORKDIR /usr/src
RUN apk add git; \
git clone https://github.com/secvisogram/secvisogram.git; \
cd secvisogram; \
npm ci; \
npm run build

# Build Stage 2
# This build takes the production build from staging build
#

FROM nginx:1.23-alpine
COPY --from=build /usr/src/secvisogram/app/dist /usr/share/nginx/html
EXPOSE 80
7 changes: 7 additions & 0 deletions docker/secvisogram/appspecific/de.bsi.secvisogram.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"loginAvailable": true,
"loginUrl": "/oauth2/sign_in?rd=http%3A%2F%2Flocalhost",
"logoutUrl": "/oauth2/sign_out?rd=http://localhost/realms/csaf/protocol/openid-connect/logout?post_logout_redirect_uri=http%3A%2F%2Flocalhost&client_id=secvisogram",
"userInfoUrl": "/oauth2/userinfo",
"validatorUrl": "/validate"
}
2 changes: 1 addition & 1 deletion keycloak/csaf-realm.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
"registrationAllowed": false,
"verifyEmail": false,
"attributes" : {
"frontendUrl": "http://localhost:9000/"
"frontendUrl": "http://localhost/"
},
"roles": {
"client": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ public class MainController {
)
public String about() {
LOG.info("about");
return "{version:\"" + buildProperties.getVersion() + "\"}";
return "{\"version\":\"" + buildProperties.getVersion() + "\"}";
}

}
Loading
Loading