Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

route "stdout" contains reference to undefined or misconfigured template "raw-json"." #496

Closed
gsoeldner opened this issue Nov 16, 2022 · 33 comments · Fixed by #556
Closed

Comments

@gsoeldner
Copy link

Description

What did you expect to happen?

The installation succeeded. We expected to see results of the scan in the container log.

What happened instead?

We received error from the logs regarding the template.

Output of run with POSTEE_DEBUG=true:

#################################################

ERROR MESSAGE

#################################################
{
insertId: "q35i4olmi0ay1o75"
labels: {5}
logName: "projects/t-2004107-aquaops-0000/logs/stderr"
receiveTimestamp: "2022-11-16T10:55:12.840010372Z"
resource: {2}
severity: "ERROR"
textPayload: "2022/11/16 10:55:08 route "stdout" contains reference to undefined or misconfigured template "raw-json"."
timestamp: "2022-11-16T10:55:08.364027297Z"
}

(paste your output here)

Additional details (environment setup, networking info...):

We copied the yaml file from the GitHub page and checked it several times. There are no syntax errors inside. Also the config map after the deployment exactly resembles the documentation.

@simar7
Copy link
Member

simar7 commented Nov 18, 2022

@AndreyLevchenko can you take a look at this?

@DmitriyLewen
Copy link
Collaborator

DmitriyLewen commented Nov 18, 2022

Hello @gsoeldner
Thanks for your report!

You can check your config file, does this file have raw-json template in template block?
i mean:

- name: raw-json                 
  rego-package: postee.rawmessage.json

Can you also tell which version of Postee you are using?

Regards, Dmitriy

@gsoeldner
Copy link
Author

Hi Dmitry,

we are using 2.9.0...we used the values.yaml as and removed all the unnecessary stuff:
https://github.com/aquasecurity/postee/blob/main/deploy/helm/postee/values.yaml

Best
Guido

@gsoeldner
Copy link
Author

Here is the config:
####################################################

Installation: helm install app -n postee aqua/postee --values values.yaml

#################################################### values.yaml
image:
repository: xxx:5200/aquasec/postee
pullPolicy: Always
tag: "2.8.4-amd64"
imageInit:
repository: xxx:5200/busybox
pullPolicy: IfNotPresent
tag: "1.34"

posteUi:
port: 8000
user: ""
pass: ""
image: xxx:5200/aquasec/postee-ui
tag: "2.8.4-amd64"

Default values for postee.

This is a YAML-formatted file.

Declare variables to be passed into your templates.

replicaCount: 1
posteeConfig: |

The configuration file contains a general settings section,

routes, templates and actions sections.

name: aqua-test # The tenant name
aqua-server: xxx
max-db-size: 1000MB # Max size of DB. pattern is used, such as "300MB" or "1GB". If empty or 0 then unlimited
db-verify-interval: 1 # How often to check the DB size. By default, Postee checks every 1 hour

Routes are used to define how to handle an incoming message

routes:

  • name: stdout
    actions: [ splunk ]
    template: raw-json

Templates are used to format a message

templates:

  • name: raw-json # route message "As Is" to external webhook
    rego-package: postee.rawmessage.json

Actions are target services that should consume the messages

actions:

  • name: splunk
    type: splunk
    enable: true
    url: xxx
    token: xxx # Mandatory. a HTTP Event Collector Token
    size-limit: 10000 # Optional. Maximum scan length, in bytes. Default: 10000

@DmitriyLewen
Copy link
Collaborator

Hello @gsoeldner

Can you check config log(start of Postee)?
From Starting Router.... to Starting WebServer....
We need to check that template has been configured.

my configuration example:

2022/11/21 10:12:29 Starting Router....
2022/11/21 10:12:29 Loading alerts configuration file ./test_config/cfg.yaml ....
2022/11/21 10:12:29 Configuring template raw-json 
2022/11/21 10:12:29 No aggregation package configured!!!
2022/11/21 10:12:29 Configured with Rego package postee.rawmessage.json
2022/11/21 10:12:29 Action stdout is configured
2022/11/21 10:12:29 Starting ServiceNow action "my-servicenow"....
2022/11/21 10:12:29 Your ServiceNow Table is "incident" on 'xxx'
2022/11/21 10:12:29 Action my-servicenow is configured
2022/11/21 10:12:29 Starting WebServer....

@g-soeldner
Copy link

Hi,

here is the thconfig log:

2022/11/21 14:18:12 Starting Router....
2022/11/21 14:18:12 Loading alerts configuration file /data/cfg.yaml ....
2022/11/21 14:18:12 Configuring template raw-json
2022/11/21 14:18:12 Can not initialize template raw-json: 1 error occurred during loading: open rego-templates/custom/lost+found: permission denied
2022/11/21 14:18:12 &router.ActionSettings{Name:"splunk", Type:"splunk", RunsOn:"", Enable:true, Url:"", User:"", Password:"", TlsVerify:false, ProjectKey:"", IssueType:"", BoardName:"", Priority:"", Assignee:[]string(nil), Summary:"", FixVersions:[]string(nil), AffectsVersions:[]string(nil), Labels:[]string(nil), Sprint:"", Unknowns:map[string]string(nil), Host:"", Port:0, Recipients:[]string(nil), Sender:"", Token:"dfc15c07-8440-4b0e-b809-69c7257ee3c9", UseMX:false, InstanceName:"", SizeLimit:10000, InputFile:"", ExecScript:"", Env:[]string(nil), BodyFile:"", BodyContent:"", Method:"", Timeout:"", Headers:map[string][]string(nil), OrganizationId:"", KubeConfigFile:"", KubeLabelSelector:"", KubeActions:map[string]map[string]string(nil), KubeNamespace:"", DockerImageName:"", DockerNetwork:"", DockerCmd:[]string(nil), DockerVolumes:map[string]string(nil), DockerEnv:[]string(nil), Tags:[]string(nil), Alias:"", Entity:""}
2022/11/21 14:18:12 Starting Action "splunk": "splunk"
2022/11/21 14:18:12 Starting Splunk action "splunk"....
2022/11/21 14:18:12 Action splunk is configured
2022/11/21 14:18:12 Starting WebServer....
2022/11/21 14:18:13 Listening for HTTPS on 0.0.0.0:8445
2022/11/21 14:18:13 Listening for HTTP on 0.0.0.0:8082

@DmitriyLewen
Copy link
Collaborator

Hello!

Thanks a lot.

Looks like there is problem with access to templates.

can you try to get list of templates in /server/rego-templates folder?

example commands(on my PC):

➜  kubectl get pods -n postee2                           
NAME                            READY   STATUS    RESTARTS   AGE
app-postee-0                    1/1     Running   0          6m30s
app-posteeui-68955bc58c-9775c   1/1     Running   0          6m30s
➜  kubectl exec -it --namespace postee2 app-postee-0 /bin/sh

kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Defaulted container "postee" out of: postee, setting-db (init), setting-cfg (init)
/server $ ls -hl /server/rego-templates/
total 60K    
drwxr-xr-x    1 postee   postee      4.0K Sep 14 23:26 common
drwxrwxrwx    2 root     root        4.0K Nov 22 05:33 custom
drwxr-xr-x    1 postee   postee      4.0K Sep 14 23:26 example
-rw-r--r--    1 postee   postee       199 Sep 14 23:26 raw-message-html.rego
-rw-r--r--    1 postee   postee        93 Sep 14 23:26 raw-message-json.rego

@g-soeldner
Copy link

Hi,

here is the output:
/server/rego-templates $ ls -hl /server/rego-templates/
total 60K
drwxr-xr-x 1 postee postee 4.0K Sep 14 23:26 common
drwxr-xr-x 3 root root 4.0K Sep 16 08:41 custom
drwxr-xr-x 1 postee postee 4.0K Sep 14 23:26 example
-rw-r--r-- 1 postee postee 199 Sep 14 23:26 raw-message-html.rego
-rw-r--r-- 1 postee postee 93 Sep 14 23:26 raw-message-json.rego
-rw-r--r-- 1 postee postee 416 Sep 14 23:26 tracee-html.rego
-rw-r--r-- 1 postee postee 654 Sep 14 23:26 tracee-slack.rego
-rw-r--r-- 1 postee postee 2.6K Sep 14 23:26 vuls-cyclonedx.rego
-rw-r--r-- 1 postee postee 467 Sep 14 23:26 vuls-html-aggregation.rego
-rw-r--r-- 1 postee postee 5.9K Sep 14 23:26 vuls-html.rego
-rw-r--r-- 1 postee postee 154 Sep 14 23:26 vuls-opsgenie.rego
-rw-r--r-- 1 postee postee 517 Sep 14 23:26 vuls-slack-aggregation.rego
-rw-r--r-- 1 postee postee 7.9K Sep 14 23:26 vuls-slack.rego

@DmitriyLewen
Copy link
Collaborator

I was able to get a similar error:

2022/11/24 03:57:11 Configuring template raw-html 
2022/11/24 03:57:11 Can not initialize template raw-html: 1 error occurred during loading: open rego-templates/custom/123: permission denied 

can you check /server/rego-templates/custom/ folder?
Looks like it contains file without permission.

Remove /server/rego-templates/custom folder and rerun postee.

@g-soeldner
Copy link

Hi,
how can we delete the folder. We are not root.
When using su, we have the error:
su: must be suid to work properly

@DmitriyLewen
Copy link
Collaborator

Hello @g-soeldner
Thanks for wait!

Can you do one more experiment:
Use /server/test/custom value in persistentVolume/mountPathRego field(values.yaml#L264).

persistentVolume:
  enabled: false
  mountPathDb: /server/database
  mountPathConfig: /data
  mountPathRego: /server/test/custom
  mountPathFilters: /server/rego-filters/custom
  accessModes:
    - ReadWriteOnce
  size: 1Gi
  annotations: {}

And install postee with this change.

@g-soeldner
Copy link

Ok that helps...but next Problem:

Error while sending event: Post "https://hecendpoint:8088/services/collector": x509: certificate signed by unknown authority

How can we ignore the warning?

@DmitriyLewen
Copy link
Collaborator

Looks like you need to install and update ca-certificates.

Postee currently has insecure flag only for jira.

@g-soeldner
Copy link

Well that is pretty complicated without being root.
The Helm Chart also does not allow to mount a volume from a config map with the certifcates.
How are custom certificates to designed to be uploaded?

@DmitriyLewen
Copy link
Collaborator

I found that helm provide --insecure-skip-tls-verify flag (https://helm.sh/docs/helm/helm_install/#options).

Can you try install postee with this flag?

@g-soeldner
Copy link

You mean this here:
--insecure-skip-tls-verify skip tls certificate checks for the chart download

We can use the chart, however we do not know how to import our ca-certificate into the app-postee-0 pod.

@DmitriyLewen
Copy link
Collaborator

DmitriyLewen commented Dec 6, 2022

if i understand correctly - you can add certificate using ca-file flag (https://stackoverflow.com/a/59966896).

@g-soeldner
Copy link

No, the installation is not the problem. We need to bring our certificates into the Pod in order to talk to Splunk....see here:
https://paraspatidar.medium.com/add-self-signed-or-ca-root-certificate-in-kubernetes-pod-ca-root-certificate-store-cb7863cb3f87

Two solutions:

  • you change the implementation for the Splunk Action (allow ssl-non-verify)
  • You change the kubernetes deployment / yaml file to upload the certificates via a config map.

@DmitriyLewen
Copy link
Collaborator

hm... you are right. It makes sense.
I will add insecure skip verify for `slack.

@g-soeldner
Copy link

We need it for Splunk not for the slack action

@g-soeldner
Copy link

There seems to be a merge into Postee for Enterprise branch...tls-verify-skip for Splunk.
#502

We really need it super super super urgent the new release!!!!!

@DmitriyLewen
Copy link
Collaborator

DmitriyLewen commented Dec 7, 2022

oh... my mistake...
Thank a lot you for finding this.

I created #505. We will try to merge it as soon as possible.

@linthoisana
Copy link

linthoisana commented Dec 10, 2022

Hello All,
I have something to clarify related to the logs collected by postee. I would like to know if there is a way where I can configure to turn off the logs but postee sends the scan result to the Roxy only. The logs is actually making some issue with the splunk in my current project.
please help me out guys. i am new ro aqua topics.

What I want__: Not to collect logs into container log by Postee but send out only the scan reports to webhook/recipents.

@DmitriyLewen
Copy link
Collaborator

Hello @linthoisana

I thinks it is different issue.
Can you create new issue and add some examples into new issue?
I asked for examples because postee doesn't include logs in send message.

@linthoisana
Copy link

thank you @DmitriyLewen . i have created an issue #507

@tectime
Copy link

tectime commented Jan 11, 2023

@DmitriyLewen do you have any updates to share regarding this issue?

@DmitriyLewen
Copy link
Collaborator

DmitriyLewen commented Jan 11, 2023

Hello @tectime
We merged #505

Next release will include this changes and you will be able to use tls-verify option in splunk.

@smangels
Copy link
Contributor

smangels commented Apr 6, 2023

Hi we are using Postee and run into the same issue where lost+found objects are confusing the template scanner. I've tested the fix posted in this thread (move mount point to /server/tests/custom). But we are still interested in a solution for it. Any idea how we could fix it? Can someone guide me where in the code the scanner stumbles upon this lost+found item?

@stanislavdip
Copy link

Hello, +1.
I have the same issue as posted before.
Are there some ideas on how to fix the "lost+found" issue?

@smangels
Copy link
Contributor

We could change the code so that certain folders can be ignored when recursively scanning for REGO templates.

@DmitriyLewen
Copy link
Collaborator

Hello @smangels , @stanislavdip
Thanks for you reports and sorry for long wait.

I created filter for rego-templates by lost+found name - #556

Can you check this changes?

Also I have 1 more request - can you send me permissions for lost + found in your cases?
It might be better to filter files by this value.

Regards, Dmitriy

@stanislavdip
Copy link

stanislavdip commented Apr 28, 2023

Hello @DmitriyLewen
Postee deployed in Kubernetes cluster via ArgoCD, helm chart:
version: v2.10.2
repository: https://aquasecurity.github.io/helm-charts/

Here are the permissions for the files:

/server $ ls -la /server/rego-templates/custom/
total 28
drwxr-xr-x    3 root     root          4096 Apr 27 07:48 .
drwxr-xr-x    1 postee   postee        4096 Apr 27 09:29 ..
drwx------    2 root     root         16384 Apr 27 07:48 lost+found
/server $ ls -la /server/rego-filters/custom/
total 28
drwxr-xr-x    3 root     root          4096 Apr 27 07:48 .
drwxr-xr-x    1 postee   postee        4096 Apr 27 09:29 ..
drwx------    2 root     root         16384 Apr 27 07:48 lost+found

@DmitriyLewen
Copy link
Collaborator

Hello @g-soeldner , @smangels , @stanislavdip

We merged #556.
Fix will be include to next release.

Regards, Dmtriyi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants