VNC not working and flow post processor exists with error

[aswad546]

Hey Sohom,

I opened up the crawler after some time and noticed some new commits notably the addition of the sha256 hashes to the script_flow table that seems to be a breaking change when attempting to use the flow post processor for vv8. Since the sha256 table does not exist in the script_flow table on startup. Perhaps adjustment of the schema would do some good. I fixed it manually for my use case by recreating the table with the sha256 column included.

Additionally the noVNC UI located on ports 5901/6901 just shows a black screen and does not show the browser open up. Based on what I recall this worked fine in the past (Did not investigate this issue for now). Just wanted to let you know about issues that I have seen since I am using the crawler again. Hope this helps.

Let me know if you have any questions about the broken features.

Best,
Aswad

[aswad546]

For the first issue I think you forgot to add sha256 as a column in the other_postprocessor_models.py file in backend/app/database_models

And in the /vv8_backend_database/init/1_postgres_schema.sql

[Ale0x78]

script_flow should have had that one field, see this commit, so might be worth git pulling on your copy of the crawler!

Problem is currently if you update your post-processor version, if you already have the database created, you will have to remake-it or run

ALTER TABLE script_flow
ADD sha256 BYTEA

to add the extra field

We should consider either pinning the post-processor version via the crawler's CLI or have some kind of migration logic.

[sohomdatta1]

I had setup an upgrade mechanism through SQLAlchemy alembic system, however, I forgot to add an upgrade path for this change. #6 should fix that and running alembic upgrade head in the backend/ folder on the latest version should automagically add the sha256 column :) -- I'll make a mental note of adding upgrade paths for any upcoming changes.

Additionally the noVNC UI located on ports 5901/6901 just shows a black screen and does not show the browser open up. Based on what I recall this worked fine in the past (Did not investigate this issue for now). Just wanted to let you know about issues that I have seen since I am using the crawler again. Hope this helps.

Hmm, I cannot reproduce this issue, can you send a screenshot and/or docker logs for the docker container?

We should consider either pinning the post-processor version via the crawler's CLI

Currently, rebuilding using the setup command force pulls the latest postprocessors, maybe we should instead pin it to a specific version which can be updated based on change?

[aswad546]

out.txt

Here is my log file it may contain some extra info but I don't see why noVnc fails for me.

[Ale0x78]

vv8_worker-1         | Starting webserver and WebSockets proxy on port 6901
vv8_worker-1         | /usr/bin/xauth:  file /home/vv8/.Xauthority does not exist
vv8_worker-1         | /opt/noVNC/utils/websockify/websockify/websocket.py:31: UserWarning: no 'numpy' module, HyBi protocol will be slower
vv8_worker-1         |   warnings.warn("no 'numpy' module, HyBi protocol will be slower")

Only warning that feels relevant. I wonder if this is a client-side issue with NoVNC.

Do you see anything in your browser console when you load the page in your browser? What browser are you using.

[aswad546]

It used to work for safari but now it does not work for safari or chrome, here is the console output:

<meta name="apple-mobile-web-app-capable" content="yes"> is deprecated. Please include <meta name="mobile-web-app-capable" content="yes">Understand this warningAI
localhost/:1 The resource http://localhost:6901/app/images/warning.svg was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.Understand this warningAI
localhost/:1 The resource http://localhost:6901/app/images/error.svg was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriate `as` value and it is preloaded intentionally.Understand this warningAI

Nothing special here I think.

Here is what the UI looks like:

[Ale0x78]

Have these containers been running for a while or are you re-running docker compose up --build every time?

[aswad546]

I run docker compose up --build relatively often whenever I need to modify the code.