Skip to content

Local Vestec Server

Jump to bottom
H. C. Kruse edited this page Jan 5, 2021 · 3 revisions

Vestec WP5 Docker Ubuntu 18.04

Written for Vestec commit 3e414b4 03.12.2020

Docker Setup (optional)

Remove the default Docker installation that ships with Ubuntu.

sudo apt-get remove docker docker-engine docker.io containerd runc

sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common

# Add the GPG Key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# Add the repository
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

# Install docker
apt-get update
apt-get install docker-ce docker-ce-cli containerd.io

Docker Compose (required)

Download the current stable release:

sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

sudo chmod +x /usr/local/bin/docker-compose

# Optionally symlink to bin
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

Vestec Setup

Clone the vestec repository from https://github.com/EPCCed/vestec-wp5.git

$ git clone https://github.com/EPCCed/vestec-wp5.git

Fix #30

diff --git a/ExternalServices/requirements.txt b/ExternalServices/requirements.txt
index 150cd55..48c631b 100644
--- a/ExternalServices/requirements.txt
+++ b/ExternalServices/requirements.txt
@@ -4,7 +4,7 @@ chardet==3.0.4
 Click==7.0
 Flask==1.0.2
 Flask-RESTful==0.3.7
-flask_jwt_extended==3.20.0
+flask_jwt_extended==3.25.0
 gunicorn==19.9.0
 idna==2.8
 itsdangerous==1.1.0

Fix the MachineInterface Dockerfile

diff --git a/MachineInterface/Dockerfile b/MachineInterface/Dockerfile
index ebb4805..ff35aaa 100644
--- a/MachineInterface/Dockerfile
+++ b/MachineInterface/Dockerfile
@@ -2,6 +2,6 @@ FROM python:3.7-slim
 COPY . /app
 WORKDIR /app/MachineInterface
 RUN apt-get update;apt-get install -y openssh-client vim
-RUN cat /app/MachineInterface/misc/ssh_archer_config >> /etc/ssh/ssh_config
+#RUN cat /app/MachineInterface/misc/ssh_archer_config >> /etc/ssh/ssh_config
 RUN pip install -r /app/MachineInterface/requirements.txt
 CMD python interface.py --config machines.yaml

Update required paths in DockerFiles/docker-compose.yml

diff --git a/DockerFiles/docker-compose.yml b/DockerFiles/docker-compose.yml
index 03d902a..a958a97 100644
--- a/DockerFiles/docker-compose.yml
+++ b/DockerFiles/docker-compose.yml
@@ -10,7 +10,7 @@ services:
       - 80:80
       - 443:443
     volumes:
-      - /var/vestec/certs:/var/vestec/certs
+#      - /var/vestec/certs:/var/vestec/certs
       - ../Nginx/static/:/var/www/
     links:
       - externalservices:externalservices
@@ -127,7 +127,8 @@ services:
     environment:
       MYSQL_ROOT_PASSWORD: "changeme"
     volumes:
-      - "/var/vestec/db:/var/lib/mysql"
+      - "../var/vestec/db:/var/lib/mysql"
     networks:
       - vestec

Update the default nginx configuration to work locally

diff --git a/Nginx/default.conf b/Nginx/default.conf
index c081bee..5fbbc1f 100644
--- a/Nginx/default.conf
+++ b/Nginx/default.conf
@@ -1,17 +1,18 @@
-server {
-    listen 80 default_server;
-    server_name vestec.epcc.ed.ac.uk;
-    return 301 https://$host$request_uri;
-}
+#server {
+#    listen 80 default_server;
+#    server_name vestec.epcc.ed.ac.uk;
+#    return 301 https://$host$request_uri;
+#}

 server {
-    listen 443 ssl;
+    listen 80 default_server; #443 ssl;
     root /var/www/;
-    server_name vestec.epcc.ed.ac.uk;
-    ssl_certificate     /var/vestec/certs/vestec.epcc.ed.ac.uk.crt;
-    ssl_certificate_key /var/vestec/certs/vestec.epcc.ed.ac.uk.key;
-    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
-    ssl_ciphers         HIGH:!aNULL:!MD5;
+    server_name _;
+#    server_name vestec.epcc.ed.ac.uk;
+#    ssl_certificate     /var/vestec/certs/vestec.epcc.ed.ac.uk.crt;
+#   ssl_certificate_key /var/vestec/certs/vestec.epcc.ed.ac.uk.key;
+#    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
+#    ssl_ciphers         HIGH:!aNULL:!MD5;
     index login.html;

     location / {
@@ -22,7 +23,7 @@ server {
         #
         # Custom headers and headers various browsers *should* be OK with but aren't
         #
-        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
+        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization';
         #
         # Tell client that this pre-flight info is valid for 20 days
         #
@@ -34,13 +35,13 @@ server {
        if ($request_method = 'POST') {
         add_header 'Access-Control-Allow-Origin' '*';
         add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
-        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
+        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization';
         add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
        }
        if ($request_method = 'GET') {
         add_header 'Access-Control-Allow-Origin' '*';
         add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
-        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
+        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization';
         add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
        }
     }
@@ -58,7 +59,7 @@ server {
         #
         # Custom headers and headers various browsers *should* be OK with but aren't
         #
-        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
+        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization';
         #
         # Tell client that this pre-flight info is valid for 20 days
         #
@@ -70,18 +71,19 @@ server {
        if ($request_method = 'POST') {
         add_header 'Access-Control-Allow-Origin' '*';
         add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
-        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
+        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization';
         add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
        }
        if ($request_method = 'GET') {
         add_header 'Access-Control-Allow-Origin' '*';
         add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
-        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
+        add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization';
         add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
        }
     }

     location /EDI {
+        client_max_body_size 256M;
         proxy_pass http://externalservices:8000;
         proxy_redirect off;
        if ($request_method = 'OPTIONS') {
@@ -116,4 +118,4 @@ server {
     location /home {
         try_files $uri /base.html;
     }
}

Change into the DockerFiles directory and build all container. If you want to change the database name or password edit environment_variables.env.

$ cd DockerFiles
$ sudo docker-compose pull
$ sudo docker-compose build

Start the mariadb container and create the vestec database (in a second shell).

$ sudo docker-compose up maria
$ sudo docker exec -it mariadb /bin/bash
$ mysql -u root -p -h 0.0.0.0
$ CREATE DATABASE testdbbp;
$ exit;
$ exit
$ sudo docker-compose down

Start all container

$ sudo docker-compose up -d

The nginx instance is listening on port 80. Visit the site: http://localhost:80 and create a user. The user needs to be activated and set to admin.

$ sudo docker exec -it msm /bin/bash
$ cd ../CommandLine
$ python manage_user.py -e USERNAME
$ ptyhon manage_user.py -admin USERNAME

Usage

Login through the website and create a workflow at Admin -> Workflows. Choose a workflow name that matches one of the implemented workflows: https://github.com/EPCCed/vestec-wp5/tree/master/WorkflowManager/workflows For the simple workflow the name should be simple. Take the queue names from the workflow python file. For the simple workflow the names are as following:

  • Initiation: initialise_simple
  • Shutdown: shutdown_simple
  • Data: add_data_simple

Granting access to a workflow

Access to a workflow needs to be explicitly given to users through the webinterface. Visit Admin -> Users and click click here to edit on the desired user. In the popup select the created workflow from the dropdown and select Add Workflow.

Creating an incident

After access to a workflow was granted, the user can now add incidents. Newly created incidents are in PENDING state, to activate an incident click the eye button on the home screen and press the Activate Incident button.

Adding Data to an incident

Only activated incidents can have data added to them.
To make data usable by the CosmoScout Incident Node visit the documentation.

Clone this wiki locally