Skip to content

Commit

Permalink
restructure the repo and dockerise the frontend
Browse files Browse the repository at this point in the history
  • Loading branch information
jamesamcl committed Oct 18, 2022
1 parent 3c55e59 commit cebb5ca
Show file tree
Hide file tree
Showing 158 changed files with 96 additions and 21 deletions.
51 changes: 43 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,44 +13,79 @@ Version 4 of the EMBL-EBI Ontology Lookup Service (OLS), featuring:
* React frontend using Redux and Tailwind
* Backwards compatibility with the OLS3 API

This repository contains both the dataloader (`dataload` directory) and the API/webapp server (`server` directory).
This repository contains three projects:

* The dataloader (`dataload` directory)
* The API server (`backend` directory)
* The React frontend (`frontend` directory)

# Deploying OLS4

Deployment instructions will go here. OLS4 is still under heavy development, so currently we only have instructions for developers below.
Deployment instructions will go here. OLS4 is still under heavy development, so currently we only have detailed instructions for developers below.

However, if you just want to try it out, this should get you going:

export OLS4_CONFIG=./dataload/configs/efo.json
docker-compose up

You should now be able to access the OLS4 frontend at `http://localhost:8080`.



# Developing OLS4

OLS is different to most webapps in that its API provides both full text search and recursive graph queries, neither of which are possible and/or performant using traditional RDBMS.
It therefore uses two specialized database servers: [**Solr**](https://solr.apache.org), a Lucene server similar to ElasticSearch; and [**Neo4j**](https://neo4j.com), a graph database.

The `dataload` directory contains the code which turns OWL ontologies into JSON and CSV datasets which can be loaded into Solr and Neo4j, respectively; and some minimal bash scripts which help with loading them.

The `server` directory contains (1) a Spring Boot application which hosts the OLS API over the above Solr and Neo4j instances; and (2) a React frontend built upon the OLS API.
* The `dataload` directory contains the code which turns OWL ontologies into JSON and CSV datasets which can be loaded into Solr and Neo4j, respectively; and some minimal bash scripts which help with loading them.
* The `backend` directory contains a Spring Boot application which hosts the OLS API over the above Solr and Neo4j instances
* The `frontend` directory contains the React frontend built upon the `backend` above.

![OLS4 overview](docs/overview.png)

## Running Solr and Neo4j using Docker
## Running OLS4 using Docker

You can run OLS4, or any combination of its consistuent parts (dataload, backend, frontend) in Docker. When developing, it is often useful to run, for example, just Solr and Neo4j in Docker, while running the API server locally; or to run Solr, Neo4j, and the backend API server in Docker while running the frontend locally.

First install the latest version of Docker Desktop if you are on Mac or Windows. This now includes the `docker compose` command. If you are on Linux, make sure you have the `docker compose` plugin installed (`apt install docker.io docker-compose-plugin` on Ubuntu).

You will need a config file, which configures the ontologies to load into OLS4. You can provide this to `docker compose` using the `OLS4_CONFIG` environment variable. For example:

export OLS4_CONFIG=./dataload/configs/efo.json

Then, build and run Solr and Neo4j:
Then, start up the components you would like to run. For example, Solr and Neo4j only (to develop the backend API server and/or frontend):

docker compose up --force-recreate --build --always-recreate-deps --attach-dependencies ols4-solr ols4-neo4j

This will build and run the dataload, and start up Solr and Neo4j with your new dataset on ports 8983 and 7474, respectively. Now you can run the API server Spring Boot application located in `server` and frontend for development. Set the following environment variables to point it at your local (Dockerized) Solr and Neo4j servers:
This will build and run the dataload, and start up Solr and Neo4j with your new dataset on ports 8983 and 7474, respectively. To start Solr and Neo4j **AND** the backend API server (to develop the frontend):

docker compose up --force-recreate --build --always-recreate-deps --attach-dependencies ols4-solr ols4-neo4j ols4-backend

To start everything, including the frontend:

docker compose up --force-recreate --build --always-recreate-deps --attach-dependencies ols4-solr ols4-neo4j ols4-backend ols4-frontend

## Running OLS4 locally

Alternatively, you can run OLS4 or any of its constituent parts locally, which is more useful for development.

### Running the dataload locally

TODO

### Running the API server backend locally

The API server Spring Boot application located in `backend`. Set the following environment variables to point it at your local (Dockerized) Solr and Neo4j servers:

OLS_SOLR_HOST=http://localhost:8983
OLS_NEO4J_HOST=bolt://localhost:7687


### Running the frontend locally

The frontend is a React application in `server/frontend`. TODO instructions!


## Updating `testcases_expected_output`

If you change something that results in the test output changing (e.g. adding new tests, changing what the output looks like), the CI on this repo will fail.
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
19 changes: 14 additions & 5 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ services:
depends_on:
ols4-dataload:
condition: service_completed_successfully
ols4-server:
build: ./server
ols4-backend:
build: ./backend
ports:
- 8080:8080
environment:
Expand All @@ -46,17 +46,26 @@ services:
links:
- ols4-solr
- ols4-neo4j
ols4-frontend:
build:
context: ./frontend
ports:
- 8081:8080
depends_on:
- ols4-backend
links:
- ols4-backend
run-api-tests:
profiles: ["run-api-tests"]
build: ./apitester4
volumes:
- ${OLS4_APITEST_OUTDIR:-./to_be_filled_by_script_outdir}:/mnt/out:rw
- ${OLS4_APITEST_COMPAREDIR:-./to_be_filled_by_script_comparedir}:/mnt/compare:rw
command: ["bash", "-c", "java -jar /opt/apitester/target/apitester-1.0-SNAPSHOT-jar-with-dependencies.jar --url http://ols4-server:8080 --outDir /mnt/out --compareDir /mnt/compare > /mnt/out/apitester4.log"]
command: ["bash", "-c", "java -jar /opt/apitester/target/apitester-1.0-SNAPSHOT-jar-with-dependencies.jar --url http://ols4-backend:8080 --outDir /mnt/out --compareDir /mnt/compare > /mnt/out/apitester4.log"]
depends_on:
- ols4-server
- ols4-backend
links:
- ols4-server
- ols4-backend
volumes:
ols4-solr-data:
ols4-neo4j-data:
Expand Down
4 changes: 4 additions & 0 deletions frontend/.env.docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
REACT_APP_APIURL="http://localhost:8080/"
PUBLIC_URL="/"


File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 2 additions & 0 deletions frontend/.yarnrc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
nodeLinker: node-modules

11 changes: 11 additions & 0 deletions frontend/Caddyfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
auto_https off
}

:8080 {
root * /opt/ols-www
try_files {path} /
file_server
}


20 changes: 20 additions & 0 deletions frontend/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@


FROM node:16 AS build


RUN mkdir /opt/ols-frontend
COPY . /opt/ols-frontend

WORKDIR /opt/ols-frontend

RUN yarn install

RUN npm run build:docker

FROM caddy:2-alpine

COPY --from=build /opt/ols-frontend/build /opt/ols-www
COPY Caddyfile /etc/caddy/Caddyfile


File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes
File renamed without changes
File renamed without changes.
File renamed without changes.
File renamed without changes.
1 change: 1 addition & 0 deletions server/frontend/package.json → frontend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
"build:css": "tailwindcss -i ./src/index.css -o ./public/main.css --minify",
"build:sandbox": "REACT_APP_ENV=sandbox npm run build",
"build:prod": "REACT_APP_ENV=prod npm run build",
"build:docker": "REACT_APP_ENV=docker npm run build",
"test": "react-scripts test",
"eject": "react-scripts eject"
},
Expand Down
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes.
File renamed without changes.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

<modules>
<module>dataload</module>
<module>server</module>
<module>backend</module>
</modules>

</project>
Expand Down
4 changes: 0 additions & 4 deletions server/frontend/.env

This file was deleted.

3 changes: 0 additions & 3 deletions server/frontend/.yarnrc.yml

This file was deleted.

0 comments on commit cebb5ca

Please sign in to comment.