Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bacpop-25 use gps v6 db #42

Merged
merged 47 commits into from
Feb 20, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
b803baf
use v6 db
EmmaLRussell Oct 10, 2023
1e74854
pull api image on first run
EmmaLRussell Jan 27, 2024
72d966f
squash lint warning in test file
EmmaLRussell Jan 29, 2024
a0d8b57
correct cache dependency path
EmmaLRussell Jan 29, 2024
44065ef
expect 404 on unexpected sample
EmmaLRussell Jan 29, 2024
8234376
poll more sensibly in test
EmmaLRussell Jan 29, 2024
2405525
debug
EmmaLRussell Jan 29, 2024
5f345a7
bump up timeout
EmmaLRussell Jan 29, 2024
095143d
missing ;
EmmaLRussell Jan 29, 2024
1d46810
add some extra logging to problem test
EmmaLRussell Jan 29, 2024
10e7225
logging fix
EmmaLRussell Jan 30, 2024
790eb67
throw if any error status responses
EmmaLRussell Jan 30, 2024
189cea9
more logging
EmmaLRussell Jan 30, 2024
de3e888
remove previous
EmmaLRussell Jan 30, 2024
97ffd74
comment
EmmaLRussell Jan 30, 2024
000c6da
updates for docker run
EmmaLRussell Feb 5, 2024
f34e7ce
rollback erroneous changes, and revert api branch pin
EmmaLRussell Feb 5, 2024
c6e2223
add github docker env vars
EmmaLRussell Feb 5, 2024
a7bd996
fix broken server container
EmmaLRussell Feb 5, 2024
ab366fd
added docker oauth app for github
EmmaLRussell Feb 5, 2024
0163d1e
add google ids
EmmaLRussell Feb 6, 2024
1883ba5
remove api container on exit
EmmaLRussell Feb 6, 2024
e9da2a9
allow for project hash not yet refistered when check status
EmmaLRussell Feb 6, 2024
e9bdc7c
merge with bacpop-25
EmmaLRussell Feb 6, 2024
d4187f5
npm ci in run scripts, and use db location when run deps
EmmaLRussell Feb 6, 2024
d37f9b3
null check
EmmaLRussell Feb 6, 2024
943648f
polling debug
EmmaLRussell Feb 6, 2024
bda569e
remove debug
EmmaLRussell Feb 6, 2024
2332bae
timeout
EmmaLRussell Feb 6, 2024
0ca15a3
bump up universal timeout but do not allow more than five consecutive…
EmmaLRussell Feb 6, 2024
9a070c0
show details of latest error response when exceed consecutive threshold
EmmaLRussell Feb 6, 2024
f344503
update for latest docker compose
EmmaLRussell Feb 7, 2024
12673ff
Merge pull request #46 from bacpop/bacpop-145
richfitz Feb 12, 2024
9ddfa88
Merge branch 'main' into bacpop-25
richfitz Feb 12, 2024
67a3389
fix smoke test
EmmaLRussell Feb 12, 2024
90c7319
re-add vars to docker-compose
EmmaLRussell Feb 12, 2024
ac27364
use more informative py branch
EmmaLRussell Feb 12, 2024
758e8a5
check assign job id on submit
EmmaLRussell Feb 12, 2024
ae44116
revert api
EmmaLRussell Feb 12, 2024
4bb8a94
use bacpop-151
EmmaLRussell Feb 16, 2024
1036aa3
use debug branch of py and include tmate
EmmaLRussell Feb 19, 2024
2060c5f
try putting tmate on the right workflow...
EmmaLRussell Feb 19, 2024
f5159bb
relax test for specific error string
EmmaLRussell Feb 19, 2024
8a74952
use unique project hash for each test#
EmmaLRussell Feb 19, 2024
12dc93e
use a single worker
EmmaLRussell Feb 19, 2024
248458b
remove allow status failures and target main py branch
EmmaLRussell Feb 19, 2024
c5a050c
remove tmate
EmmaLRussell Feb 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/jestCI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ jobs:
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
cache-dependency-path: ./app/client/package-lock.json
cache-dependency-path: ./app/server/package-lock.json
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This path just seemed to be wrong!

- run: npm ci
- name: get config file
run: |
Expand Down
4 changes: 2 additions & 2 deletions app/client/tests/unit/components/SavedProjects.spec.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
/* eslint-disable import/first */
import EditProjectName from "@/components/projects/EditProjectName.vue";
import EditProjectName from "@/components/projects/EditProjectName.vue"

const { toLocaleString } = Date.prototype;
// eslint-disable-next-line no-extend-native
// eslint-disable-next-line no-extend-native, func-names
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disabling this rule here to remove lint warning .

Date.prototype.toLocaleString = function (locale: any = undefined, ...args: any) {
const options = args[0];
return toLocaleString.call(this, "en-GB", { ...options, timeZone: "UTC" });
Expand Down
8 changes: 4 additions & 4 deletions app/server/tests/integration/errors.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import {flushRedis, get, post, saveRedisHash, saveRedisSet} from "./utils";
import {uid} from "uid";
import {setTimeout} from "timers/promises";
import {testSample} from "./testSample";

describe("Error handling", () => {
Expand Down Expand Up @@ -43,19 +44,18 @@ describe("Error handling", () => {
testSample.projectId = projectId;
const poppunkRes = await post(`poppunk`, testSample, connectionCookie);
expect(poppunkRes.status).toBe(200);
const counter = 0;
let counter = 0;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This counter hadn't been counting!

let finished = false;
while (!finished && counter < 100) {
await new Promise(resolve => {
setTimeout(() => {resolve(""), 1000})
});
await setTimeout(2000);
Copy link
Collaborator Author

@EmmaLRussell EmmaLRussell Jan 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous attempt at delays between polling hadn't been working, so we were getting some intermittent test failures, which I think were related to spamming the endpoint. Bumping up the delay as well to avoid intermittent errors on first poll .

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use fake timers here instead if possible?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately not, as we're waiting for a popunk job to finish and polling the real beebop_py api.

const statusRes = await post("status", {hash: testSample.projectHash}, connectionCookie);
expect(statusRes.status).toBe(200);
const statusValues = statusRes.data.data;
if (statusValues.assign === "finished" && statusValues.microreact === "finished" && statusValues.network === "finished") {
finished = true;
break;
}
counter = counter + 1;
}
expect(finished).toBe(true);
return projectId;
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ services:
environment:
- REDIS_HOST=beebop-redis
- STORAGE_LOCATION=./storage
- DB_LOCATION=./storage/GPS_v4_references
- DB_LOCATION=./storage/GPS_v6_references
beebop-redis:
image: redis:5.0
beebop-py-worker:
Expand Down
8 changes: 4 additions & 4 deletions scripts/run_dependencies
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ NETWORK=beebop_nw
VOLUME=beebop-storage
NAME_REDIS=beebop-redis
NAME_API=beebop-py-api
API_BRANCH=main
API_BRANCH=bacpop-25
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leaving comment so can be changed before merge

NAME_WORKER=beebop-py-worker
PORT=5000

Expand All @@ -14,13 +14,13 @@ docker network create $NETWORK > /dev/null || /bin/true
docker run -d --rm --name $NAME_REDIS --network=$NETWORK -p 6379:6379 redis:5.0
docker run -d --rm --name $NAME_WORKER --network=$NETWORK \
--env=REDIS_HOST="$NAME_REDIS" \
--pull always \
-v $VOLUME:/beebop/storage \
mrcide/beebop-py:$API_BRANCH rqworker
docker run -d --rm --name $NAME_API --network=$NETWORK \
--pull always \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this pull not be retained? also the --rm?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rm should! I'll re-add that. I've moved the pull to previous docker run, which is for the same image - mrcide/beebop-py:$API_BRANCH

docker run -d --name $NAME_API --network=$NETWORK \
--env=REDIS_HOST="$NAME_REDIS" \
--env=STORAGE_LOCATION="./storage" \
--env=DB_LOCATION="./storage/GPS_v4_references" \
--env=DB_LOCATION="./storage/GPS_v6_references" \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will we ever run with v4 again?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not. Are you thinking we could just have a fixed storage location?

-v $VOLUME:/beebop/storage \
-p $PORT:5000 \
mrcide/beebop-py:$API_BRANCH
Loading