Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy JupyterLab for better user experience #31

Open
wants to merge 1 commit into
base: docket-active-dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 70 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,58 +8,67 @@ Run these commands from the root DOCKET directory that contains the docker-compo

To build the DOCKET docker image:

```bash
docker-compose build
```

This docker container is built off the jupyter/scipy-notebook
base image and installs java 8, nextflow and specific perl
and python libraries used by DOCKET
This docker container is built off the jupyter/scipy-notebook base image and installs java 8, nextflow and specific perl and python libraries used by DOCKET

https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-scipy-notebook

jupyter/scipy-notebook comes installed with gcc
pandas, matplotlib, scipy, seaborn, scikit-learn, cython, Click
uses apt-get to install system packages
uses conda to install python libraries
jupyter/scipy-notebook comes installed with gcc, pandas, matplotlib, scipy, seaborn, scikit-learn, cython, Click, uses apt-get to install system packages, uses conda to install python libraries

## Run

To run the DOCKET docker container:

```bash
docker-compose up
```

## Connect

To connect to the DOCKET docker container:

docker-compose exec docket bash
```bash
docker-compose exec docket bash
```


## Nextflow

Connect to the container and run:

nextflow run docket_study --infile test/dataset1.txt --docket test/ds1_test
```bash
nextflow run docket_study --infile test/dataset1.txt --docket test/ds1_test
```

## Python sample

Connect to the container and run:

python scripts/hello_docket.py --files test/hello_docket.txt
```bash
python scripts/hello_docket.py --files test/hello_docket.txt
```

## Tests

Connect to the container and run:

pip install -r common/tests/requirements.txt
python -m pytest common/tests
```bash
pip install -r common/tests/requirements.txt
python -m pytest common/tests
```

## Jupyter notebook

Jupyter notebook runs on port 8888.
To see the token, either start the container without the detached flag
or check the logs using:

docker-compose logs -f docket
```bash
docker-compose logs -f docket
```

## Data

Expand All @@ -84,19 +93,51 @@ If this is the configuration you wanted, you would add this line to the docker-c

/Data/datasets:/datasets

An example docker-compose.yml

version: "2.1"
services:
docket:
build: "."
image: "docket-dev"
environment:
DEBUG: 1
PYTHONUNBUFFERED: 1
PYTHONPATH: '/app'
ports:
- 8888:8888
volumes:
- ./:/app
- /Data/datasets:/datasets
An example `docker-compose.yml`

```yaml
version: "2.1"
services:
docket:
build: "."
image: "docket-dev"
environment:
DEBUG: 1
PYTHONUNBUFFERED: 1
PYTHONPATH: '/app'
JUPYTER_ENABLE_LAB: 'yes'
ports:
- 8888:8888
volumes:
- ./:/app
- /Data/datasets:/datasets
```
## Password

Generate a hash to safely define your password by executing the following 2 lines in a Jupyter Notebook:

```python
from notebook.auth import passwd
passwd()
```

Use this hash to define the password in the command run when the container is started:

```yaml
version: "2.1"
services:
docket:
build: "."
image: "docket-dev"
command: start-notebook.sh --NotebookApp.password='sha1:9316432938f9:93985dffbb854d31308dfe0602a51db947fb7d80'
environment:
DEBUG: 1
PYTHONUNBUFFERED: 1
PYTHONPATH: '/app'
JUPYTER_ENABLE_LAB: 'yes'
ports:
- 8888:8888
volumes:
- ./:/app
```

3 changes: 3 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,13 @@ services:
docket:
build: "."
image: "docket-dev"
# sha1 hash used to define password ("test" here)
command: start-notebook.sh --NotebookApp.password='sha1:9316432938f9:93985dffbb854d31308dfe0602a51db947fb7d80'
environment:
DEBUG: 1
PYTHONUNBUFFERED: 1
PYTHONPATH: '/app'
JUPYTER_ENABLE_LAB: 'yes'
ports:
- 8888:8888
volumes:
Expand Down
3 changes: 2 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
annoy==1.16.3
cyjupyter==0.2.0
plotly==4.5.2
git+git://github.com/gglusman/data-fingerprints#egg=datafingerprint
git+git://github.com/gglusman/data-fingerprints#egg=datafingerprint
jupyterlab