Skip to content

Commit

Permalink
Merge pull request #418 from AllenInstitute/dev
Browse files Browse the repository at this point in the history
Dev
  • Loading branch information
rcpeene authored Jul 2, 2024
2 parents 88b5fcc + 4894bd2 commit 5a72d1b
Show file tree
Hide file tree
Showing 10 changed files with 43 additions and 26 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
jupyter-book clean ./docs
jupyter-book build ./docs
- name: Deploy book to Github pages
- name: Deploy book to GitHub pages
uses: peaceiris/[email protected]
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
Expand Down
6 changes: 3 additions & 3 deletions data/contributors.csv
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
Name,ORCID,Institution,Email,Github,Funding,Role
Name,ORCID,Institution,Email,GitHub,Funding,Role
Katrina Ager,,Arizona State University,[email protected],https://github.com/katrinaager,,"Processing"
Shailaja Akella,0000-0002-8333-2021,Allen Institute,[email protected],https://github.com/shailajaAkella,,"Review"
Ahad Bawany,,Allen Institute,[email protected],https://github.com/Ahad-Allen,,"Processing"
Corbett Bennett,0009-0001-2847-7754,Allen Institute,[email protected],https://github.com/corbennett,,"Processing"
Benjamin Dichter,0000-0001-5725-6910,CatalystNeuro,[email protected],https://github.com/bendichter,,"Conceptualization"
Yaroslav Halchenko,,Dartmouth College,,https://github.com/yarikoptic,,"Processing"
Yaroslav O. Halchenko,0000-0003-3456-2493,Dartmouth College,[email protected],https://github.com/yarikoptic,NIH R24MH117295,"Processing"
Daniel Hulsey,0000-0003-3243-6282,University of Oregon,[email protected],https://github.com/dhulsey,,"Processing"
Santiago Jaramillo,0000-0002-6595-8450,University of Oregon,[email protected],https://github.com/sjara,,"Processing"
Satrajit Ghosh,0000-0002-5312-6729,Massachusetts Institute of Technology,[email protected],https://github.com/satra,NIH R24MH117295,"Conceptualization"
Expand All @@ -19,7 +19,7 @@ Stephanie Prince,0000-0002-3083-6955,Lawrence Berkeley National Laboratory,smpri
Jason Pina,0000-0003-1385-8762,York University,[email protected],https://github.com/jayepi,,"Conceptualization"
Hyeyoung Shin,,Seoul National University,[email protected],https://github.com/hs13,,"Conceptualization"
Josh Siegle,,Allen Institute,[email protected],https://github.com/jsiegle,NIH U24,"Processing"
Jiatai Song,,Tsingua University,,https://github.com/Laohusong,,"Processing"
Jiatai Song,0009-0004-4910-1141,Tsingua University,,https://github.com/Laohusong,,"Processing"
Shih-Yi Tseng,0000-0002-5029-433X,"University of California, San Francisco",[email protected],https://github.com/sytseng,,"Processing"
Jacob Westerberg,0000-0001-5331-8707,Netherlands Institute for Neuroscience,[email protected],https://github.com/jakewesterberg,,"Conceptualization"
Alex Williams,0000-0001-5853-103X,New York University,[email protected],https://github.com/ahwillia,,"Review"
13 changes: 11 additions & 2 deletions databook_utils/insert_authors_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,16 @@ def run(self):

line_block = nodes.line_block()
for property in properties[1:]:
if property != "":
if not property:
continue
elif property.startswith("https://") or property.startswith("http://"):
link_node = nodes.reference(text=property, refuri=property)
line_block.append(link_node)
elif "@" in property:
mailto_link = f"mailto:{property}"
email_node = nodes.reference(text=property, refuri=mailto_link)
line_block.append(email_node)
else:
line_block.append(nodes.line(text=property))
line_block.append(nodes.line(text=""))

Expand All @@ -153,4 +162,4 @@ def setup(app):
'version': '0.1',
'parallel_read_safe': True,
'parallel_write_safe': True,
}
}
12 changes: 10 additions & 2 deletions docs/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The Openscope Databook is a great place to demonstrate the capabilities and some
As described in [This Jupyter blog post](https://blog.jupyter.org/mybinder-org-reducing-capacity-c93ccfc6413f), Binder no longer has the support of Google, and therefore shows reduced performance. Launches may fail or take a long time. There is no working solution to this except trying to launch again. An alternative would be to launch Databook notebooks with Google Colab.

**How can I store my work on the Databook and come back to it later?**\
Launching the Databook with [Dandihub](https://hub.dandiarchive.org/) will allow your files to be stored persistently and contain all the Databook's notebooks together. Additionally, you can clone the [Github repo](https://github.com/AllenInstitute/openscope_databook) and run our files locally. These are both explained in further detail on the [front page](https://alleninstitute.github.io/openscope_databook/intro.html).
Launching the Databook with [Dandihub](https://hub.dandiarchive.org/) will allow your files to be stored persistently and contain all the Databook's notebooks together. Additionally, you can clone the [GitHub repo](https://github.com/AllenInstitute/openscope_databook) and run our files locally. These are both explained in further detail on the [front page](https://alleninstitute.github.io/openscope_databook/intro.html).

**How do you recommend using the Databook?**\
The Databook can be used to reproduce analysis on files, as a starting point for investigating a dataset, or as an educational resource to get more familiar with NWB files or particular kinds of data. In all of these cases, the code can be modified, copied, and interactively run to gain a better understanding of the data. For educational use, the databook may be run remotely with Thebe, Binder, or Google Colab as simple demonstrations. For more advanced usage and analysis, it may behoove you to download an individual notebook and run it locally.
Expand All @@ -39,7 +39,15 @@ If local installation is failing, it is recommended that you attempt to clone an
### Contribution

**How can I contribute to this project?**\
Contributing to this project can be as simple as forking the [Github repo](https://github.com/AllenInstitute/openscope_databook), making your changes, and issuing a PR on Github. However, it would be advised to reach out to [Jerome Lecoq](https://github.com/jeromelecoq) or [Carter Peene](https://github.com/rcpeene) and discuss if you wish to make a significant contribution.
Contributing to this project can be as simple as forking the [GitHub repo](https://github.com/AllenInstitute/openscope_databook), making your changes, and issuing a PR on GitHub. However, it would be advised to reach out to [Jerome Lecoq](https://github.com/jeromelecoq) or [Carter Peene](https://github.com/rcpeene) and discuss if you wish to make a significant contribution.

### Citing

**How can I cite the Databook?**\
The Databook has a DOI through Zenodo, and can be cited accordingly 10.5281/zenodo.12614663.


### Additional Questions

**I have a question that isn't addressed here**\
Questions, bugs, or any other topics of interest can be discussed by filing an issue on the Github repo's [issues page](https://github.com/AllenInstitute/openscope_databook/issues).
6 changes: 3 additions & 3 deletions docs/contribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
The OpenScope Databook is a community project. It aims to serve as a collection of contributed analyses from many different experts and to include a comprehensive set of useful and reproducible analyses. Contributors are added to the authors list and have an opportunity to share their figures, results, and datasets through their notebooks in the Databook. Together the project can grow and continue to shape the growing reproducible neuroscience ecosystem.

## Fork the OpenScope Databook
The Databook can be forked via the Github Web UI from the Databook's [Github repo](https://github.com/AllenInstitute/openscope_databook). Press the [fork button](https://github.com/AllenInstitute/openscope_databook/fork) or click this link.
The Databook can be forked via the GitHub Web UI from the Databook's [GitHub repo](https://github.com/AllenInstitute/openscope_databook). Press the [fork button](https://github.com/AllenInstitute/openscope_databook/fork) or click this link.

## Initialize Locally
A local repo can be made by pressing the `code` button on the front page of the forked repo, and copying the HTTPS url. Then locally, run the command `git clone <copied_url_here>`. For more information on cloning Github repos, check out Github's [Cloning a Repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) Page.
A local repo can be made by pressing the `code` button on the front page of the forked repo, and copying the HTTPS url. Then locally, run the command `git clone <copied_url_here>`. For more information on cloning GitHub repos, check out GitHub's [Cloning a Repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) Page.

Then the environment must be set up. You may set up a conda environment if you don't want to interfere with your local environment. After installing conda, this can be done with the commands `conda create --name databook python=3.9` followed by `activate databook` (Windows) or `source activate databook` (Mac/Linux). Within or without the conda environment, the dependencies for the databook can be installed by navigating to the openscope_databook directory and running `pip install -e . --user`.

Expand Down Expand Up @@ -37,7 +37,7 @@ Once the pull request is made, it will be reviewed for merging into the Databook


## Reviewing the Databook
Reviews of the Databook are also made on Github via Git commits. We recommend you make a local fork of this repository and create a PR with your edits that either correct or extend a given notebook as discussed in the various sections above. If you feel like a notebook should be discussed, please first make a PR request and enter your reviewing comment on the PR form on Github with ReviewNB. You are free to choose any page of the databook for reviews, depending on your expertise and time. According to the authorship section below, reviewers gain authorship if they propose modifications that are eventually merged into the main branch.
Reviews of the Databook are also made on GitHub via Git commits. We recommend you make a local fork of this repository and create a PR with your edits that either correct or extend a given notebook as discussed in the various sections above. If you feel like a notebook should be discussed, please first make a PR request and enter your reviewing comment on the PR form on GitHub with ReviewNB. You are free to choose any page of the databook for reviews, depending on your expertise and time. According to the authorship section below, reviewers gain authorship if they propose modifications that are eventually merged into the main branch.


## Authorship
Expand Down
10 changes: 5 additions & 5 deletions docs/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ aliases: Carter Peene--R. Carter Peene, colleenjg--Colleen J. Gillon, shailajaAk
````
````{tab-item} Commits
**Committed code to the primary Github repository**
**Committed code to the primary GitHub repository**
```{committers}
---
blacklist: Publishing Bot, github-actions[bot], GitHub Authors Action, Ross Carter Peene, rcpeene
Expand Down Expand Up @@ -67,7 +67,7 @@ aliases: Carter Peene--R. Carter Peene, colleenjg--Colleen J. Gillon, shailajaAk

Reproducibility is a significant challenge in neuroscience, as analysis and visualization methods are often difficult to replicate due to a lack of accessible code, separation of code from published figures, or unavailability of code altogether. This issue may arise from the complex nature of neuroscience research, the use of diverse data formats and analysis techniques, and insufficient emphasis on open-source, collaborative practices. In addition, key neuroscience analyses are typically rewritten at the start of new scientific projects, slowing down the initiation of research efforts.

Four key components are essential for reproducible analysis: accessible data, accessible computational resources, a reproducible environment, and usage documentation. The OpenScope Databook, provided by the Allen Institute's OpenScope Project, offers a solution to these challenges by facilitating the analysis and visualization of brain data, primarily using [NWB files](https://www.nwb.org/) and the [DANDI archive](https://dandiarchive.org/). Hosted on Github, the entire publication – including code, data access, text, references, and revisions from reviewers and contributors – is readily available for collaboration and version control, promoting transparency and collective knowledge growth. The OpenScope Databook addresses these components by leveraging a combination of open-source Python libraries, such as DANDI, [Binder](https://mybinder.org/), [Jupyter Book](https://jupyterbook.org/en/stable/intro.html), [Google Colab](https://colab.research.google.com/), LaTeX references, Python scripts, Git versioning, and scientific revision through approved pull requests. The entire publication can be recreated by running the code locally, on distributed servers such as Binder, [DandiHub](https://hub.dandiarchive.org/), or Google Colab, or on any host running Jupyter notebooks.
Four key components are essential for reproducible analysis: accessible data, accessible computational resources, a reproducible environment, and usage documentation. The OpenScope Databook, provided by the Allen Institute's OpenScope Project, offers a solution to these challenges by facilitating the analysis and visualization of brain data, primarily using [NWB files](https://www.nwb.org/) and the [DANDI archive](https://dandiarchive.org/). Hosted on GitHub, the entire publication – including code, data access, text, references, and revisions from reviewers and contributors – is readily available for collaboration and version control, promoting transparency and collective knowledge growth. The OpenScope Databook addresses these components by leveraging a combination of open-source Python libraries, such as DANDI, [Binder](https://mybinder.org/), [Jupyter Book](https://jupyterbook.org/en/stable/intro.html), [Google Colab](https://colab.research.google.com/), LaTeX references, Python scripts, Git versioning, and scientific revision through approved pull requests. The entire publication can be recreated by running the code locally, on distributed servers such as Binder, [DandiHub](https://hub.dandiarchive.org/), or Google Colab, or on any host running Jupyter notebooks.

We cover several broadly used analyses across the community, providing a missing component for system neuroscience. Our key analyses are organized into chapters, including NWB basics such as downloading, streaming, and visualizing NWB files from data archives. We document essential analyses typically performed in all neuroscience laboratories, such as temporal alignment, alignment to sensory stimuli, and association with experimental metadata. We cover the two leading neuronal recording techniques: two-photon calcium imaging and electrophysiological recordings, and share example analyses of stimulus-averaged responses. Advanced first-order analyses include showing receptive fields, identifying optotagged units, current source density analysis, and cell matching across days.

Expand All @@ -87,13 +87,13 @@ Binder will automatically setup the environment with [repo2docker](https://githu
[Thebe](https://github.com/executablebooks/thebe) uses Binder in the backend to prepare the environment and run the kernel. It allows users to run notebooks embedded directly within the Databook's web UI. It can be used by hovering over the `Launch` button in the top-right of a notebook and selecting `Live Code`. Thebe is a work-in-progress project and has room for improvement. It is also worth noting that, like Binder, starting the Jupyter Kernel can sometimes take many minutes.

### Dandihub
[Dandihub](https://hub.dandiarchive.org/) is an instance of JupyterHub hosted by DANDI. Dandihub does not automatically reproduce the environment required for these notebooks, but importantly, Dandihub allows for persistent storage of your files, so you can leave your work and come back to it later. It can be used by hovering over the `Launch` button in the top-right of a notebook and selecting `JupyterHub`. In order to run notebooks on Dandihub, you must sign in with your Github account. To set up the correct environment on Dandihub, open a `terminal` tab, navigate to the directory `openscope_databook` and run the command
[Dandihub](https://hub.dandiarchive.org/) is an instance of JupyterHub hosted by DANDI. Dandihub does not automatically reproduce the environment required for these notebooks, but importantly, Dandihub allows for persistent storage of your files, so you can leave your work and come back to it later. It can be used by hovering over the `Launch` button in the top-right of a notebook and selecting `JupyterHub`. In order to run notebooks on Dandihub, you must sign in with your GitHub account. To set up the correct environment on Dandihub, open a `terminal` tab, navigate to the directory `openscope_databook` and run the command
```
pip install -e .
```

### Locally
You can download an individual notebook by pressing the `Download` button in the top-right and selecting `.ipynb`. Alternatively, you can clone the repo to your machine and access the files there. The repo can be found by hovering over the the `Github` button in the top-right and selecting `repository`. When run locally, the environment can be replicated with our [requirements.txt](https://github.com/AllenInstitute/openscope_databook/blob/main/requirements.txt) file using the command
You can download an individual notebook by pressing the `Download` button in the top-right and selecting `.ipynb`. Alternatively, you can clone the repo to your machine and access the files there. The repo can be found by hovering over the the `GitHub` button in the top-right and selecting `repository`. When run locally, the environment can be replicated with our [requirements.txt](https://github.com/AllenInstitute/openscope_databook/blob/main/requirements.txt) file using the command
```
pip install -e .
```
Expand Down Expand Up @@ -126,7 +126,7 @@ The Databook leverages a number of technologies to combine those components into
Data is accessed from The [DANDI archive](https://dandiarchive.org/) and downloaded via the [DANDI Python API](https://dandi.readthedocs.io/en/latest/modref/index.html) within notebooks. Most notebooks make use of publicly available datasets on DANDI, but for some notebooks, there is not yet sufficient publicly-available data to demonstrate our analysis. For these, it is encouraged to use your own NWB Files that are privately stored on DANDI.

### Computation
This project utilizes [Binder](https://mybinder.org/), as the host for the environment and the provider of computational resources. Conveniently, Binder has support for effectively replicating a computational environment from a Github Repo. Users of the Databook don't have to worry about managing the environment if they prefer to use our integrated Binder functionality. However, the Databook can be run locally or on other hosts. Details about the different ways to run this code can be found in the section [How Can I Use It?](Usage) below.
This project utilizes [Binder](https://mybinder.org/), as the host for the environment and the provider of computational resources. Conveniently, Binder has support for effectively replicating a computational environment from a GitHub Repo. Users of the Databook don't have to worry about managing the environment if they prefer to use our integrated Binder functionality. However, the Databook can be run locally or on other hosts. Details about the different ways to run this code can be found in the section [How Can I Use It?](Usage) below.

### Environment
As mentioned above, Binder is capable of reproducing the environment in which to run this code. There are also other options for creating the necessary environment. Instructions for running this code locally can be found in the section [How Can I Use It?](Usage) below.
Expand Down
Loading

0 comments on commit 5a72d1b

Please sign in to comment.