From 3a505acb62412689acafcda6a6b6e0a1463c2eb1 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 1 Jul 2024 22:40:46 -0400 Subject: [PATCH 1/5] [DATALAD RUNCMD] Use correct capitalization of GitHub name === Do not change lines below === { "chain": [], "cmd": "git-sedi Github GitHub", "exit": 0, "extra_inputs": [], "inputs": [], "outputs": [], "pwd": "." } ^^^ Do not change lines above ^^^ --- .github/workflows/deploy.yml | 2 +- data/contributors.csv | 2 +- docs/FAQ.md | 6 +++--- docs/contribution.md | 6 +++--- docs/intro.md | 10 +++++----- docs/methods/environments.md | 2 +- docs/methods/github.md | 14 +++++++------- docs/methods/jupyter_book.md | 2 +- docs/visualization/visualize_2p_VR_behavior.ipynb | 2 +- 9 files changed, 23 insertions(+), 23 deletions(-) diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml index a767a6da..cbf39d3d 100644 --- a/.github/workflows/deploy.yml +++ b/.github/workflows/deploy.yml @@ -36,7 +36,7 @@ jobs: jupyter-book clean ./docs jupyter-book build ./docs - - name: Deploy book to Github pages + - name: Deploy book to GitHub pages uses: peaceiris/actions-gh-pages@v3.6.1 with: github_token: ${{ secrets.GITHUB_TOKEN }} diff --git a/data/contributors.csv b/data/contributors.csv index 127f6c01..c24bba76 100644 --- a/data/contributors.csv +++ b/data/contributors.csv @@ -1,4 +1,4 @@ -Name,ORCID,Institution,Email,Github,Funding,Role +Name,ORCID,Institution,Email,GitHub,Funding,Role Katrina Ager,,Arizona State University,katrinaager12@gmail.com,https://github.com/katrinaager,,"Processing" Shailaja Akella,0000-0002-8333-2021,Allen Institute,shailaja.akella@alleninstitute.org,https://github.com/shailajaAkella,,"Review" Ahad Bawany,,Allen Institute,ahad.bawany@alleninstitute.org,https://github.com/Ahad-Allen,,"Processing" diff --git a/docs/FAQ.md b/docs/FAQ.md index bcb456d3..765caf18 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -21,7 +21,7 @@ The Openscope Databook is a great place to demonstrate the capabilities and some As described in [This Jupyter blog post](https://blog.jupyter.org/mybinder-org-reducing-capacity-c93ccfc6413f), Binder no longer has the support of Google, and therefore shows reduced performance. Launches may fail or take a long time. There is no working solution to this except trying to launch again. An alternative would be to launch Databook notebooks with Google Colab. **How can I store my work on the Databook and come back to it later?**\ -Launching the Databook with [Dandihub](https://hub.dandiarchive.org/) will allow your files to be stored persistently and contain all the Databook's notebooks together. Additionally, you can clone the [Github repo](https://github.com/AllenInstitute/openscope_databook) and run our files locally. These are both explained in further detail on the [front page](https://alleninstitute.github.io/openscope_databook/intro.html). +Launching the Databook with [Dandihub](https://hub.dandiarchive.org/) will allow your files to be stored persistently and contain all the Databook's notebooks together. Additionally, you can clone the [GitHub repo](https://github.com/AllenInstitute/openscope_databook) and run our files locally. These are both explained in further detail on the [front page](https://alleninstitute.github.io/openscope_databook/intro.html). **How do you recommend using the Databook?**\ The Databook can be used to reproduce analysis on files, as a starting point for investigating a dataset, or as an educational resource to get more familiar with NWB files or particular kinds of data. In all of these cases, the code can be modified, copied, and interactively run to gain a better understanding of the data. For educational use, the databook may be run remotely with Thebe, Binder, or Google Colab as simple demonstrations. For more advanced usage and analysis, it may behoove you to download an individual notebook and run it locally. @@ -39,7 +39,7 @@ If local installation is failing, it is recommended that you attempt to clone an ### Contribution **How can I contribute to this project?**\ -Contributing to this project can be as simple as forking the [Github repo](https://github.com/AllenInstitute/openscope_databook), making your changes, and issuing a PR on Github. However, it would be advised to reach out to [Jerome Lecoq](https://github.com/jeromelecoq) or [Carter Peene](https://github.com/rcpeene) and discuss if you wish to make a significant contribution. +Contributing to this project can be as simple as forking the [GitHub repo](https://github.com/AllenInstitute/openscope_databook), making your changes, and issuing a PR on GitHub. However, it would be advised to reach out to [Jerome Lecoq](https://github.com/jeromelecoq) or [Carter Peene](https://github.com/rcpeene) and discuss if you wish to make a significant contribution. **I have a question that isn't addressed here**\ -Questions, bugs, or any other topics of interest can be discussed by filing an issue on the Github repo's [issues page](https://github.com/AllenInstitute/openscope_databook/issues). +Questions, bugs, or any other topics of interest can be discussed by filing an issue on the GitHub repo's [issues page](https://github.com/AllenInstitute/openscope_databook/issues). diff --git a/docs/contribution.md b/docs/contribution.md index 43e90f06..4bbb2903 100644 --- a/docs/contribution.md +++ b/docs/contribution.md @@ -3,10 +3,10 @@ The OpenScope Databook is a community project. It aims to serve as a collection of contributed analyses from many different experts and to include a comprehensive set of useful and reproducible analyses. Contributors are added to the authors list and have an opportunity to share their figures, results, and datasets through their notebooks in the Databook. Together the project can grow and continue to shape the growing reproducible neuroscience ecosystem. ## Fork the OpenScope Databook -The Databook can be forked via the Github Web UI from the Databook's [Github repo](https://github.com/AllenInstitute/openscope_databook). Press the [fork button](https://github.com/AllenInstitute/openscope_databook/fork) or click this link. +The Databook can be forked via the GitHub Web UI from the Databook's [GitHub repo](https://github.com/AllenInstitute/openscope_databook). Press the [fork button](https://github.com/AllenInstitute/openscope_databook/fork) or click this link. ## Initialize Locally -A local repo can be made by pressing the `code` button on the front page of the forked repo, and copying the HTTPS url. Then locally, run the command `git clone `. For more information on cloning Github repos, check out Github's [Cloning a Repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) Page. +A local repo can be made by pressing the `code` button on the front page of the forked repo, and copying the HTTPS url. Then locally, run the command `git clone `. For more information on cloning GitHub repos, check out GitHub's [Cloning a Repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) Page. Then the environment must be set up. You may set up a conda environment if you don't want to interfere with your local environment. After installing conda, this can be done with the commands `conda create --name databook python=3.9` followed by `activate databook` (Windows) or `source activate databook` (Mac/Linux). Within or without the conda environment, the dependencies for the databook can be installed by navigating to the openscope_databook directory and running `pip install -e . --user`. @@ -37,7 +37,7 @@ Once the pull request is made, it will be reviewed for merging into the Databook ## Reviewing the Databook -Reviews of the Databook are also made on Github via Git commits. We recommend you make a local fork of this repository and create a PR with your edits that either correct or extend a given notebook as discussed in the various sections above. If you feel like a notebook should be discussed, please first make a PR request and enter your reviewing comment on the PR form on Github with ReviewNB. You are free to choose any page of the databook for reviews, depending on your expertise and time. According to the authorship section below, reviewers gain authorship if they propose modifications that are eventually merged into the main branch. +Reviews of the Databook are also made on GitHub via Git commits. We recommend you make a local fork of this repository and create a PR with your edits that either correct or extend a given notebook as discussed in the various sections above. If you feel like a notebook should be discussed, please first make a PR request and enter your reviewing comment on the PR form on GitHub with ReviewNB. You are free to choose any page of the databook for reviews, depending on your expertise and time. According to the authorship section below, reviewers gain authorship if they propose modifications that are eventually merged into the main branch. ## Authorship diff --git a/docs/intro.md b/docs/intro.md index 81e0a075..faafc6cc 100644 --- a/docs/intro.md +++ b/docs/intro.md @@ -37,7 +37,7 @@ aliases: Carter Peene--R. Carter Peene, colleenjg--Colleen J. Gillon, shailajaAk ```` ````{tab-item} Commits -**Committed code to the primary Github repository** +**Committed code to the primary GitHub repository** ```{committers} --- blacklist: Publishing Bot, github-actions[bot], GitHub Authors Action, Ross Carter Peene, rcpeene @@ -67,7 +67,7 @@ aliases: Carter Peene--R. Carter Peene, colleenjg--Colleen J. Gillon, shailajaAk Reproducibility is a significant challenge in neuroscience, as analysis and visualization methods are often difficult to replicate due to a lack of accessible code, separation of code from published figures, or unavailability of code altogether. This issue may arise from the complex nature of neuroscience research, the use of diverse data formats and analysis techniques, and insufficient emphasis on open-source, collaborative practices. In addition, key neuroscience analyses are typically rewritten at the start of new scientific projects, slowing down the initiation of research efforts. -Four key components are essential for reproducible analysis: accessible data, accessible computational resources, a reproducible environment, and usage documentation. The OpenScope Databook, provided by the Allen Institute's OpenScope Project, offers a solution to these challenges by facilitating the analysis and visualization of brain data, primarily using [NWB files](https://www.nwb.org/) and the [DANDI archive](https://dandiarchive.org/). Hosted on Github, the entire publication – including code, data access, text, references, and revisions from reviewers and contributors – is readily available for collaboration and version control, promoting transparency and collective knowledge growth. The OpenScope Databook addresses these components by leveraging a combination of open-source Python libraries, such as DANDI, [Binder](https://mybinder.org/), [Jupyter Book](https://jupyterbook.org/en/stable/intro.html), [Google Colab](https://colab.research.google.com/), LaTeX references, Python scripts, Git versioning, and scientific revision through approved pull requests. The entire publication can be recreated by running the code locally, on distributed servers such as Binder, [DandiHub](https://hub.dandiarchive.org/), or Google Colab, or on any host running Jupyter notebooks. +Four key components are essential for reproducible analysis: accessible data, accessible computational resources, a reproducible environment, and usage documentation. The OpenScope Databook, provided by the Allen Institute's OpenScope Project, offers a solution to these challenges by facilitating the analysis and visualization of brain data, primarily using [NWB files](https://www.nwb.org/) and the [DANDI archive](https://dandiarchive.org/). Hosted on GitHub, the entire publication – including code, data access, text, references, and revisions from reviewers and contributors – is readily available for collaboration and version control, promoting transparency and collective knowledge growth. The OpenScope Databook addresses these components by leveraging a combination of open-source Python libraries, such as DANDI, [Binder](https://mybinder.org/), [Jupyter Book](https://jupyterbook.org/en/stable/intro.html), [Google Colab](https://colab.research.google.com/), LaTeX references, Python scripts, Git versioning, and scientific revision through approved pull requests. The entire publication can be recreated by running the code locally, on distributed servers such as Binder, [DandiHub](https://hub.dandiarchive.org/), or Google Colab, or on any host running Jupyter notebooks. We cover several broadly used analyses across the community, providing a missing component for system neuroscience. Our key analyses are organized into chapters, including NWB basics such as downloading, streaming, and visualizing NWB files from data archives. We document essential analyses typically performed in all neuroscience laboratories, such as temporal alignment, alignment to sensory stimuli, and association with experimental metadata. We cover the two leading neuronal recording techniques: two-photon calcium imaging and electrophysiological recordings, and share example analyses of stimulus-averaged responses. Advanced first-order analyses include showing receptive fields, identifying optotagged units, current source density analysis, and cell matching across days. @@ -87,13 +87,13 @@ Binder will automatically setup the environment with [repo2docker](https://githu [Thebe](https://github.com/executablebooks/thebe) uses Binder in the backend to prepare the environment and run the kernel. It allows users to run notebooks embedded directly within the Databook's web UI. It can be used by hovering over the `Launch` button in the top-right of a notebook and selecting `Live Code`. Thebe is a work-in-progress project and has room for improvement. It is also worth noting that, like Binder, starting the Jupyter Kernel can sometimes take many minutes. ### Dandihub -[Dandihub](https://hub.dandiarchive.org/) is an instance of JupyterHub hosted by DANDI. Dandihub does not automatically reproduce the environment required for these notebooks, but importantly, Dandihub allows for persistent storage of your files, so you can leave your work and come back to it later. It can be used by hovering over the `Launch` button in the top-right of a notebook and selecting `JupyterHub`. In order to run notebooks on Dandihub, you must sign in with your Github account. To set up the correct environment on Dandihub, open a `terminal` tab, navigate to the directory `openscope_databook` and run the command +[Dandihub](https://hub.dandiarchive.org/) is an instance of JupyterHub hosted by DANDI. Dandihub does not automatically reproduce the environment required for these notebooks, but importantly, Dandihub allows for persistent storage of your files, so you can leave your work and come back to it later. It can be used by hovering over the `Launch` button in the top-right of a notebook and selecting `JupyterHub`. In order to run notebooks on Dandihub, you must sign in with your GitHub account. To set up the correct environment on Dandihub, open a `terminal` tab, navigate to the directory `openscope_databook` and run the command ``` pip install -e . ``` ### Locally -You can download an individual notebook by pressing the `Download` button in the top-right and selecting `.ipynb`. Alternatively, you can clone the repo to your machine and access the files there. The repo can be found by hovering over the the `Github` button in the top-right and selecting `repository`. When run locally, the environment can be replicated with our [requirements.txt](https://github.com/AllenInstitute/openscope_databook/blob/main/requirements.txt) file using the command +You can download an individual notebook by pressing the `Download` button in the top-right and selecting `.ipynb`. Alternatively, you can clone the repo to your machine and access the files there. The repo can be found by hovering over the the `GitHub` button in the top-right and selecting `repository`. When run locally, the environment can be replicated with our [requirements.txt](https://github.com/AllenInstitute/openscope_databook/blob/main/requirements.txt) file using the command ``` pip install -e . ``` @@ -126,7 +126,7 @@ The Databook leverages a number of technologies to combine those components into Data is accessed from The [DANDI archive](https://dandiarchive.org/) and downloaded via the [DANDI Python API](https://dandi.readthedocs.io/en/latest/modref/index.html) within notebooks. Most notebooks make use of publicly available datasets on DANDI, but for some notebooks, there is not yet sufficient publicly-available data to demonstrate our analysis. For these, it is encouraged to use your own NWB Files that are privately stored on DANDI. ### Computation -This project utilizes [Binder](https://mybinder.org/), as the host for the environment and the provider of computational resources. Conveniently, Binder has support for effectively replicating a computational environment from a Github Repo. Users of the Databook don't have to worry about managing the environment if they prefer to use our integrated Binder functionality. However, the Databook can be run locally or on other hosts. Details about the different ways to run this code can be found in the section [How Can I Use It?](Usage) below. +This project utilizes [Binder](https://mybinder.org/), as the host for the environment and the provider of computational resources. Conveniently, Binder has support for effectively replicating a computational environment from a GitHub Repo. Users of the Databook don't have to worry about managing the environment if they prefer to use our integrated Binder functionality. However, the Databook can be run locally or on other hosts. Details about the different ways to run this code can be found in the section [How Can I Use It?](Usage) below. ### Environment As mentioned above, Binder is capable of reproducing the environment in which to run this code. There are also other options for creating the necessary environment. Instructions for running this code locally can be found in the section [How Can I Use It?](Usage) below. diff --git a/docs/methods/environments.md b/docs/methods/environments.md index 430c4aea..9cc5a2ac 100644 --- a/docs/methods/environments.md +++ b/docs/methods/environments.md @@ -2,6 +2,6 @@ As briefly mentioned in [Jupyter/Jupyter Book](./jupyter_book.md), Jupyter Book allows users to launch notebooks in the Databook onto other computing platforms. Namely, [Dandihub](hub.dandiarchive.org), [Binder](mybinder.org), and [Google Colab](colab.research.google.com). In the case of Binder, clicking the launch button automatically produces a docker container with the necessary environment installed by using the files [apt.txt](https://github.com/AllenInstitute/openscope_databook/blob/main/apt.txt), [setup.py](https://github.com/AllenInstitute/openscope_databook/blob/main/setup.py), and [postBuild](https://github.com/AllenInstitute/openscope_databook/blob/main/postBuild) from the root directory. In the case of Dandihub, once a user has made an account, the environment should be persistent between runtimes, but it must still be setup once. As for Google Colab, the environment must be setup every runtime. If a notebook is being run locally, the environment may or may not be installed depending on the user. -The measure we employed to solve this is to have an *"Environment Setup"* cell at the top of every notebook. This cell checks to see if the `databook_utils` package exists. databook_utils consists of merely a few small Python modules which handle DANDI operations that are used in nearly every Notebook in the databook. It can be presumed that the presence of databook_utils is a reliable indicator of whether or not the proper environment of the Databook is installed in the local environment of a notebook. Therefore, if importing databook_utils succeeds, the notebook continues. If it fails, the databook is cloned from Github and the necessary environment is installed from within the Databook's root directory using the command `pip install -e .`. This installs all the necessary pip dependencies as well as the databook_utils package which is stored within the Databook at [/databook_utils](https://github.com/AllenInstitute/openscope_databook/tree/main/databook_utils). However, the Jupyter kernel does not update its environment with the local environment until it has been restarted. In the case that the environment was installed by the Environment Setup, the Jupyter kernel must be restarted by the user and run again. +The measure we employed to solve this is to have an *"Environment Setup"* cell at the top of every notebook. This cell checks to see if the `databook_utils` package exists. databook_utils consists of merely a few small Python modules which handle DANDI operations that are used in nearly every Notebook in the databook. It can be presumed that the presence of databook_utils is a reliable indicator of whether or not the proper environment of the Databook is installed in the local environment of a notebook. Therefore, if importing databook_utils succeeds, the notebook continues. If it fails, the databook is cloned from GitHub and the necessary environment is installed from within the Databook's root directory using the command `pip install -e .`. This installs all the necessary pip dependencies as well as the databook_utils package which is stored within the Databook at [/databook_utils](https://github.com/AllenInstitute/openscope_databook/tree/main/databook_utils). However, the Jupyter kernel does not update its environment with the local environment until it has been restarted. In the case that the environment was installed by the Environment Setup, the Jupyter kernel must be restarted by the user and run again. Finally, Jupyter Book allows for [Thebe integration](https://jupyterbook.org/en/stable/interactive/thebe.html). Thebe is a platform which allows Jupyter Book to run notebooks within its own UI, using Binder as its backend runtime. Just as with the other launch buttons, Thebe can be activate by pressing the its button, labeled "Live Code" in the top right of the UI. Just as with Binder, the environment is setup automatically using files from the Databook repository. diff --git a/docs/methods/github.md b/docs/methods/github.md index 0b18ce63..99f319b8 100644 --- a/docs/methods/github.md +++ b/docs/methods/github.md @@ -1,13 +1,13 @@ -# Git/Github +# Git/GitHub -The main code repository and development location of the Databook is on [Github](https://github.com/), and the project is version-controlled with [Git](https://git-scm.com/). The Databook is developed with version-control practices in mind. Branches are used for developing separate features and commits are pushed from local machines to the remote repo. A `dev` branch is used as the basis for all continuous integration and merging separate feature branches. During deployments, if the dev branch passes the build test, it may be merged into the `main` branch, intended to host the latest working release of the Databook. Finally, from the main branch, the Databook is deployed to the public website and a release is made manually using the Github release feature. +The main code repository and development location of the Databook is on [GitHub](https://github.com/), and the project is version-controlled with [Git](https://git-scm.com/). The Databook is developed with version-control practices in mind. Branches are used for developing separate features and commits are pushed from local machines to the remote repo. A `dev` branch is used as the basis for all continuous integration and merging separate feature branches. During deployments, if the dev branch passes the build test, it may be merged into the `main` branch, intended to host the latest working release of the Databook. Finally, from the main branch, the Databook is deployed to the public website and a release is made manually using the GitHub release feature. -We also utilize Github in concert with [ReviewNB](https://www.reviewnb.com/) for a formal review process, by which other teammembers can review changes prior to merging them into dev or main. Because Github does not provide a useful platform to evaluate the changes made to Jupyter Notebooks, ReviewNB is integrated into our repository. Anytime a Pull Request is made, ReviewNB generates a link displaying all the changes to notebooks contained within the Pull Request, and a teammember can view these changes and provide comments pertaining to specific changes or whole sections. This way, teammembers can evaluate the scientific validity of a notebook, critique the methodology of the analysis or the usability of the plots, as well as provide feedback on ways to improve the educational value of the content. Changes can continually be made leveraging feedback through ReviewNB until the notebook is prepared to be merged. +We also utilize GitHub in concert with [ReviewNB](https://www.reviewnb.com/) for a formal review process, by which other teammembers can review changes prior to merging them into dev or main. Because GitHub does not provide a useful platform to evaluate the changes made to Jupyter Notebooks, ReviewNB is integrated into our repository. Anytime a Pull Request is made, ReviewNB generates a link displaying all the changes to notebooks contained within the Pull Request, and a teammember can view these changes and provide comments pertaining to specific changes or whole sections. This way, teammembers can evaluate the scientific validity of a notebook, critique the methodology of the analysis or the usability of the plots, as well as provide feedback on ways to improve the educational value of the content. Changes can continually be made leveraging feedback through ReviewNB until the notebook is prepared to be merged. -Since the Databook is intended to be a community-oriented project, Github issues from the public are encouraged. Users may submit issues reporting bugs, suggesting enhancements, or asking questions regarding any part of the Databook. Furthermore, the Databook may accept changes from community members. For instance, the Databook repository has been forked and a new chapter called replication was made. After providing a review of the Pull Request and changes were made, this enhancement was merged and integrated into the Databook. Users who contribute at least one commit to the databook are automatically added to the dynamic authors list, discussed below. +Since the Databook is intended to be a community-oriented project, GitHub issues from the public are encouraged. Users may submit issues reporting bugs, suggesting enhancements, or asking questions regarding any part of the Databook. Furthermore, the Databook may accept changes from community members. For instance, the Databook repository has been forked and a new chapter called replication was made. After providing a review of the Pull Request and changes were made, this enhancement was merged and integrated into the Databook. Users who contribute at least one commit to the databook are automatically added to the dynamic authors list, discussed below. -Testing, deployment, and generating the dynamic authors list are managed by Github Workflows. Two workflows are used for testing, [test.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/.github/workflows/test.yml) and [build.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/.github/workflows/build.yml). test.yml is run before any changes are pushed or merged to `dev`. It sets up the environment within the provided Ubuntu Github runner using our repo's [requirements.txt](https://github.com/AllenInstitute/openscope_databook/blob/main/requirements.txt) file, and uses the Github workflow [Ana06/get-changed-files](https://github.com/Ana06/get-changed-files) to retrieve a list of the Jupyter notebooks which were changed and then runs Pytest with nbmake to run every changed notebook. If all the changed notebooks run without error, the test passes. Otherwise, the test fails. build.yml is used before any changes are pushed or merged to `main`. Similarly to test.yml it sets up the environment and uses pytest and nbmake. However, build.yml runs all notebooks in the repository, and also builds the Jupyter Book to validate the configuration and table of contents. +Testing, deployment, and generating the dynamic authors list are managed by GitHub Workflows. Two workflows are used for testing, [test.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/.github/workflows/test.yml) and [build.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/.github/workflows/build.yml). test.yml is run before any changes are pushed or merged to `dev`. It sets up the environment within the provided Ubuntu GitHub runner using our repo's [requirements.txt](https://github.com/AllenInstitute/openscope_databook/blob/main/requirements.txt) file, and uses the GitHub workflow [Ana06/get-changed-files](https://github.com/Ana06/get-changed-files) to retrieve a list of the Jupyter notebooks which were changed and then runs Pytest with nbmake to run every changed notebook. If all the changed notebooks run without error, the test passes. Otherwise, the test fails. build.yml is used before any changes are pushed or merged to `main`. Similarly to test.yml it sets up the environment and uses pytest and nbmake. However, build.yml runs all notebooks in the repository, and also builds the Jupyter Book to validate the configuration and table of contents. -Because there are some notebooks which require a secure API key to access embargoed data on DANDI, we use Github secrets to store such a key. Both test.yml and build.yml assign the the Github secret key to an environment variable, called DANDI_API_KEY, before running the notebooks. Within the relevant notebooks, Python's OS module is used to access DANDI_API_KEY from the runtime environment, and the embargoed files are accessible. +Because there are some notebooks which require a secure API key to access embargoed data on DANDI, we use GitHub secrets to store such a key. Both test.yml and build.yml assign the the GitHub secret key to an environment variable, called DANDI_API_KEY, before running the notebooks. Within the relevant notebooks, Python's OS module is used to access DANDI_API_KEY from the runtime environment, and the embargoed files are accessible. -Deployment is performed by the [deploy.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/.github/workflows/deploy.yml) workflow, which is dispatched manually. This workflow first runs another workflow, [insert_authors.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/.github/workflows/insert_authors.yml). insert_authors.yml runs a Python file which, in Github's Ubuntu runner, checks out the latest version of the Databook repository's main branch, parses a list of contributors from `git shortlog`, inserts them into a markdown header in the [Intro.md](https://github.com/AllenInstitute/openscope_databook/blob/main/docs/intro.md) file, and makes a commit. After insert_authors.yml is finished, the Jupyter Book is built, and then the deployment uses [peaceiris/actions-gh-pages](https://github.com/peaceiris/actions-gh-pages) to deploy the notebook to Github pages. Github pages serves to reliably host the Jupyter book website for free. +Deployment is performed by the [deploy.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/.github/workflows/deploy.yml) workflow, which is dispatched manually. This workflow first runs another workflow, [insert_authors.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/.github/workflows/insert_authors.yml). insert_authors.yml runs a Python file which, in GitHub's Ubuntu runner, checks out the latest version of the Databook repository's main branch, parses a list of contributors from `git shortlog`, inserts them into a markdown header in the [Intro.md](https://github.com/AllenInstitute/openscope_databook/blob/main/docs/intro.md) file, and makes a commit. After insert_authors.yml is finished, the Jupyter Book is built, and then the deployment uses [peaceiris/actions-gh-pages](https://github.com/peaceiris/actions-gh-pages) to deploy the notebook to GitHub pages. GitHub pages serves to reliably host the Jupyter book website for free. diff --git a/docs/methods/jupyter_book.md b/docs/methods/jupyter_book.md index 97e1a7aa..9b194be6 100644 --- a/docs/methods/jupyter_book.md +++ b/docs/methods/jupyter_book.md @@ -4,6 +4,6 @@ The Databook primarily functions as a repository comprised of [Jupyter notebooks In its deployed form, the Databook is a website powered by [Jupyter Book](jupyterbook.org). Jupyter Book handles the web-based functionality of the Databook and manages a neat and simple UI. All Jupyter book files exist within the [/docs](https://github.com/AllenInstitute/openscope_databook/tree/main/docs) directory of the Databook. Jupyter Book is configurable with a set of yaml files, namely [_config.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/docs/_config.yml) and [_toc.yml](https://github.com/AllenInstitute/openscope_databook/blob/main/docs/_toc.yml). To build the repository into the Jupyter Book, the command `jupyter-book build ./docs` is used within the root directory. The build generates a `_build` directory which contains all the HTML and Javascript files for a functional website, which is used automatically when the Databook is deployed. -Within the Table of Contents, we specified the Introductory page, various chapters of the Databook, and their respective notebook/markdown files. Within the config file, the project Title, logo, and links to the Github repository and Github issues were specified. Also through the config we specified the available launch buttons and their links as [Dandihub](hub.dandiarchive.org) (Jupyterhub), [Binder](mybinder.org), and [Google Colab](colab.research.google.com). We also included the option to allow for [Thebe](https://jupyterbook.org/en/stable/interactive/thebe.html) functionality. +Within the Table of Contents, we specified the Introductory page, various chapters of the Databook, and their respective notebook/markdown files. Within the config file, the project Title, logo, and links to the GitHub repository and GitHub issues were specified. Also through the config we specified the available launch buttons and their links as [Dandihub](hub.dandiarchive.org) (Jupyterhub), [Binder](mybinder.org), and [Google Colab](colab.research.google.com). We also included the option to allow for [Thebe](https://jupyterbook.org/en/stable/interactive/thebe.html) functionality. Additionally, Jupyter Book has integrated citation and bibliography functionality, such that when a [references.bib](https://github.com/AllenInstitute/openscope_databook/blob/main/docs/references.bib) file is included and specified in the config, and citations appear in the markdown of the Databook's pages in the form `{cite}citation_name_here`, a bibliography page is generated which we put in the final chapter of the Table of Contents. As new works are cited in notebooks, they can be manually added to the references.bib file after which they will show up automatically in the bibliography when the Jupyter Book is built. diff --git a/docs/visualization/visualize_2p_VR_behavior.ipynb b/docs/visualization/visualize_2p_VR_behavior.ipynb index 7a912532..f8568bb2 100644 --- a/docs/visualization/visualize_2p_VR_behavior.ipynb +++ b/docs/visualization/visualize_2p_VR_behavior.ipynb @@ -26,7 +26,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This notebook is modified from this [tutorial](https://github.com/sytseng/Tutorial_notebook_for_Dandiset_000579/blob/main/notebook/Tutorial_for_reading_NWB_files_from_Dandiset_000579.ipynb). For more information on this dataset, one can visit this [Github repository](https://github.com/sytseng/Tutorial_notebook_for_Dandiset_000579/tree/main), or a [related repo](https://github.com/sytseng/GLM_Tensorflow_2) on fitting **generalized linear models (GLM)** to the data. " + "This notebook is modified from this [tutorial](https://github.com/sytseng/Tutorial_notebook_for_Dandiset_000579/blob/main/notebook/Tutorial_for_reading_NWB_files_from_Dandiset_000579.ipynb). For more information on this dataset, one can visit this [GitHub repository](https://github.com/sytseng/Tutorial_notebook_for_Dandiset_000579/tree/main), or a [related repo](https://github.com/sytseng/GLM_Tensorflow_2) on fitting **generalized linear models (GLM)** to the data. " ] }, { From 4630bb2869df636921ef04d9c08e55e4ede59a92 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 1 Jul 2024 22:42:20 -0400 Subject: [PATCH 2/5] Do use O. middle initial for me I use it in "official" works --- data/contributors.csv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data/contributors.csv b/data/contributors.csv index 127f6c01..d7fd9125 100644 --- a/data/contributors.csv +++ b/data/contributors.csv @@ -4,7 +4,7 @@ Shailaja Akella,0000-0002-8333-2021,Allen Institute,shailaja.akella@alleninstitu Ahad Bawany,,Allen Institute,ahad.bawany@alleninstitute.org,https://github.com/Ahad-Allen,,"Processing" Corbett Bennett,0009-0001-2847-7754,Allen Institute,corbettb@alleninstitute.org,https://github.com/corbennett,,"Processing" Benjamin Dichter,0000-0001-5725-6910,CatalystNeuro,ben.dichter@gmail.com,https://github.com/bendichter,,"Conceptualization" -Yaroslav Halchenko,,Dartmouth College,,https://github.com/yarikoptic,,"Processing" +Yaroslav O. Halchenko,,Dartmouth College,,https://github.com/yarikoptic,,"Processing" Daniel Hulsey,0000-0003-3243-6282,University of Oregon,dhulsey@uoregon.edu,https://github.com/dhulsey,,"Processing" Santiago Jaramillo,0000-0002-6595-8450,University of Oregon,sjara@uoregon.edu,https://github.com/sjara,,"Processing" Satrajit Ghosh,0000-0002-5312-6729,Massachusetts Institute of Technology,satra@mit.edu,https://github.com/satra,NIH R24MH117295,"Conceptualization" From 7be1a00a9417d45158600db7ae8c133fde50e9b2 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 1 Jul 2024 22:45:19 -0400 Subject: [PATCH 3/5] Provide ORCID and email for myself, point to DANDI grant as the funding --- data/contributors.csv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data/contributors.csv b/data/contributors.csv index d7fd9125..de325bb0 100644 --- a/data/contributors.csv +++ b/data/contributors.csv @@ -4,7 +4,7 @@ Shailaja Akella,0000-0002-8333-2021,Allen Institute,shailaja.akella@alleninstitu Ahad Bawany,,Allen Institute,ahad.bawany@alleninstitute.org,https://github.com/Ahad-Allen,,"Processing" Corbett Bennett,0009-0001-2847-7754,Allen Institute,corbettb@alleninstitute.org,https://github.com/corbennett,,"Processing" Benjamin Dichter,0000-0001-5725-6910,CatalystNeuro,ben.dichter@gmail.com,https://github.com/bendichter,,"Conceptualization" -Yaroslav O. Halchenko,,Dartmouth College,,https://github.com/yarikoptic,,"Processing" +Yaroslav O. Halchenko,0000-0003-3456-2493,Dartmouth College,yoh@dartmouth.edu,https://github.com/yarikoptic,NIH R24MH117295,"Processing" Daniel Hulsey,0000-0003-3243-6282,University of Oregon,dhulsey@uoregon.edu,https://github.com/dhulsey,,"Processing" Santiago Jaramillo,0000-0002-6595-8450,University of Oregon,sjara@uoregon.edu,https://github.com/sjara,,"Processing" Satrajit Ghosh,0000-0002-5312-6729,Massachusetts Institute of Technology,satra@mit.edu,https://github.com/satra,NIH R24MH117295,"Conceptualization" From bf53465ad55e4f36ff60a9b525bc67f00cd8b370 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 1 Jul 2024 23:19:08 -0400 Subject: [PATCH 4/5] Make contributor records to be hyperlink if email or URL email matching is quite rudimentary but I think should work here. Note: did not test if works, mostly chatgpt creation ;-) --- databook_utils/insert_authors_version.py | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/databook_utils/insert_authors_version.py b/databook_utils/insert_authors_version.py index 6e99b1a1..01a90e9e 100644 --- a/databook_utils/insert_authors_version.py +++ b/databook_utils/insert_authors_version.py @@ -133,7 +133,16 @@ def run(self): line_block = nodes.line_block() for property in properties[1:]: - if property != "": + if not property: + continue + elif property.startswith("https://") or property.startswith("http://"): + link_node = nodes.reference(text=property, refuri=property) + line_block.append(link_node) + elif "@" in property: + mailto_link = f"mailto:{property}" + email_node = nodes.reference(text=property, refuri=mailto_link) + line_block.append(email_node) + else: line_block.append(nodes.line(text=property)) line_block.append(nodes.line(text="")) @@ -153,4 +162,4 @@ def setup(app): 'version': '0.1', 'parallel_read_safe': True, 'parallel_write_safe': True, - } \ No newline at end of file + } From a11d4d83e8bc4daa59e612a97ef7db7e1c31b61a Mon Sep 17 00:00:00 2001 From: Carter Peene Date: Tue, 2 Jul 2024 13:44:33 -0700 Subject: [PATCH 5/5] add citation instructions to FAQ --- data/contributors.csv | 2 +- docs/FAQ.md | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/data/contributors.csv b/data/contributors.csv index 127f6c01..fcd33df9 100644 --- a/data/contributors.csv +++ b/data/contributors.csv @@ -19,7 +19,7 @@ Stephanie Prince,0000-0002-3083-6955,Lawrence Berkeley National Laboratory,smpri Jason Pina,0000-0003-1385-8762,York University,jaypina@yorku.ca,https://github.com/jayepi,,"Conceptualization" Hyeyoung Shin,,Seoul National University,shinehyeyoung@gmail.com,https://github.com/hs13,,"Conceptualization" Josh Siegle,,Allen Institute,joshs@alleninstitute.org,https://github.com/jsiegle,NIH U24,"Processing" -Jiatai Song,,Tsingua University,,https://github.com/Laohusong,,"Processing" +Jiatai Song,0009-0004-4910-1141,Tsingua University,,https://github.com/Laohusong,,"Processing" Shih-Yi Tseng,0000-0002-5029-433X,"University of California, San Francisco",shihyitseng41@gmail.com,https://github.com/sytseng,,"Processing" Jacob Westerberg,0000-0001-5331-8707,Netherlands Institute for Neuroscience,j.westerberg@nin.knaw.nl,https://github.com/jakewesterberg,,"Conceptualization" Alex Williams,0000-0001-5853-103X,New York University,alex.h.williams@nyu.edu,https://github.com/ahwillia,,"Review" diff --git a/docs/FAQ.md b/docs/FAQ.md index bcb456d3..7717fba8 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -43,3 +43,9 @@ Contributing to this project can be as simple as forking the [Github repo](https **I have a question that isn't addressed here**\ Questions, bugs, or any other topics of interest can be discussed by filing an issue on the Github repo's [issues page](https://github.com/AllenInstitute/openscope_databook/issues). + + +### Citing + +**How can I cite the Databook?**\ +The Databook has a DOI through Zenodo, and can be cited accordingly 10.5281/zenodo.12614663. \ No newline at end of file