Dissociate requirements-min and setup requirements (#619)

.circleci/config.yml

@@ -365,6 +365,7 @@ jobs:
           command: |
             python -m venv ../venv
             . ../venv/bin/activate
+            # use click<8 until https://github.com/j0057/github-release/issues/62 is resolved
             pip install "click<8" githubrelease
             githubrelease release $CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME \
               create $CIRCLE_TAG --name $CIRCLE_TAG \

.github/ISSUE_TEMPLATE/bug_report.md

@@ -20,9 +20,9 @@ assignees: ''
 <!--Please describe your environment according to the following bullet points.-->
 
     Python Executable: Conda or Python
-    Python Version: Python 3.5, 3.6, 3.7, or 3.8
+    Python Version: Python 3.6, 3.7, or 3.8
     Operating System: Windows, macOS or Linux
-    HDMF Version: Version of HDMF used
+    HDMF Version:
 
 ## Checklist
 

.github/PULL_REQUEST_TEMPLATE/release.md

@@ -1,21 +1,28 @@
 Prepare for release of HDMF [version]
 
 ### Before merging:
-- [ ] Minor releases: Update package versions in `requirements.txt`, `requirements-dev.txt`, `requirements-doc.txt`, `requirements-min.txt` as needed. See https://requires.io/github/hdmf-dev/hdmf/requirements/?branch=dev
-- [ ] Check legal file dates and information in `Legal.txt`, `license.txt`, `README.rst`, `docs/source/conf.py`, and any other locations as needed
+- [ ] Major and minor releases: Update package versions in `requirements.txt`, `requirements-dev.txt`,
+  `requirements-doc.txt`, `requirements-min.txt`, `setup.py` as needed
+  See https://requires.io/github/hdmf-dev/hdmf/requirements/?branch=dev
+- [ ] Check legal file dates and information in `Legal.txt`, `license.txt`, `README.rst`, `docs/source/conf.py`,
+  and any other locations as needed
 - [ ] Update `setup.py` as needed
 - [ ] Update `README.rst` as needed
-- [ ] Update `src/hdmf/common/hdmf-common-schema` submodule as needed. Check the version number manually.
+- [ ] Update `src/hdmf/common/hdmf-common-schema` submodule as needed. Check the version number and commit SHA manually
 - [ ] Update changelog (set release date) in `CHANGELOG.md` and any other docs as needed
-- [ ] Run tests locally including gallery tests and inspect all warnings and outputs (`python test.py -v > out.txt`)
-- [ ] Run PyNWB tests locally including gallery tests and inspect all warnings and outputs (`python test.py -v > out.txt`)
+- [ ] Run tests locally including gallery tests, and inspect all warnings and outputs
+  (`python test.py -v > out.txt`)
+- [ ] Run PyNWB tests locally including gallery and validation tests, and inspect all warnings and outputs
+  (`python test.py -v > out.txt`)
 - [ ] Test docs locally (`make apidoc`, `make html`)
 - [ ] Push changes to this PR and make sure all PRs to be included in this release have been merged
 - [ ] Check that the readthedocs build for this PR succeeds (build latest to pull the new branch, then activate and
   build docs for new branch): https://readthedocs.org/projects/hdmf/builds/
 
 ### After merging:
 1. Create release by following steps in `docs/source/make_a_release.rst` or use alias `git pypi-release [tag]` if set up
-2. After the CI bot creates the new release (wait ~10 min), update the release notes on the [GitHub releases page](https://github.com/hdmf-dev/hdmf/releases) with the changelog
+2. After the CI bot creates the new release (wait ~10 min), update the release notes on the
+   [GitHub releases page](https://github.com/hdmf-dev/hdmf/releases) with the changelog
 3. Check that the readthedocs "latest" and "stable" builds run and succeed
-4. Update [conda-forge/hdmf-feedstock](https://github.com/conda-forge/hdmf-feedstock)
+4. Update [conda-forge/hdmf-feedstock](https://github.com/conda-forge/hdmf-feedstock) with the latest version number
+   and SHA256 retrieved from PyPI > HDMF > Download Files > View hashes for the `.tar.gz` file. Re-render as needed

CHANGELOG.md

@@ -4,6 +4,7 @@
 
 ### Bug fix
 - Raise minimum version of pandas from 0.23 to 1.0.5 to be compatible with numpy 1.20. @rly (#618)
+- Update documentation and update structure of requirements files. @rly (#619)
 
 ## HDMF 2.5.5 (May 17, 2021)
 

docs/source/conf.py

@@ -313,7 +313,7 @@ def run_apidoc(_):
     apidoc_main(['-f', '-e', '--no-toc', '-o', out_dir, src_dir])
 
 
-from abc import abstractmethod, abstractproperty
+from abc import abstractproperty
 
 def skip(app, what, name, obj, skip, options):
     if isinstance(obj, abstractproperty) or getattr(obj, '__isabstractmethod__', False):

docs/source/software_process.rst

@@ -45,22 +45,24 @@ codecov_, which shows line by line which lines are covered by the tests.
 Requirement Specifications
 --------------------------
 
-There are 4 kinds of requirements specification in HDMF.
+There are 5 kinds of requirements specification in HDMF.
 
-The first one is the requirements-min.txt_ file, which lists the package dependencies and their minimum versions for
-installing HDMF. These dependencies are read by setup.py_ into the `install_requires` key, with the adjustment that
-the `'=='` listed in `requirements-min.txt` are replaced with `'>='` to reflect that they are minimum versions.
+The first one is requirements-min.txt_, which lists the package dependencies and their minimum versions for
+installing HDMF.
 
-The second one is requirements.txt_ which contain a list of pinned (concrete) dependencies to reproduce
+The second one is requirements.txt_, which lists the pinned (concrete) dependencies to reproduce
 an entire development environment to use HDMF.
 
-The third one is requirements-dev.txt_ which contain a list of pinned (concrete) dependencies to reproduce
+The third one is requirements-dev.txt_, which list the pinned (concrete) dependencies to reproduce
 an entire development environment to use HDMF, run HDMF tests, check code style, compute coverage, and create test
 environments.
 
-The final one is requirements-doc.txt_ which contain a list of dependencies to generate the documentation for HDMF.
+The fourth one is requirements-doc.txt_, which lists the dependencies to generate the documentation for HDMF.
 Both this file and `requirements.txt` are used by ReadTheDocs_ to initialize the local environment for Sphinx to run.
 
+The final one is within setup.py_, which contains a list of package dependencies and their version ranges allowed for
+running HDMF.
+
 In order to check the status of the required packages, requires.io_ is used to create a badge on the project
 README_. If all the required packages are up to date, a green badge appears.
 

docs/source/update_requirements.rst

@@ -46,7 +46,7 @@ the following scripts:
 
    cd hdmf
 
-   # Set the requirements file to update
+   # Set the requirements file to update: requirements-dev.txt or requirements-doc.txt
    target_requirements=requirements-dev.txt
 
    mkvirtualenv hdmf-requirements
@@ -74,5 +74,3 @@ Minimum requirements should be updated manually if a new feature or bug fix is a
 for proper running of HDMF. Minimum requirements should also be updated if a user requests that HDMF be installable
 with an older version of a dependency, all tests pass using the older version, and there is no valid reason for the
 minimum version to be as high as it is.
-
-These minimum requirements are used by ``setup.py`` which replaces "==" with ">=".

requirements-min.txt

@@ -1,9 +1,8 @@
-# package dependencies and their minimum versions for installing HDMF
-# the requirements here specify '==' for testing; setup.py replaces '==' with '>='
-h5py==2.9,<3  # support for setting attrs to lists of utf-8 added in 2.9
-numpy==1.16,<1.21
-scipy==1.1,<2
-pandas==1.0.5,<2
-ruamel.yaml==0.15,<1
-jsonschema==2.6.0,<4
+# minimum versions of package dependencies for installing HDMF
+h5py==2.9  # support for setting attrs to lists of utf-8 added in 2.9
+numpy==1.16
+scipy==1.1
+pandas==1.0.5
+ruamel.yaml==0.15
+jsonschema==2.6.0
 setuptools

setup.py

@@ -1,5 +1,4 @@
 # -*- coding: utf-8 -*-
-
 from setuptools import setup, find_packages
 
 import versioneer
@@ -12,10 +11,15 @@
 
 schema_dir = 'common/hdmf-common-schema/common'
 
-with open('requirements-min.txt', 'r') as fp:
-    # replace == with >= and remove trailing comments and spaces
-    reqs = [x.replace('==', '>=').split('#')[0].strip() for x in fp]
-    reqs = [x for x in reqs if x]  # remove empty strings
+reqs = [
+    'h5py>=2.9,<3',
+    'numpy>=1.16,<1.21',
+    'scipy>=1.1,<2',
+    'pandas>=1.0.5,<2',
+    'ruamel.yaml>=0.15,<1',
+    'jsonschema>=2.6.0,<4',
+    'setuptools',
+]
 
 print(reqs)