Move python related files to root directory

We want to add more python tools in the ramen source that will run from
the root directory, so various files need to be in the root directory,
and new make target need to be in the main Makefile.

Files moved to root directory:

- test/requirements.txt - require packages for testing python
- test/.flake8 - flake8 configuration for linting python files
- test/.vimrc.example - example for configuring vim for Black formatter
- test/coverage.pth - coverage configuration for getting full test
  coverage from multiple processes

New make targets:

- `make venv` - create python virtual environment with all the tools

Files moved to hack directory:

- test/scripts/make-venv - used by `make venv`

make-venv changes:

- When creating the virtual environment, we install now the required
  packages and configure coverage to get full test coverage.

Documentation changes:

- The instructions on creating the virtual environment moved to the quick
  start guide.
- Add missing note about activating the virtual environment to the quick
  start guide.

Signed-off-by: Nir Soffer <[email protected]>

.github/workflows/ci.yaml

@@ -102,10 +102,10 @@ jobs:
           kubectl version --client --output=yaml
 
       - name: Install tools
-        run: pip install -r test/requirements.txt
+        run: pip install -r requirements.txt
 
       - name: Enable coverage for child processes
-        run: cp test/coverage.pth $(python -m site --user-site)
+        run: cp coverage.pth $(python -m site --user-site)
 
       - name: Install drenv
         run: pip install -e test

.gitignore

@@ -29,11 +29,13 @@ config/hub/bundle
 *.swo
 *~
 .vscode
+.vimrc
+
+# Generated files
+venv
 
 # Test enviromemnt generated files
 test/drenv.egg-info/
 test/**/__pycache__/
 test/.coverage
 test/.coverage.*
-test/.vimrc
-test/venv

Makefile

@@ -169,6 +169,10 @@ test-drpc: generate manifests envtest
 test-util-pvc: generate manifests envtest
 	KUBEBUILDER_ASSETS=$(KUBEBUILDER_ASSETS) go test ./controllers/util -coverprofile cover.out $(GO_TEST_GINKGO_ARGS) -ginkgo.focus PVCS_Util
 
+.PHONY: venv
+venv:
+	hack/make-venv
+
 test-drenv:
 	$(MAKE) -C test
 

docs/devel-quick-start.md

@@ -58,6 +58,20 @@ enough resources:
    chmod +x .git/hooks/commit-msg
    ```
 
+1. Create python virtual environment
+
+   The *Ramen* project use python tool to create and provision test
+   environment and run tests. The create a virtual environment including
+   the tools run:
+
+   ```
+   make venv
+   ```
+
+   This create a virtual environment in `~/.venv/ramen` and a symbolic
+   link `venv` for activating the environment. To activate the
+   environment use:
+
 That's all! You are ready to submit your first pull request!
 
 ## Setting up the `drenv` tool
@@ -69,6 +83,13 @@ to set up the `drenv` tool.
 
 ## Starting the test environment
 
+Before using the `drenv` tool to start a test environment, you need to
+activate the python virtual environment:
+
+```
+source venv
+```
+
 *Ramen* supports many configurations, but the only development
 environment available now is the `regional-dr.yaml`. This environment
 simulates a Regional DR setup, when the managed clusters are located in

test/scripts/make-venv → hack/make-venv

@@ -13,7 +13,13 @@ echo "Upgrading pip..."
 $venv/bin/pip install --upgrade pip
 
 echo "Installing drenv..."
-$venv/bin/pip install -e .
+$venv/bin/pip install -e test
+
+echo "Installing development tools"
+$venv/bin/pip install -r requirements.txt
+
+echo "Enabling full test coverage"
+cp coverage.pth $venv/lib/python*/site-packages
 
 echo "Adding venv symlink..."
 ln -sf $venv/bin/activate venv

test/Makefile

@@ -16,9 +16,6 @@ sources := $(wildcard \
 
 all: flake8 pylint black test coverage
 
-venv:
-	scripts/make-venv
-
 flake8:
 	python3 -m flake8 $(sources)
 

test/README.md

@@ -10,6 +10,9 @@ environment.
 
 ## Setup
 
+1. Setup a development environment as describe in
+   [developer quick start guide](../docs/devel-quick-start.md)
+
 1. Add yourself to the libvirt group (required for minikube kvm2 driver).
 
    ```
@@ -49,27 +52,6 @@ environment.
    myusername ALL=(ALL) NOPASSWD: /usr/bin/podman
    ```
 
-1. Clone the `ramen` source
-
-   *Ramen* test environment is part of the ramen source. You need to
-   clone it locally to create a test environment and run the tests:
-
-   ```
-   git clone https://github.com/RamenDR/ramen.git
-   ```
-
-1. Create a `ramen` virtual environment
-
-   Run this once in the ramen test directory
-
-   ```
-   make venv
-   ```
-
-   This creates a python virtual environment in `~/.venv/ramen`, and
-   install the `drenv` tool using develop mode, so changes in `drenv`
-   source are available immediately without reinstalling the tool.
-
 ### Testing that drenv is healthy
 
 Run this script to make sure `drenv` works:
@@ -91,7 +73,7 @@ The shell prompt will change to reflect that the `ramen` virtual
 environment is active:
 
 ```
-(ramen) [user@host test]$
+(ramen) [user@host ramen]$
 ```
 
 Change directory to the test directory:
@@ -570,21 +552,6 @@ simpler and faster to work with a minimal environment.
 
 ## Testing drenv
 
-### Installing development tools
-
-```
-pip install -r requirements.txt
-```
-
-### Enabling full test coverage
-
-To enable test coverage for child processes, copy the `coverage.pth`
-file to the python installation:
-
-```
-cp coverage.pth ~/.venv/ramen/lib/python*/site-packages
-```
-
 ### Running the tests
 
 Run all linters and tests and report test coverage: