fixing various issues with running tests, importing utils package and…

… installing this package

.github/workflows/workflow.yml

@@ -15,7 +15,7 @@ jobs:
         python-version: '3.12'
 
     - name: Install requirements
-      run: pip install -r requirements.txt
+      run: pip install .
 
     - name: Run tests and collect coverage
       run: pytest --cov .

README.md

@@ -9,7 +9,16 @@ The purpose of this repo is to deploy AWS Lambda function that scrapes rising an
     1. Install Terraform CLI
     2. Install AWS CLI and run `aws configure` and enter in your aws credentials.
 3. Clone this repository 
-4. From within this repository run the following:
+4. You can run the tests locally yourself by doing the following (it is recommended that you manage your python environments with something like [asdf](https://asdf-vm.com/) and use python==3.12.3 as your local runtime):
+
+    ```sh
+    python -m venv venv  # this sets up a local virtual env using the current python runtime
+    source ./venv//bin/activate  # activates the virtual env
+    pip install -e .  # installs this packages in local env with dependencies
+    pytest . -r f -s   # -r f shows extra info for failures, -s disables capturing
+    ```
+
+5. From within this repository run the following:
 
     ```sh
     terraform init

lambdaFunctions/getRedditDataFunction/test_lambda.py

@@ -8,7 +8,7 @@
 import os
 THIS_DIR = os.path.dirname(os.path.abspath(__file__))
 sys.path.append(os.path.join(THIS_DIR, '../../'))
-import configUtils as cu
+import viral_reddit_posts_utils.configUtils as cu
 import pickle
 from moto import mock_dynamodb
 

pyproject.toml

@@ -0,0 +1,33 @@
+[build-system]
+requires = ["setuptools >= 61.0"]
+build-backend = "setuptools.build_meta"
+
+# see https://packaging.python.org/en/latest/guides/writing-pyproject-toml/
+[project]
+name = "Reddit-Scraping"
+
+dynamic = ["version"]
+
+dependencies = [
+    "boto3==1.26.117",
+    "moto[dynamodb,s3]==4.1.8",
+    "pre-commit==2.21.0",
+    "praw==7.7.0",
+    "pytest==7.3.1",
+    "pytest-cov==4.0.0",
+    "viral_reddit_posts_utils@git+https://github.com/ViralRedditPosts/Utils.git@main"
+]
+
+requires-python = "== 3.12.3"
+
+authors = [
+  {name = "Kenneth Myers", email = "[email protected]"},
+]
+
+description = "This project scrapes Reddit data and loads it to a DynamoDB. It is intended to be run through an AWS Lambda function."
+
+readme = "README.md"
+
+
+
+

requirements.txt

@@ -4,3 +4,4 @@ pre-commit==2.21.0
 praw==7.7.0
 pytest==7.3.1
 pytest-cov==4.0.0
+git+https://github.com/ViralRedditPosts/Utils.git@main

scripts/zipLambdaFunction.sh

@@ -25,7 +25,6 @@ cd $SCRIPT_PATH
 cd ./zippedLambdaFunction/
 rm -r ./${function_name} || true
 cp -r ../../lambdaFunctions/${function_name} ./  # copy lambda function files here
-cp ../../configUtils.py ./${function_name}/  # copy configUtils here
 rm -rf ${function_name}.zip # remove first if it exists
 cd ./${function_name}/  # for some reason you have to zip from within this folder or it wont work, it otherwise wraps it in another folder
 #rm -rf ./*.ipynb*  # remove any notebook stuff