06-PythonFromTheTerminalPrompt.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d7063624-7ff3-4b66-97aa-cadf66f733d0",
   "metadata": {},
   "source": [
    "# Python From the Terminal Prompt\n",
    "## Why? I LOVE JupyterHub!\n",
    "JupyterHub is great for developing code and running short scripts, but it's not the best option for developing re-useable code or processing large datasets. A few reasons why:\n",
    "- Jupyter does take up a bit of overhead when running code and adds a layer of complexity to the Python kernals it runs.\n",
    "- Because of this overhead and added complexity, it's not the best option for processing large datasets.\n",
    "- Jupyter doesn't enforce modular coding techniques.  In fact, I'd argue that it encourages long, unstructured scripts.\n",
    "- If you want to shift to a different geographic region or different time window you have to actually edit the code, making your process prone to errors.\n",
    "- Jupyter doesn't exist everywhere... and requires additional setup then just Python."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac5f00e3-d1ca-4f08-aa15-1faed5617ac2",
   "metadata": {},
   "source": [
    "## Putting Our Workflow in a .py File\n",
    "Now that we have our workflow running relatively well in Jupyter, it's time to structure it in a .py file so we can reuse it in other notebooks and other projects.  It's really as easy as cut and paste.\n",
    "\n",
    "[Python File](python/uber-workflow.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a70722e-577e-4c71-b354-a2c83e635ddc",
   "metadata": {},
   "source": [
    "## Running Python from the Command Line\n",
    "Now that we have our .py file, we can open up a terminal in JupyterHub and type ```python uber-workflow.py``` from the python directory."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e4fd4df-c29c-4bc9-bd3a-72015793a716",
   "metadata": {},
   "source": [
    "## Organizing our Workflow Code\n",
    "As your code base gets bigger, you'll find that having all of your code in a single file makes functions hard to find. It also makes working on a coding project as a team more difficult as everyone will be editing the same file. (See the Git Workshop this afternoon!). So, what often is done is that code is divided up into files based on it's function and then the core workflow code also gets its own file. You might hear other programming tutorials refer to this core workflow file as the \"driver module\".  Below is a link to the files split up into workflow.py and modules.py.  Notice how the workflow.py file imports the modules file to find those.  As your code grows, you'll probably want to continue to break the code up into managable-sized files. One option here might be to put the acquisition modules in one file, the manipulation modules in another and the visualization modules in a third.\n",
    "\n",
    "[workflow.py](python/workflow.py)  \n",
    "[modules.py](python/modules.py)\n",
    "\n",
    "You run this from the command line just like before... with ```python workflow.py```.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79634019-8e3c-4c6e-8236-3d2c0a803ebb",
   "metadata": {},
   "source": [
    "## Supercharging the Terminal Prompt: Command Line Arguments\n",
    "So, now we can run our code from the command line, but we still have to modify our code every time we want to change the data we're loading, so how is this better? I thought you said this is one reason why JupyterHub isn't so great.\n",
    "\n",
    "Introducing... command line arguments. When you type ```python <filename>``` at the terminal prompt, you can pass additional arguments to the file to tell it what to do. Let's, for a quick example, pass our start and end date to the file. We'll call this new command line argument supporting version workflowCLI.py for Command Line Interface. You'll see this as a common way to refer to the use of command line arguments to control how a program runs.\n",
    "\n",
    "[workflowCLI.py](python/workflowCLI.py)\n",
    "\n",
    "You'll see we used the sys.argv[] built-in Python list to retrieve our command line arguments in the code. Note that the first element in the list, \\[0\\], is the filename (here, workflowCLI.py), so normally you'll want to start with the second element in the list, \\[1\\]. We'll assume the first argument is the start date, and the second the end date, but if we had more time, we might add some code to print out what we're expecting for command line arguments.\n",
    "\n",
    "Now, we can call our code and specify the start and end dates without ever touching the actual code with something like ```python workflowCLI.py 2016-06-01 2016-11-01```. Note that by default, a space will seperate the command line arguments so if you need a space in an argument, you can put the arguments in quotes.\n",
    "\n",
    "## Python argparse Module\n",
    "For a more sophisticated CLI, you can use the Python built-in argparse module. That allows you to do command line arguments in key-value pairs like ```python workflowCLIv2.py startDate=2016-06-01 endDate=2016-11-01```.  More info at the links below.\n",
    "\n",
    "[Python Docs](https://docs.python.org/3/library/argparse.html)  \n",
    "[Python Tutorial](https://docs.python.org/3/howto/argparse.html)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}