01.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "451f5c30-9d6d-420d-9480-1b2a919d9bb7",
   "metadata": {},
   "source": [
    "# PyTorch Workflow\n",
    "\n",
    "Let's explore a an eample of an end-to-end workflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "efce9095-643f-4b0a-85f4-94f22c703017",
   "metadata": {},
   "outputs": [],
   "source": [
    "what_were_covering = {\n",
    "    1:\"Data (prepare and load)\",\n",
    "    2:\"build model\",\n",
    "    3:\"fitting the model to data (training)\",\n",
    "    4:\"making predictions and evaluation (inference)\",\n",
    "    5:\"saving and loading a model\",\n",
    "    6:\"putting it all together\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a81513aa-0c1d-4c2f-b509-112aca295c80",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.9.1'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn # nn contains all of PyTorch's building blocs for neural networks\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import torchvision\n",
    "\n",
    "device = torch.device('cuda')\n",
    "\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edd46ee3-635c-4d0d-bca1-a891c7b5110e",
   "metadata": {},
   "source": [
    "## 1. Data (preparing and loading)\n",
    "\n",
    "Data can be almost anything.... in machine learning\n",
    "\n",
    "* Excel spreadsheet\n",
    "* CSV File\n",
    "* JSON Files\n",
    "* Images\n",
    "* Audio\n",
    "* Videos\n",
    "* Text\n",
    "* DNA\n",
    "\n",
    "Machien learn is a game of two parts:\n",
    "1. Get data into a numerical representation\n",
    "2. Build a model to learn pattern in that numerical representation\n",
    "\n",
    "To showcase this, let's create some *knowne* data using the linear regression formula\n",
    "\n",
    "We'll use a linear regression formula to make a straight line with know parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "807ba16c-47f6-40a9-991f-7c2c26314f33",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the known parameters\n",
    "\n",
    "# y = bX + a\n",
    "weight = 0.7 # b\n",
    "bias = 0.3 # a\n",
    "\n",
    "# Create data\n",
    "\n",
    "start = 0.0\n",
    "end = 1.0\n",
    "step = 0.02\n",
    "X = torch.arange(start, end ,step).unsqueeze(dim=1).to('cuda')\n",
    "y = (weight * X + bias).to('cuda')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "81a3acd6-c916-4f3e-b2b6-bd0c2841fc30",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[0.0000],\n",
       "         [0.0200],\n",
       "         [0.0400],\n",
       "         [0.0600],\n",
       "         [0.0800]], device='cuda:0'),\n",
       " tensor([[0.3000],\n",
       "         [0.3140],\n",
       "         [0.3280],\n",
       "         [0.3420],\n",
       "         [0.3560]], device='cuda:0'))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X[:5], y[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e453091d-c198-4497-866c-1dc3965dffe6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50, 50)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(X), len(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4da77b2-be2d-4084-8563-dd27836307a8",
   "metadata": {},
   "source": [
    "So in AI what happens is that the Model checks the difference between the y predictions and the true y and changes its weights and biases according and finds the correct correlation between the 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "696226fd-20f2-4d86-ba9b-8ed8004d8222",
   "metadata": {},
   "source": [
    "### Splitting data into trainings and test sets\n",
    "\n",
    "- Training set - the training part\n",
    "- Practising (checking the model out) - practise exam t\n",
    "- test set - the final part"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1a2668c0-7c57-4133-bbe9-20ac23460544",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "476d6631-953f-4127-9cd3-e42a3cb11c4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_split = int(0.8 * len(X))\n",
    "X_train, y_train = X[:train_split],y[:train_split]\n",
    "X_test,y_test = X[train_split:], y[train_split:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e595e261-4a8f-4a72-934c-6d8d50f2e02e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(40, 40, 10, 10)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(X_train),len(y_train), len(X_test), len(y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c6b2a15-c079-4d17-bcb5-68c681093378",
   "metadata": {},
   "source": [
    "how might we better visualize our data?\n",
    "\n",
    "This is where the data explorer's motto comes in!\n",
    "\n",
    "\"Visaulize, Visualize, Visualize!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "42191dd2-ff54-450c-b558-abb56bb06032",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_predictions(train_data=X_train,train_labels=y_train,test_data=X_test,test_labels=y_test,predictions=None):\n",
    "    \"\"\"\n",
    "    Plots trainings data, test data and compares predictions\n",
    "    \"\"\"\n",
    "    plt.figure(figsize=(10,7))\n",
    "    \n",
    "    # Plot trainings data in blue\n",
    "    plt.scatter(train_data.cpu(),train_labels.cpu(), c='b', s=10, label='Trainings data')\n",
    "    \n",
    "    # Plot test data in green\n",
    "    plt.scatter(test_data.cpu(), test_labels.cpu(), c='g', s=10, label='Testing data')\n",
    "    \n",
    "    if predictions is not None:\n",
    "        # Plot the predictions\n",
    "        plt.scatter(test_data.cpu(),predictions.cpu(),c='r', s=10, label='predictions')\n",
    "    plt.legend(prop={\"size\":14})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "3d4b535f-acbd-4e69-bd10-1cb8b65cb442",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAGbCAYAAADgEhWsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAApWUlEQVR4nO3dfXRV9Z3v8c8XApLwqCXIyHMxoohYIaKsWVasik8Y9HJVoA/Q4apdkt46RSo+VAUGb6tYxnvBKq2C1Q6igiVFCnQYrNURJSBYIaAoKCCV4HRqK9Ug+d4/TkiTeJKcZO/ztM/7tVZWss/e5+xfsgN82L+9P8fcXQAAAGidNukeAAAAQDYjTAEAAARAmAIAAAiAMAUAABAAYQoAACCAvHTtuHv37t6/f/907R4AACBhmzZtOuTuhfHWpS1M9e/fX+Xl5enaPQAAQMLM7L3G1jHNBwAAEABhCgAAIADCFAAAQACEKQAAgAAIUwAAAAE0ezefmT0maYykg+4+JM56k/SgpMslHZY02d03Bx3Yxx9/rIMHD+rIkSNBXwo5rl27durRo4e6dOmS7qEAACIokWqExZLmS/pFI+svk1RU83GOpJ/WfG61jz/+WB9++KF69eql/Px8xfIa0HLurr/97W/av3+/JBGoAACha3aaz91flPRfTWwyVtIvPGaDpG5m9g9BBnXw4EH16tVLBQUFBCkEYmYqKChQr169dPDgwXQPBwAQQWFcM9VL0t46y/tqHmu1I0eOKD8/P9CggLry8/OZMgYAJEVKL0A3sxvMrNzMyisrK5vbNkWjQi7g9wkAkCxhhKn9kvrUWe5d89gXuPtCdy929+LCwrhvbwMAAJBVwghTZZK+ZTHnSvqzux8I4XUBAAAyXrNhysyWSHpF0iAz22dmU8zsO2b2nZpNVkl6V9IuST+TdFPSRpuDJk+erDFjxrToOaNGjVJpaWmSRtR65eXlMjPt2bMn3UMBACA0zVYjuPuEZta7pKmhjShLNXdNzqRJk7R48eIWv+6DDz6o2I84ccuXL1e7du1avK9MNGrUKA0ZMkTz589P91AAAIgrkZ4pJODAgb/PbK5cuVLXX399vcca3p145MiRhAJP165dWzyWE044ocXPAQAgG5XtLNPad9Zq9MDRKhlUkpYx8HYyIenZs2ftR7du3eo99umnn6pbt25asmSJvva1ryk/P1+PPPKIPvroI02YMEG9e/dWfn6+Tj/9dC1atKje6zac5hs1apRuuukm3X777erevbt69OihW265RdXV1fW2qTvN179/f/3Lv/yLbrzxRnXp0kW9e/fW/fffX28/b731ls4//3x16NBBgwYN0qpVq9SpU6d6Z9NmzZqlfv366bjjjlPPnj31rW99q8mfyerVq3XqqaeqQ4cOOu+88/TWW2/VW9/c9z958mT97ne/04IFC2RmtVOER48e1ZQpUzRgwADl5+erqKhI9913X72fAQAg+sp2lmnCsglasHGBJiyboLKdZWkZR+TDVFmZVFoa+5xut912m2666SZt375dV111lT799FMNGzZMK1eu1LZt2/S9731PN954o9atW9fk6/zyl79UXl6e/vM//1Pz58/Xv/7rv2rp0qVNPmfevHk644wztHnzZt166636wQ9+oFdeeUWSVF1drauvvlp5eXnasGGDFi9erJkzZ+qzzz6rff6yZcs0d+5cPfTQQ3r77be1cuVKjRgxotH97d27V1dddZUuvvhibdmyRd/97nf1gx/8oN42zX3/Dz74oEaOHKlvf/vbOnDggA4cOKA+ffqourpavXr10tNPP62KigrNmTNH99577xeCKAAg2ta+s1aHjxyWJB0+clhr31mbnoG4e1o+hg8f7o3Zvn17o+taYsUK94ICdyn2ecWKUF62Wc8884zHfrQxu3fvdkk+d+7cZp973XXX+ZQpU2qXJ02a5FdccUXt8vnnn+/nnntuvedcdNFF9Z5z/vnn+9SpU2uX+/Xr5+PHj6/3nJNPPtlnz57t7u6rV6/2tm3b+r59+2rXv/zyyy7JFy1a5O7uDzzwgJ9yyileVVXV7Pfg7n7bbbd5UVGRV1dX1z42e/Zsl+S7d+9u9HkNv/+G30tjbr31Vr/wwgub3Cas3ysAQGZYsWOFF8wpcN0jL5hT4Ct2JO8feknl3kimifSZqbVrpcOxwKrDh2PL6VRcXFxv+ejRo5ozZ46GDh2qL33pS+rUqZOWL1+u999/v8nXGTp0aL3lk046qdm3SmnqOTt27NBJJ52kXr3+Xlx/9tlnq02bv/96XHPNNfr00081YMAATZkyRc8880y9M1cNVVRU6Nxzz613Yf7IkSPrbdPa71+SHn74YRUXF6uwsFCdOnXSvHnzEnoeACA6SgaVaMm4JZp69lQtGbeEa6aSYfRoqaAg9nVBQWw5nTp27Fhvee7cuXrggQc0ffp0rVu3Tlu2bNFVV12lqqqqJl+n4YXrZtbs9UKteU5dffr00c6dO/XII4+oS5cumjZtmoYPH65PPvkk4ddoqLXf/9KlS3XzzTdr8uTJWrNmjbZs2aKbbrqp2ecBAKKnZFCJ5l8+P21BSor43XwlJdKSJbEzUqNHx5YzyUsvvaQrr7xS3/zmNyXFplzfeuut2gvYU+XUU0/VBx98oA8++EAnnXSSpFgnVMOw1aFDB11xxRW64oorNGPGDPXs2VMvv/yyRsdJqaeddpqWLVsmd689O7Vhw4Z62yTy/bdv315Hjx79wvPOOeecehfZv/POO63/AQAAEECkz0xJsQA1f37mBSlJOuWUU7Ru3Tq99NJL2rFjh0pLS7V79+6Uj+Piiy/WoEGDNGnSJG3dulUbNmzQ97//feXl5dUGocWLF+vnP/+5/vCHP2j37t1atGiR2rVrp6Kioriv+Z3vfEd79uzRzTffrJ07d+rZZ5/Vww8/XG+bRL7//v3767XXXtOePXt06NAhVVdX65RTTtHmzZv1m9/8Rm+//bZmz56t3/3ud8n54QAAUq5sZ5lKV5Wm7e68lop8mMpkd955p0aMGKHLLrtMX/3qV9WxY0d9/etfT/k42rRpo+eee06fffaZRowYoUmTJumOO+6QmalDhw6SpG7duunRRx/VeeedpyFDhmjZsmVavny5BgwYEPc1+/btq+XLl2v16tU688wzNW/ePP3oRz+qt00i3/8tt9yi9u3ba/DgwSosLNT777+vG2+8Uddee60mTpyos88+W3v27NG0adOS88MBAKRUptQdtIR5C9u1w1JcXOzl5eVx11VUVOi0005L8YhQ19atW/WVr3xF5eXlGj58eLqHEwp+rwAg85WuKtWCjQtql6eePVXzL0//u2CY2SZ3L463jjNTkCQ999xzWrt2rXbv3q3169dr8uTJOvPMMzVs2LB0Dw0AkENGDxytgnaxu8cK2hVo9MA03z2WgEhfgI7E/eUvf9Gtt96qvXv36vjjj9eoUaM0b968Zt9zEACAMB2rO0j3W8S0BNN8yBn8XgEAWotpPgAAgCQhTAEAgJTItsqDRBGmAABA0mVj5UGiCFMAACDp1r6zVoePxN4w9/CRw1r7TprfMDdEhCkAAJB02Vh5kCiqEQAAQNJlY+VBojgzlcX69++vuXPnpmXfY8aM0eTJk9OybwBAdioZVKL5l8+PVJCSCFOhMbMmP4IEj3vuuUdDhgz5wuMbN27UTTfdFGDUqfPCCy/IzHTo0KF0DwUAgFAxzReSAwcO1H69cuVKXX/99fUey8/PD32fhYWFob8mAAAtVbazLJLTd4nizFRIevbsWfvRrVu3Lzz24osvavjw4erQoYMGDBigO+64Q1VVVbXPX758uYYOHar8/HydcMIJOv/88/Xhhx9q8eLFmjlzprZt21Z7lmvx4sWSvjjNZ2ZauHChrrnmGnXs2FFf/vKX9eSTT9Yb56uvvqphw4apQ4cOOuuss7Rq1SqZmV544YVGv7fDhw9r8uTJ6tSpk0488UTde++9X9jmySef1Nlnn63OnTurR48euuaaa7R//35J0p49e3TBBRdIigXAumfqVq9erfPOO0/HH3+8TjjhBF1yySWqqKho6Y8fAJAmUa48SFTkw1QmFIStWbNGX//611VaWqpt27bpscce07PPPqvbb79dkvTHP/5R48eP16RJk1RRUaEXX3xR3/zmNyVJ1113naZNm6ZBgwbpwIEDOnDggK677rpG9zVr1iyNHTtWW7du1XXXXad/+qd/0vvvvy9J+utf/6oxY8bo1FNP1aZNm3Tfffdp+vTpzY7/lltu0W9/+1stW7ZM69at0+uvv64XX3yx3jZVVVWaOXOmtm7dqpUrV+rQoUOaMGGCJKlPnz5atmyZJGnbtm06cOCAHnzwQUnSJ598optvvlmvvfaaXnjhBXXt2lVXXnllvaAJAMhcUa48SJi7p+Vj+PDh3pjt27c3uq4lVuxY4QVzClz3yAvmFPiKHStCed3mPPPMMx770cacd955PmvWrHrbPPfcc96xY0evrq72TZs2uSTfs2dP3Ne7++67/fTTT//C4/369fP777+/dlmSz5gxo3b5yJEjnp+f70888YS7uz/88MN+/PHH++HDh2u3+eUvf+mSfP369XH3/Ze//MXbt2/vTz75ZL3Hunbt6pMmTWr0Z1BRUeGSfO/eve7uvn79epfklZWVjT7H3f2vf/2rt2nTxn//+983uV1rhPV7BQD4u3T9W5tqksq9kUwT6TNTmZKWN23apDlz5qhTp061HxMnTtQnn3yiP/7xjzrzzDN10UUXaciQIRo3bpx++tOfqrKyslX7Gjp0aO3XeXl5Kiws1MGDByVJO3bs0JAhQ+pdv3XOOec0+XrvvPOOqqqqNHLkyNrHOnXqpDPOOKPedps3b9bYsWPVr18/de7cWcXFsfeCPHZWrKnXnzhxogYOHKguXbroxBNPVHV1dbPPAwBkhmOVB1PPnqol45ZwzVTUZEpBWHV1te6++25t2bKl9uONN97Q22+/rcLCQrVt21Zr167V2rVrNXToUD366KMqKirS1q1bW7yvdu3a1Vs2M1VXV4f1rcT1ySef6JJLLlFBQYGeeOIJbdy4UatXr5akZqfrxowZo8rKSj3yyCN69dVX9frrrysvL49pPgDIIlGtPEhUpO/my5SCsGHDhmnHjh06+eSTG93GzDRy5EiNHDlSd911l04//XQtXbpUZ555ptq3b6+jR48GHsepp56qxx9/XH/7299qz0699tprTT5n4MCBateunTZs2KAvf/nLkmLh6c0339TAgQMlxc54HTp0SPfee68GDBggKXZBfV3t27eXpHrfx0cffaQdO3booYceqr1AffPmzfr8888Df68AAKRKpMOUFAtU6U7Kd911l8aMGaN+/frp2muvVV5ent5880299tpruu+++7Rhwwb9+7//uy655BKdeOKJev3117V3714NHjxYUuyuvffee0+bN29W37591blzZx133HEtHsfEiRN155136vrrr9ftt9+uDz74oPbOPDOL+5xOnTppypQpuvXWW1VYWKiTTjpJs2bNqheK+vbtq+OOO07z58/X1KlTVVFRoR/+8If1Xqdfv34yMz3//PO68sorlZ+fr+OPP17du3fXz372M/Xp00f79+/X9OnTlZcX+V9LAMh4uV530BKRnubLFJdccomef/55rV+/XiNGjNCIESP0ox/9SH379pUkde3aVS+//LLGjBmjoqIiTZs2TT/84Q/1jW98Q5I0btw4XX755brwwgtVWFioJUuWtGocnTt31q9//Wtt27ZNZ511lqZPn6577rlHktShQ4dGnzd37lxdcMEFuvrqq3XBBRdoyJAh+upXv1q7vrCwUI8//rh+9atfafDgwZo5c6Z+8pOf1HuNXr16aebMmbrjjjt04oknqrS0VG3atNHSpUv1xhtvaMiQIZo6dapmz57dqqAIAAgPdQctY7EL1FOvuLjYy8vL466rqKjQaaedluIR5aYVK1bo6quv1sGDB9W9e/d0Dyep+L0CgMSUrirVgo0Lapennj1V8y+fn8YRpZ+ZbXL34njrODOVYx5//HH9/ve/1549e7Ry5UrdfPPNuvLKKyMfpAAAicuUG7iyBRen5JgPP/xQd999tw4cOKCePXvqiiuu0I9//ON0DwsAkEEy5QaubME0H3IGv1cAgNZimg8AACBJMjZMJbtoErmF3ycAiMmE96yNmowMUx07dtT+/ftVVVWldE1DIhrcXVVVVdq/f786duyY7uEAQFpReZAcGXkBeu/evXXo0CG99957tGEjsLy8PHXt2pU7FgHkvHjvWcvF5cFlZJhq06aNevTooR49eqR7KAAARMbogaO1aMsiHT5ymMqDEGVkmAIAAOGj8iA5CFMAAOSQTHjP2qjJyAvQAQAAskVCYcrMLjWznWa2y8xmxFnfz8zWmdkbZvaCmfUOf6gAAKAxVB6kT7NhyszaSlog6TJJgyVNMLPBDTabK+kX7j5U0ixJ/yfsgQIAgPioPEivRM5MjZC0y93fdfcqSU9JGttgm8GS/qPm6/Vx1gMAgCSJV3mA1EkkTPWStLfO8r6ax+raKul/1Hx9taTOZvalhi9kZjeYWbmZlVdWVrZmvAAAoIHRA0eroF2BJFF5kAZh3c13i6T5ZjZZ0ouS9ks62nAjd18oaaEUe6PjkPYNAEBOo/IgvRIJU/sl9amz3LvmsVru/oFqzkyZWSdJ49z9v0MaIwAAaAaVB+mTyDTfRklFZjbAzNpLGi+p3pVtZtbdzI691m2SHgt3mAAAAJmp2TDl7p9LKpW0RlKFpKfdfZuZzTKzYxF4lKSdZvaWpBMlzUnSeAEAyBnUHWQHc0/PpUvFxcVeXl6eln0DAJDpjtUdHHsfvSXjljCNl0Zmtsndi+OtowEdAIAMRN1B9iBMAQCQgag7yB680TEAABmIuoPsQZgCACBDUXeQHZjmAwAACIAwBQBAilF5EC2EKQAAUuhY5cGCjQs0YdkEAlUEEKYAAEghKg+ihzAFAEAKUXkQPdzNBwBAClF5ED2EKQAAUozKg2hhmg8AACAAwhQAACGh8iA3EaYAAAgBlQe5izAFAEAIqDzIXYQpAABCQOVB7uJuPgAAQkDlQe4iTAEAEBIqD3IT03wAAAABEKYAAGhCWZlUWhr7DMRDmAIAoBFlZdKECdKCBbHPBCrEQ5gCAKARa9dKh2NtBzp8OLYMNESYAgCgEaNHSwWxtgMVFMSWgYa4mw8AgEaUlEhLlsTOSI0eHVsGGiJMAQDQhJISQhSaxjQfAABAAIQpAEBOovIAYSFMAQByDpUHCBNhCgCQc6g8QJgIUwCAnEPlAcLE3XwAgJxD5QHCRJgCAOQkKg8QFqb5AAAAAiBMAQAig7oDpANhCgAQCdQdIF0IUwCASKDuAOlCmAIARAJ1B0gX7uYDAEQCdQdIF8IUACAyqDtAOjDNBwAAEABhCgCQ8ag8QCZLKEyZ2aVmttPMdpnZjDjr+5rZejN73czeMLPLwx8qACAXUXmATNdsmDKztpIWSLpM0mBJE8xscIPN7pT0tLufJWm8pIfCHigAIDdReYBMl8iZqRGSdrn7u+5eJekpSWMbbOOSutR83VXSB+ENEQCQy6g8QKZL5G6+XpL21lneJ+mcBtvcI2mtmX1XUkdJF8V7ITO7QdINktS3b9+WjhUAkIOoPECmC6saYYKkxe7+gJmNlPSEmQ1x9+q6G7n7QkkLJam4uNhD2jcAIOKoPEAmS2Sab7+kPnWWe9c8VtcUSU9Lkru/IqmDpO5hDBAAACCTJRKmNkoqMrMBZtZesQvMG95L8b6kCyXJzE5TLExVhjlQAED0UHmAKGg2TLn755JKJa2RVKHYXXvbzGyWmR076TpN0vVmtlXSEkmT3Z1pPABAo6g8QFQkdM2Uu6+StKrBY3fV+Xq7pH8Md2gAgCiLV3nAdVHIRjSgAwDSgsoDRAVvdAwASAsqDxAVhCkAQNpQeYAoYJoPAAAgAMIUACBU1B0g1xCmAAChoe4AuYgwBQAITby6AyDqCFMAgNBQd4BcxN18AIDQUHeAXESYAgCEiroD5Bqm+QAAAAIgTAEAAARAmAIAJIT+KCA+whQAoFn0RwGNI0wBAJpFfxTQOMIUAKBZ9EcBjaMaAQDQLPqjgMYRpgAACaE/CoiPaT4AAIAACFMAkOOoPACCIUwBQA6j8gAIjjAFADmMygMgOMIUAOQwKg+A4LibDwByGJUHQHCEKQDIcVQeAMEwzQcAABAAYQoAIoi6AyB1CFMAEDHUHQCpRZgCgIih7gBILcIUAEQMdQdAanE3HwBEDHUHQGoRpgAggqg7AFKHaT4AAIAACFMAkEWoPAAyD2EKALIElQdAZiJMAUCWoPIAyEyEKQDIElQeAJmJu/kAIEtQeQBkJsIUAGQRKg+AzMM0HwAAQACEKQDIAFQeANmLMAUAaUblAZDdEgpTZnapme00s11mNiPO+nlmtqXm4y0z++/QRwoAEUXlAZDdmg1TZtZW0gJJl0kaLGmCmQ2uu427/7O7f8XdvyLp/0lanoSxAkAkUXkAZLdE7uYbIWmXu78rSWb2lKSxkrY3sv0ESXeHMzwAiD4qD4DslkiY6iVpb53lfZLOibehmfWTNEDSfzSy/gZJN0hS3759WzRQAIgyKg+A7BX2BejjJT3r7kfjrXT3he5e7O7FhYWFIe8aAAAg9RIJU/sl9amz3LvmsXjGS1oSdFAAEAXUHQC5IZEwtVFSkZkNMLP2igWmL/zVYGanSjpe0ivhDhEAsg91B0DuaDZMufvnkkolrZFUIelpd99mZrPMrO4M/3hJT7m7J2eoAJA9qDsAckdC783n7qskrWrw2F0Nlu8Jb1gAkN1Gj5YWLYoFKeoOgGjjjY4BIAmoOwByB2EKAJKEugMgN/DefAAAAAEQpgCghag8AFAXYQoAWoDKAwANEaYAoAWoPADQEGEKAFpg9OhY1YFE5QGAGO7mA4AWoPIAQEOEKQBoISoPANTFNB8AAEAAhCkAqEHlAYDWIEwBgKg8ANB6hCkAEJUHAFqPMAUAovIAQOtxNx8AiMoDAK1HmAKAGlQeAGgNpvkAAAACIEwBiDTqDgAkG2EKQGRRdwAgFQhTACKLugMAqUCYAhBZ1B0ASAXu5gMQWdQdAEgFwhSASKPuAECyMc0HAAAQAGEKQFai8gBApiBMAcg6VB4AyCSEKQBZh8oDAJmEMAUg61B5ACCTcDcfgKxD5QGATEKYApCVqDwAkCmY5gMAAAiAMAUgo1B5ACDbEKYAZAwqDwBkI8IUgIxB5QGAbESYApAxqDwAkI24mw9AxqDyAEA2IkwByChUHgDINkzzAQAABECYApB01B0AiDLCFICkou4AQNQRpgAkFXUHAKIuoTBlZpea2U4z22VmMxrZ5loz225m28zs38IdJoBsRd0BgKhr9m4+M2sraYGkiyXtk7TRzMrcfXudbYok3SbpH939T2bWI1kDBpBdqDsAEHWJVCOMkLTL3d+VJDN7StJYSdvrbHO9pAXu/idJcveDYQ8UQPai7gBAlCUyzddL0t46y/tqHqvrFEmnmNnLZrbBzC6N90JmdoOZlZtZeWVlZetGDAAAkEHCugA9T1KRpFGSJkj6mZl1a7iRuy9092J3Ly4sLAxp1wDShcoDAEgsTO2X1KfOcu+ax+raJ6nM3Y+4+25JbykWrgBEFJUHABCTSJjaKKnIzAaYWXtJ4yU1/GvzV4qdlZKZdVds2u/d8IYJINNQeQAAMc2GKXf/XFKppDWSKiQ97e7bzGyWmR27pHSNpI/MbLuk9ZKmu/tHyRo0gPSj8gAAYszd07Lj4uJiLy8vT8u+AYSjrIzKAwC5wcw2uXtxvHWJVCMAQFxUHgAAbycDAAAQCGEKQD3UHQBAyxCmANSi7gAAWo4wBaAWdQcA0HKEKQC1qDsAgJbjbj4AtUpKpCVLqDsAgJYgTAGoh7oDAGgZpvkAAAACIEwBOYLKAwBIDsIUkAOoPACA5CFMATmAygMASB7CFJADqDwAgOThbj4gB1B5AADJQ5gCcgSVBwCQHEzzAQAABECYArIclQcAkF6EKSCLUXkAAOlHmAKyGJUHAJB+hCkgi1F5AADpx918QBaj8gAA0o8wBWQ5Kg8AIL2Y5gMAAAiAMAVkIOoOACB7EKaADEPdAQBkF8IUkGGoOwCA7EKYAjIMdQcAkF24mw/IMNQdAEB2IUwBGYi6AwDIHkzzAQAABECYAgAACIAwBaQQ/VEAED2EKSBF6I8CgGgiTAEpQn8UAEQTYQpIEfqjACCaqEYAUoT+KACIJsIUkEL0RwFA9DDNBwAAEABhCggBlQcAkLsIU0BAVB4AQG4jTAEBUXkAALmNMAUEROUBAOS2hMKUmV1qZjvNbJeZzYizfrKZVZrZlpqP/xX+UIHMdKzyYOrU2Gfu1gOA3NJsNYKZtZW0QNLFkvZJ2mhmZe6+vcGmS929NAljBDIelQcAkLsSOTM1QtIud3/X3askPSVpbHKHBQAAkB0SCVO9JO2ts7yv5rGGxpnZG2b2rJn1ifdCZnaDmZWbWXllZWUrhgukDnUHAIBEhHUB+q8l9Xf3oZJ+K+nxeBu5+0J3L3b34sLCwpB2DYSPugMAQKISCVP7JdU909S75rFa7v6Ru39Ws/hzScPDGR6QHtQdAAASlUiY2iipyMwGmFl7SeMl1ft/upn9Q53FEkkV4Q0RSD3qDgAAiWr2bj53/9zMSiWtkdRW0mPuvs3MZkkqd/cySf/bzEokfS7pvyRNTuKYgaQ7Vnewdm0sSHGnHgCgMebuadlxcXGxl5eXp2XfAAAALWFmm9y9ON46GtABAAACIEwh51B5AAAIE2EKOYXKAwBA2AhTyClUHgAAwkaYQk6h8gAAELZmqxGAKKHyAAAQNsIUck5JCSEKABAepvkAAAACIEwhMqg8AACkA2EKkUDlAQAgXQhTiAQqDwAA6UKYQiRQeQAASBfu5kMkUHkAAEgXwhQig8oDAEA6MM0HAAAQAGEKGY26AwBApiNMIWNRdwAAyAaEKWQs6g4AANmAMIWMRd0BACAbcDcfMhZ1BwCAbECYQkaj7gAAkOmY5gMAAAiAMIW0oPIAABAVhCmkHJUHAIAoIUwh5ag8AABECWEKKUflAQAgSribDylH5QEAIEoIU0gLKg8AAFHBNB8AAEAAhCmEisoDAECuIUwhNFQeAAByEWEKoaHyAACQiwhTCA2VBwCAXMTdfAgNlQcAgFxEmEKoqDwAAOQapvkAAAACIEyhWdQdAADQOMIUmkTdAQAATSNMoUnUHQAA0DTCFJpE3QEAAE3jbj40iboDAACaltCZKTO71Mx2mtkuM5vRxHbjzMzNrDi8ISLdSkqk+fMJUgAAxNNsmDKztpIWSLpM0mBJE8xscJztOkv6nqRXwx4kAABApkrkzNQISbvc/V13r5L0lKSxcbabLenHkj4NcXxIIioPAAAILpEw1UvS3jrL+2oeq2VmwyT1cffnm3ohM7vBzMrNrLyysrLFg0V4qDwAACAcge/mM7M2kn4iaVpz27r7QncvdvfiwsLCoLtGAFQeAAAQjkTC1H5Jfeos96557JjOkoZIesHM9kg6V1IZF6FnNioPAAAIRyLVCBslFZnZAMVC1HhJE4+tdPc/S+p+bNnMXpB0i7uXhztUhInKAwAAwtFsmHL3z82sVNIaSW0lPebu28xslqRyd+dqmyxVUkKIAgAgqIRKO919laRVDR67q5FtRwUfFgAAQHbg7WQiiMoDAABShzAVMVQeAACQWoSpiKHyAACA1CJMRQyVBwAApFZCF6Aje1B5AABAahGmIojKAwAAUodpPgAAgAAIU1mCugMAADITYSoLUHcAAEDmIkxlAeoOAADIXISpLEDdAQAAmYu7+bIAdQcAAGQuwlSWoO4AAIDMxDQfAABAAISpNKPyAACA7EaYSiMqDwAAyH6EqTSi8gAAgOxHmEojKg8AAMh+3M2XRlQeAACQ/QhTaUblAQAA2Y1pPgAAgAAIU0lA3QEAALmDMBUy6g4AAMgthKmQUXcAAEBuIUyFjLoDAAByC3fzhYy6AwAAcgthKgmoOwAAIHcwzQcAABAAYaoFqDwAAAANEaYSROUBAACIhzCVICoPAABAPISpBFF5AAAA4uFuvgRReQAAAOIhTLUAlQcAAKAhpvkAAAACIEyJygMAANB6OR+mqDwAAABB5HyYovIAAAAEkfNhisoDAAAQRM7fzUflAQAACCLnw5RE5QEAAGi9hKb5zOxSM9tpZrvMbEac9d8xsz+Y2RYze8nMBoc/VAAAgMzTbJgys7aSFki6TNJgSRPihKV/c/cz3P0rku6T9JOwBwoAAJCJEjkzNULSLnd/192rJD0laWzdDdz94zqLHSV5eENsHbqjAABAKiRyzVQvSXvrLO+TdE7DjcxsqqTvS2ov6WuhjK6VjnVHHT4sLVoUu8Cca6IAAEAyhFaN4O4L3H2gpFsl3RlvGzO7wczKzay8srIyrF1/Ad1RAAAgVRIJU/sl9amz3LvmscY8JemqeCvcfaG7F7t7cWFhYcKDbCm6owAAQKokMs23UVKRmQ1QLESNlzSx7gZmVuTub9csXiHpbaUR3VEAACBVmg1T7v65mZVKWiOpraTH3H2bmc2SVO7uZZJKzewiSUck/UnSpGQOOhF0RwEAgFRIqLTT3VdJWtXgsbvqfP29kMcFAACQFXL+vfkAAACCIEwBAAAEQJgCAAAIgDAFAAAQAGEKAAAgAMIUAABAAIQpAACAAAhTAAAAARCmAAAAAiBMAQAABECYAgAACIAwBQAAEIC5e3p2bFYp6b0k76a7pENJ3gdaj+OTuTg2mY3jk9k4PpkryLHp5+6F8VakLUylgpmVu3txuseB+Dg+mYtjk9k4PpmN45O5knVsmOYDAAAIgDAFAAAQQNTD1MJ0DwBN4vhkLo5NZuP4ZDaOT+ZKyrGJ9DVTAAAAyRb1M1MAAABJRZgCAAAIIBJhyswuNbOdZrbLzGbEWX+cmS2tWf+qmfVPwzBzVgLH5/tmtt3M3jCzdWbWLx3jzEXNHZs6240zMzczbvdOoUSOj5ldW/PnZ5uZ/Vuqx5irEvh7ra+ZrTez12v+brs8HePMRWb2mJkdNLM3G1lvZvZ/a47dG2Y2LOg+sz5MmVlbSQskXSZpsKQJZja4wWZTJP3J3U+WNE/Sj1M7ytyV4PF5XVKxuw+V9Kyk+1I7ytyU4LGRmXWW9D1Jr6Z2hLktkeNjZkWSbpP0j+5+uqSbUz3OXJTgn507JT3t7mdJGi/podSOMqctlnRpE+svk1RU83GDpJ8G3WHWhylJIyTtcvd33b1K0lOSxjbYZqykx2u+flbShWZmKRxjLmv2+Lj7enc/XLO4QVLvFI8xVyXyZ0eSZiv2H5BPUzk4JHR8rpe0wN3/JEnufjDFY8xViRwbl9Sl5uuukj5I4fhymru/KOm/mthkrKRfeMwGSd3M7B+C7DMKYaqXpL11lvfVPBZ3G3f/XNKfJX0pJaNDIsenrimSfpPUEeGYZo9NzenvPu7+fCoHBkmJ/dk5RdIpZvaymW0ws6b+N47wJHJs7pH0DTPbJ2mVpO+mZmhIQEv/XWpWXqDhACEys29IKpZ0frrHAsnM2kj6iaTJaR4KGpen2FTFKMXO6L5oZme4+3+nc1CQJE2QtNjdHzCzkZKeMLMh7l6d7oEhfFE4M7VfUp86y71rHou7jZnlKXbK9aOUjA6JHB+Z2UWS7pBU4u6fpWhsua65Y9NZ0hBJL5jZHknnSirjIvSUSeTPzj5JZe5+xN13S3pLsXCF5Erk2EyR9LQkufsrkjoo9ia7SL+E/l1qiSiEqY2SisxsgJm1V+xCv7IG25RJmlTz9f+U9B9OW2mqNHt8zOwsSY8oFqS45iN1mjw27v5nd+/u7v3dvb9i17OVuHt5eoabcxL5u+1Xip2Vkpl1V2za790UjjFXJXJs3pd0oSSZ2WmKhanKlI4SjSmT9K2au/rOlfRndz8Q5AWzfprP3T83s1JJayS1lfSYu28zs1mSyt29TNKjip1i3aXYRWnj0zfi3JLg8blfUidJz9TcF/C+u5ekbdA5IsFjgzRJ8PiskTTazLZLOippurtz1j3JEjw20yT9zMz+WbGL0Sfzn/jUMLMliv0no3vNNWt3S2onSe7+sGLXsF0uaZekw5K+HXifHFsAAIDWi8I0HwAAQNoQpgAAAAIgTAEAAARAmAIAAAiAMAUAABAAYQoAACAAwhQAAEAA/x+0q541Vd1Y6gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_predictions()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b83224cb-f737-4f8a-93aa-9fdb61db2490",
   "metadata": {},
   "source": [
    "## 2. Build Model\n",
    "\n",
    "Our first PyTorch Model!\n",
    "\n",
    "What our model does:\n",
    "* Start with random values (weights & bias)\n",
    "* Look at training data and adjust the random values to better represent the actual weights and bias\n",
    "\n",
    "They do this through\n",
    "1. Gradient Descent\n",
    "2. Backpropagation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "bab653e7-077d-437d-9c3e-ad4001a6bed7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a linear regression model class\n",
    "\n",
    "class LinearRegressionModel(nn.Module): # <- almost everythhing in pytorch inherits from nn.Module\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.weights = nn.Parameter(torch.randn(1,requires_grad=True,dtype=torch.float))\n",
    "        self.bias = nn.Parameter(torch.randn(1,requires_grad=True,dtype=torch.float))\n",
    "        \n",
    "    def forward(self,X):\n",
    "        return self.weights * X + self.bias"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "6a4e0319-9e4e-4a3c-81f9-19ffdc0031fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = LinearRegressionModel().to('cuda')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2759abd7-b6ad-46de-bb47-720e72d81f2b",
   "metadata": {},
   "source": [
    "### PyTorch model building essentials\n",
    "\n",
    "* torch.nn - contains all of the buildings for computational graphs (another word for neural networks)\n",
    "* torch.nn.Parameter - what parameters should our model try and learn, often a pytorch layer from torch.nn will set these for us\n",
    "* torch.nn.Module - the base class for all neural network modules, if you subclass it, you should overwrite forward()\n",
    "* torch.optim - this is where the optimizers in PyTorch live, they will help you with gradient descent\n",
    "* def forward() - All nn.Module subclasses are required for you to overwrtite, forward(), this method defines what happens when data is passed to the model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f90ee21-af4c-4693-8e4d-4941e29ca438",
   "metadata": {},
   "source": [
    "### Checking the contents of our PyTorch Model\n",
    "\n",
    "Now we've created a model, let's see what's inside...\n",
    "\n",
    "SO we can check our model parameters or what's inside our model using '.parameters()'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "2c67d36d-b116-4f30-9010-fc5e866cc3c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Parameter containing:\n",
       " tensor([0.3367], device='cuda:0', requires_grad=True),\n",
       " Parameter containing:\n",
       " tensor([0.1288], device='cuda:0', requires_grad=True)]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.manual_seed(42)\n",
    "\n",
    "# Create instance of the model\n",
    "model_0 = LinearRegressionModel().to('cuda')\n",
    "\n",
    "list(model_0.parameters()) # the values are the values they are becz we got random val"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e53f9cc9-ae53-4475-90f6-ea396b9e445c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('weights', tensor([0.3367], device='cuda:0')),\n",
       "             ('bias', tensor([0.1288], device='cuda:0'))])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_0.state_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2396bc2-ba48-4407-913c-8035d4490429",
   "metadata": {},
   "source": [
    "### Making predictions using `torch.inference_mode()`\n",
    "\n",
    "To check our models predictive poweerer, let's see how well it predictions  `y_test` based on `X_test`\n",
    "\n",
    "When we pass data thhrough our moodel, it's going to run it through the forrward() mmeethod"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "baa3c018-d9e3-4608-9436-81a50d17bd43",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAGbCAYAAADgEhWsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAw8UlEQVR4nO3de3RU9b338c+Xe8LdErCEO+IFERUjyuqy4NHiBQz6eFTQWqJ4T3qkFUTFG3D0sUq19IFWbRXwUkQFS4qIeDggakUJNzUEEAQFpBI8bVVSDZDv88eEnCROkklmMtf3a62sZM/eM/NLdhg+2b+9P2PuLgAAADRMk1gPAAAAIJERpgAAAMJAmAIAAAgDYQoAACAMhCkAAIAwNIvVE3fq1Ml79eoVq6cHAAAI2dq1a/e7e0awdTELU7169VJBQUGsnh4AACBkZvZpTeuY5gMAAAgDYQoAACAMhCkAAIAwEKYAAADCQJgCAAAIQ51X85nZ05JGStrn7gOCrDdJMyRdKKlEUo67rwt3YF999ZX27dungwcPhvtQSHHNmzdX586d1a5du1gPBQCQhEKpRpgjaaakZ2pYf4GkfuUfZ0j6ffnnBvvqq6/0xRdfKDMzU2lpaQrkNaD+3F3/+te/tGfPHkkiUAEAIq7OaT53XyXpf2rZZJSkZzxgtaQOZvbDcAa1b98+ZWZmKj09nSCFsJiZ0tPTlZmZqX379sV6OACAJBSJc6YyJe2qtLy7/LYGO3jwoNLS0sIaFFBZWloaU8YAgEYR1RPQzewGMysws4Li4uK6to3SqJAK+H0CADSWSISpPZK6V1ruVn7b97j7k+6e5e5ZGRlB394GAAAgoUQiTOVL+pkFnCnpn+6+NwKPCwAAEPfqDFNmNk/Su5KOM7PdZjbOzG4ys5vKN1ki6RNJ2yT9QdItjTbaFJSTk6ORI0fW6z7Dhg1TXl5eI42o4QoKCmRm2rlzZ6yHAgBAxNRZjeDuY+pY75JyIzaiBFXXOTljx47VnDlz6v24M2bMUOBHHLqFCxeqefPm9X6ueDRs2DANGDBAM2fOjPVQAAAIKpSeKYRg797/ndlcvHixrr/++iq3Vb868eDBgyEFnvbt29d7LEcddVS97wMAQCLK35KvZduXaXjf4co+LjsmY+DtZCLk6KOPrvjo0KFDldu+/fZbdejQQfPmzdO//du/KS0tTU888YS+/PJLjRkzRt26dVNaWppOPPFEzZ49u8rjVp/mGzZsmG655Rbddddd6tSpkzp37qwJEyaorKysyjaVp/l69eql//zP/9SNN96odu3aqVu3bnrkkUeqPM/WrVs1dOhQtWrVSscdd5yWLFmiNm3aVDmaNnXqVPXs2VMtW7bU0UcfrZ/97Ge1/kyWLl2q448/Xq1atdJZZ52lrVu3Vllf1/efk5OjN998U7NmzZKZVUwRHj58WOPGjVPv3r2Vlpamfv366eGHH67yMwAAJL/8Lfkas2CMZq2ZpTELxih/S35MxpH0YSo/X8rLC3yOtTvvvFO33HKLNm3apIsvvljffvutBg0apMWLF6uwsFC33nqrbrzxRi1fvrzWx3n++efVrFkz/fWvf9XMmTP1m9/8RvPnz6/1Po899phOOukkrVu3TpMmTdLtt9+ud999V5JUVlamSy65RM2aNdPq1as1Z84cTZkyRd99913F/RcsWKDp06frd7/7nT7++GMtXrxYgwcPrvH5du3apYsvvlg/+clPtGHDBv385z/X7bffXmWbur7/GTNmaMiQIbrmmmu0d+9e7d27V927d1dZWZkyMzP14osvqqioSA888IAefPDB7wVRAEByW7Z9mUoOlkiSSg6WaNn2ZbEZiLvH5OO0007zmmzatKnGdfWxaJF7erq7FPi8aFFEHrZOL730kgd+tAE7duxwST59+vQ673vFFVf4uHHjKpbHjh3rI0aMqFgeOnSon3nmmVXuc+6551a5z9ChQz03N7diuWfPnj569Ogq9znmmGN82rRp7u6+dOlSb9q0qe/evbti/TvvvOOSfPbs2e7u/utf/9qPPfZYLy0trfN7cHe/8847vV+/fl5WVlZx27Rp01yS79ixo8b7Vf/+q38vNZk0aZKfc845tW4Tqd8rAEB8WLR5kac/kO66X57+QLov2tx4/9FLKvAaMk1SH5latkwqCQRWlZQElmMpKyuryvLhw4f1wAMPaODAgfrBD36gNm3aaOHChfrss89qfZyBAwdWWe7atWudb5VS2302b96srl27KjPzf4vrTz/9dDVp8r+/Hpdddpm+/fZb9e7dW+PGjdNLL71U5chVdUVFRTrzzDOrnJg/ZMiQKts09PuXpMcff1xZWVnKyMhQmzZt9Nhjj4V0PwBA8sg+LlvzLp2n3NNzNe/SeZwz1RiGD5fS0wNfp6cHlmOpdevWVZanT5+uX//615o4caKWL1+uDRs26OKLL1ZpaWmtj1P9xHUzq/N8oYbcp7Lu3btry5YteuKJJ9SuXTvddtttOu2003TgwIGQH6O6hn7/8+fP1/jx45WTk6PXX39dGzZs0C233FLn/QAAySf7uGzNvHBmzIKUlORX82VnS/PmBY5IDR8eWI4nb7/9ti666CJdffXVkgJTrlu3bq04gT1ajj/+eH3++ef6/PPP1bVrV0mBTqjqYatVq1YaMWKERowYoTvuuENHH3203nnnHQ0PklJPOOEELViwQO5ecXRq9erVVbYJ5ftv0aKFDh8+/L37nXHGGVVOst++fXvDfwAAAIQhqY9MSYEANXNm/AUpSTr22GO1fPlyvf3229q8ebPy8vK0Y8eOqI/jJz/5iY477jiNHTtWGzdu1OrVq/XLX/5SzZo1qwhCc+bM0R//+Ed9+OGH2rFjh2bPnq3mzZurX79+QR/zpptu0s6dOzV+/Hht2bJFL7/8sh5//PEq24Ty/ffq1Uvvv/++du7cqf3796usrEzHHnus1q1bp9dee00ff/yxpk2bpjfffLNxfjgAgKjL35KvvCV5Mbs6r76SPkzFs7vvvluDBw/WBRdcoB//+Mdq3bq1rrrqqqiPo0mTJnrllVf03XffafDgwRo7dqwmT54sM1OrVq0kSR06dNBTTz2ls846SwMGDNCCBQu0cOFC9e7dO+hj9ujRQwsXLtTSpUt18skn67HHHtNDDz1UZZtQvv8JEyaoRYsW6t+/vzIyMvTZZ5/pxhtv1OWXX64rr7xSp59+unbu3KnbbrutcX44AICoipe6g/owr2e7dqRkZWV5QUFB0HVFRUU64YQTojwiVLZx40adcsopKigo0GmnnRbr4UQEv1cAEP/yluRp1ppZFcu5p+dq5oWxfxcMM1vr7lnB1nFkCpKkV155RcuWLdOOHTu0YsUK5eTk6OSTT9agQYNiPTQAQAoZ3ne40psHrh5Lb56u4X1jfPVYCJL6BHSE7uuvv9akSZO0a9cudezYUcOGDdNjjz1W53sOAgAQSUfqDmL9FjH1wTQfUga/VwCAhmKaDwAAoJEQpgAAQFQkWuVBqAhTAACg0SVi5UGoCFMAAKDRLdu+TCUHA2+YW3KwRMu2x/gNcyOIMAUAABpdIlYehIpqBAAA0OgSsfIgVByZSmC9evXS9OnTY/LcI0eOVE5OTkyeGwCQmLKPy9bMC2cmVZCSCFMRY2a1foQTPO6//34NGDDge7evWbNGt9xySxijjp6VK1fKzLR///5YDwUAgIhimi9C9u7dW/H14sWLdf3111e5LS0tLeLPmZGREfHHBACgvvK35Cfl9F2oODIVIUcffXTFR4cOHb5326pVq3TaaaepVatW6t27tyZPnqzS0tKK+y9cuFADBw5UWlqajjrqKA0dOlRffPGF5syZoylTpqiwsLDiKNecOXMkfX+az8z05JNP6rLLLlPr1q3Vp08fPffcc1XG+d5772nQoEFq1aqVTj31VC1ZskRmppUrV9b4vZWUlCgnJ0dt2rRRly5d9OCDD35vm+eee06nn3662rZtq86dO+uyyy7Tnj17JEk7d+7U2WefLSkQACsfqVu6dKnOOussdezYUUcddZTOO+88FRUV1ffHDwCIkWSuPAhV0oepeCgIe/3113XVVVcpLy9PhYWFevrpp/Xyyy/rrrvukiT97W9/0+jRozV27FgVFRVp1apVuvrqqyVJV1xxhW677TYdd9xx2rt3r/bu3asrrriixueaOnWqRo0apY0bN+qKK67Qtddeq88++0yS9M0332jkyJE6/vjjtXbtWj388MOaOHFineOfMGGC3njjDS1YsEDLly/X+vXrtWrVqirblJaWasqUKdq4caMWL16s/fv3a8yYMZKk7t27a8GCBZKkwsJC7d27VzNmzJAkHThwQOPHj9f777+vlStXqn379rrooouqBE0AQPxK5sqDkLl7TD5OO+00r8mmTZtqXFcfizYv8vQH0l33y9MfSPdFmxdF5HHr8tJLL3ngRxtw1lln+dSpU6ts88orr3jr1q29rKzM165d65J8586dQR/vvvvu8xNPPPF7t/fs2dMfeeSRimVJfscdd1QsHzx40NPS0vzZZ591d/fHH3/cO3bs6CUlJRXbPP/88y7JV6xYEfS5v/76a2/RooU/99xzVW5r3769jx07tsafQVFRkUvyXbt2ubv7ihUrXJIXFxfXeB9392+++cabNGnib731Vq3bNUSkfq8AAP8rVv/XRpukAq8h0yT1kal4Sctr167VAw88oDZt2lR8XHnllTpw4ID+9re/6eSTT9a5556rAQMG6NJLL9Xvf/97FRcXN+i5Bg4cWPF1s2bNlJGRoX379kmSNm/erAEDBlQ5f+uMM86o9fG2b9+u0tJSDRkypOK2Nm3a6KSTTqqy3bp16zRq1Cj17NlTbdu2VVZW4L0gjxwVq+3xr7zySvXt21ft2rVTly5dVFZWVuf9AADx4UjlQe7puZp36TzOmUo28VIQVlZWpvvuu08bNmyo+Pjggw/08ccfKyMjQ02bNtWyZcu0bNkyDRw4UE899ZT69eunjRs31vu5mjdvXmXZzFRWVhapbyWoAwcO6LzzzlN6erqeffZZrVmzRkuXLpWkOqfrRo4cqeLiYj3xxBN67733tH79ejVr1oxpPgBIIMlaeRCqpL6aL14KwgYNGqTNmzfrmGOOqXEbM9OQIUM0ZMgQ3XvvvTrxxBM1f/58nXzyyWrRooUOHz4c9jiOP/54zZ07V//6178qjk69//77td6nb9++at68uVavXq0+ffpICoSnjz76SH379pUUOOK1f/9+Pfjgg+rdu7ekwAn1lbVo0UKSqnwfX375pTZv3qzf/e53FSeor1u3TocOHQr7ewUAIFqSOkxJgUAV66R87733auTIkerZs6cuv/xyNWvWTB999JHef/99Pfzww1q9erX+67/+S+edd566dOmi9evXa9euXerfv7+kwFV7n376qdatW6cePXqobdu2atmyZb3HceWVV+ruu+/W9ddfr7vuukuff/55xZV5Zhb0Pm3atNG4ceM0adIkZWRkqGvXrpo6dWqVUNSjRw+1bNlSM2fOVG5uroqKinTPPfdUeZyePXvKzPTqq6/qoosuUlpamjp27KhOnTrpD3/4g7p37649e/Zo4sSJatYs6X8tASDupXrdQX0k9TRfvDjvvPP06quvasWKFRo8eLAGDx6shx56SD169JAktW/fXu+8845Gjhypfv366bbbbtM999yjn/70p5KkSy+9VBdeeKHOOeccZWRkaN68eQ0aR9u2bfWXv/xFhYWFOvXUUzVx4kTdf//9kqRWrVrVeL/p06fr7LPP1iWXXKKzzz5bAwYM0I9//OOK9RkZGZo7d67+/Oc/q3///poyZYoeffTRKo+RmZmpKVOmaPLkyerSpYvy8vLUpEkTzZ8/Xx988IEGDBig3NxcTZs2rUFBEQAQOdQd1I8FTlCPvqysLC8oKAi6rqioSCeccEKUR5SaFi1apEsuuUT79u1Tp06dYj2cRsXvFQCEJm9JnmatmVWxnHt6rmZeODOGI4o9M1vr7lnB1nFkKsXMnTtXb731lnbu3KnFixdr/Pjxuuiii5I+SAEAQhcvF3AlCk5OSTFffPGF7rvvPu3du1dHH320RowYoV/96lexHhYAII7EywVciYJpPqQMfq8AAA3FNB8AAEAjIUwBAJBC4uE9a5MNYQoAgBRB5UHjIEwBAJAi4uU9a5MNYQoAgBRB5UHjoBoBAIAUQeVB4yBMJaiRI0eqU6dOmjNnjiRp2LBhGjBggGbObHhDbU5Ojvbv36/FixdHaJQAgHgTD+9Zm2wIU0li4cKFat68eUjbrly5UmeffbaKi4urNJ/PmDFDseodAwAgUYV0zpSZnW9mW8xsm5ndEWR9TzNbbmYfmNlKM+sW+aEmn9LS0og91lFHHaW2bduG9Rjt27dXhw4dIjMgAEBUUXkQO3WGKTNrKmmWpAsk9Zc0xsz6V9tsuqRn3H2gpKmS/m+kB5oIhg0bpptuukm33nqrOnbsqI4dO2rixIkqKyuTJPXq1Uv333+/rr32WnXo0EFXXXWVJOmvf/2rhg4dqvT0dGVmZurmm2/WV199VfG4JSUlysnJUZs2bdSlSxc9+OCDQZ87Ly+vYrm0tFR33XWXevbsqZYtW6pPnz767W9/q507d+rss8+WJGVkZMjMlJOTIykwzTdy5MiKx/juu+80fvx4denSRa1atdKZZ56pt99+u2L9ypUrZWZavny5zjjjDKWnpysrK0vr1q2r2Oaf//ynrr76anXu3FmtWrVSnz599Jvf/Cb8HzYAoAKVB7EVypGpwZK2ufsn7l4q6QVJo6pt01/Sf5d/vSLI+tjJz5fy8gKfo+D5559XWVmZ3n33XT3xxBN68sknq4SHRx99VMcff7wKCgr04IMP6sMPP9Tw4cOVnZ2tjRs3auHChdqwYYOuvfbaivtMmDBBb7zxhhYsWKDly5dr/fr1WrVqVa3jGDt2rJ555hk9+uijKioq0lNPPaUOHTqoe/fuWrBggSSpsLBQe/fu1YwZM4I+xu2336758+fr6aef1vr163XSSSfp/PPP1969e6tsd+edd+qhhx7SunXr9IMf/EBXXXVVxXTh3XffrQ8//FCLFy/Wli1b9PTTTyszM7MhP1oAQA2oPIgxd6/1Q9K/S/pjpeWrJc2sts2fJN1a/vX/keSSfhDksW6QVCCpoEePHl6TTZs21biuXhYtck9Pd5cCnxctiszj1mDo0KHer18/Lysrq7ht2rRpnpmZ6e7uPXv29JEjR1a5z9VXX+3XXnttldvWr1/vkvyLL77wr7/+2lu0aOHPPfdcxfqvv/7a27dv72PHjq3y3Lm5ue7uvnXrVpfkr732WtBxrlixwiV5cXFxldvHjh3rI0aMcHf3b775xps3b+5z586tWH/o0CHv06ePT548ucrjLF26tGKbt99+2yX5rl273N39oosu8muuuaaWn1r0ROz3CgDizKLNizz9gXTX/fL0B9J90ebG/f8uFUkq8BqyUqR6piZIGmpm6yUNlbRH0uEgwe1Jd89y96yMjIwIPXUtli2TSgJJXSUlgeVGduaZZ8rMKpaHDBmiPXv2VEzbZWVVfY/EtWvX6rnnnlObNm0qPn70ox9JkrZv367t27ertLRUQ4YMqbhPmzZtdNJJJ9U4hvXr16tJkyYV03kNsX37dh08eLBiLJLUtGlTDRkyRJs2baqy7cCBAyu+7tq1qyRp3759kqSbb75Z8+fP18knn6wJEybozTffbPCYAADBHak8yD09V/MuncfVelEWytV8eyR1r7Tcrfy2Cu7+uQJHpGRmbSRd6u7/iNAYG274cGn27ECQSk8PLMdY69atqyyXlZXpuuuu0y9+8YvvbZuZmamtW7dGa2ghqxwWJVW5ivDIuiPniV1wwQX69NNP9dprr2n58uUaMWKELrvsMs2ePTt6AwaAFEDlQeyEcmRqjaR+ZtbbzFpIGi2pyglIZtbJzI481p2Sno7sMBsoO1uaN0/KzQ18zm78X7L33nuvSr3A6tWr1bVrV7Vr1y7o9oMGDVJhYaGOOeaY732kpaWpb9++at68uVavXl1xnwMHDuijjz6qcQynnHKKysrKtGLFiqDrW7RoIUk6fPh7Bw8r9O3bVy1atNA777xTcdvhw4f17rvvqn//6tcf1K5Tp066+uqrNWfOHD311FOaO3euvvvuu3o9BgAA8arOMOXuhyTlSXpdUpGkF9290MymmtmRdDJM0hYz2yqpi6QHGmm89ZedLc2cGZUgJUmff/65xo8fry1btujll1/WI488EvSo0xGTJk3S+++/r5tuuknr16/Xtm3btHjxYt14442SAlN648aN06RJk/TGG2+osLBQ1157ba1B6Nhjj9Xll1+u6667TgsWLNCOHTv01ltv6dlnn5Uk9ezZU2amV199VcXFxfrmm2++9xitW7fWzTffrEmTJmnJkiUqKirSzTffrC+++EK33HJLyD+Pe++9V3/+85/18ccfq6ioSAsXLlSfPn3UsmXLkB8DAFIVdQeJIaTSTndfImlJtdvurfT1y5JejuzQEtNVV12lw4cP64wzzpCZady4cbWGqYEDB2rVqlW6++67NXToUB0+fFh9+vTRJZdcUrHN9OnTdeDAAV1yySVKT0/Xz3/+cx04cKDWcTzzzDO655579B//8R/av3+/unXrVjGOzMxMTZkyRZMnT9Z1112nn/3sZxVN6pX96le/kiRdc801+sc//qFTTz1VS5cu1Q9/+MOQfx4tW7bU5MmTtWPHjop6hb/85S8h3x8AUtWRuoOSgyWavWE250LFMfMYNV5nZWV5QUFB0HVFRUU64YQTojyi8EXiLV3QeBL19wpAaspbkqdZa2ZVLOeenquZF/L/S6yY2Vp3zwq2LlJX8wEAgAga3ne40punS5LSm6dreN/YX0SF4HhvPgAA4tCRuoNl25dpeN/hTPHFMcJUBK1cuTLWQwAAJBHqDhID03wAAABhiNswdaT0EYgEfp8AxBMqD5JLXIap1q1ba8+ePSotLVWsrjZEcnB3lZaWas+ePd9rnweAWDhSeTBrzSyNWTCGQJUE4vKcqW7dumn//v369NNPdejQoVgPBwmuWbNmat++vTp16hTroQCAlm1fppKDgfeNLTlYomXbl3FeVIKLyzDVpEkTde7cWZ07d471UAAAiKjhfYdr9obZKjlYQuVBkojLMAUAQLKi8iD5EKYAAIgyKg+SS1yegA4AAJAoCFMAAEQIlQepiTAFAEAEUHmQughTAABEQLDKA6QGwhQAABEwvO9wpTdPlyQqD1IMV/MBABABVB6kLsIUAAARQuVBamKaDwAAIAyEKQAAapGfL+XlBT4DwRCmAACoQX6+NGaMNGtW4DOBCsEQpgAAqMGyZVJJoO1AJSWBZaA6whQAADUYPlxKD7QdKD09sAxUx9V8AADUIDtbmjcvcERq+PDAMlAdYQoAgFpkZxOiUDum+QAAAMJAmAIApCQqDxAphCkAQMqh8gCRRJgCAKQcKg8QSYQpAEDKofIAkcTVfACAlEPlASKJMAUASElUHiBSmOYDAAAIA2EKAJA0qDtALBCmAABJgboDxAphCgCQFKg7QKwQpgAASYG6A8QKV/MBAJICdQeIFcIUACBpUHeAWGCaDwAAIAyEKQBA3KPyAPEspDBlZueb2RYz22ZmdwRZ38PMVpjZejP7wMwujPxQAQCpiMoDxLs6w5SZNZU0S9IFkvpLGmNm/attdrekF939VEmjJf0u0gMFAKQmKg8Q70I5MjVY0jZ3/8TdSyW9IGlUtW1cUrvyr9tL+jxyQwQApDIqDxDvQrmaL1PSrkrLuyWdUW2b+yUtM7OfS2ot6dxgD2RmN0i6QZJ69OhR37ECAFIQlQeId5GqRhgjaY67/9rMhkh61swGuHtZ5Y3c/UlJT0pSVlaWR+i5AQBJjsoDxLNQpvn2SOpeablb+W2VjZP0oiS5+7uSWknqFIkBAgAAxLNQwtQaSf3MrLeZtVDgBPPq11J8JukcSTKzExQIU8WRHCgAIPlQeYBkUGeYcvdDkvIkvS6pSIGr9grNbKqZHTnoepuk681so6R5knLcnWk8AECNqDxAsgjpnCl3XyJpSbXb7q309SZJP4rs0AAAySxY5QHnRSER0YAOAIgJKg+QLHijYwBATFB5gGRBmAIAxAyVB0gGTPMBAACEgTAFAIgo6g6QaghTAICIoe4AqYgwBQCImGB1B0CyI0wBACKGugOkIq7mAwBEDHUHSEWEKQBARFF3gFTDNB8AAEAYCFMAAABhIEwBAEJCfxQQHGEKAFAn+qOAmhGmAAB1oj8KqBlhCgBQJ/qjgJpRjQAAqBP9UUDNCFMAgJDQHwUExzQfAABAGAhTAJDiqDwAwkOYAoAURuUBED7CFACkMCoPgPARpgAghVF5AISPq/kAIIVReQCEjzAFACmOygMgPEzzAQAAhIEwBQBJiLoDIHoIUwCQZKg7AKKLMAUASYa6AyC6CFMAkGSoOwCii6v5ACDJUHcARBdhCgCSEHUHQPQwzQcAABAGwhQAJBAqD4D4Q5gCgARB5QEQnwhTAJAgqDwA4hNhCgASBJUHQHziaj4ASBBUHgDxiTAFAAmEygMg/jDNBwAAEAbCFADEASoPgMRFmAKAGKPyAEhsIYUpMzvfzLaY2TYzuyPI+sfMbEP5x1Yz+0fERwoASYrKAyCx1RmmzKyppFmSLpDUX9IYM+tfeRt3/4W7n+Lup0j6f5IWNsJYASApUXkAJLZQruYbLGmbu38iSWb2gqRRkjbVsP0YSfdFZngAkPyoPAASWyhhKlPSrkrLuyWdEWxDM+spqbek/65h/Q2SbpCkHj161GugAJDMqDwAElekT0AfLelldz8cbKW7P+nuWe6elZGREeGnBgAAiL5QwtQeSd0rLXcrvy2Y0ZLmhTsoAEgG1B0AqSGUMLVGUj8z621mLRQITN97aTCz4yV1lPRuZIcIAImHugMgddQZptz9kKQ8Sa9LKpL0orsXmtlUM6s8wz9a0gvu7o0zVABIHNQdAKkjpPfmc/clkpZUu+3easv3R25YAJDYhg+XZs8OBCnqDoDkxhsdA0AjoO4ASB2EKQBoJNQdAKmB9+YDAAAIA2EKAOqJygMAlRGmAKAeqDwAUB1hCgDqgcoDANURpgCgHoYPD1QdSFQeAAjgaj4AqAcqDwBUR5gCgHqi8gBAZUzzAQAAhIEwBQDlqDwA0BCEKQAQlQcAGo4wBQCi8gBAwxGmAEBUHgBoOK7mAwBReQCg4QhTAFCOygMADcE0HwAAQBgIUwCSGnUHABobYQpA0qLuAEA0EKYAJC3qDgBEA2EKQNKi7gBANHA1H4CkRd0BgGggTAFIatQdAGhsTPMBAACEgTAFICFReQAgXhCmACQcKg8AxBPCFICEQ+UBgHhCmAKQcKg8ABBPuJoPQMKh8gBAPCFMAUhIVB4AiBdM8wEAAISBMAUgrlB5ACDREKYAxA0qDwAkIsIUgLhB5QGARESYAhA3qDwAkIi4mg9A3KDyAEAiIkwBiCtUHgBINEzzAQAAhIEwBaDRUXcAIJkRpgA0KuoOACQ7whSARkXdAYBkF1KYMrPzzWyLmW0zsztq2OZyM9tkZoVm9qfIDhNAoqLuAECyq/NqPjNrKmmWpJ9I2i1pjZnlu/umStv0k3SnpB+5+9/NrHNjDRhAYqHuAECyC6UaYbCkbe7+iSSZ2QuSRknaVGmb6yXNcve/S5K774v0QAEkLuoOACSzUKb5MiXtqrS8u/y2yo6VdKyZvWNmq83s/GAPZGY3mFmBmRUUFxc3bMQAAABxJFInoDeT1E/SMEljJP3BzDpU38jdn3T3LHfPysjIiNBTA4gVKg8AILQwtUdS90rL3cpvq2y3pHx3P+juOyRtVSBcAUhSVB4AQEAoYWqNpH5m1tvMWkgaLan6y+afFTgqJTPrpMC03yeRGyaAeEPlAQAE1Bmm3P2QpDxJr0sqkvSiuxea2VQzO3JK6euSvjSzTZJWSJro7l821qABxB6VBwAQYO4ekyfOysrygoKCmDw3gMjIz6fyAEBqMLO17p4VbF0o1QgAEBSVBwDA28kAAACEhTAFoArqDgCgfghTACpQdwAA9UeYAlCBugMAqD/CFIAK1B0AQP1xNR+ACtnZ0rx51B0AQH0QpgBUQd0BANQP03wAAABhIEwBKYLKAwBoHIQpIAVQeQAAjYcwBaQAKg8AoPEQpoAUQOUBADQeruYDUgCVBwDQeAhTQIqg8gAAGgfTfAAAAGEgTAEJjsoDAIgtwhSQwKg8AIDYI0wBCYzKAwCIPcIUkMCoPACA2ONqPiCBUXkAALFHmAISHJUHABBbTPMBAACEgTAFxCHqDgAgcRCmgDhD3QEAJBbCFBBnqDsAgMRCmALiDHUHAJBYuJoPiDPUHQBAYiFMAXGIugMASBxM8wEAAISBMAUAABAGwhQQRfRHAUDyIUwBUUJ/FAAkJ8IUECX0RwFAciJMAVFCfxQAJCeqEYAooT8KAJITYQqIIvqjACD5MM0HAAAQBsIUEAFUHgBA6iJMAWGi8gAAUhthCggTlQcAkNoIU0CYqDwAgNQWUpgys/PNbIuZbTOzO4KszzGzYjPbUP5xXeSHCsSnI5UHubmBz1ytBwCppc5qBDNrKmmWpJ9I2i1pjZnlu/umapvOd/e8RhgjEPeoPACA1BXKkanBkra5+yfuXirpBUmjGndYAAAAiSGUMJUpaVel5d3lt1V3qZl9YGYvm1n3YA9kZjeYWYGZFRQXFzdguED0UHcAAAhFpE5A/4ukXu4+UNIbkuYG28jdn3T3LHfPysjIiNBTA5FH3QEAIFShhKk9kiofaepWflsFd//S3b8rX/yjpNMiMzwgNqg7AACEKpQwtUZSPzPrbWYtJI2WVOXvdDP7YaXFbElFkRsiEH3UHQAAQlXn1XzufsjM8iS9LqmppKfdvdDMpkoqcPd8Sf9hZtmSDkn6H0k5jThmoNEdqTtYtiwQpLhSDwBQE3P3mDxxVlaWFxQUxOS5AQAA6sPM1rp7VrB1NKADAACEgTCFlEPlAQAgkghTSClUHgAAIo0whZRC5QEAINIIU0gpVB4AACKtzmoEIJlQeQAAiDTCFFJOdjYhCgAQOUzzAQAAhIEwhaRB5QEAIBYIU0gKVB4AAGKFMIWkQOUBACBWCFNIClQeAABihav5kBSoPACAFJWfH/MXf3P3mDxxVlaWFxQUxOS5AQBAEjhywmxJSWBaYt68RgtUZrbW3bOCrWOaDwAAJKY4OWGWMIW4Rt0BAKBGcXLCLOdMIW5VPno7e3ajHr0FAMSTUM+DipMTZglTiFvBjt4SpgAgydX3L+k4eI8wpvkQt+Lk6C0AIJri5Dyo+iBMIW4dOXqbm8sUHwAkhVBOhE3Av6SpRgAAAI2vPjUGcdAdVV1t1QicMwUAABpffU6EjYPzoOqDaT7EBJUHAJBiEnD6LlQcmULUUXkAAEkmlGm5OKkxaAyEKUQdlQcAkETq8xdygk3fhYppPkRdEh/pBYDUk4BVBpFGmELUUXkAAAkg1JNb+QuZagQAAFBNfWoMjmyfhOdCVUY1AgAACF19T25N0nOhQsU0HyKKygMAiHNJ2kIeS0zzIWLqe1QYABBlCd5CHku1TfNxZAoRwwUdABDn6vNCnZ0tzZxJkAoBYQoRw1FhAIhzvFA3Ck5AR8QkcbktAMS/FG8hjyXOmQIAINFx0mqj45wpAACSGSetxhRhCnWi7gAAYoQW8oTANB9qxZFjAIgRWsjjCtN8aDCOHANAjNT3BZgqg5ghTKFWHDkGgBjhBThhUI2AWnEVLQA0AmoMkkpI50yZ2fmSZkhqKumP7v5QDdtdKullSae7e60nRHHOFAAgJXEyakIK65wpM2sqaZakCyT1lzTGzPoH2a6tpFslvRfecAEASGKcjJp0QjlnarCkbe7+ibuXSnpB0qgg202T9CtJ30ZwfGhEVB4AQISF8sLKuVBJJ5QwlSlpV6Xl3eW3VTCzQZK6u/urtT2Qmd1gZgVmVlBcXFzvwSJyjhxlnjUr8JlABQBhCvWF9ci5ULm5TPElibCv5jOzJpIelXRbXdu6+5PunuXuWRkZGeE+NcLAUWYAiLD6vLBSY5BUQglTeyR1r7Tcrfy2I9pKGiBppZntlHSmpHwzC3qSFuIDR5kBIMJ4YU1ZoVQjrJHUz8x6KxCiRku68shKd/+npE5Hls1spaQJdV3Nh9jiilsAiDBeWFNWnWHK3Q+ZWZ6k1xWoRnja3QvNbKqkAnfnbJsElZ3Nv3UAiCheWFNSSKWd7r5E0pJqt91bw7bDwh8WAABAYuDtZJIQlQcAAEQPYSrJUHkAAEB0EaaSDJUHAABEF2EqyXBlLgAA0RXSCehIHFyZCwBAdBGmkhBX5gIAED1M8wEAAISBMJUgqDsAACA+EaYSAHUHAADEL8JUAqDuAACA+EWYSgDUHQAAEL+4mi8BUHcAAED8IkwlCOoOAACIT0zzAQAAhIEwFWNUHgAAkNgIUzFE5QEAAImPMBVDVB4AAJD4CFMxROUBAACJj6v5YojKAwAAEh9hKsaoPAAAILExzQcAABAGwlQjoO4AAIDUQZiKMOoOAABILYSpCKPuAACA1EKYijDqDgAASC1czRdh1B0AAJBaCFONgLoDAABSB9N8AAAAYSBM1QOVBwAAoDrCVIioPAAAAMEQpkJE5QEAAAiGMBUiKg8AAEAwXM0XIioPAABAMISpeqDyAAAAVMc0HwAAQBgIU6LyAAAANFzKhykqDwAAQDhSPkxReQAAAMKR8mGKygMAABCOlL+aj8oDAAAQjpQPUxKVBwAAoOFCmuYzs/PNbIuZbTOzO4Ksv8nMPjSzDWb2tpn1j/xQAQAA4k+dYcrMmkqaJekCSf0ljQkSlv7k7ie5+ymSHpb0aKQHCgAAEI9COTI1WNI2d//E3UslvSBpVOUN3P2rSoutJXnkhtgwdEcBAIBoCOWcqUxJuyot75Z0RvWNzCxX0i8ltZD0bxEZXQMd6Y4qKZFmzw6cYM45UQAAoDFErBrB3We5e19JkyTdHWwbM7vBzArMrKC4uDhST/09dEcBAIBoCSVM7ZHUvdJyt/LbavKCpIuDrXD3J909y92zMjIyQh5kfdEdBQAAoiWUab41kvqZWW8FQtRoSVdW3sDM+rn7x+WLIyR9rBiiOwoAAERLnWHK3Q+ZWZ6k1yU1lfS0uxea2VRJBe6eLynPzM6VdFDS3yWNbcxBh4LuKAAAEA0hlXa6+xJJS6rddm+lr2+N8LgAAAASQsq/Nx8AAEA4CFMAAABhIEwBAACEgTAFAAAQBsIUAABAGAhTAAAAYSBMAQAAhIEwBQAAEAbCFAAAQBgIUwAAAGEgTAEAAISBMAUAABAGc/fYPLFZsaRPG/lpOkna38jPgYZj/8Qv9k18Y//EN/ZP/Apn3/R094xgK2IWpqLBzArcPSvW40Bw7J/4xb6Jb+yf+Mb+iV+NtW+Y5gMAAAgDYQoAACAMyR6mnoz1AFAr9k/8Yt/EN/ZPfGP/xK9G2TdJfc4UAABAY0v2I1MAAACNijAFAAAQhqQIU2Z2vpltMbNtZnZHkPUtzWx++fr3zKxXDIaZskLYP780s01m9oGZLTeznrEYZyqqa99U2u5SM3Mz43LvKApl/5jZ5eX/fgrN7E/RHmOqCuF1rYeZrTCz9eWvbRfGYpypyMyeNrN9ZvZRDevNzH5bvu8+MLNB4T5nwocpM2sqaZakCyT1lzTGzPpX22ycpL+7+zGSHpP0q+iOMnWFuH/WS8py94GSXpb0cHRHmZpC3Dcys7aSbpX0XnRHmNpC2T9m1k/SnZJ+5O4nShof7XGmohD/7dwt6UV3P1XSaEm/i+4oU9ocSefXsv4CSf3KP26Q9PtwnzDhw5SkwZK2ufsn7l4q6QVJo6ptM0rS3PKvX5Z0jplZFMeYyurcP+6+wt1LyhdXS+oW5TGmqlD+7UjSNAX+APk2moNDSPvnekmz3P3vkuTu+6I8xlQVyr5xSe3Kv24v6fMoji+lufsqSf9TyyajJD3jAasldTCzH4bznMkQpjIl7aq0vLv8tqDbuPshSf+U9IOojA6h7J/Kxkl6rVFHhCPq3Dflh7+7u/ur0RwYJIX2b+dYScea2TtmttrMavtrHJETyr65X9JPzWy3pCWSfh6doSEE9f1/qU7NwhoOEEFm9lNJWZKGxnoskMysiaRHJeXEeCioWTMFpiqGKXBEd5WZneTu/4jloCBJGiNpjrv/2syGSHrWzAa4e1msB4bIS4YjU3skda+03K38tqDbmFkzBQ65fhmV0SGU/SMzO1fSZEnZ7v5dlMaW6uraN20lDZC00sx2SjpTUj4noUdNKP92dkvKd/eD7r5D0lYFwhUaVyj7ZpykFyXJ3d+V1EqBN9lF7IX0/1J9JEOYWiOpn5n1NrMWCpzol19tm3xJY8u//ndJ/+20lUZLnfvHzE6V9IQCQYpzPqKn1n3j7v90907u3svdeylwPlu2uxfEZrgpJ5TXtj8rcFRKZtZJgWm/T6I4xlQVyr75TNI5kmRmJygQpoqjOkrUJF/Sz8qv6jtT0j/dfW84D5jw03zufsjM8iS9LqmppKfdvdDMpkoqcPd8SU8pcIh1mwInpY2O3YhTS4j75xFJbSS9VH5dwGfunh2zQaeIEPcNYiTE/fO6pOFmtknSYUkT3Z2j7o0sxH1zm6Q/mNkvFDgZPYc/4qPDzOYp8EdGp/Jz1u6T1FyS3P1xBc5hu1DSNkklkq4J+znZtwAAAA2XDNN8AAAAMUOYAgAACANhCgAAIAyEKQAAgDAQpgAAAMJAmAIAAAgDYQoAACAM/x/wmWSI9w00vgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# predictions = inference\n",
    "with torch.inference_mode():\n",
    "    y_preds = model_0(X_test.to('cuda'))\n",
    "plot_predictions(predictions=y_preds.cpu())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "577b7e2a-6ea8-4331-8b64-93e82a893c6a",
   "metadata": {},
   "source": [
    "## 3. Train Model\n",
    "\n",
    "The whole idea of training is for a model to move from *uknown* paamters to know parameters\n",
    "\n",
    "or in other words from a poor representation of the data to a better representation of the data\n",
    "\n",
    "One way to measure how badly the models predicions are is using a loss function\n",
    "\n",
    "Loss functions also called \"cost function\" or \"criterion\"\n",
    "\n",
    "Things we need to train : \n",
    "\n",
    "*Loss Fnction* is a function to measure how wrong the model's prediction is\n",
    "\n",
    "*Optimizer* takes into accont the loss of a model and adjusts the model's parameters (weights and bias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f02c3d46-9f3e-48d4-8256-9fec0e11d899",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([Parameter containing:\n",
       "  tensor([0.3367], device='cuda:0', requires_grad=True),\n",
       "  Parameter containing:\n",
       "  tensor([0.1288], device='cuda:0', requires_grad=True)],\n",
       " OrderedDict([('weights', tensor([0.3367], device='cuda:0')),\n",
       "              ('bias', tensor([0.1288], device='cuda:0'))]))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(model_0.parameters()),model_0.state_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "6b25eae3-633f-4cdc-bf4c-a250fbf628c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "loss_fn = nn.L1Loss()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "623b5940-c4ed-407e-88e6-ec02aaed0751",
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = torch.optim.SGD(params=model_0.parameters(),lr=0.01) \n",
    "# learning rate = possibily th emost important hyper parameter, it is a value that we set our selves, the learning rate is \n",
    "# smaller the learning rate the smaller the change done to parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bdf6a2e-95ff-477d-8f12-3f59b8b523c7",
   "metadata": {},
   "source": [
    "### Building a training loop and testing loop\n",
    "\n",
    "A couple things we need is:\n",
    "1. Loop through the data\n",
    "2. Forward pass (data moving through `forward()` function and make predictions\n",
    "3. Calculate the loss (compare expected and predicted vals)\n",
    "4. Loss backward - backward propagration, calculation works backwards throguh the network to calculate teh gradients of each the poarameters of our models with resepct to the loss (backpropagation)\n",
    "5. optimizer step- use the optimizer to adjust the model's parameters (gradient decent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0855147c-f458-4498-bf27-a4e6fe9109d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm import tqdm\n",
    "# the model gets to see the data once\n",
    "epochs = 100\n",
    "epoch_count = []\n",
    "loss_values = []\n",
    "test_loss_values = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "0f3d7f1f-b38a-4262-bb70-f070b73e6d8d",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████████████████████████████| 100/100 [00:00<00:00, 1944.71it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0 | Test : 0.48106518387794495 | Loss : 0.31288135051727295\n",
      "Epoch: 10 | Test : 0.3463551998138428 | Loss : 0.1976713389158249\n",
      "Epoch: 20 | Test : 0.21729658544063568 | Loss : 0.08908725529909134\n",
      "Epoch: 30 | Test : 0.14464019238948822 | Loss : 0.0531485341489315\n",
      "Epoch: 40 | Test : 0.11360953003168106 | Loss : 0.04543796926736832\n",
      "Epoch: 50 | Test : 0.09919948130846024 | Loss : 0.04167863354086876\n",
      "Epoch: 60 | Test : 0.08886633068323135 | Loss : 0.03818932920694351\n",
      "Epoch: 70 | Test : 0.0805937647819519 | Loss : 0.03476089984178543\n",
      "Epoch: 80 | Test : 0.07232122868299484 | Loss : 0.03132382780313492\n",
      "Epoch: 90 | Test : 0.06473556160926819 | Loss : 0.02788739837706089\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "for epoch in tqdm(range(epochs)):\n",
    "    # Set model to training model\n",
    "    model_0.train() # turns on gradient tracking\n",
    "    \n",
    "    # Forward pass\n",
    "    y_preds = model_0(X_train)\n",
    "    \n",
    "    # Calculaate the loss\n",
    "    loss = loss_fn(y_preds,y_train)\n",
    "    \n",
    "    # Optimizer zero grad\n",
    "    optimizer.zero_grad() # make sure that optimizer data doesnt add up, think as if its \"C\" so that it changes in every iteration\n",
    "    \n",
    "    # Back propagration\n",
    "    loss.backward()\n",
    "    \n",
    "    # Gradient Descent\n",
    "    optimizer.step()\n",
    "    \n",
    "    model_0.eval() # turns off  stf like dropout and batchnormalisation\n",
    "    \n",
    "    with torch.inference_mode(): # turns off gradient tracking \n",
    "        # 1. Do the forward pass\n",
    "        test_pred = model_0(X_test)\n",
    "        \n",
    "        # Calculate the loss\n",
    "        test_loss = loss_fn(test_pred,y_test)\n",
    "    \n",
    "    if epoch % 10 == 0:\n",
    "        epoch_count.append(epoch)\n",
    "        loss_values.append(loss.cpu().detach())\n",
    "        test_loss_values.append(test_loss.cpu().detach())\n",
    "        print(f\"Epoch: {epoch} | Test : {test_loss} | Loss : {loss}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "955515b4-974c-4782-9145-5d893503e41b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[0.3518],\n",
       "         [0.3633],\n",
       "         [0.3749],\n",
       "         [0.3864],\n",
       "         [0.3979]], device='cuda:0', grad_fn=<SliceBackward>),\n",
       " tensor([[0.3000],\n",
       "         [0.3140],\n",
       "         [0.3280],\n",
       "         [0.3420],\n",
       "         [0.3560]], device='cuda:0'))"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_preds[:5],y_train[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "0f193b3d-8725-417e-86fe-53300e530d53",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAGbCAYAAADgEhWsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxMUlEQVR4nO3deXiU9b338c+XnbBbApawIy6IqBhRrl4WrDYuYNDHo4LWEsU96ZFWEBU34OjjQrX2gVZtFXApooIlRUQ8HBC1ooRNDQEEQQGpBE9blVQD5Pv8MSFN4iSZZCazvl/XlSu5575n5pdMgA/37/59xtxdAAAAaJgmsR4AAABAIiNMAQAAhIEwBQAAEAbCFAAAQBgIUwAAAGFoFqsn7ty5s/fu3TtWTw8AABCyNWvW7HP39GD7YhamevfurYKCglg9PQAAQMjM7NOa9jHNBwAAEAbCFAAAQBgIUwAAAGEgTAEAAISBMAUAABCGOlfzmdnTkkZK2uvuA4PsN0mPSTpfUomkHHdfG+7AvvrqK+3du1cHDhwI96GQ4po3b64uXbqoffv2sR4KACAJhVKNMFvSDEnP1LD/PEn9yz9Ok/T78s8N9tVXX+mLL75QRkaGWrdurUBeA+rP3fWvf/1Lu3fvliQCFQAg4uqc5nP3lZL+t5ZDRkl6xgNWSepoZj8MZ1B79+5VRkaG0tLSCFIIi5kpLS1NGRkZ2rt3b6yHAwBIQpG4ZipD0s5K27vKb2uwAwcOqHXr1mENCqisdevWTBkDABpFVC9AN7PrzKzAzAqKi4vrOjZKo0Iq4PcJANBYIhGmdkvqUWm7e/lt3+PuT7p7prtnpqcHfXsbAACAhBKJMJUv6ecWcLqkf7r7ngg8LgAAQNyrM0yZ2VxJ70o6xsx2mdk4M7vBzG4oP2SxpE8kbZX0B0k3NdpoU1BOTo5GjhxZr/sMHz5ceXl5jTSihisoKJCZaceOHbEeCgAAEVNnNYK7j6ljv0vKjdiIElRd1+SMHTtWs2fPrvfjPvbYYwr8iEO3YMECNW/evN7PFY+GDx+ugQMHasaMGbEeCgAAQYXSM4UQ7Nnz75nNRYsW6dprr61yW/XViQcOHAgp8HTo0KHeYzniiCPqfR8AABJR/uZ8Ld22VFn9spR9THZMxsDbyUTIkUceWfHRsWPHKrd9++236tixo+bOnauf/OQnat26tZ544gl9+eWXGjNmjLp3767WrVvr+OOP16xZs6o8bvVpvuHDh+umm27SHXfcoc6dO6tLly6aMGGCysrKqhxTeZqvd+/e+q//+i9df/31at++vbp3766HH364yvNs2bJFw4YNU6tWrXTMMcdo8eLFatu2bZWzaVOnTlWvXr3UsmVLHXnkkfr5z39e689kyZIlOvbYY9WqVSudccYZ2rJlS5X9dX3/OTk5evPNNzVz5kyZWcUU4aFDhzRu3Dj16dNHrVu3Vv/+/fXQQw9V+RkAAJJf/uZ8jZk/RjNXz9SY+WOUvzk/JuNI+jCVny/l5QU+x9rtt9+um266SRs3btSFF16ob7/9VoMHD9aiRYtUWFiom2++Wddff72WLVtW6+M8//zzatasmf76179qxowZ+s1vfqN58+bVep9HH31UJ5xwgtauXatJkybp1ltv1bvvvitJKisr00UXXaRmzZpp1apVmj17tqZMmaLvvvuu4v7z58/X9OnT9bvf/U4ff/yxFi1apCFDhtT4fDt37tSFF16on/70p1q/fr1+8Ytf6NZbb61yTF3f/2OPPaahQ4fqqquu0p49e7Rnzx716NFDZWVlysjI0IsvvqiioiLdd999uv/++78XRAEAyW3ptqUqOVAiSSo5UKKl25bGZiDuHpOPU045xWuycePGGvfVx8KF7mlp7lLg88KFEXnYOr300kse+NEGbN++3SX59OnT67zvZZdd5uPGjavYHjt2rI8YMaJie9iwYX766adXuc/ZZ59d5T7Dhg3z3Nzciu1evXr56NGjq9znqKOO8mnTprm7+5IlS7xp06a+a9euiv3vvPOOS/JZs2a5u/uvf/1rP/roo720tLTO78Hd/fbbb/f+/ft7WVlZxW3Tpk1zSb59+/Ya71f9+6/+vdRk0qRJftZZZ9V6TKR+rwAA8WHhpoWedl+a61552n1pvnBT4/1DL6nAa8g0SX1maulSqSQQWFVSEtiOpczMzCrbhw4d0n333adBgwbpBz/4gdq2basFCxbos88+q/VxBg0aVGW7W7dudb5VSm332bRpk7p166aMjH8X15966qlq0uTfvx6XXHKJvv32W/Xp00fjxo3TSy+9VOXMVXVFRUU6/fTTq1yYP3To0CrHNPT7l6THH39cmZmZSk9PV9u2bfXoo4+GdD8AQPLIPiZbcy+eq9xTczX34rlcM9UYsrKktLTA12lpge1YatOmTZXt6dOn69e//rUmTpyoZcuWaf369brwwgtVWlpa6+NUv3DdzOq8Xqgh96msR48e2rx5s5544gm1b99et9xyi0455RTt378/5MeorqHf/7x58zR+/Hjl5OTo9ddf1/r163XTTTfVeT8AQPLJPiZbM86fEbMgJSX5ar7sbGnu3MAZqayswHY8efvtt3XBBRfoyiuvlBSYct2yZUvFBezRcuyxx+rzzz/X559/rm7dukkKdEJVD1utWrXSiBEjNGLECN1222068sgj9c477ygrSEo97rjjNH/+fLl7xdmpVatWVTkmlO+/RYsWOnTo0Pfud9ppp1W5yH7btm0N/wEAABCGpD4zJQUC1IwZ8RekJOnoo4/WsmXL9Pbbb2vTpk3Ky8vT9u3boz6On/70pzrmmGM0duxYbdiwQatWrdKvfvUrNWvWrCIIzZ49W3/84x/14Ycfavv27Zo1a5aaN2+u/v37B33MG264QTt27ND48eO1efNmvfzyy3r88cerHBPK99+7d2+9//772rFjh/bt26eysjIdffTRWrt2rV577TV9/PHHmjZtmt58883G+eEAAKIuf3O+8hbnxWx1Xn0lfZiKZ3feeaeGDBmi8847Tz/+8Y/Vpk0bXXHFFVEfR5MmTfTKK6/ou+++05AhQzR27FhNnjxZZqZWrVpJkjp27KinnnpKZ5xxhgYOHKj58+drwYIF6tOnT9DH7NmzpxYsWKAlS5boxBNP1KOPPqoHHnigyjGhfP8TJkxQixYtNGDAAKWnp+uzzz7T9ddfr0svvVSXX365Tj31VO3YsUO33HJL4/xwAABRFS91B/VhXs927UjJzMz0goKCoPuKiop03HHHRXlEqGzDhg066aSTVFBQoFNOOSXWw4kIfq8AIP7lLc7TzNUzK7ZzT83VjPNj/y4YZrbG3TOD7ePMFCRJr7zyipYuXart27dr+fLlysnJ0YknnqjBgwfHemgAgBSS1S9Lac0Dq8fSmqcpq1+MV4+FIKkvQEfovv76a02aNEk7d+5Up06dNHz4cD366KN1vucgAACRdLjuINZvEVMfTPMhZfB7BQBoKKb5AAAAGglhCgAAREWiVR6EijAFAAAaXSJWHoSKMAUAABrd0m1LVXIg8Ia5JQdKtHRbjN8wN4IIUwAAoNElYuVBqKhGAAAAjS4RKw9CxZmpBNa7d29Nnz49Js89cuRI5eTkxOS5AQCJKfuYbM04f0ZSBSmJMBUxZlbrRzjB495779XAgQO/d/vq1at10003hTHq6FmxYoXMTPv27Yv1UAAAiCim+SJkz549FV8vWrRI1157bZXbWrduHfHnTE9Pj/hjAgBQX/mb85Ny+i5UnJmKkCOPPLLio2PHjt+7beXKlTrllFPUqlUr9enTR5MnT1ZpaWnF/RcsWKBBgwapdevWOuKIIzRs2DB98cUXmj17tqZMmaLCwsKKs1yzZ8+W9P1pPjPTk08+qUsuuURt2rRR37599dxzz1UZ53vvvafBgwerVatWOvnkk7V48WKZmVasWFHj91ZSUqKcnBy1bdtWXbt21f333/+9Y5577jmdeuqpateunbp06aJLLrlEu3fvliTt2LFDZ555pqRAAKx8pm7JkiU644wz1KlTJx1xxBE655xzVFRUVN8fPwAgRpK58iBUSR+m4qEg7PXXX9cVV1yhvLw8FRYW6umnn9bLL7+sO+64Q5L0t7/9TaNHj9bYsWNVVFSklStX6sorr5QkXXbZZbrlllt0zDHHaM+ePdqzZ48uu+yyGp9r6tSpGjVqlDZs2KDLLrtMV199tT777DNJ0jfffKORI0fq2GOP1Zo1a/TQQw9p4sSJdY5/woQJeuONNzR//nwtW7ZM69at08qVK6scU1paqilTpmjDhg1atGiR9u3bpzFjxkiSevToofnz50uSCgsLtWfPHj322GOSpP3792v8+PF6//33tWLFCnXo0EEXXHBBlaAJAIhfyVx5EDJ3j8nHKaec4jXZuHFjjfvqY+GmhZ52X5rrXnnafWm+cNPCiDxuXV566SUP/GgDzjjjDJ86dWqVY1555RVv06aNl5WV+Zo1a1yS79ixI+jj3XPPPX788cd/7/ZevXr5ww8/XLEtyW+77baK7QMHDnjr1q392WefdXf3xx9/3Dt16uQlJSUVxzz//PMuyZcvXx70ub/++mtv0aKFP/fcc1Vu69Chg48dO7bGn0FRUZFL8p07d7q7+/Lly12SFxcX13gfd/dvvvnGmzRp4m+99VatxzVEpH6vAAD/Fqt/a6NNUoHXkGmS+sxUvKTlNWvW6L777lPbtm0rPi6//HLt379ff/vb33TiiSfq7LPP1sCBA3XxxRfr97//vYqLixv0XIMGDar4ulmzZkpPT9fevXslSZs2bdLAgQOrXL912mmn1fp427ZtU2lpqYYOHVpxW9u2bXXCCSdUOW7t2rUaNWqUevXqpXbt2ikzM/BekIfPitX2+Jdffrn69eun9u3bq2vXriorK6vzfgCA+HC48iD31FzNvXgu10wlm3gpCCsrK9M999yj9evXV3x88MEH+vjjj5Wenq6mTZtq6dKlWrp0qQYNGqSnnnpK/fv314YNG+r9XM2bN6+ybWYqKyuL1LcS1P79+3XOOecoLS1Nzz77rFavXq0lS5ZIUp3TdSNHjlRxcbGeeOIJvffee1q3bp2aNWvGNB8AJJBkrTwIVVKv5ouXgrDBgwdr06ZNOuqoo2o8xsw0dOhQDR06VHfffbeOP/54zZs3TyeeeKJatGihQ4cOhT2OY489VnPmzNG//vWvirNT77//fq336devn5o3b65Vq1apb9++kgLh6aOPPlK/fv0kBc547du3T/fff7/69OkjKXBBfWUtWrSQpCrfx5dffqlNmzbpd7/7XcUF6mvXrtXBgwfD/l4BAIiWpA5TUiBQxTop33333Ro5cqR69eqlSy+9VM2aNdNHH32k999/Xw899JBWrVql//7v/9Y555yjrl27at26ddq5c6cGDBggKbBq79NPP9XatWvVs2dPtWvXTi1btqz3OC6//HLdeeeduvbaa3XHHXfo888/r1iZZ2ZB79O2bVuNGzdOkyZNUnp6urp166apU6dWCUU9e/ZUy5YtNWPGDOXm5qqoqEh33XVXlcfp1auXzEyvvvqqLrjgArVu3VqdOnVS586d9Yc//EE9evTQ7t27NXHiRDVrlvS/lgAQ91K97qA+knqaL16cc845evXVV7V8+XINGTJEQ4YM0QMPPKCePXtKkjp06KB33nlHI0eOVP/+/XXLLbforrvu0s9+9jNJ0sUXX6zzzz9fZ511ltLT0zV37twGjaNdu3b6y1/+osLCQp188smaOHGi7r33XklSq1atarzf9OnTdeaZZ+qiiy7SmWeeqYEDB+rHP/5xxf709HTNmTNHf/7znzVgwABNmTJFjzzySJXHyMjI0JQpUzR58mR17dpVeXl5atKkiebNm6cPPvhAAwcOVG5urqZNm9agoAgAiBzqDurHAheoR19mZqYXFBQE3VdUVKTjjjsuyiNKTQsXLtRFF12kvXv3qnPnzrEeTqPi9woAQpO3OE8zV8+s2M49NVczzp8RwxHFnpmtcffMYPs4M5Vi5syZo7feeks7duzQokWLNH78eF1wwQVJH6QAAKGLlwVciYKLU1LMF198oXvuuUd79uzRkUceqREjRujBBx+M9bAAAHEkXhZwJQqm+ZAy+L0CADQU03wAACA55edLeXmBzzFCmAIAIIXEw3vWRkx+vjRmjDRzZuBzjAIVYQoAgBSRdJUHS5dKJYG3jVNJSWA7BghTAACkiHh5z9qIycqS0gKrDpWWFtiOAcIUAAApIukqD7KzpblzpdzcwOfs2Kw6pBoBAIAUkZSVB9nZMQtRh3FmKkGNHDlSOTk5FdvDhw9XXl5eWI+Zk5OjkSNHhjkyAEA8yz4mWzPOnxHfQSoOVujVB2emksSCBQvUvHnzkI5dsWKFzjzzTBUXF1dpPn/ssccUq94xAAAk/XuFXkmJNGtWTKfvQhXSmSkzO9fMNpvZVjO7Lcj+Xma2zMw+MLMVZtY98kNNPqWlpRF7rCOOOELt2rUL6zE6dOigjh07RmZAAICoSprKgzhZoVcfdYYpM2sqaaak8yQNkDTGzAZUO2y6pGfcfZCkqZL+b6QHmgiGDx+uG264QTfffLM6deqkTp06aeLEiSorK5Mk9e7dW/fee6+uvvpqdezYUVdccYUk6a9//auGDRumtLQ0ZWRk6MYbb9RXX31V8bglJSXKyclR27Zt1bVrV91///1Bn7vyNF9paanuuOMO9erVSy1btlTfvn3129/+Vjt27NCZZ54pSUpPT5eZVUwXVp/m++677zR+/Hh17dpVrVq10umnn6633367Yv+KFStkZlq2bJlOO+00paWlKTMzU2vXrq045p///KeuvPJKdenSRa1atVLfvn31m9/8JvwfNgCgQlJVHsTJCr36COXM1BBJW939E3cvlfSCpFHVjhkg6X/Kv14eZH/sRHne9fnnn1dZWZneffddPfHEE3ryySerhIdHHnlExx57rAoKCnT//ffrww8/VFZWlrKzs7VhwwYtWLBA69ev19VXX11xnwkTJuiNN97Q/PnztWzZMq1bt04rV66sdRxjx47VM888o0ceeURFRUV66qmn1LFjR/Xo0UPz58+XJBUWFmrPnj167LHHgj7Grbfeqnnz5unpp5/WunXrdMIJJ+jcc8/Vnj17qhx3++2364EHHtDatWv1gx/8QFdccUXFdOGdd96pDz/8UIsWLdLmzZv19NNPKyMjoyE/WgBADZKq8iBOVujVi7vX+iHpPyT9sdL2lZJmVDvmT5JuLv/6/0hyST8I8ljXSSqQVNCzZ0+vycaNG2vcVy8LF7qnpblLgc8LF0bmcWswbNgw79+/v5eVlVXcNm3aNM/IyHB39169evnIkSOr3OfKK6/0q6++uspt69atc0n+xRdf+Ndff+0tWrTw5557rmL/119/7R06dPCxY8dWee7c3Fx3d9+yZYtL8tdeey3oOJcvX+6SvLi4uMrtY8eO9REjRri7+zfffOPNmzf3OXPmVOw/ePCg9+3b1ydPnlzlcZYsWVJxzNtvv+2SfOfOne7ufsEFF/hVV11Vy08teiL2ewUAcWbhpoWedl+a61552n1pvnBT4/5712ALF7rn5jb6v8eNQVKB15CVIrWab4KkYWa2TtIwSbslHQoS3J5090x3z0xPT4/QU9ciBvOup59+usysYnvo0KHavXt3xbRdZmbV90hcs2aNnnvuObVt27bi40c/+pEkadu2bdq2bZtKS0s1dOjQivu0bdtWJ5xwQo1jWLdunZo0aVIxndcQ27Zt04EDByrGIklNmzbV0KFDtXHjxirHDho0qOLrbt26SZL27t0rSbrxxhs1b948nXjiiZowYYLefPPNBo8JABDc4cqD3FNzNffiufG5Ui9O3vqlMYSymm+3pB6VtruX31bB3T9X4IyUzKytpIvd/R8RGmPDZWUFVgKUlMTNvGubNm2qbJeVlemaa67RL3/5y+8dm5GRoS1btkRraCGrHBYlVVlFeHjf4evEzjvvPH366ad67bXXtGzZMo0YMUKXXHKJZs2aFb0BA0AKyD4mOz5D1GHBTnAkwhReCEI5M7VaUn8z62NmLSSNllQlTppZZzM7/Fi3S3o6ssNsoBjMu7733ntV6gVWrVqlbt26qX379kGPHzx4sAoLC3XUUUd976N169bq16+fmjdvrlWrVlXcZ//+/froo49qHMNJJ52ksrIyLV++POj+Fi1aSJIOHfreycMK/fr1U4sWLfTOO+9U3Hbo0CG9++67GjCg+vqD2nXu3FlXXnmlZs+eraeeekpz5szRd999V6/HAAAkuAS8sDxUdYYpdz8oKU/S65KKJL3o7oVmNtXMDqeT4ZI2m9kWSV0l3ddI462/7Gxpxoyopd/PP/9c48eP1+bNm/Xyyy/r4YcfDnrW6bBJkybp/fff1w033KB169Zp69atWrRoka6//npJgSm9cePGadKkSXrjjTdUWFioq6++utYgdPTRR+vSSy/VNddco/nz52v79u1666239Oyzz0qSevXqJTPTq6++quLiYn3zzTffe4w2bdroxhtv1KRJk7R48WIVFRXpxhtv1BdffKGbbrop5J/H3XffrT//+c/6+OOPVVRUpAULFqhv375q2bJlyI8BAKkqaeoOpMS8sDxEIZV2uvtiSYur3XZ3pa9flvRyZIeWmK644godOnRIp512msxM48aNqzVMDRo0SCtXrtSdd96pYcOG6dChQ+rbt68uuuiiimOmT5+u/fv366KLLlJaWpp+8YtfaP/+/bWO45lnntFdd92l//zP/9S+ffvUvXv3inFkZGRoypQpmjx5sq655hr9/Oc/1+zZs7/3GA8++KAk6aqrrtI//vEPnXzyyVqyZIl++MMfhvzzaNmypSZPnqzt27dX1Cv85S9/Cfn+AJCqDtcdlBwo0az1s+L3Wqj6iIO3fmkM5jFqvM7MzPSCgoKg+4qKinTcccdFeUThGz58uAYOHKgZM2bEeigIIlF/rwCkprzFeZq5embFdu6puZpxfpz++5KfH7gGKisrKcOSJJnZGnfPDLaP9+YDACAOZfXLUlrzwDVGac3TlNUvTq8xSuJVeqHivfkAAIhDh+sOlm5bqqx+WfE7xZfEq/RCRZiKoBUrVsR6CACAJBL3dQdSXNYQRRthCgAANNzhVXpJfs1UbeI2TJWVlalJEy7pQmQcLhEFgHiQvzk//qfv6iNJV+mFKi7TSps2bbR7926VlpYqVqsNkRzcXaWlpdq9e/f32ucBIBYOVx7MXD1TY+aPid8Oqfx8KS8vJS8or6+4PDPVvXt37du3T59++qkOHjwY6+EgwTVr1kwdOnRQ586dYz0UANDSbUtVciBwwXbJgRIt3bY0/s5OHV6hV1ISuB4qyUo2Iy0uw1STJk3UpUsXdenSJdZDAQAgorL6ZWnW+lkqOVASv5UHrNCrl7ic5gMAIFkdrjzIPTU3flvNk/h99BpDXDagAwCAGEuBVvP6qK0BPS6n+QAAQIyl+Aq9+mCaDwCACMnfnK+8xXnxu0JPYpVeIyBMAQAQAQlRecD76DUKwhQAABEQrPIg7gRbpYewEaYAAIiArH5ZSmseWAEXt5UHrNJrFKzmAwAgQhLibWJYpdcgta3mI0wBAADUobYwxTQfAADJgFV6MUOYAgCgFgmRUVilF1OEKQAAapAwGYVVejFFmAIAoAYJk1FYpRdThCkAAGqQMBklO1uaO1fKzQ18ZpVeVLGaDwCAWtAkAIk3OgYAoMFi+n6/JLmEwDQfAADxKGGufgdhCgCQkuK+8iBhrn4HYQoAkHIS4qRPwlz9DsIUACDlJMRJH1boJQzCFAAg5cT8pE+oc4zZ2dKMGQSpOMdqPgBAyjl80icmC+UOzzGWlEizZnHWKQkQpgAAKSlmlQfB5hgJUwmNaT4AAKIp5nOMiDTOTAEAkkZCdFzGdI4RjYG3kwEAJIXKlyKlpXEpEiKrtreTYZoPAJAU4qLuIO6bQNEYCFMAgKQQ80uREqIJFI2BMAUASAox77iMi1NjiAUuQAcAJI2Y1R1IgVNhs2b9+6ItVumlDMIUAACRwCq9lEWYAgDEvYSoPJBifGoMsRLSNVNmdq6ZbTazrWZ2W5D9Pc1suZmtM7MPzOz8yA8VAJCKuK4b8a7OMGVmTSXNlHSepAGSxpjZgGqH3SnpRXc/WdJoSb+L9EABAKmJ67oR70I5MzVE0lZ3/8TdSyW9IGlUtWNcUvvyrztI+jxyQwQApLKYVx4AdQjlmqkMSTsrbe+SdFq1Y+6VtNTMfiGpjaSzgz2QmV0n6TpJ6tmzZ33HCgBIQVzXjXgXqQvQx0ia7e6/NrOhkp41s4HuXlb5IHd/UtKTUuDtZCL03ACAJMd13YhnoUzz7ZbUo9J29/LbKhsn6UVJcvd3JbWS1DkSAwQAAIhnoYSp1ZL6m1kfM2uhwAXm1ddSfCbpLEkys+MUCFPFkRwoACD58FZ2SAZ1hil3PygpT9LrkooUWLVXaGZTzezwSddbJF1rZhskzZWU4+5M4wEAakTlAZJFSNdMuftiSYur3XZ3pa83SvpRZIcGAEhmwSoPuC4KiYg3OgYAxASVB0gWvJ0MACAmqDxAsiBMAQBihsoDJAOm+QAAAMJAmAIARBR1B0g1hCkAQMRQd4BURJgCAERMsLoDINkRpgAAEUPdAVIRq/kAABFD3QFSEWEKABBR1B0g1TDNBwAAEAbCFAAAQBgIUwCAkNAfBQRHmAIA1In+KKBmhCkAQJ3ojwJqRpgCANSJ/iigZlQjAADqRH8UUDPCFAAgJPRHAcExzQcAABAGwhQApDgqD4DwEKYAIIVReQCEjzAFACmMygMgfIQpAEhhVB4A4WM1HwCkMCoPgPARpgAgxVF5AISHaT4AAIAwEKYAIAlRdwBED2EKAJIMdQdAdBGmACDJUHcARBdhCgCSDHUHQHSxmg8Akgx1B0B0EaYAIAlRdwBED9N8AAAAYSBMAUACofIAiD+EKQBIEFQeAPGJMAUACYLKAyA+EaYAIEFQeQDEJ1bzAUCCoPIAiE+EKQBIIFQeAPGHaT4AAIAwEKYAIA5QeQAkLsIUAMQYlQdAYgspTJnZuWa22cy2mtltQfY/ambryz+2mNk/Ij5SAEhSVB4Aia3OMGVmTSXNlHSepAGSxpjZgMrHuPsv3f0kdz9J0v+TtKARxgoASYnKAyCxhbKab4ikre7+iSSZ2QuSRknaWMPxYyTdE5nhAUDyo/IASGyhhKkMSTsrbe+SdFqwA82sl6Q+kv6nhv3XSbpOknr27FmvgQJAMqPyAEhckb4AfbSkl939ULCd7v6ku2e6e2Z6enqEnxoAACD6QglTuyX1qLTdvfy2YEZLmhvuoAAgGVB3AKSGUMLUakn9zayPmbVQIDB9768GMztWUidJ70Z2iACQeKg7AFJHnWHK3Q9KypP0uqQiSS+6e6GZTTWzyjP8oyW94O7eOEMFgMRB3QGQOkJ6bz53XyxpcbXb7q62fW/khgUAiS0rS5o1KxCkqDsAkhtvdAwAjYC6AyB1EKYAoJFQdwCkBt6bDwAAIAyEKQCoJyoPAFRGmAKAeqDyAEB1hCkAqAcqDwBUR5gCgHrIygpUHUhUHgAIYDUfANQDlQcAqiNMAUA9UXkAoDKm+QAAAMJAmAKAclQeAGgIwhQAiMoDAA1HmAIAUXkAoOEIUwAgKg8ANByr+QBAVB4AaDjCFACUo/IAQEMwzQcAABAGwhSApEbdAYDGRpgCkLSoOwAQDYQpAEmLugMA0UCYApC0qDsAEA2s5gOQtKg7ABANhCkASY26AwCNjWk+AACAMBCmACQkKg8AxAvCFICEQ+UBgHhCmAKQcKg8ABBPCFMAEg6VBwDiCav5ACQcKg8AxBPCFICEROUBgHjBNB8AAEAYCFMA4gqVBwASDWEKQNyg8gBAIiJMAYgbVB4ASESEKQBxg8oDAImI1XwA4gaVBwASEWEKQFyh8gBAomGaDwAAIAyEKQCNjroDAMmMMAWgUVF3ACDZEaYANCrqDgAku5DClJmda2abzWyrmd1WwzGXmtlGMys0sz9FdpgAEhV1BwCSXZ2r+cysqaSZkn4qaZek1WaW7+4bKx3TX9Ltkn7k7n83sy6NNWAAiYW6AwDJLpRqhCGStrr7J5JkZi9IGiVpY6VjrpU0093/LknuvjfSAwWQuKg7AJDMQpnmy5C0s9L2rvLbKjta0tFm9o6ZrTKzc4M9kJldZ2YFZlZQXFzcsBEDAADEkUhdgN5MUn9JwyWNkfQHM+tY/SB3f9LdM909Mz09PUJPDSBWqDwAgNDC1G5JPSptdy+/rbJdkvLd/YC7b5e0RYFwBSBJUXkAAAGhhKnVkvqbWR8zayFptKTqf23+WYGzUjKzzgpM+30SuWECiDdUHgBAQJ1hyt0PSsqT9LqkIkkvunuhmU01s8OXlL4u6Usz2yhpuaSJ7v5lYw0aQOxReQAAAebuMXnizMxMLygoiMlzA4iM/HwqDwCkBjNb4+6ZwfaFUo0AAEFReQAAvJ0MAABAWAhTAKqg7gAA6ocwBaACdQcAUH+EKQAVqDsAgPojTAGoQN0BANQfq/kAVMjOlubOpe4AAOqDMAWgCuoOAKB+mOYDAAAIA2EKSBFUHgBA4yBMASmAygMAaDyEKSAFUHkAAI2HMAWkACoPAKDxsJoPSAFUHgBA4yFMASmCygMAaBxM8wEAAISBMAUkOCoPACC2CFNAAqPyAABijzAFJDAqDwAg9ghTQAKj8gAAYo/VfEACo/IAAGKPMAUkOCoPACC2mOYDAAAIA2EKiEPUHQBA4iBMAXGGugMASCyEKSDOUHcAAImFMAXEGeoOACCxsJoPiDPUHQBAYiFMAXGIugMASBxM8wEAAISBMAUAABAGwhQQRfRHAUDyIUwBUUJ/FAAkJ8IUECX0RwFAciJMAVFCfxQAJCeqEYAooT8KAJITYQqIIvqjACD5MM0HAAAQBsIUEAFUHgBA6iJMAWGi8gAAUhthCggTlQcAkNoIU0CYqDwAgNQWUpgys3PNbLOZbTWz24LszzGzYjNbX/5xTeSHCsSnw5UHubmBz6zWA4DUUmc1gpk1lTRT0k8l7ZK02szy3X1jtUPnuXteI4wRiHtUHgBA6grlzNQQSVvd/RN3L5X0gqRRjTssAACAxBBKmMqQtLPS9q7y26q72Mw+MLOXzaxHsAcys+vMrMDMCoqLixswXCB6qDsAAIQiUheg/0VSb3cfJOkNSXOCHeTuT7p7prtnpqenR+ipgcij7gAAEKpQwtRuSZXPNHUvv62Cu3/p7t+Vb/5R0imRGR4QG9QdAABCFUqYWi2pv5n1MbMWkkZLqvL/dDP7YaXNbElFkRsiEH3UHQAAQlXnaj53P2hmeZJel9RU0tPuXmhmUyUVuHu+pP80s2xJByX9r6ScRhwz0OgO1x0sXRoIUqzUAwDUxNw9Jk+cmZnpBQUFMXluAACA+jCzNe6eGWwfDegAAABhIEwh5VB5AACIJMIUUgqVBwCASCNMIaVQeQAAiDTCFFIKlQcAgEirsxoBSCZUHgAAIo0whZSTnU2IAgBEDtN8AAAAYSBMIWlQeQAAiAXCFJIClQcAgFghTCEpUHkAAIgVwhSSApUHAIBYYTUfkgKVBwCAWCFMIWlQeQAAiAWm+QAAAMJAmEJco+4AABDvCFOIW9QdAAASAWEKcYu6AwBAIiBMIW5RdwAASASs5kPcou4AAJAICFOIa9QdAADiHdN8AAAAYSBMISaoPAAAJAvCFKKOygMAQDIhTCHqqDwAACQTwhSijsoDAEAyYTUfoo7KAwBAMiFMISaoPAAAJAum+QAAAMJAmEJEUXkAAEg1hClEDJUHAIBURJhCxFB5AABIRYQpRAyVBwCAVMRqPkQMlQcAgFREmEJEUXkAAEg1TPMBAACEgTCFOlF3AABAzQhTqBV1BwAA1I4whVpRdwAAQO0IU6gVdQcAANSO1XyoFXUHAADULqQzU2Z2rpltNrOtZnZbLcddbGZuZpmRGyJiLTtbmjGDIAUAQDB1hikzaypppqTzJA2QNMbMBgQ5rp2kmyW9F+lBAgAAxKtQzkwNkbTV3T9x91JJL0gaFeS4aZIelPRtBMeHRkTlAQAA4QslTGVI2llpe1f5bRXMbLCkHu7+am0PZGbXmVmBmRUUFxfXe7CIHCoPAACIjLBX85lZE0mPSLqlrmPd/Ul3z3T3zPT09HCfGmGg8gAAgMgIJUztltSj0nb38tsOaydpoKQVZrZD0umS8rkIPb5ReQAAQGSEUo2wWlJ/M+ujQIgaLenywzvd/Z+SOh/eNrMVkia4e0Fkh4pIovIAAIDIqDNMuftBM8uT9LqkppKedvdCM5sqqcDdudomQWVnE6IAAAhXSKWd7r5Y0uJqt91dw7HDwx8WAABAYuDtZJIQlQcAAEQPYSrJUHkAAEB0EaaSDJUHAABEF2EqyVB5AABAdIV0AToSB5UHAABEF2EqCVF5AABA9DDNBwAAEAbCVIKg7gAAgPhEmEoA1B0AABC/CFMJgLoDAADiF2EqAVB3AABA/GI1XwKg7gAAgPhFmEoQ1B0AABCfmOYDAAAIA2Eqxqg8AAAgsRGmYojKAwAAEh9hKoaoPAAAIPERpmKIygMAABIfq/liiMoDAAASH2Eqxqg8AAAgsTHNBwAAEAbCVCOg7gAAgNRBmIow6g4AAEgthKkIo+4AAIDUQpiKMOoOAABILazmizDqDgAASC2EqUZA3QEAAKmDaT4AAIAwEKbqgcoDAABQHWEqRFQeAACAYAhTIaLyAAAABEOYChGVBwAAIBhW84WIygMAABAMYaoeqDwAAADVMc0HAAAQBsKUqDwAAAANl/JhisoDAAAQjpQPU1QeAACAcKR8mKLyAAAAhCPlV/NReQAAAMKR8mFKovIAAAA0XEjTfGZ2rpltNrOtZnZbkP03mNmHZrbezN42swGRHyoAAED8qTNMmVlTSTMlnSdpgKQxQcLSn9z9BHc/SdJDkh6J9EABAADiUShnpoZI2urun7h7qaQXJI2qfIC7f1Vps40kj9wQG4buKAAAEA2hXDOVIWlnpe1dkk6rfpCZ5Ur6laQWkn4SkdE10OHuqJISadaswAXmXBMFAAAaQ8SqEdx9prv3kzRJ0p3BjjGz68yswMwKiouLI/XU30N3FAAAiJZQwtRuST0qbXcvv60mL0i6MNgOd3/S3TPdPTM9PT3kQdYX3VEAACBaQpnmWy2pv5n1USBEjZZ0eeUDzKy/u39cvjlC0seKIbqjAABAtNQZptz9oJnlSXpdUlNJT7t7oZlNlVTg7vmS8szsbEkHJP1d0tjGHHQo6I4CAADREFJpp7svlrS42m13V/r65giPCwAAICGk/HvzAQAAhIMwBQAAEAbCFAAAQBgIUwAAAGEgTAEAAISBMAUAABAGwhQAAEAYCFMAAABhIEwBAACEgTAFAAAQBsIUAABAGAhTAAAAYTB3j80TmxVL+rSRn6azpH2N/BxoOF6f+MVrE994feIbr0/8Cue16eXu6cF2xCxMRYOZFbh7ZqzHgeB4feIXr0184/WJb7w+8auxXhum+QAAAMJAmAIAAAhDsoepJ2M9ANSK1yd+8drEN16f+MbrE78a5bVJ6mumAAAAGluyn5kCAABoVIQpAACAMCRFmDKzc81ss5ltNbPbguxvaWbzyve/Z2a9YzDMlBXC6/MrM9toZh+Y2TIz6xWLcaaiul6bSsddbGZuZiz3jqJQXh8zu7T8z0+hmf0p2mNMVSH8vdbTzJab2bryv9vOj8U4U5GZPW1me83soxr2m5n9tvy1+8DMBof7nAkfpsysqaSZks6TNEDSGDMbUO2wcZL+7u5HSXpU0oPRHWXqCvH1WScp090HSXpZ0kPRHWVqCvG1kZm1k3SzpPeiO8LUFsrrY2b9Jd0u6Ufufryk8dEeZyoK8c/OnZJedPeTJY2W9LvojjKlzZZ0bi37z5PUv/zjOkm/D/cJEz5MSRoiaau7f+LupZJekDSq2jGjJM0p//plSWeZmUVxjKmsztfH3Ze7e0n55ipJ3aM8xlQVyp8dSZqmwH9Avo3m4BDS63OtpJnu/ndJcve9UR5jqgrltXFJ7cu/7iDp8yiOL6W5+0pJ/1vLIaMkPeMBqyR1NLMfhvOcyRCmMiTtrLS9q/y2oMe4+0FJ/5T0g6iMDqG8PpWNk/Rao44Ih9X52pSf/u7h7q9Gc2CQFNqfnaMlHW1m75jZKjOr7X/jiJxQXpt7Jf3MzHZJWizpF9EZGkJQ33+X6tQsrOEAEWRmP5OUKWlYrMcCycyaSHpEUk6Mh4KaNVNgqmK4Amd0V5rZCe7+j1gOCpKkMZJmu/uvzWyopGfNbKC7l8V6YIi8ZDgztVtSj0rb3ctvC3qMmTVT4JTrl1EZHUJ5fWRmZ0uaLCnb3b+L0thSXV2vTTtJAyWtMLMdkk6XlM9F6FETyp+dXZLy3f2Au2+XtEWBcIXGFcprM07Si5Lk7u9KaqXAm+wi9kL6d6k+kiFMrZbU38z6mFkLBS70y692TL6kseVf/4ek/3HaSqOlztfHzE6W9IQCQYprPqKn1tfG3f/p7p3dvbe791bgerZsdy+IzXBTTih/t/1ZgbNSMrPOCkz7fRLFMaaqUF6bzySdJUlmdpwCYao4qqNETfIl/bx8Vd/pkv7p7nvCecCEn+Zz94NmlifpdUlNJT3t7oVmNlVSgbvnS3pKgVOsWxW4KG107EacWkJ8fR6W1FbSS+XrAj5z9+yYDTpFhPjaIEZCfH1el5RlZhslHZI00d05697IQnxtbpH0BzP7pQIXo+fwn/joMLO5Cvwno3P5NWv3SGouSe7+uALXsJ0vaaukEklXhf2cvLYAAAANlwzTfAAAADFDmAIAAAgDYQoAACAMhCkAAIAwEKYAAADCQJgCAAAIA2EKAAAgDP8fyu08rex0z9YAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x504 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# predictions = inference\n",
    "with torch.inference_mode():\n",
    "    y_preds = model_0(X_test.to('cuda'))\n",
    "plot_predictions(predictions=y_preds.cpu())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "b70779ab-3dc1-4a8c-8d3c-2ba36fb91cd4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('weights', tensor([0.5784], device='cuda:0')),\n",
       "             ('bias', tensor([0.3513], device='cuda:0'))])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_0.state_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "b97ff918-88f2-4b60-bd1c-ea7f0f5eb7e4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.7, 0.3)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weight,bias"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "b21fa2c9-8f97-4ce4-a81c-b4d119a889a9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7fc2ec6c9490>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxvklEQVR4nO3deXhU5dnH8e+dyUoW1rAkQQOKCwQSEEGxyOIOKqKiKFZt9bXYIq4orlVcWqtVxNqibdVW24KiKIrivoCgssgqIIsBwhq2JBBCtvv945wJE0ggQCZnkrk/1zVXZs6Z5c4w5DfPcp4jqooxxpjwFeF1AcYYY7xlQWCMMWHOgsAYY8KcBYExxoQ5CwJjjAlzFgTGGBPmLAiMMSbMWRCYek1EskXkbI9eu4eIfCAiO0Vku4h8LyK/8qIWY46GBYExR0BETgc+B74CjgeaAzcDFxzh8/lqrzpjDo8FgWmQRCRGRMaKyAb3MlZEYtx9LUTk/YBv8tNFJMLdd4+IrBeRAhFZLiJnVfMSTwH/UtUnVXWrOuaq6hXu81wvIjP2q0lF5Hj3+qsi8je3RbEbuEtENgUGgogMFpGF7vUIERktIqtEZJuIvCEizdx9sSLyurt9p4jMFpFWtfyWmgbMgsA0VPcDpwFZQCbQA3jA3XcnkAMkA62A+wAVkROBEcCpqpoInAdk7//EItIIOB2YdJQ1Xg08DiQCzwG7gf777f+ve/0W4BKgD5AC7ABecPddBzQG2uK0TIYDe46yNhNGLAhMQzUMGKOqW1Q1F3gE+KW7rwRoAxyrqiWqOl2dRbfKgBigo4hEqWq2qq6q4rmb4vzf2XiUNb6rqt+oarmqFgH/A64CEJFEYIC7DZw/7verao6q7gUeBi4XkUj392kOHK+qZW7LJP8oazNhxILANFQpwJqA22vcbeB066wEPhaR1SIyGkBVVwK34fyR3SIiE0QkhQPtAMpxwuRorNvv9n+BS90urEuBearq/x2OBSa7XT87gaU4wdUKeA34CJjgdoP9SUSijrI2E0YsCExDtQHnj6ffMe42VLVAVe9U1fbAxcAd/rEAVf2vqv7CfawCT+7/xKpaCMwCLjvI6+8GGvlviEjrKu5TaelfVf0RJ7AuoHK3EDihcYGqNgm4xKrqerdV84iqdgR6ARcC1x6kNmMqsSAwDUGUO2Dqv0TidKk8ICLJItICeAh4HUBELhSR40VEgDycb9blInKiiPR3v5EX4fSzl1fzmncD14vIKBFp7j5vpohMcPcvADqJSJaIxOK0Mmriv8CtwJnAmwHbxwOPi8ix7msli8gg93o/EensDjTn43QVVVe3MQewIDANwQc4f7T9l4eBx4A5wEJgETDP3QbQAfgU2IXzzf6vqvoFzvjAH4GtwCagJXBvVS+oqjNxBnb7A6tFZDvwklsLqvoTMMZ9nRXAjKqepwr/wxkQ/lxVtwZsfw6YgtOdVQB8C/R097XGGbjOx+ky+gqnu8iYGhE7MY0xxoQ3axEYY0yYsyAwxpgwZ0FgjDFhLqhBICLnu4fpr/TP1d5v//Uikisi893LjcGsxxhjzIEig/XE7lS2F4BzcA7nny0iU9y50oEmquqImj5vixYtND09vfYKNcaYMDB37tytqppc1b6gBQHO2i4rVXU1gDu/ehCwfxAclvT0dObMmVML5RljTPgQkTXV7Qtm11AqlQ+hz3G37e8yEVkoIpNEpG1VTyQiN4nIHBGZk5ubG4xajTEmbHk9WPwekK6qXYBPgH9VdSdVfUlVu6tq9+TkKls2xhhjjlAwg2A9zrK4fmnutgqqus1dSRHgH8ApQazHGGNMFYI5RjAb6CAi7XACYCjOQloVRKSNqvqX8r0Y5/B4Y0wYKSkpIScnh6KiIq9LaRBiY2NJS0sjKqrmC9AGLQhUtVRERuAsj+sDXlbVJSIyBpijqlOAkSJyMVAKbAeuD1Y9xpjQlJOTQ2JiIunp6TjrAJojpaps27aNnJwc2rVrV+PHBbNFgKp+gLsIV8C2hwKu30s1i3oZY8JDUVGRhUAtERGaN2/O4U6q8Xqw2BhjLARq0ZG8l+ETBJsWwye/B1tt1RhjKgmfIMieAd+MheUfHPKuxpjwsW3bNrKyssjKyqJ169akpqZW3C4uLj7oY+fMmcPIkSMP6/XS09PZunXroe9Yh4I6RhBSTr0B5rwMH90Px58NkTFeV2SMCQHNmzdn/vz5ADz88MMkJCRw1113VewvLS0lMrLqP5Xdu3ene/fudVFmUIVPi8AXBec/ATt+hu/Ge12NMSaEXX/99QwfPpyePXty99138/3333P66afTtWtXevXqxfLlywH48ssvufDCCwEnRH7961/Tt29f2rdvz7hx42r8etnZ2fTv358uXbpw1llnsXbtWgDefPNNMjIyyMzM5MwzzwRgyZIl9OjRg6ysLLp06cKKFSuO+vcNnxYBOC2BDufBV09B5lWQ0NLriowxAR55bwk/bsiv1efsmJLE7y/qdNiPy8nJYebMmfh8PvLz85k+fTqRkZF8+umn3Hfffbz11lsHPGbZsmV88cUXFBQUcOKJJ3LzzTfXaD7/LbfcwnXXXcd1113Hyy+/zMiRI3nnnXcYM2YMH330EampqezcuROA8ePHc+uttzJs2DCKi4spKys77N9tf+HTIvA77wko3QOfjfG6EmNMCBsyZAg+nw+AvLw8hgwZQkZGBrfffjtLliyp8jEDBw4kJiaGFi1a0LJlSzZv3lyj15o1axZXX+0cb/vLX/6SGTOcU1yfccYZXH/99fz973+v+IN/+umn88QTT/Dkk0+yZs0a4uLijvZXDbMWAUCL46HncJj1Apx6I6RkeV2RMcZ1JN/cgyU+Pr7i+oMPPki/fv2YPHky2dnZ9O3bt8rHxMTsG3v0+XyUlpYeVQ3jx4/nu+++Y+rUqZxyyinMnTuXq6++mp49ezJ16lQGDBjAiy++SP/+/Y/qdcKvRQBw5iho1AymjbbppMaYQ8rLyyM11Vk8+dVXX6315+/VqxcTJkwA4D//+Q+9e/cGYNWqVfTs2ZMxY8aQnJzMunXrWL16Ne3bt2fkyJEMGjSIhQsXHvXrh2cQxDWB/g/C2lmwZLLX1RhjQtzdd9/NvffeS9euXY/6Wz5Aly5dSEtLIy0tjTvuuIPnn3+eV155hS5duvDaa6/x3HPPATBq1Cg6d+5MRkYGvXr1IjMzkzfeeIOMjAyysrJYvHgx11577VHXI1rPvhF3795da+XENOVl8GIfKNoJI2ZD1NH3sxljDt/SpUs5+eSTvS6jQanqPRWRuapa5VzX8GwRAET44Pw/QN46mPm819UYY4xnwjcIANr1hpMvhhnPQv4Gr6sxxhhPhHcQAJz7qNNN9OnDXldijDGesCBomg69RsDCibButtfVGGNMnbMgAPjFHZDQGqbdA+XlXldjjDF1yoIAICYBzv49rJ8Li97wuhpjjKlTFgR+XYZCSjfnnAV7d3ldjTGmjhzNMtTgLDw3c+bMKve9+uqrjBgxorZLrnUWBH4REXDBk7BrkzOLyBgTFvzLUM+fP5/hw4dz++23V9yOjo4+5OMPFgT1hQVBoLY9oPMQ57iCHWu8rsYY45G5c+fSp08fTjnlFM477zw2btwIwLhx4+jYsSNdunRh6NChZGdnM378eJ599lmysrKYPn16jZ7/mWeeISMjg4yMDMaOHQvA7t27GThwIJmZmWRkZDBx4kQARo8eXfGagedJqE3ht+jcoZz9CCybCp88CFf82+tqjAkvH46GTYtq9zlbd4YL/ljju6sqt9xyC++++y7JyclMnDiR+++/n5dffpk//vGP/Pzzz8TExLBz506aNGnC8OHDDziZzcHMnTuXV155he+++w5VpWfPnvTp04fVq1eTkpLC1KlTAWd9o23btjF58mSWLVuGiFQsRV3brEWwv8apcMZt8OO7zuktjTFhZe/evSxevJhzzjmHrKwsHnvsMXJycgBnjaBhw4bx+uuvV3vWskOZMWMGgwcPJj4+noSEBC699FKmT59O586d+eSTT7jnnnuYPn06jRs3pnHjxsTGxnLDDTfw9ttv06hRo9r8VStYi6AqvW6Bef92Vie96StnOQpjTPAdxjf3YFFVOnXqxKxZsw7YN3XqVL7++mvee+89Hn/8cRYtqr3WywknnMC8efP44IMPeOCBBzjrrLN46KGH+P777/nss8+YNGkSf/nLX/j8889r7TX9rEVQlehGcO4Yp4n6w2teV2OMqUMxMTHk5uZWBEFJSQlLliyhvLycdevW0a9fP5588kny8vLYtWsXiYmJFBQU1Pj5e/fuzTvvvENhYSG7d+9m8uTJ9O7dmw0bNtCoUSOuueYaRo0axbx589i1axd5eXkMGDCAZ599lgULFgTld7YWQXU6XQrf/x0+exQ6DYbYxl5XZIypAxEREUyaNImRI0eSl5dHaWkpt912GyeccALXXHMNeXl5qCojR46kSZMmXHTRRVx++eW8++67PP/88xXnEvB79dVXeeeddypuf/vtt1x//fX06NEDgBtvvJGuXbvy0UcfMWrUKCIiIoiKiuJvf/sbBQUFDBo0iKKiIlSVZ555Jii/c/guQ10TG36Al/o5S1Cc+1jdvKYxYcaWoa59tgx1bUrpCl2HwbfjYdsqr6sxxpigsCA4lP4PQWQsfHS/15UYY0xQWBAcSmIrOPNO+OlDWPmZ19UY0yDVty7qUHYk76UFQU2c9lto2g4+ug/Kjv58pcaYfWJjY9m2bZuFQS1QVbZt20ZsbOxhPc5mDdVEZIwzWDxxGMx5GXre5HVFxjQYaWlp5OTkkJub63UpDUJsbCxpaWmH9RgLgpo6aSC0OxO+eBw6Xw6NmnldkTENQlRUFO3atfO6jLBmXUM1JQLn/xH25sOXf/C6GmOMqTUWBIejVSc45Vcw+5+wZanX1RhjTK0IahCIyPkislxEVorI6IPc7zIRURGp8mCHkNLvfueMZtPuBRvcMsY0AEELAhHxAS8AFwAdgatEpGMV90sEbgW+C1YttSq+OfS9F1Z/AT9N87oaY4w5asFsEfQAVqrqalUtBiYAg6q436PAk0BREGupXafeCC1OcA4yKz30qeyMMSaUBTMIUoF1Abdz3G0VRKQb0FZVpwaxjtrni4LznoDtq+D7F72uxhhjjopng8UiEgE8A9xZg/veJCJzRGROyMw17nAOdDgXvvoT7AqRmowx5ggEMwjWA20Dbqe52/wSgQzgSxHJBk4DplQ1YKyqL6lqd1XtnpycHMSSD9N5T0BJIXz+qNeVGGPMEQtmEMwGOohIOxGJBoYCU/w7VTVPVVuoarqqpgPfAherah2tMV0LWnSAHjc5ZzPbuNDraowx5ogELQhUtRQYAXwELAXeUNUlIjJGRC4O1uvWuT53O0cZTxtt00mNMfVSUJeYUNUPgA/22/ZQNfftG8xagiauqXNswdQ7nBPed7rE64qMMeaw2JHFtaHbddCyE3zyIJTs8boaY4w5LBYEtcEXCef/AXauhVl/8boaY4w5LBYEtaV9HzjpQpj+LORv9LoaY4ypMQuC2nTuY1BeAp894nUlxhhTYxYEtalZOzj9d7Dgf5BTf2bBGmPCmwVBbet9JyS0sumkxph6w4KgtsUkwlkPQc5sWPSm19UYY8whWRAEQ+bV0CYLPvk9FO/2uhpjjDkoC4JgiIiAC56Egg0wY6zX1RhjzEFZEATLMadBxmUwc5xzfIExxoQoC4JgOvsRQOCTKlfVMMaYkGBBEExN2sIZt8KSybBmptfVGGNMlSwIgu2MWyEpFT68B8rLvK7GGGMOYEEQbNGN4JwxsGkhzP+P19UYY8wBLAjqQsZl0LYnfDYGivK9rsYYYyqxIKgLIs7qpLtzYfrTXldjjDGVWBDUldRTIGsYzPorbFvldTXGGFPBgqAunfUQRMbAxw94XYkxxlSwIKhLia3hzLtg+Qfw08deV2OMMYAFQd077XfQvAN8eDeUFHldjTHGWBDUuchoGPAn2PEzzHze62qMMcaCwBPH9YeOlzgziHas8boaY0yYsyDwynlPgPhg2r1eV2KMCXMWBF5pnAp9RsHyqfDTR15XY4wJYxYEXrKBY2NMCLAg8FJkNAx4CnZkO+ctMMYYD1gQeO24ftBpMEz/sxMIxhhTxywIQsG5j9vAsTHGMxYEoaBxKvS52z3i2AaOjTF1y4IgVJz2W2hxgg0cG2PqnAVBqAgcOP7mOa+rMcaEkbAJgrJyZVFOntdlHFz7vtDpUpjxDGz/2etqjDFhImyC4LlPf+Ky8TNZuaXA61IO7tzHbODYGFOnwiYIfnl6OvHRPu58cyGlZeVel1O9xqnQ9x746UNYPs3raowxYSBsgiA5MYZHL8lgwbqdvPj1aq/LObieN0OLE92B4z1eV2OMaeCCGgQicr6ILBeRlSIyuor9w0VkkYjMF5EZItIxmPVc2CWFgZ3bMPbTn1i2KYRPIu8fON65xgaOjTFBF7QgEBEf8AJwAdARuKqKP/T/VdXOqpoF/Al4Jlj1+I0Z1Imk2CjuenMBJaHcRdS+jzNwPN0Gjo0xwRXMFkEPYKWqrlbVYmACMCjwDqoa+LU8HtAg1gNA84QYHh/cmcXr8/nrFyF+EvnzHoeISJh2QGPKGGNqTTCDIBVYF3A7x91WiYj8TkRW4bQIRlb1RCJyk4jMEZE5ubm5R13Y+RmtGZSVwvOfr2DJhhCeUpqUAn1Hw0/TYPmHXldjjGmgPB8sVtUXVPU44B7ggWru85KqdlfV7snJybXyug9f1Imm8dHc+cYCiktDuIvotJsh+ST48B4bODbGBEUwg2A90Dbgdpq7rToTgEuCWE8lTeOj+cPgzizbVMBfPl9RVy97+HxR+waOZ4z1uhpjTAMUzCCYDXQQkXYiEg0MBaYE3kFEOgTcHAjU6V/kszu24rJuabzw5SoW5uysy5c+PO3OhIzLYMazsD3Ep74aY+qdoAWBqpYCI4CPgKXAG6q6RETGiMjF7t1GiMgSEZkP3AFcF6x6qvPQRR1pkeB0Ee0tLavrl6+5cx9zWgcf3gMa9DF1Y0wYCeoYgap+oKonqOpxqvq4u+0hVZ3iXr9VVTupapaq9lPVJcGspyqN46J48rIurNiyi7GfhnAXkX/geMXHNnBsjKlVng8Wh4K+J7Zk6KltefGrVfywdofX5VSv53Bn4HiaDRwbY2qPBYHr/oEn0zopljvfXEBRSYh2EfmiYMDTsHOtM15gjDG1wILAlRgbxZ8uz2R17m7+/PFyr8upXrvekHG5M4NoW4gfEGeMqRcsCAL8okMLhvU8hn/M+Jk52du9Lqd6NnBsjKlFFgT7uW/AyaQ2ieOuNxewpzhEu4iS2kDfe2HlJ855jo0x5ihYEOwnPiaSpy7PJHtbIU9OW+Z1OdXr+RtIPhk+HA3FhV5XY4ypx2oUBCISLyIR7vUTRORiEYkKbmneOf245lzfK51XZ2bz7eptXpdTNV8UDHwa8mzg2BhzdGraIvgaiBWRVOBj4JfAq8EqKhTcff6JpDdvxKhJC9i9t9TrcqqW/gvoPAS+GWsDx8aYI1bTIBBVLQQuBf6qqkOATsEry3uNoiN5akgmOTv28IcPl3pdTvXOeRR8MTZwbIw5YjUOAhE5HRgGTHW3+YJTUug4Nb0ZN5zRjte/XcuMFVu9LqdqSW2gnw0cG2OOXE2D4DbgXmCyu15Qe+CLoFUVQu4670TaJ8dzz1sLKSgq8bqcqvW4CVp2tIFjY8wRqVEQqOpXqnqxqj7pDhpvVdUqTyLT0MRG+Xh6SCYb8/bwxAch2kXkP+I4by3MCPrZPo0xDUxNZw39V0SSRCQeWAz8KCKjglta6Oh2TFNuOvM4/vf9Or5cvsXrcqqWfgZ0vsI52b0NHBtjDkNNu4Y6uucXvgT4EGiHM3MobNx2dgc6tExg9FuLyNsTol1E5/oHju+2gWNjTI3VNAii3OMGLgGmqGoJdXCi+VASG+Xjz1dkkrtrL4++/6PX5VQtsTX0uw9WfgrLph76/sYYQ82D4EUgG4gHvhaRY4H8YBUVqrqkNeG3fY9j0twcPlu62etyquYfOJ5mA8fGmJqp6WDxOFVNVdUB6lgD9AtybSHplv4dOKl1IqPfXsTOwmKvyzmQL9IdOF4H0//sdTXGmHqgpoPFjUXkGRGZ417+jNM6CDvRkRH8+YpMduwu5uEpdX5CtZpJPwO6XAkzx8HWlV5XY4wJcTXtGnoZKACucC/5wCvBKirUdUppzC39O/DO/A1MW7zJ63Kqds6jEBlrA8fGmEOqaRAcp6q/V9XV7uURoH0wCwt1v+13HJ1SknjgnUVs3x2CXUSJrZyB41WfwbL3va7GGBPCahoEe0TkF/4bInIGENYnzY3yOV1EeXtKePDdxV6XU7VT/w9adoJp90Lxbq+rMcaEqJoGwXDgBRHJFpFs4C/Ab4JWVT1xUuskbjv7BKYu3Mj7Czd4Xc6BfJHuUtU2cGyMqV5NZw0tUNVMoAvQRVW7Av2DWlk98Zsz25OZ1pgH31lMbsFer8s50LG9oMtQ+MYGjo0xVTusM5Spar57hDHAHUGop96J9EXw9JBMdheX8cA7i9BQHJg9ZwxExcGHo2zg2BhzgKM5VaXUWhX1XIdWidx5zgl8tGQzUxaEYBdRxcDx57D0Pa+rMcaEmKMJAvtqGeDG3u3pdkwTHnp3CVvyi7wu50A2cGyMqcZBg0BECkQkv4pLAZBSRzXWC74I4ekhmRSVlHHf5BDsIvIPHOfnwNdPe12NMSaEHDQIVDVRVZOquCSqamRdFVlftE9O4O7zT+LTpVt4a956r8s50LG9IPMqmPk8bF3hdTXGmBBxNF1Dpgq/6pVOj/RmPPLeEjbmheChFv6B4w9s4NgY47AgqGUREcJTQ7pQWqaMfisEu4gSWkK/+2H1F7BkstfVGGNCgAVBEBzbPJ57B5zEVz/lMnH2Oq/LOdCpN0KbTHjnZljxidfVGGM8ZkEQJNf0PJbT2zfnsalLydkRYucF8EXCNW9DixPgf0Nh8dteV2SM8ZAFQZBERAh/urwLqso9by0MvS6i+BZw/fuQdipM+jXM/ZfXFRljPGJBEERtmzXi/oEd+WblNv7z3VqvyzlQbGOnZXD82fDeSGc2kTEm7FgQBNlVPdrSu0MLnvhgKWu3hVgXEUB0Ixj6X+g0GD5+AD5/zGYTGRNmghoEInK+iCwXkZUiMrqK/XeIyI8islBEPnPPhdygiAhPXtYFnwijJi2gvDwE/8hGRsNl/4Ru18LXTzknsykv97oqY0wdCVoQiIgPeAG4AOgIXCUiHfe72w9Ad1XtAkwC/hSseryU0iSOBy/qyHc/b+ffs7K9LqdqET64aBycPgK+f8mZUVRW6nVVxpg6EMwWQQ9gpXtGs2JgAjAo8A6q+oWq+vtLvgXSgliPp4ackka/E5P547RlZG8N0bV+RODcx6D/A7BwArx5HZSE4LpJxphaFcwgSAUCJ9HnuNuqcwPwYVU7ROQmEZkjInNyc3NrscS6IyL84dIuRPsiuPPNBZSWhWjXiwicOQoueMo5xeV/h8DeXV5XZYwJopAYLBaRa4DuwFNV7VfVl1S1u6p2T05OrtvialHrxrE8ekkGc9fs4IUvVnldzsH1vAkGvwjZ38C/B0Hhdq8rMsYESTCDYD3QNuB2mrutEhE5G7gfuFhVQ/AUX7VrUFYql2SlMO7zFcxds8Prcg4ucyhc8W/YtBBevRAKNntdkTEmCIIZBLOBDiLSTkSigaHAlMA7iEhX4EWcENgSxFpCyphLMmjTOJbbJv5AQVGJ1+Uc3MkXwrA3YUc2vHwe7FjjdUXGmFoWtCBQ1VJgBPARsBR4Q1WXiMgYEbnYvdtTQALwpojMF5Ep1Txdg5IUG8VzQ7PYsLOI37+7xOtyDq19X7j2XdizHV4+H3KXe12RMaYWScgtfXAI3bt31zlz5nhdRq0Y++lPjP10Bc8NzWJQ1sHG0UPEpsXw2mDQMueI5JQsrysyxtSQiMxV1e5V7QuJweJwNaLf8ZxybFMemLyYddtD8Kjj/bXOgF9Pg6h4+NdFsGam1xUZY2qBBYGHIn0RjL0yC4DbJ84P3SmlgZof54RBYmundfDTx15XZIw5ShYEHmvbrBGPXpLBnDU7+OuXIT6l1K9xKvzqQ0g+ESZcBYvf8roiY8xRsCAIAZd0daaUPvdZPZhS6hffAq57D9J6wKQbYO6rXldkjDlCFgQhol5NKfWLbQzXvOUuY30rfDPO64qMMUfAgiBEJMVGMfbKLNbv2MPvp9SDKaV+FctYXwqfPAifPWrLWBtTz1gQhJDu6c24pX8H3p63nikLNnhdTs1FRsNl/4Bu18H0p+GDUbaMtTH1iAVBiLmlvzOl9P7Ji0LvXMcHE+GDi56DXiNh9t/hneG2jLUx9YQFQYjxTylVrUdTSv1E4JwxcNZDsHAivHGtLWNtTD1gQRCCnCmlnZidXY+mlPqJQO87YcDTsHyqu4x1gddVGWMOwoIgRA3umsYgd0rpvLX1ZEppoB7/B4NfcpexvsSWsTYmhFkQhLBH/VNKJ8yvP1NKA2VeCVe+5i5jPRAKNnldkTGmChYEIcw/pTRnR2H9mlIa6KSB7jLWa5yVS20Za2NCjgVBiKu3U0oDte8L102BPTtsGWtjQpAFQT1wS//j6XZMk/o3pTRQWndnfSItc8Jgww9eV2SMcVkQ1AORvgieG9q1YkppWXk9PXK3VUdn5dKYBHj1Imcg2RjjOQuCeqLSlNIvVnpdzpFr1h5+/REkpcDrl9oy1saEAAuCesQ/pXRsfZ1S6peU4i5jfZKzjPWnj9ggsjEesiCoZx69JIPWSfV4SqlffHNnGeuTBsI3Y+G5THj9Mlj6HpTV49/LmHrIgqCe8Z/4vl5PKfWLTYIr/g23LoQ+98DmH2HiNfBshrOKqbUSjKkTFgT1UPf0Zoyo71NKAzVpC/3uhdsWwVUTICULZjxjrQRj6ohoPVs7vnv37jpnzhyvy/BcaVk5V7w4ixVbdvHhrb1Ja9rI65Jq18518MPrMO/fULABElpD12ug27XQ9FivqzOm3hGRuaravap91iKopwKnlN4xcUH9nVJanf1bCW0yrZVgTJBYENRj/iml32dvr99TSg/GFwknXgDD3nDHEu62sQRjapkFQT13SVYqF2c2gCmlNdGkLfS7z2klDP2f00qY/mdrJRhzlGyMoAHILyrhgrHT8UUIH9zam4SYSK9Lqjs718EPr8G81/aNJXT7JXT9pY0lGBPAxggauEpTSt+t51NKD1dVrYSvnw5oJbxvrQRjDsGCoIHwTyl9a14O7zWEKaWHyxcJJw1wxhJuW+SOJSyBicOcsYTPH7OxBGOqYV1DDUhpWTlDXpzFyoY6pfRwlZXCio9h7iuw4hNn2/FnwSm/ghPOd8LDmDBhXUNhItIXwXNXNuAppYeropXwZhWthE5OK2HnWq+rNMZzFgQNzDHNGzFmkDOl9G9fNtAppUeiYixhsTuW0MUZSxjbBV6/3JlxZOdVNmHK2sYN0OCuqXy5PJdnP13BGce3oOsxTb0uKXT4WwknDXBaA/Nec2YdTbzG2Z+UBq07u5cM52eTdIiw70ym4bIxggYqb08JA54L0ymlh6usFNbMgA3zYfNi2LQItv4EWu7sj06AVhn7gqF1Z2jZEaLiPC3bmMNxsDECC4IGbHb2dq58cRaDu6bx5ysyvS6nfinZA1uWOqHgD4dNi6G4wNkvEdC8Q+VwaNUZElt5W7cx1ThYEAT1a6KInA88B/iAf6jqH/fbfyYwFugCDFXVScGsJ9ycmt6MEf2OZ9znK+l7YjIXZaZ4XVL9ERUHqd2ci195OexcUzkc1s2GxW/tu098y4Bw6OK0JJofbzOUTEgL2qdTRHzAC8A5QA4wW0SmqOqPAXdbC1wP3BWsOsLdyLM6MH3lVu6bvIhuxzYltYl1ZxyxiAho1s65dLx43/Y9O5zZSJsW7bt8+zcoK3b2R8ZCy5P3tRpad4ZWnZzzMRgTAoL5NaUHsFJVVwOIyARgEFARBKqa7e4rD2IdYc0/pXTAuOncPmE+/7vpNHwR4nVZDUtcU0j/hXPxKytxxhkCw2Hp+86y2n5N0yuHQ+vO0DgNxP59TN0KZhCkAusCbucAPY/kiUTkJuAmgGOOOeboKwsz/imld7yxgL99uZIR/Tt4XVLD54tyvvW36gSZQ51tqlCwsXI4+AMCd6wupjE0S4em7ZygCLw0TnOe15haVi86LlX1JeAlcAaLPS6nXhrcNZUvbEqpt0QgKcW5nHDevu17d8GWH51Q2PIj7Mh2xiCWTYXygHWSxOeEQdN0p3tq/6CIs39Tc2SCGQTrgbYBt9PcbcYDIsJjl2Qwb80Obps4n6kjbUppyIhJgLY9nEug8jKnBbEje99l+8/Oz6XvQ+HWyvePbRwQDO2sNWFqLJh/CWYDHUSkHU4ADAWuDuLrmUNoHBfF2KFZXPniLB6esoSnh9iU0pAW4bYAGqdVHn/w21vgLKRXERRuSGxeAss/3DdYDZVbE1W1KKw1EdaCFgSqWioiI4CPcKaPvqyqS0RkDDBHVaeIyKnAZKApcJGIPKKqnYJVk6k8pTQjJYlrTjuWSJ8dNVsvxSS6U1UzDtxXVWvC36JYNvUQrYl0aHKs04WV2AaSUqFRczu6ugGzA8rCUGlZOdf88zu+Xb2dts3i+L/e7RlySlvion1el2bqygGtiex9LYqdayu3JgAiotxQSIGkNpCYst/1Ns7+yJi6/11MjdiRxeYA5eXKJ0s3M/6rVfywdifN4qO5vlc6155+LE0aRXtdnvFSeRns2gz5G52zvuVvhPz1Tgsjf4NzKdgIJYUHPrZRCycUklIDgiNl3/XENk7rw6bI1jkLAlMtVeX7n7fz4ter+XzZFhpF+7jy1Lbc2Lu9HXxmqqcKRXluKPjDoorrhdsOfGxU/L4WRFJq1S2MhJbOGImpNRYEpkaWbcrnpa9WM8U9w9nFmSnc1Kc9J7W2I2DNESrde2BLYv/rBRuhvLTy48QHia3dsDhIC8MW/qsxCwJzWNbv3MM/p//MhNlrKSwuo9+Jyfymz3H0bNcMsSa9qW3l5c7gdf76gO6oDft1TW3Yt+BfoLim+8YoklICrgcER1xT64rCgsAcoZ2Fxbw2aw2vzsxm2+5isto2YXif4zi3YysibJkKU9f2FlQ/XuG/vjuXiqO0/SJjq25NBAZHQusGvzCgBYE5KkUlZbw5Zx0vTV/Nuu17aN8inpvObM/gbqnERFo/rgkhZSVQsGm/8Qp/cARc339WlEQ4K8dWak0EXE9s4ywxHpNUb1sXFgSmVpSWlfPh4k2M/2oVSzbkk5wYw6/PaMew044hKdaOWjX1hKpzWtLAlkXBxoCuKfd6Ud6Bj42McwIhofVBfraGuGYhd9yFBYGpVarKNyu3Mf6rVcxYuZXEmEiuPu0YbjijHS2TYr0uz5jaUbzbbV2sh4LNTkDs2uxsC/y5N//Ax0ZEOTOfElsfPDTik+usS8qCwATN4vV5jP9qFR8s2khkRASDu6ZyU5/2HJec4HVpxtSN4kLYtckJi4P9rGoqrUQ4x15U1apIaFX551EerGdBYIJuzbbd/GP6z7wxZx3FZeWc27EVv+lzHN1slVNjHKXFsHtLQOuimtDYvWXf+bIDxTWFcx+HrsOO6OUtCEyd2bprL/+amc2/Z60hb08JPdo1Y3if9vQ7saVNPTWmJsrLYPfWqoMi4zI4ttcRPa0Fgalzu/eWMmH2Ov45fTUb8oo4sVUiv+nTnosyU4iyRe6MqXMWBMYzJWXlvLdgAy9+tZrlmwtIaRzLDb3bM/TUtsTb+RCMqTMWBMZzqsoXy7cw/qvVfP/zdhrHRXHt6cdyXa90WiTYipXGBJsFgQkp89buYPyXq/hk6WaifRGc1CaJVokxtEyKoVViLC2TYmiZFEvLxBhaJsbSPD7ajmQ25igdLAisbW7qXLdjmvLStd1ZuWUXr3+7hlW5u1izrZDvs7ezs7DkgPtHRggtEmJolRRDcmIsrZKcgGiV5IRHSzc8msfH4LPAMOawWRAYzxzfMoGHL658QrqikjJyC/aypWAvW/KLnJ8FRWzOd7bl7Chk3todbN9dfMDz+SKEFgnRFSFxQGhUBEa0nZXNmAAWBCakxEb5aNusEW2bNTro/YpLy8ndtZfN+UVsyd9LbkVYOD/X7yzih7U72VZFYEQINHdbGBWhkRBDYmwU8TGRxMf4aBTt/IyPjqzYFh8TSaMon4WIaXAsCEy9FB0ZQWqTuEOePKe4tJytu/a1MDYX7CU3f19obMorYmFOHtt276Wmw2UxkREkxETSqFJQRBIf7QRIQoyPRjGRzn2ife4+5/7+bc5P5z6xURF2jIXxlAWBadCiIyNIaRJHyiECo6xcKSwupbC4jF17Sync6/4sLmV3cRm795a6lzIKi0vdfe724lLy95SwceceZ1uxc9+SspoliwhuoPgqAqNRlPvTDZfKP/e1WOKinKCJi3YfG7DfAsbUlAWBMTjjC4mxUSTGRtGqlp6zuLS8IiiqDJi9TsgU7i1lV0DA7HHDZMfuYnJ2lFXcLiwuo7i0iqUHqiECjaKc1smBQeIER5zbYnECxUdctL9ls+/+cf7gCbhug/INiwWBMUESHRlBdGQ0TeOja+05S8vKKSwpo9ANjsLisopWyB63heLftm+/v/VSxp6SUgqKStmSv3ffY4pLKSqpecCA0z22f7jEBYRLVS2YwPtUXHdDyN+FFhNprRgvWBAYU49E+iJI8kXU+vkfysqVPSVO6yQwWCoHyr7rga0U53oZe4pL2ZRfVGlfYXEZZeU1P1Zp/1aMv+sr8HpctG+/lk7VoeO/7g8mW9qkehYExhh8EUKCO8Bdm1SV4rLyg4ZKods9VljihorbcnHGY5zru/Y6rZjCktKA+5QdVi3Rvgi3RbKvi6vK8HBnh/lbLAeE0H4tn7goX71vxVgQGGOCRkSIifQRE+mjycFnBB+28nKlqNQNhQNaKDVv0eTu2kthcWFF19qekrIaD/Q7vyMV3Vv7d4HVtJsssPUS704WiIv2ER1ZN60YCwJjTL0UESHuH9Xa/zNWXOq2YtyWyR5/iASMz+wp8YeQvztt33V/6GzdtbfS7T0lZTWepgwQ5RM3ZJxZZLeffQIXZabU+u9rQWCMMftxBvojaEztjsWoKkUl5ZVaLNW2Xip1lznXmzaqvYkHgSwIjDGmjogIcW63UCixYXRjjAlzFgTGGBPmLAiMMSbMWRAYY0yYsyAwxpgwZ0FgjDFhzoLAGGPCnAWBMcaEOdHDOd45BIhILrDmCB/eAthai+XUd/Z+VGbvxz72XlTWEN6PY1U1uaod9S4IjoaIzFHV7l7XESrs/ajM3o997L2orKG/H9Y1ZIwxYc6CwBhjwly4BcFLXhcQYuz9qMzej33svaisQb8fYTVGYIwx5kDh1iIwxhizHwsCY4wJc2ETBCJyvogsF5GVIjLa63rqkoi0FZEvRORHEVkiIre625uJyCcissL92dTrWuuSiPhE5AcRed+93U5EvnM/IxNFJDingwpBItJERCaJyDIRWSoip4fr50NEbnf/nywWkf+JSGxD/2yERRCIiA94AbgA6AhcJSIdva2qTpUCd6pqR+A04Hfu7z8a+ExVOwCfubfDya3A0oDbTwLPqurxwA7gBk+q8sZzwDRVPQnIxHlfwu7zISKpwEigu6pmAD5gKA38sxEWQQD0AFaq6mpVLQYmAIM8rqnOqOpGVZ3nXi/A+U+eivMe/Mu927+ASzwp0AMikgYMBP7h3hagPzDJvUvYvB8i0hg4E/gngKoWq+pOwvfzEQnEiUgk0AjYSAP/bIRLEKQC6wJu57jbwo6IpANdge+AVqq60d21CWjlVV0eGAvcDZS7t5sDO1W11L0dTp+RdkAu8IrbVfYPEYknDD8fqroeeBpYixMAecBcGvhnI1yCwAAikgC8BdymqvmB+9SZRxwWc4lF5EJgi6rO9bqWEBEJdAP+pqpdgd3s1w0ULp8PdxxkEE44pgDxwPmeFlUHwiUI1gNtA26nudvChohE4YTAf1T1bXfzZhFp4+5vA2zxqr46dgZwsYhk43QT9sfpI2/idgdAeH1GcoAcVf3OvT0JJxjC8fNxNvCzquaqagnwNs7npUF/NsIlCGYDHdyR/2icwZ8pHtdUZ9z+738CS1X1mYBdU4Dr3OvXAe/WdW1eUNV7VTVNVdNxPgufq+ow4Avgcvdu4fR+bALWiciJ7qazgB8Jz8/HWuA0EWnk/r/xvxcN+rMRNkcWi8gAnH5hH/Cyqj7ubUV1R0R+AUwHFrGvT/w+nHGCN4BjcJb2vkJVt3tSpEdEpC9wl6peKCLtcVoIzYAfgGtUda+H5dUZEcnCGTiPBlYDv8L5ohh2nw8ReQS4Eme23Q/AjThjAg32sxE2QWCMMaZq4dI1ZIwxphoWBMYYE+YsCIwxJsxZEBhjTJizIDDGmDBnQWCMS0TKRGR+wKXWFlkTkXQRWVxbz2dMbYo89F2MCRt7VDXL6yKMqWvWIjDmEEQkW0T+JCKLROR7ETne3Z4uIp+LyEIR+UxEjnG3txKRySKywL30cp/KJyJ/d9e6/1hE4tz7j3TPFbFQRCZ49GuaMGZBYMw+cft1DV0ZsC9PVTsDf8E5Qh3geeBfqtoF+A8wzt0+DvhKVTNx1uxZ4m7vALygqp2AncBl7vbRQFf3eYYH51czpnp2ZLExLhHZpaoJVWzPBvqr6mp38b5NqtpcRLYCbVS1xN2+UVVbiEgukBa4BIG7/Pcn7kleEJF7gChVfUxEpgG7gHeAd1R1V5B/VWMqsRaBMTWj1Vw/HIFr05Sxb4xuIM4Z9LoBswNWuTSmTlgQGFMzVwb8nOVen4mzeinAMJyF/cA5rePNUHFe5MbVPamIRABtVfUL4B6gMXBAq8SYYLJvHsbsEyci8wNuT1NV/xTSpiKyEOdb/VXutltwzuo1CucMX79yt98KvCQiN+B8878Z52xXVfEBr7thIcA49zSRxtQZGyMw5hDcMYLuqrrV61qMCQbrGjLGmDBnLQJjjAlz1iIwxpgwZ0FgjDFhzoLAGGPCnAWBMcaEOQsCY4wJc/8P2R/7PDCqWccAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the loss curves\n",
    "plt.plot(epoch_count,loss_values,label='Train Loss')\n",
    "plt.plot(epoch_count,test_loss_values,label='Test Loss')\n",
    "plt.title(\"Loss Curves\")\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fcfa9fa-a744-4fae-97e2-5b6f36d67a20",
   "metadata": {},
   "source": [
    "## Saving a model in pytorch\n",
    "\n",
    "There are three main methods you should about for saving and loading models in PyTorch.\n",
    "\n",
    "1. `torch.save()` - allows you save a pytorch in pickle method\n",
    "2. `torch.load()` - load a saved pytorch object\n",
    "3. `torch.nn.Module.load_state_dict()` - allows to load a model's state dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "08deda11-b872-497a-9ce3-0f209fa47076",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Saving out PyTorch model\n",
    "torch.save(model.state_dict(),'./models/01/00.pth')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "90ae4d54-5391-4d13-adbe-76fa6b924509",
   "metadata": {},
   "outputs": [],
   "source": [
    "state_dict_00 = torch.load(\"./models/01/00.pth\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "ab80e1ea-faab-44d2-9de5-eb51d6e373cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_01 = LinearRegressionModel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9bb82ba3-ff7f-4b1a-b57f-78212265a1f1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<All keys matched successfully>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_01.load_state_dict(state_dict_00)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc527976-c912-43f3-82b0-100d046a391c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}