BFGS for Single Variable Linear Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "16943853",
   "metadata": {},
   "source": [
    "## Single Variable Linear Regression\n",
    "### Use BFGS Method to optimize a LR model for the data provided in the attached file.\n",
    "### Use Multivariable gradient descent to solve the same problem.\n",
    "### Evaluate Gradient descent solution and BFGS solution using r2_score.\n",
    "### Plot the following curves:\n",
    "#### - Cost vs. iterations.\n",
    "#### - Cost vs. each parameter.\n",
    "### Find the solution using sklearn.linear_model.LinearRegression model.\n",
    "### Compare the results from Gradient descent, BFGS, and sklearn models. Write your conclusion in brief."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f29dc59",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "aa98dffd",
   "metadata": {},
   "source": [
    "# Abdelrahman Tarek Mahmoud"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c42fe01",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "bceadbf5",
   "metadata": {},
   "source": [
    "## **Imports**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4a6ee84a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.metrics import r2_score\n",
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "de0e8253",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('RegData.csv', header = None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "6d2f162d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(13, 2)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "043dd304",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     2.9\n",
       "1     6.7\n",
       "2     4.9\n",
       "3     7.9\n",
       "4     9.8\n",
       "5     6.9\n",
       "6     6.1\n",
       "7     6.2\n",
       "8     6.0\n",
       "9     5.1\n",
       "10    4.7\n",
       "11    4.4\n",
       "12    5.8\n",
       "Name: 0, dtype: float64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = df.iloc[:,0]\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "27b1c15b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(13, 2)\n",
      "[[1.         2.9000001 ]\n",
      " [1.         6.69999981]\n",
      " [1.         4.9000001 ]\n",
      " [1.         7.9000001 ]\n",
      " [1.         9.80000019]\n",
      " [1.         6.9000001 ]\n",
      " [1.         6.0999999 ]\n",
      " [1.         6.19999981]\n",
      " [1.         6.        ]\n",
      " [1.         5.0999999 ]\n",
      " [1.         4.69999981]\n",
      " [1.         4.4000001 ]\n",
      " [1.         5.80000019]]\n"
     ]
    }
   ],
   "source": [
    "x = np.array(x).reshape(-1,1)\n",
    "X = np.hstack((np.ones((len(x),1)),x))\n",
    "print(X.shape)\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e686dfc8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(13, 1)\n",
      "[[4.        ]\n",
      " [7.4000001 ]\n",
      " [5.        ]\n",
      " [7.19999981]\n",
      " [7.9000001 ]\n",
      " [6.0999999 ]\n",
      " [6.        ]\n",
      " [5.80000019]\n",
      " [5.19999981]\n",
      " [4.19999981]\n",
      " [4.        ]\n",
      " [4.4000001 ]\n",
      " [5.19999981]]\n"
     ]
    }
   ],
   "source": [
    "y = df.iloc[:,1]\n",
    "y = np.array(y).reshape(-1,1)\n",
    "print(y.shape)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f476eb35",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAUZ0lEQVR4nO3df4zc9X3n8efrbEdZUND2wibBCweNhHxVEoG5FYEioSQ0cUkR+BAnESmiQadzyKE0qXSu4n8S3f2Tntw/8kvC8tHLJUpC1VDjoopgolZpo5PgtLYhJgGrhJDgNYFN7gwlrA7jvu+PHXPr8ezud+xZz853nw9pNN/5fD/znbdG9mtnPt/PZ76pKiRJo+9fDLsASdJgGOiS1BIGuiS1hIEuSS1hoEtSS6wf1gtfeOGFddlllw3r5SVpJO3fv/9XVTXRa9/QAv2yyy5jenp6WC8vSSMpyc8X29doyCXJHyf5cZInk9yX5K1d+5PkK0meSfKjJFedbdGSpP4sG+hJJoE/Aqaq6r3AOuD2rm43Apd3btuAewZcpyRpGU1Piq4HxpKsB84DjnbtvwX4Zs17FBhPctEA65QkLWPZQK+qGeDPgF8ALwAvV9UjXd0mgecXPD7SaTtFkm1JppNMz87OnnnVkqTTNBly+S3mP4H/NrAROD/Jx7u79XjqaT8SU1W7q2qqqqYmJnqepJUknaEms1x+D/hZVc0CJNkD/C7wrQV9jgCXLHh8MacPy0jSmrb34Aw79x3m6LE5No6PsX3LJrZuPm0w44w1GUP/BXBNkvOSBLgBeKqrz4PAHZ3ZLtcwPyzzwsCqlKQRt/fgDDv2HGLm2BwFzBybY8eeQ+w9ODOw12gyhv4YcD9wADjUec7uJHcluavT7SHgWeAZ4L8B/3FgFUpSC+zcd5i54ydOaZs7foKd+w4P7DUaLSyqqi8AX+hq3rVgfwF3D6wqSWqZo8fm+mo/E/6WiySdAxvHx/pqPxMGuiSdA9u3bGJsw7pT2sY2rGP7lk0De42h/ZaLJK0lJ2ezrOQsFwNdks6RrZsnBxrg3RxykaSWMNAlqSUMdElqCQNdklrCQJekljDQJaklDHRJagkDXZJawkCXpJYw0CWpJQx0SWoJA12SWqLJRaI3JXl8we2VJJ/t6vOBJC8v6PP5FatYktTTsr+2WFWHgSsBkqwDZoAHenT9YVXdNNDqJEmN9TvkcgPw06r6+UoUI0k6c/0G+u3AfYvsuzbJE0m+l+Q9Z1mXJKlPjQM9yVuAm4Hv9th9ALi0qq4AvgrsXeQY25JMJ5menZ09g3IlSYvp5xP6jcCBqnqxe0dVvVJVr3a2HwI2JLmwR7/dVTVVVVMTExNnXLQk6XT9XILuYywy3JLkXcCLVVVJrmb+D8WvB1CfNDL2HpxZ0etFSstpFOhJzgM+DHxyQdtdAFW1C7gN+FSSN4A54PaqqsGXK61Oew/OsGPPIeaOnwBg5tgcO/YcAjDUdc40CvSqeg14e1fbrgXbXwO+NtjSpNGxc9/hN8P8pLnjJ9i577CBrnPGlaLSABw9NtdXu7QSDHRpADaOj/XVLq0EA10agO1bNjG2Yd0pbWMb1rF9y6YhVaS1qJ9ZLpIWcXKc3FkuGiYDXRqQrZsnDXANlUMuktQSBroktYRDLpIW5erX0WKgS+rJ1a+jxyEXST0ttfpVq5OBLqknV7+OHgNdUk+ufh09Brqknlz9Ono8KSqpJ1e/jh4DXdKiXP06WhxykaSWMNAlqSWWDfQkm5I8vuD2SpLPdvVJkq8keSbJj5JctWIVS5J6WnYMvaoOA1cCJFkHzAAPdHW7Ebi8c3s/cE/nXpJ0jvQ75HID8NOq+nlX+y3AN2veo8B4kosGUqEkqZF+A/124L4e7ZPA8wseH+m0nSLJtiTTSaZnZ2f7fGlJ0lIaB3qStwA3A9/ttbtHW53WULW7qqaqampiYqJ5lZKkZfXzCf1G4EBVvdhj3xHgkgWPLwaOnk1hkqT+9BPoH6P3cAvAg8Adndku1wAvV9ULZ12dJKmxRitFk5wHfBj45IK2uwCqahfwEPBR4BngNeDOgVcqSVpSo0CvqteAt3e17VqwXcDdgy1NktQPV4pKUksY6JLUEga6JLWEgS5JLWGgS1JLGOiS1BIGuiS1hIEuSS1hoEtSSxjoktQSBroktYSBLkktYaBLUksY6JLUEga6JLWEgS5JLdEo0JOMJ7k/ydNJnkpybdf+DyR5OcnjndvnV6ZcSdJiGl2xCPgy8HBV3ZbkLcB5Pfr8sKpuGlxpkqR+LBvoSS4Argc+AVBVrwOvr2xZkqR+NRlyeTcwC3w9ycEk9yY5v0e/a5M8keR7Sd7T60BJtiWZTjI9Ozt7NnVLkro0CfT1wFXAPVW1GfgN8LmuPgeAS6vqCuCrwN5eB6qq3VU1VVVTExMTZ161JOk0TQL9CHCkqh7rPL6f+YB/U1W9UlWvdrYfAjYkuXCglUqSlrRsoFfVL4Hnk2zqNN0A/GRhnyTvSpLO9tWd4/56wLVKkpbQdJbLp4Fvd2a4PAvcmeQugKraBdwGfCrJG8AccHtV1UoULEnqLcPK3ampqZqenh7Ka0vSqEqyv6qmeu1zpagktYSBLkktYaBLUksY6JLUEga6JLWEgS5JLWGgS1JLGOiS1BIGuiS1hIEuSS1hoEtSSzT9cS5JA7D34Aw79x3m6LE5No6PsX3LJrZunhx2WWoJA106R/YenGHHnkPMHT8BwMyxOXbsOQRgqGsgHHKRzpGd+w6/GeYnzR0/wc59h4dUkdrGQJfOkaPH5vpql/ploEvnyMbxsb7apX41CvQk40nuT/J0kqeSXNu1P0m+kuSZJD9KctVix5LWqu1bNjG2Yd0pbWMb1rF9y6ZFniH1p+lJ0S8DD1fVbZ3L0J3Xtf9G4PLO7f3APZ17aehWy8ySk6+5GmpROy0b6EkuAK4HPgFQVa8Dr3d1uwX4Zuc6oo92PtFfVFUvDLheqS+rbWbJ1s2TBrhWTJMhl3cDs8DXkxxMcm+S87v6TALPL3h8pNMmDZUzS7SWNAn09cBVwD1VtRn4DfC5rj7p8bzTrj6dZFuS6STTs7OzfRcr9cuZJVpLmgT6EeBIVT3WeXw/8wHf3eeSBY8vBo52H6iqdlfVVFVNTUxMnEm9Ul+cWaK1ZNlAr6pfAs8nOXkq/gbgJ13dHgTu6Mx2uQZ42fFzrQbOLNFa0nSWy6eBb3dmuDwL3JnkLoCq2gU8BHwUeAZ4DbhzBWqV+ubMEq0lmZ+Ycu5NTU3V9PT0UF5bkkZVkv1VNdVrnytFJaklDHRJagl/PldrzmpZOSoNmoGuNWW1rRyVBskhF60prhxVmxnoWlNcOao2M9C1prhyVG1moGtNceWo2syTolpTXDmqNjPQteb4m+RqK4dcJKklDHRJagkDXZJawkCXpJYw0CWpJQx0SWoJA12SWqLRPPQkzwH/BJwA3ui+WkaSDwB/Dfys07Snqv7LwKqUJC2rn4VFH6yqXy2x/4dVddPZFiRJOjMOuUhSSzQN9AIeSbI/ybZF+lyb5Ikk30vynl4dkmxLMp1kenZ29owKliT11nTI5bqqOprkHcD3kzxdVf+wYP8B4NKqejXJR4G9wOXdB6mq3cBugKmpqTq70iVJCzX6hF5VRzv3LwEPAFd37X+lql7tbD8EbEhy4YBrlSQtYdlAT3J+kred3AY+AjzZ1eddSdLZvrpz3F8PvlxJ0mKaDLm8E3igk9frge9U1cNJ7gKoql3AbcCnkrwBzAG3V5VDKpJ0Di0b6FX1LHBFj/ZdC7a/BnxtsKVJkvrhtEVJagkDXZJawkCXpJYw0CWpJQx0SWoJA12SWqKfX1uUhmrvwRl27jvM0WNzbBwfY/uWTWzdPNl3H6mtDHSNhL0HZ9ix5xBzx08AMHNsjh17DgG8GdhN+kht5pCLRsLOfYffDOqT5o6fYOe+w331kdrMQNdIOHpsbtn2Jn2kNjPQNRI2jo8t296kj9RmBrpGwvYtmxjbsO6UtrEN69i+ZVNffaQ286SoRsLJk5pLzWBp0kdqswzrV26npqZqenp6KK8tSaMqyf6qmuq1zyEXSWoJA12SWqLRGHqS54B/Ak4Ab3R/3O9cfu7LwEeB14BPVNWBwZaqc8GVltLo6uek6Aer6leL7LsRuLxzez9wT+deI8SVltJoG9SQyy3AN2veo8B4kosGdGydI660lEZb00Av4JEk+5Ns67F/Enh+weMjnbZTJNmWZDrJ9OzsbP/VakW50lIabU0D/bqquor5oZW7k1zftT89nnPafMiq2l1VU1U1NTEx0WepWmmutJRGW6NAr6qjnfuXgAeAq7u6HAEuWfD4YuDoIArUueNKS2m0LRvoSc5P8raT28BHgCe7uj0I3JF51wAvV9ULA69WK2rr5km+eOv7mBwfI8Dk+BhfvPV9nhCVRkSTWS7vBB6Yn5nIeuA7VfVwkrsAqmoX8BDzUxafYX7a4p0rU65W2tbNkwa4NKKWDfSqeha4okf7rgXbBdw92NIkSf1wpagktYSBLkktYaBLUksY6JLUEga6JLWEgS5JLWGgS1JLGOiS1BIGuiS1hIEuSS1hoEtSSxjoktQSBroktYSBLkktYaBLUksY6JLUEo0DPcm6JAeT/E2PfR9I8nKSxzu3zw+2TEnScppcgu6kzwBPARcssv+HVXXT2ZckSToTjT6hJ7kY+APg3pUtR5J0ppoOuXwJ+BPgn5foc22SJ5J8L8l7zroySVJflg30JDcBL1XV/iW6HQAuraorgK8Cexc51rYk00mmZ2dnz6ReSdIimnxCvw64OclzwF8AH0ryrYUdquqVqnq1s/0QsCHJhd0HqqrdVTVVVVMTExNnX70k6U3LnhStqh3ADpifzQL8p6r6+MI+Sd4FvFhVleRq5v9Q/Hrg1epNew/OsHPfYY4em2Pj+Bjbt2xi6+bJYZclaYj6meVyiiR3AVTVLuA24FNJ3gDmgNurqgZTorrtPTjDjj2HmDt+AoCZY3Ps2HMIwFCX1rAMK3enpqZqenp6KK896q77079j5tjcae2T42P8z899aAgVSTpXkuyvqqle+1wpOoKO9gjzpdolrQ0G+gjaOD7WV7uktcFAH0Hbt2xibMO6U9rGNqxj+5ZNQ6pI0mpwxidFNTwnT3w6y0XSQgb6iNq6edIAl3QKh1wkqSUMdElqCYdc1BdXqEqrl4GuxlyhKq1uDrmosZ37Dr8Z5ifNHT/Bzn2Hh1SRpIUMdDXmClVpdTPQ1ZgrVKXVzUBXY65QlVY3T4qqMVeoSqubga6+uEJVWr0ccpGkljDQJaklGg+5JFkHTAMzVXVT174AXwY+CrwGfKKqDgyyUHCV4tny/ZParZ8x9M8ATwEX9Nh3I3B55/Z+4J7O/cC4SvHs+P5J7ddoyCXJxcAfAPcu0uUW4Js171FgPMlFA6oRcJXi2fL9k9qv6Rj6l4A/Af55kf2TwPMLHh/ptJ0iybYk00mmZ2dn+6nTVYpnyfdPar9lAz3JTcBLVbV/qW492uq0hqrdVTVVVVMTExN9lOkqxbPl+ye1X5NP6NcBNyd5DvgL4ENJvtXV5whwyYLHFwNHB1Jhh6sUz47vn9R+ywZ6Ve2oqour6jLgduDvqurjXd0eBO7IvGuAl6vqhUEWunXzJF+89X1Mjo8RYHJ8jC/e+j5P6DXk+ye13xmvFE1yF0BV7QIeYn7K4jPMT1u8cyDVdXGV4tnx/ZPara9Ar6ofAD/obO9a0F7A3YMsTJLUH1eKSlJLGOiS1BIGuiS1hIEuSS2R+fOZQ3jhZBb4+QAPeSHwqwEeb6VZ78qy3pVlvStvsZovraqeKzOHFuiDlmS6qqaGXUdT1ruyrHdlWe/KO5OaHXKRpJYw0CWpJdoU6LuHXUCfrHdlWe/Kst6V13fNrRlDl6S1rk2f0CVpTTPQJaklRjrQk7w1yf9K8kSSHyf5z8OuqYkk65IcTPI3w66liSTPJTmU5PEk08OuZzlJxpPcn+TpJE8luXbYNS0myabO+3ry9kqSzw67rqUk+ePO/7cnk9yX5K3DrmkpST7TqfXHq/G9TfLfk7yU5MkFbf8yyfeT/GPn/reaHGukAx34v8CHquoK4Erg9zu/x77anbzg9ij5YFVdOSJzeb8MPFxV/xq4glX8XlfV4c77eiXwb5j/+ekHhlvV4pJMAn8ETFXVe4F1zF8nYVVK8l7gPwBXM/9v4aYklw+3qtP8D+D3u9o+B/xtVV0O/G3n8bJGOtA7F6V+tfNwQ+e2qs/yNrjgts5CkguA64E/B6iq16vq2FCLau4G4KdVNcgV1CthPTCWZD1wHgO+OtmA/Q7waFW9VlVvAH8P/Nsh13SKqvoH4H93Nd8CfKOz/Q1ga5NjjXSgw5vDF48DLwHfr6rHhlzScr7E0hfcXo0KeCTJ/iTbhl3MMt4NzAJf7wxr3Zvk/GEX1dDtwH3DLmIpVTUD/BnwC+AF5q9O9shwq1rSk8D1Sd6e5DzmL8RzyTLPWQ3eefKqb537dzR50sgHelWd6HxdvRi4uvMVa1VqeMHt1ei6qroKuBG4O8n1wy5oCeuBq4B7qmoz8Bsafl0dpiRvAW4GvjvsWpbSGcu9BfhtYCNwfpLuS1KuGlX1FPBfge8DDwNPAG8MtagVNPKBflLna/UPOH0sajVpcsHtVaeqjnbuX2J+fPfq4Va0pCPAkQXf1O5nPuBXuxuBA1X14rALWcbvAT+rqtmqOg7sAX53yDUtqar+vKquqqrrmR/a+Mdh19TAi0kuAujcv9TkSSMd6Ekmkox3tseY/8f29FCLWkLDC26vKknOT/K2k9vAR5j/GrsqVdUvgeeTbOo03QD8ZIglNfUxVvlwS8cvgGuSnJckzL+/q/akM0CSd3Tu/xVwK6PxPj8I/GFn+w+Bv27ypDO+SPQqcRHwjSTrmP/j9JdVNRJTAUfIO4EH5v/vsh74TlU9PNySlvVp4NudYYxnWaGLlg9KZ2z3w8Anh13LcqrqsST3AweYH7o4yOpfVv9XSd4OHAfurqr/M+yCFkpyH/AB4MIkR4AvAH8K/GWSf8/8H9F/1+hYLv2XpHYY6SEXSdL/Z6BLUksY6JLUEga6JLWEgS5JLWGgS1JLGOiS1BL/D2IXKbbVCLgmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(x,y);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4abb939",
   "metadata": {},
   "source": [
    "## **BFGS Linear Regression Model**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "178df069",
   "metadata": {},
   "outputs": [],
   "source": [
    "def BFGS_LR(X,Y,tol = 0.001, iterations = 40, learning_rate = 0.1):\n",
    "   \n",
    "    costs = []\n",
    "    thetas = []\n",
    "    hypothesis = []\n",
    "    b = np.eye(2)\n",
    "    \n",
    "    # randomly initialize theta\n",
    "    theta = np.random.rand(2,1).reshape(2,1)\n",
    "    theta_prev = theta - 0.5\n",
    "    \n",
    "    for i in range(iterations):\n",
    "        # calculate hypothesis\n",
    "        y_hat = np.dot(X,theta)\n",
    "        y_hat_prev = np.dot(X,theta_prev)\n",
    "        \n",
    "        hypothesis.append(y_hat)\n",
    "        # calculate error\n",
    "        e = y_hat - y\n",
    "        e_prev = y_hat_prev - y\n",
    "        # mean square error\n",
    "        mse = (np.dot(e.T,e)/(2*len(y)))[0][0]\n",
    "        mse_prev = (np.dot(e_prev.T,e_prev)/(2*len(y)))[0][0]\n",
    "        \n",
    "        costs.append(mse)\n",
    "        # calculate gradients\n",
    "        gradient = np.dot(X.T,e)/len(y)\n",
    "        gradient_prev = np.dot(X.T,e_prev)/len(y)\n",
    "        # update theta\n",
    "        delta_gradient = gradient-gradient_prev\n",
    "        delta_theta = theta - theta_prev\n",
    "        \n",
    "        # b inverse plus equation with substitution of y with delta gradient and x_delta with theta delta\n",
    "        prt_1 = (np.eye(2) - ((delta_theta@delta_gradient.T)/(delta_gradient.T@delta_theta)))\n",
    "        prt_2 = ((np.eye(2) - (delta_gradient @ delta_theta.T)/(delta_gradient.T@delta_theta)))                                                            \n",
    "        prt_3 = ((delta_theta@delta_theta.T)/(delta_gradient.T@delta_theta))\n",
    "\n",
    "        B_inv_plus = prt_1 @ b @ prt_2 + prt_3  \n",
    " \n",
    "        thetas.append(theta)\n",
    "        theta_new = theta - learning_rate * np.dot(B_inv_plus,gradient)\n",
    "        \n",
    "        # Stop condition\n",
    "        if(i>0 and np.abs(costs[i]-costs[i-1])<0.001):\n",
    "            print(\"stopped at iteration {}\".format(i))\n",
    "            break\n",
    "            \n",
    "        theta_prev=theta      \n",
    "        theta=theta_new\n",
    "    return np.array(thetas), costs, hypothesis\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e45780e3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "stopped at iteration 39\n"
     ]
    }
   ],
   "source": [
    "theta,costs ,hypothesis= BFGS_LR(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "152688d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEWCAYAAABhffzLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAoY0lEQVR4nO3dd3wc9Z3/8ddnVS1blmxJlm0kY1wwxsSmyICBEDok1HBJCAmEHJcfPxLIj7RLv1za3ZHGhRQuIdSEfgESIAkloYVuYWyDK+BeJXfJsup+fn/syF4rkr2SrJ3Vzvv5YB87O7M7894BPjP6zuz3a+6OiIhERyzsACIikl4q/CIiEaPCLyISMSr8IiIRo8IvIhIxKvwiIhGjwi+Dipl92sw2mlmjmZWlcbtfN7Nb0rW9TGdm483MzSw37CzSeyr80idm9jEzqw0K8Hoz+4uZndTPda4wszP2sTwPuAE4y92Hufvm/mxvH9s5xczWJM9z9/90908NxPYOhKAI7wz+fXQ+vhx2LslMOlpLr5nZF4CvAlcDTwCtwDnAhcALA7jpSqAQWDCA2xjMZrj7O2GHkMynM37pFTMrAb4LXOPuD7n7Tndvc/dH3f1fg/cUmNlPzWxd8PipmRUEy8rN7DEz22ZmW8zs72YWM7PfAeOAR7s7WzWzQ4ElwcttZvZ0d80NZvasmX0qmP6kmb1gZj82s61mttzM3p/03pFmdnuQcauZ/cHMhgJ/AcYmnTmPNbNvm9ldSZ+9wMwWBN/jWTObmrRshZl9yczmm9l2M7vfzAq72ZcFweePSJpXYWa7zGxUT/uqD//Ovm1mvw9yNJjZHDObkbR8avAdtgXf6YKkZUPM7CdmtjL4Li+Y2ZCk1X/czFaZ2SYz+0Zvs0k4VPilt2aROOt+eB/v+QZwPHAkMAM4FvhmsOyLwBqggsQZ/NcBd/fLgVXA+UEzzg+TV+juS4FpwctSdz8txbzHkThglAM/BG41MwuW/Q4oCtY7Cvhvd98JvB9YF+QY5u7rklcYHITuBT4XfI8/kzhg5Se97SMk/go6BJgOfLJrMHdvAR4CLu3yuefcvY4e9lWK37urC4H/BUYC9wB/MLO8oPnsUeDJYB98FrjbzKYEn/sxcAxwQvDZLwPxpPWeBEwBTge+lXwAlMylwi+9VQZscvf2fbzn48B33b3O3euB7wCXB8vagDHAwcFfCn/3ge0waqW7/8bdO4A7g21XmtkYEgX+anffGmR5LsV1XgL8yd2fcvc2EsVxCIni2Oln7r7O3beQKKxH9rCue9i78H8smAe931dzgrP2zsfZScted/ffB3lvIHHwPj54DAOud/dWd38aeAy4NPjr4krgOndf6+4d7v5ScMDq9B133+Xu84B5JA70kuFU+KW3NgPl+7mbYyywMun1ymAewI+Ad4AnzWyZmX11YGLutqFzwt2bgslhQDWwxd239mGde30/d48Dq4GDutsu0BRssztPA0PM7DgzO5jEAaLzr6ne7quj3b006fFE0rLVXfKuCb7HWGB1MK/TyuC7lJM4QLy7j22m+j0lg6jwS2+9DDQDF+3jPeuAg5Nejwvm4e4N7v5Fd58AnA98wcxOD97X2zP/ncFzUdK80Sl+djUw0sxKu1m2vxx7fb+g6agaWJvitvdsKFFwHyBx1v8x4DF3bwiW7Wtf9VZ1Ut4YUBV8j3VAdZdrB+OC77KJxL/riX3cpmQoFX7pFXffDnwL+KWZXWRmRUFb8fvNrLNd/l7gm8GFyvLg/XcBmNl5ZjYpKJY7gI7gAbARmNCLLPUkCtRlZpZjZleSYpFy9/UkLuLeZGYjgu9wclKOMktcyO7OA8C5ZnZ60Eb+RaAFeCnV7F3cQ6L56OPsaebZ377qrWPM7OLgL7XPBXlfAV4lcQD9crAPTiFxkLkvOCjdBtwQXODOMbNZFlyol8FLhV96zd1vAL5A4oJtPYmz52uBPwRv+T5QC8wH3gTmBPMAJgN/BRpJ/PVwk7s/Gyz7LxIHjG1m9qUU4/wf4F9JNEFNo3fF93IS7eiLgToSBRF3X0zi4LUsyDI2+UPuvgS4DPg5ibPi80lclG7txbaT19dZfMeSOBh12te+6s482/s+/p8mLfsjiYPLVhLf++LgukErcAGJ6x2bgJuATwT7AOBLJP4dzga2AD9AdWPQMw3EIpLdzOzbwCR3vyzsLJIZdOQWEYkYFX4RkYhRU4+ISMTojF9EJGIGRSdt5eXlPn78+LBjiIgMKq+//vomd6/oOn9QFP7x48dTW1sbdgwRkUHFzFZ2N19NPSIiEaPCLyISMSr8IiIRo8IvIhIxKvwiIhGjwi8iEjEq/CIiEZPVhf+ZxXX88pl3wo4hIpJRsrrwv/TuJm7829u0tsf3/2YRkYjI6sI/vaqU1vY4Szc2hB1FRCRjZHXhn1FVCsC8NdtCzSEikkmyuvBXjxzCiKI85q3eFnYUEZGMkdWF38yYXlXK/DXbw44iIpIxsrrwA8yoKmHpxgaaWtvDjiIikhGyvvBPryol7rBg3Y6wo4iIZITsL/zVJQBq5xcRCWR94R9VXMjYkkLmqZ1fRASIQOEHggu828KOISKSEaJR+KtLWLm5iW1NrWFHEREJXSQKf+cPuXRbp4hIRAr/e6p0gVdEpFMkCv/wwjwmVAzVBV4RESJS+CHR3KMLvCIiESr806tKqGtoYcP25rCjiIiEasAKv5ndZmZ1ZvZW0rwfmdliM5tvZg+bWelAbb+rGdWJTc1VO7+IRNxAnvHfAZzTZd5TwBHuPh1YCnxtALe/l8PHDCc3ZmruEZHIG7DC7+7PA1u6zHvS3Tt7S3sFqBqo7XdVmJfDlNHFuqVTRCIvzDb+K4G/9LTQzK4ys1ozq62vrz8gG+z8Ba+7H5D1iYgMRqEUfjP7BtAO3N3Te9z9ZnevcfeaioqKA7LdI6tL2NHczorNTQdkfSIig1HaC7+ZXQGcB3zc03zqPb1zKEZd4BWRCEtr4Tezc4CvABe4e9pPuyePGkZhXkxj8IpIpA3k7Zz3Ai8DU8xsjZn9C/ALoBh4yszmmtmvBmr73cnNiXHE2BJd4BWRSMsdqBW7+6XdzL51oLaXqhnVpdz96kraOuLk5UTm92siIrtFrvJNryqhuS3O0o0NYUcREQlF5Aq/umgWkaiLXOE/uKyIkiF5+gWviERW5Aq/mTG9qoR5q3XGLyLRFLnCD4nmniUbG9jV2hF2FBGRtItk4Z9eVUJH3Fm4Xmf9IhI9kSz8nV00q7lHRKIokoW/cnghlcMLdIFXRCIpkoUfEu38GoNXRKIouoW/upTlm3ayfVdb2FFERNIqsoV/elUJAG/qrF9EIia6hf+gUgD11CkikRPZwl9SlMch5UM1+LqIRE5kCz9AzcEjmL1iC/G4hmIUkeiIdOE/YVIZ25raWLRhR9hRRETSJtKFf9aEcgBefndzyElERNIn0oV/dEkhE8qH8pIKv4hESKQLP8CsiWW8tnwL7R3xsKOIiKSFCv/EMhpb2nlzre7nF5FoiHzhP35CGYCae0QkMiJf+MuHFXDY6GJeWabCLyLRMGCF38xuM7M6M3srad5IM3vKzN4OnkcM1PZ7Y9bEMmav2EJLuwZmEZHsN5Bn/HcA53SZ91Xgb+4+Gfhb8Dp0syaU0dwWZ+6qbWFHEREZcANW+N39eWBLl9kXAncG03cCFw3U9nvjuAllxEzt/CISDelu46909/UAwfOont5oZleZWa2Z1dbX1w9oqJIheRxxUAkvq51fRCIgYy/uuvvN7l7j7jUVFRUDvr1ZE8p4Y9VWDcAuIlkv3YV/o5mNAQie69K8/R7NmlhGW4dTu7Jr65SISHZJd+F/BLgimL4C+GOat9+jmeNHkhsz9dsjIllvIG/nvBd4GZhiZmvM7F+A64Ezzext4MzgdUYYWpDLjOpSXeAVkayXO1ArdvdLe1h0+kBts79OmFjGTc++S0NzG8WFeWHHEREZEBl7cTcMsyaW0RF3Zq9QO7+IZC8V/iRHjxtBfm6Ml95Rc4+IZC8V/iSFeTkcM26E2vlFJKup8HdxwsQyFm3YwdadrWFHEREZECr8XZwwqQx3eHW5zvpFJDup8HcxvaqUovwcNfeISNZS4e8iLyfGzPEj9UMuEclaKvzdOGFiGW/XNVLX0Bx2FBGRA06FvxuzJiaGY9RZv4hkIxX+bkwbW0JxYa4Kv4hkJRX+buTEjOMnlKl/fhHJSir8PZg1oYyVm5tYs7Up7CgiIgeUCn8PTpikdn4RyU4q/D04dFQxFcUFPLtkYId9FBFJt/0WfjM70cyeMrOlZrbMzJab2bJ0hAtTLGacMbWSZ5fU0dym4RhFJHukcsZ/K3ADcBIwE6gJnrPe2dMq2dnawYvvbAo7iojIAZNK4d/u7n9x9zp339z5GPBkGeCEieUUF+TyxIINYUcRETlgUhmB6xkz+xHwENDSOdPd5wxYqgyRnxvjtKmj+OuiOto74uTm6JKIiAx+qRT+44LnmqR5Dpx24ONknrOnjeaPc9dRu3Irx08oCzuOiEi/7bfwu/up6QiSqd53aAX5uTGeWLBBhV9EskIqd/WUmNkNZlYbPH5iZiXpCJcJhhbkcvLkcp5csBF3DzuOiEi/pdJofRvQAHwkeOwAbu/PRs3s82a2wMzeMrN7zaywP+sbaGdNG83abbt4a+2OsKOIiPRbKoV/orv/u7svCx7fASb0dYNmdhDw/4Aadz8CyAE+2tf1pcMZUyuJGbq7R0SyQiqFf5eZndT5wsxOBHb1c7u5wBAzywWKgHX9XN+AGjk0n2MPGanCLyJZIZXC/2ngl2a2wsxWAr8Aru7rBt19LfBjYBWwnsTvBJ7s+j4zu6rzukJ9ffjdJpwzbTRv1zWyrL4x7CgiIv2y38Lv7nPdfQYwHXiPux/l7vP6ukEzGwFcCBwCjAWGmtll3Wz3ZnevcfeaioqKvm7ugDlr2mgAnliwMeQkIiL90+PtnGZ2mbvfZWZf6DIfAHe/oY/bPANY7u71wfoeAk4A7urj+tJibOkQpleV8MSCDXz6lIlhxxER6bN9nfEPDZ6Lu3kM68c2VwHHm1mRJY4ipwOL+rG+tDl72mjmrt7Ghu0ai1dEBq8ez/jd/dfB5F/d/cXkZcEF3j5x91fN7PfAHKAdeAO4ua/rS6ezp1XyoyeW8NTCDVw+a3zYcURE+iSVi7s/T3FeyoLbQw9z9yPc/XJ3b9n/p8I3aVQxEyqG8rju7hGRQWxfbfyzSLS9V3Rp5x9O4t77SDp72mhufn4Z25paKS3KDzuOiEiv7euMP59EW34ue7fv7wA+NPDRMtPZ00bTEXf+tqgu7CgiIn2yrzb+54DnzOwOd1+ZxkwZbfpBJYwpKeSJBRv4p2Oqwo4jItJrqbTx32JmpZ0vzGyEmT0xcJEyWyxmnHV4Jc+/Xc+uVg3JKCKDTyqFv9zdt3W+cPetwKgBSzQInD1tNM1tcZ5bGv4vikVEeiuVwh83s3GdL8zsYBIDsUTWsYeMpLQojyd1d4+IDEKpjMD1DeAFM3sueH0ycNXARcp8uTkxTj+skqcWbqCtI06ehmQUkUEklb56HgeOBu4HHgCOcffItvF3OntaJTua23np3UiMOy8iWSTVU9UCYAuwHTjczE4euEiDw8mHVlAyJI/fv74m7CgiIr2y36YeM/sBcAmwAIgHsx14fgBzZbzCvBw+eNRB3PPqKrbubGXEUP2YS0QGh1TO+C8Cprj7ue5+fvC4YIBzDQqXzKymtSPOw2+sDTuKiEjKUin8y4C8gQ4yGE0dM5wZ1aXcP3u1BmIXkUEjlcLfBMw1s1+b2c86HwMdbLD46MxqlmxsYO7qbWFHERFJSSqF/xHge8BLwOtJDwHOnzGWovwc7nttddhRRERSst+Lu+5+ZzqCDFbDCnI5b/oYHp2/jn87/3CGFaTy0wgRkfDs94zfzJab2bKuj3SEGywumTmOptYOHpu3LuwoIiL7lcrpaU3SdCHwYWDkwMQZnI4eV8rkUcO4b/ZqPnrsuP1/QEQkRKn8cndz0mOtu/8UOG3gow0eZsYlM6uZu3obizfsCDuOiMg+pdLUc3TSo8bMriYxIIskufjoKvJyjPtn6yKviGS2VJp6fpI03Q4sBz4yMHEGr5FD8zlr2mgefmMtXznnMArzIjs6pYhkuB7P+M3sumDy39z91OBxprtf5e5L0pRvUPnozGq2NbXx5MKNYUcREenRvpp6/jl4PuA/1jKzUjP7vZktNrNFwcDug96JE8s5qHQI989eFXYUEZEe7aupZ5GZrQAqzGx+0nwD3N2n92O7NwKPu/uHzCwfKOrHujJGLJa4yHvDU0tZtbmJcWVZ8bVEJMv0eMbv7pcCxwPvAOcnPc4LnvvEzIaTGMzl1mA7rclDOw52H66pImbwQK0u8opIZtrnXT3uvsHdZ7j7yq6PfmxzAlAP3G5mb5jZLWY2tB/ryyhjSoZwypRR/O/rq2nviO//AyIiaRbGmIG5JEb0+h93PwrYCXy165vM7CozqzWz2vr6wTWo+SUzq9m4o4Vnlwyu3CISDWEU/jXAGnd/NXj9exIHgr24+83uXuPuNRUVFWkN2F+nHTaK8mEF3Kd7+kUkA6XyA64PpzIvVe6+AVhtZlOCWacDC/u6vkyUlxPjQ8dU8cySOtZt2xV2HBGRvaRyxv+1FOf1xmeBu4O7hY4E/rOf68s4Hz9uHAbc/Lz6sxORzNLj7Zxm9n7gA8BBXQZeGU7iF7x95u5z2bvzt6xTPbKIi446iHtfW8U1p06iorgg7EgiIsC+z/jXAbVAM3sPwPIIcPbARxv8PnPKRFo74tz6wvKwo4iI7NbjGb+7zwPmmdk97t4GYGYjgGp335qugIPZhIphnPueMfzu5RVc/b4JlBblhx1JRCSlNv6nzGy4mY0E5pG4//6GAc6VNa45dRI7Wzu446UVYUcREQFSK/wl7r4DuBi43d2PAc4Y2FjZY+qY4ZwxtZLbX1xBY0u/Lo2IiBwQqRT+XDMbQ6Ir5scGOE9Wuva0SWzf1cZdr/TnB88iIgdGKoX/u8ATwLvuPtvMJgBvD2ys7HJkdSnvnVzOLX9fRnNbR9hxRCTiUhl68X/dfbq7fzp4vczd/2ngo2WXa0+dxKbGVu57TV02i0i4UvnlbpWZPWxmdWa20cweNLOqdITLJsdNKGPm+BH8+vlltLar8zYRCU8qTT23k7h3fyxwEPBoME966drTJrN+ezMPzVkTdhQRibBUCn+Fu9/u7u3B4w5gcPWaliFOnlzO9KoSbnr2XXXZLCKhSaXwbzKzy8wsJ3hcBmwe6GDZyMy45tRJrNrSxKPz14UdR0QiKpXCfyWJWzk3AOuBDwXzpA/OnFrJlMpifvnMu8TjHnYcEYmgVO7qWeXuF7h7hbuPcveL+jkCV6TFYsZnTp3IO3WNPLFgQ9hxRCSCUrmr504zK016PcLMbhvQVFnuvOljOaR8KD9/+h2d9YtI2qXS1DM9eTD0oIO2owYsUQTkxIzrTp/MwvU7NCi7iKRdKoU/FvTKCUDQWVuPvXpKai48ciwzx4/gB48vZltTa9hxRCRCUin8PwFeMrPvmdl3gZeAHw5srOxnZnzngiPYvquNnzy5NOw4IhIhqVzc/S3wT8BGoB642N1/N9DBouDwscO5/PiDufvVlby1dnvYcUQkIlI548fdF7r7L9z95+6eVQOjh+0LZ01hRFE+//7IAl3oFZG0SKnwy8ApGZLHV845jNdXbuXhN9aGHUdEIkCFPwN86Jgqjqwu5b/+spgdzW1hxxGRLKfCnwFiMeO7F05j884WfvqUhjoQkYEVWuEP+v15w8w0qhcwvaqUS48dx50vr2DJhoaw44hIFgvzjP86YFGI2884/3rWFIoLc/nWH9/CXRd6RWRghFL4g4FczgVuCWP7mWrE0Hy+dNYUXl2+hUfnrw87johkqbDO+H8KfBnosVN6M7vKzGrNrLa+vj5twcJ26bHjOOKg4fzHnxbS2NIedhwRyUJpL/xmdh5Q5+6v7+t97n6zu9e4e01FRXTGfcmJJX7Ru3FHCzf+Vb/oFZEDL4wz/hOBC8xsBXAfcJqZ3RVCjox1zMEjuPTYcdzywnJefGdT2HFEJMukvfC7+9fcvcrdxwMfBZ5298vSnSPT/dt5U5lYMYzP3T+XTY0tYccRkSyi+/gzVFF+Lr/42FFs39XGFx+Yp+4cROSACbXwu/uz7n5emBky2WGjh/Ot8w7nuaX13PLCsrDjiEiW0Bl/hvv4ceN4/xGj+eHjS3hj1daw44hIFlDhz3BmxvUXT6dyeCGfvfcNtu9SXz4i0j8q/INASVEeP7v0KNZvb+brD72pX/WKSL+o8A8Sxxw8gi+edSh/enM9983WOL0i0ncq/IPI1SdP5KRJ5Xz7kQUs3aiO3ESkb1T4B5FYzLjhkhkUF+Zy7T1z2NXaEXYkERmEVPgHmVHFhdzwkSNZurGRrzw4X/f3i0ivqfAPQicfWsFXzjmMR+at4/t/WqSLvSLSK7lhB5C+ufp9E6hvaOG2F5dTUVzAp0+ZGHYkERkkVPgHKTPjm+dOZVNjCz94fDFlw/L5SE112LFEZBBQ4R/EYjHjxx+ewdamVr720JuUDc3n9KmVYccSkQynNv5BLj83xv9cdgzTxg7nmnvm8PrKLWFHEpEMp8KfBYYV5HL7J2cypmQIV95Rq3v8RWSfVPizRNmwAn575bEU5Ma44rbXWLdtV9iRRCRDqfBnkeqRRdx55bE0Nrdz+a2vslkDuIhIN1T4s8zUMcO55Yoa1mzdxYd+9TKrtzSFHUlEMowKfxY6bkIZd3/qOLbsbOWDN73EW2u3hx1JRDKICn+Wqhk/kgc/PYuC3BiX/Ppl/v52fdiRRCRDqPBnsUmjinnoMydQPbKIf759Ng+/sSbsSCKSAVT4s1zl8EIeuHoWM8eP5PP3z+NXz72rvn1EIk6FPwKGF+Zxx5UzOX/GWK7/y2K+8+hCOtSrp0hkpb3LBjOrBn4LjAbiwM3ufmO6c0RNQW4ON15yJJXFBdzywnLqGpr58YdnUJSvXjtEoiaM/+vbgS+6+xwzKwZeN7On3H1hCFkiJRYzvnne4YwuKeQ//ryIxesb+NmlR3HEQSVhRxORNEp7U4+7r3f3OcF0A7AIOCjdOaLsU++dwN2fOo6m1g4+eNOL3PL3ZRrQRSRCQm3jN7PxwFHAq90su8rMas2str5etyIeaCdMLOcv172XU6aM4vt/WsQ/3zGb+gb90lckCkIr/GY2DHgQ+Jy77+i63N1vdvcad6+pqKhIf8AIGDE0n5svP4bvXTiNV5Zt5v03Ps9zS3WQFcl2oRR+M8sjUfTvdveHwsggCWbG5bPG88i1JzFyaD5X3PYa339sIS3tGshdJFulvfCbmQG3Aovc/YZ0b1+6N2V0MY9cexKXH38wt7ywnIt++RK1K9S3v0g2CuOM/0TgcuA0M5sbPD4QQg7pojAvh+9ddAS/+UQNW3e28qFfvczn75/Lxh3NYUcTkQMo7bdzuvsLgKV7u5K6Mw+v5MRJZdz0zLvc/Pwynliwgc+eNpkrTxpPQW5O2PFEpJ/0y13pVlF+Ll86ewpPfeFkTpxUzg8eX8zZ//08Ty/eGHY0EeknFX7Zp4PLhvKbT9Rw55XHEosZV95Ryydvf4136hrDjiYifaTCLyl536EVPH7dyXzz3KnUrtjKmf/9HNfcPUd9/YsMQjYYemqsqanx2trasGNIYFNjC7e9sJzfvbyShpZ23ndoBdecOoljDxkZdjQRSWJmr7t7zT/MV+GXvtq+q427XlnJbS8sZ/POVmaOH8FnTpnEKVMqSNy1KyJhUuGXAbOrtYMHalfz6+feZd32ZqaOGc6VJ47n3Olj1PunSIhU+GXAtbbH+ePctfz6+WW8U9fIsIJczp8xhg/XVHNUdan+ChBJMxV+SRt3p3blVu6fvZo/zV/PrrYOJo8axkdqqvng0QdRPqwg7IgikaDCL6FobGnnsXnreKB2NXNWbSM3Zpw+dRTnTR/LKVMqKC7MCzuiSNZS4ZfQvb2xgQdqV/PwG2vZ1NhKXo4xa2I5Zx1eyZmHV1I5vDDsiCJZRYVfMkZH3Hlj1VaeXLiRJxZsYOXmJgCOrC7lzMMrOevwSiaNGqZrAiL9pMIvGcndebuukScXbOCphRuZtybxg7BRxQUcP6GMWRPLOH5CGePLinQgEOklFX4ZFNZv38Uzi+t5ZdlmXl62efeoYKOHFwYHgZEcd0gZB+tAILJfKvwy6Lg7yzbt5OV3EweBV5dtZlNjKwDDC3OZXlXKe6pKmFFVwnuqShlbUqiDgUgSFX4Z9Nydd+sbmb1iK/PXbOfNtdtYvL6B9mCg+LKh+bynqoRpY4dzaGUxk0YNY2LFMArz1JW0RFNPhV8/q5RBw8yYNKqYSaOKufTYxLzmtg4Wb2jgzTXbmLdmO/PXbOPvb2+iIzgYxAzGjSxi0qhiDq0cxuTKYRxSPoyDRxZRWpSnvxAkklT4ZVArzMvhyOpSjqwu5fJgXmt7nOWbdvJ2XQNLNzbyTl0Db29s5Nkldbv/OgAoLsilemQR40YWcXBZ0e7psaVDGFNSyNAC/e8h2Un/ZUvWyc+NMWV0MVNGF+81v7U9zsrNO1mxuYmVm3eyeksTq7Y0sbSugacX19HaEd/r/cWFuYweXsjokkLGlBQyumQIo4cXUj4sn/LiAsqHFlBenK/+iGTQ0X+xEhn5uTEmVxYzubL4H5bF487GhmZWbm5iw/Zm1m9vZuOOZtZv38WG7c0s2dBAfWML3V0SG5KXQ3lxPmVDCygbmk9pUT6lRXmMKMqjpCifEUV5lA5JzCsZkkdxYS7DCnLJzdFwGBIOFX4RIBYzxpQMYUzJkB7f09YRp76hhc2NrWxqbAkerWwOpjfvbGXd9mYWrd/Btl1tNLV27HObQ/JyKC7MTRwICvMYHhwQivJzKcrPoaggh6HB9NCCxPOQvByGBM+FwWPP6xgFuTnkxHTdQvZNhV8kRXk5McaWDmFsac8Hh2Qt7R1sb2pj2642tjW1sa2ple272mhobqehuZ3Glj3TDS3tNDS3sXFHMztbOmhqbWdnawet7fH9b6iL3JhRkBujIC+HgtwY+bmx3c/5OTHycvZM5+cmXifmGbmxGLk5Rn5O4jk3FiMvx8jNiZEbM3JjRk4wnRO87lwWsz3zYp3PZuTm7FmWY4YZiemYETN2L4sFyzpfd04nHomL+8nPye839rzWBfv9C6Xwm9k5wI1ADnCLu18fRg6RgVSQm8Oo4TmM6kcfRG0dcZpagwNBSzvNbXF2tXWwq7WD5rYOdrUlnpvbEu9rbY/T0t75nJhuaY/T0paYbutwWjviNLa009oep60jHjwn5rd3xGnvnI777rujBpuuBwQS/yQODOw5gBgkLQsOHOw5eHR+BpKXJdbReXxJfn/y9vcst93zOt9Pl23sPbH3e/7zg+854KPbpb3wm1kO8EvgTGANMNvMHnH3henOIpLp8nJilAyJUTIknF5M43GnPe60x4MDQYfvPiC0dcTpSFreEXficWiPx4m709E5nTQvHocOd+JxJ+57pjvijkPwnsSyuDvuwfuC5YnXiXnue96fWBZ8Jnjf7uUOjhP8k7QsMb/zuo37nvXAnmV75gWvO9cHu5fT5XPsnt6z7r1fd788eV7nxNCCA/87lDDO+I8F3nH3ZQBmdh9wIaDCL5JhYjEjP2bkowvR2SSMf5sHAauTXq8J5u3FzK4ys1ozq62vr09bOBGRbBdG4e/uyss/NCS6+83uXuPuNRUVFWmIJSISDWEU/jVAddLrKmBdCDlERCIpjMI/G5hsZoeYWT7wUeCREHKIiERS2i/uunu7mV0LPEHids7b3H1BunOIiERVKPfxu/ufgT+HsW0RkajTPVoiIhGjwi8iEjGDYgQuM6sHVvbx4+XApgMY50BStr5Rtr5Rtr4ZzNkOdvd/uB9+UBT+/jCz2u6GHssEytY3ytY3ytY32ZhNTT0iIhGjwi8iEjFRKPw3hx1gH5Stb5Stb5Stb7IuW9a38YuIyN6icMYvIiJJVPhFRCImqwu/mZ1jZkvM7B0z+2rYeZKZ2Qoze9PM5ppZbchZbjOzOjN7K2neSDN7yszeDp5HZFC2b5vZ2mDfzTWzD4SUrdrMnjGzRWa2wMyuC+aHvu/2kS30fWdmhWb2mpnNC7J9J5ifCfutp2yh77cgR46ZvWFmjwWv+7TPsraNPxjicSlJQzwCl2bKEI9mtgKocffQfxhiZicDjcBv3f2IYN4PgS3ufn1w0Bzh7l/JkGzfBhrd/cfpztMl2xhgjLvPMbNi4HXgIuCThLzv9pHtI4S87ywx2OxQd280szzgBeA64GLC3289ZTuHzPhv7gtADTDc3c/r6/+n2XzGv3uIR3dvBTqHeJQu3P15YEuX2RcCdwbTd5IoGmnXQ7aM4O7r3X1OMN0ALCIxmlzo+24f2ULnCY3By7zg4WTGfuspW+jMrAo4F7glaXaf9lk2F/6UhngMkQNPmtnrZnZV2GG6Uenu6yFRRIBRIefp6lozmx80BYXSDJXMzMYDRwGvkmH7rks2yIB9FzRZzAXqgKfcPWP2Ww/ZIPz99lPgy0A8aV6f9lk2F/6UhngM0YnufjTwfuCaoElDUvM/wETgSGA98JMww5jZMOBB4HPuviPMLF11ky0j9p27d7j7kSRG4DvWzI4II0d3esgW6n4zs/OAOnd//UCsL5sLf0YP8eju64LnOuBhEk1TmWRj0E7c2V5cF3Ke3dx9Y/A/Zxz4DSHuu6Ad+EHgbnd/KJidEfuuu2yZtO+CPNuAZ0m0oWfEfuuUnC0D9tuJwAXBtcH7gNPM7C76uM+yufBn7BCPZjY0uOCGmQ0FzgLe2ven0u4R4Ipg+grgjyFm2Uvnf+iBDxLSvgsuBN4KLHL3G5IWhb7vesqWCfvOzCrMrDSYHgKcASwmM/Zbt9nC3m/u/jV3r3L38SRq2dPufhl93WfunrUP4AMk7ux5F/hG2HmSck0A5gWPBWFnA+4l8edrG4m/lP4FKAP+BrwdPI/MoGy/A94E5gf/4Y8JKdtJJJoP5wNzg8cHMmHf7SNb6PsOmA68EWR4C/hWMD8T9ltP2ULfb0kZTwEe688+y9rbOUVEpHvZ3NQjIiLdUOEXEYkYFX4RkYhR4RcRiRgVfhGRiFHhl0gws8bgebyZfewAr/vrXV6/dCDXL3KgqfBL1IwHelX4g55e92Wvwu/uJ/Qyk0haqfBL1FwPvDfoU/3zQYdcPzKz2UEHXP8XwMxOsUR/9veQ+OEOZvaHoFO9BZ0d65nZ9cCQYH13B/M6/7qwYN1vWWLshUuS1v2smf3ezBab2d3BL20xs+vNbGGQJdQugCV75YYdQCTNvgp8yd3PAwgK+HZ3n2lmBcCLZvZk8N5jgSPcfXnw+kp33xL8lH+2mT3o7l81s2s90alXVxeT6NRrBlAefOb5YNlRwDQS/Ue9CJxoZgtJdAdwmLt7Z9cBIgeazvgl6s4CPhF0w/sqiZ/ATw6WvZZU9AH+n5nNA14h0QHgZPbtJOBeT3TutRF4DpiZtO41nuj0ay6JJqgdQDNwi5ldDDT187uJdEuFX6LOgM+6+5HB4xB37zzj37n7TWankOiwa5a7zyDRn0thCuvuSUvSdAeQ6+7tJP7KeJDEgBqP9+J7iKRMhV+ipgEoTnr9BPDpoAtjzOzQoMfUrkqAre7eZGaHAccnLWvr/HwXzwOXBNcRKoCTgdd6Chb0nV/i7n8GPkeimUjkgFMbv0TNfKA9aLK5A7iRRDPLnOACaz3dD1/3OHC1mc0HlpBo7ul0MzDfzOa4+8eT5j8MzCLRC6sDX3b3DcGBozvFwB/NrJDEXwuf79M3FNkP9c4pIhIxauoREYkYFX4RkYhR4RcRiRgVfhGRiFHhFxGJGBV+EZGIUeEXEYmY/w9AIqQiaYE42QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(costs)\n",
    "plt.xlabel(\"Iterations\");\n",
    "plt.ylabel(\"cost function\");\n",
    "plt.title(\"Cost function vs Epoch\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "24f98be4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEWCAYAAABhffzLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAq3klEQVR4nO3dd5xU5dn/8c+1vbKwhbLAUhYEAQEFUcBYkxiNLYotmsQW1JQnmvakPOl50jSJaRZsGJOoiebJzxI1xoIFEEGaCCi9uwssbReWLdfvjznoQthlFnbmzOx836/Xee3MmTPnfHd25zpn7nPmvs3dERGR1JEWdgAREYkvFX4RkRSjwi8ikmJU+EVEUowKv4hIilHhFxFJMSr8khTM7EYze8/MdplZSRy3+y0zuyde2ztSZnaqma0LO4ckNhV+aRcz+6SZzQ4K8EYze9rMTjrCda4ysw+38Xgm8Cvgo+5e4O5bjmR7bWznP4qmu//E3a+LxfY6gpm5mQ3qoHW1e6dhZv3N7EUzqzOzJW39HSVxqPBL1Mzsy8BtwE+AHkAFcDtwfow33QPIARbFeDvSfg8Bc4ES4NvAo2ZWFm4kOSR316TpkBNQBOwCLm5jmWwiO4YNwXQbkB08Vgo8CWwDtgKvEDnweBBoBnYH6//6Aes8CqgFPHj8BaB/cD+jxXIvAdcFt68CXgVuBWqAlcBZLZYtBu4PMtYA/wDygwzNwXZ2AeXA94E/tXjueUR2QNuCbR7d4rFVwFeBBcB24BEgp5XXaRswosW8smD73Vt7rQ6ynpeD16E2yHspcCqwDvgKUAVsBK4+YNu3AmuA94A7gdw2fv9xwIwgy0bg90BWi79NPVDYYv2vADeE/f+qqe1JR/wSrfFEjrr/r41lvg2cCIwGRhEpGv8TPPYVIgWpjMgR/LcAd/dPESlC53qkGecXLVfo7u8Aw4O7Xd399CjzngAsJVJEfwHca2YWPPYgkBestzvwa3evBc4CNgQ5Ctx9Q8sVmtlRRI5wbwp+j38CT5hZVovFLgE+BgwARhLZCe3H3euBvwOXH/C8ae5eRSuv1UHWc3Jwc1SQ95Hgfk8iO+rewLXAH8ysW/DYz4kU7NHAoGCZ77bx+zcBNwev43jgDOBzwbqGAyvcfWeLWPP54O8lCUqFX6JVAmx298Y2lrkC+KG7V7l7NfAD4FPBYw1AL6Cfuze4+yvuHsuOola7+93u3gQ8EGy7h5n1IlLgbnD3miDLtCjXeSnwlLs/5+4NRI6cc4EJLZb5rbtvcPetwBNECuzB/IX9C/8ng3lw5K9VA5G/Q4O7/5PI0fuQYMf3WeBmd98aFOyfAJe1tiJ3n+PuM9290d1XAXcBpwQPFxD5ZNPSdqCwHVklBCr8Eq0tQKmZZbSxTDmwusX91cE8gFuAZcC/zGyFmX0jNjHft2nfDXevC24WAH2Bre5ecxjr3O/3c/dmYC2Ro+b/2C5QF2zzYF4Acs3sBDPrR2QHse/T1JG+VlsO2EHvy1FG5JPOHDPbZmbbgGeC+QdlZkeZ2ZNmtsnMdhDZUZQGD+8CuhzwlC7ATiShqfBLtGYAe4AL2lhmA9Cvxf2KYB7uvtPdv+LuA4FzgS+b2RnBcu098q8Nfua1mNczyueuBYrNrOtBHjtUjv1+v+AIui+wPsptf7ChyE7jr0SO+j8JPLmvyeQQr9WR2EykHX+4u3cNpiJ337dzOtjvfwewBBjs7l2INDvtazJbBAw0s5ZH+KPQSfiEp8IvUXH37cB3ibQXX2BmeWaWaWZnmdm+dvmHgP8xszIzKw2W/xOAmZ1jZoOCYrmDSNtxU/C894CB7chSTaTYXmlm6WZ2DVAZ5XM3Ak8Dt5tZt+B32NdW/h5QYmZFrTz9r8DHzeyM4BLTrxA5uTk92uwH+AuR5qMr+KCZ51Cv1YGifu2Cnc3dwK/NrHuwrd5mdmaLdR34+xcGGXaZ2VDgxhbreweYB3zPzHLM7BNEzms8Fk0eCY8Kv0TN3X8FfJnICdtqIkfPXyByVQzAj4HZRK5qWQi8GcwDGAz8m0jzwAzgdnd/KXjsp0R2GNvM7KtRxvks8DUiTVDDaV/x/RSRdvAlRK58uSn4/ZYQ2XmtCLKUt3ySuy8FrgR+R+To+VwiJ6X3tmPbLdf3OpFPL+VEdkb7tPVaHej7wANB3kui2Ox/E2lGmhk03fwbGBLkOdjv/1Uin0h2EtlpPHLA+i4DxhK5OupnwKRgxywJzGJ7fk1ERBKNjvhFRFKMCr+ISIpR4RcRSTEq/CIiKaatL+MkjNLSUu/fv3/YMUREksqcOXM2u/t/fEEvKQp///79mT17dtgxRESSipmtPth8NfWIiKQYFX4RkRSjwi8ikmJU+EVEUowKv4hIilHhFxFJMSr8IiIpplMX/unLNnPHS8vDjiEiklA6deF/6Z1qbnl2CWu21B16YRGRFNGpC/91Jw0gIy2NO1/WUb+IyD6duvB375LDpLF9eHT2Ot7bsSfsOCIiCaFTF36AG06upLG5mXtfXRl2FBGRhNDpC39FSR7njSrnTzNXs63usIZGFRHpVDp94Qe48dRB1O1tYur0VWFHEREJXUoU/iE9C/nw0T24/7VV1NY3hh1HRCRUKVH4AT53WiXbdzfw0Kw1YUcREQlVyhT+4yq6MaGyhCkvr6C+sSnsOCIioUmZwg/w+dMGUbWznsfmrA87iohIaFKq8E+oLGFUnyLunLacxqbmsOOIiIQipQq/mfG50waxZmsdTy3cGHYcEZFQpFThB/jI0T0Y3L2A219cTnOzhx1HRCTuUq7wp6UZN55aydL3dvLCkqqw44iIxF3KFX6Ac0eV06dbLn94aRnuOuoXkdSSkoU/Mz2N60+pZO6abcxcsTXsOCIicRWzwm9m95lZlZm91WLeLWa2xMwWmNn/mVnXWG3/UC4e04fSgmxuf2lZWBFEREIRyyP+qcDHDpj3HDDC3UcC7wDfjOH225STmc51HxrAK+9uZsG6bWHFEBGJu5gVfnd/Gdh6wLx/ufu+znJmAn1itf1oXHFCBV1yMrj9RQ3UIiKpI8w2/muAp1t70Mwmm9lsM5tdXV0dkwCFOZlcNaE/zyzaxLvv7YzJNkREEk0ohd/Mvg00An9ubRl3n+LuY919bFlZWcyyXDVxALmZ6dwxTUf9IpIa4l74zewzwDnAFZ4A11IW52dx+bgK/t+8DazdqkHZRaTzi2vhN7OPAf8NnOfuCVNlP3vyANIM7n5lRdhRRERiLpaXcz4EzACGmNk6M7sW+D1QCDxnZvPM7M5Ybb89ehXlctFxfXj4jbVU7dSg7CLSucXyqp7L3b2Xu2e6ex93v9fdB7l7X3cfHUw3xGr77XX9KZU0NjVz36urwo4iIhJTKfnN3YMZUJrP2cf04k8zV7N9d0PYcUREYkaFv4XPnTqIXfWNPDhjVdhRRERiRoW/hWHlXTh9aHfue20VdXs1KLuIdE4q/Af43KmVbK3dy8Oz1oYdRUQkJlT4DzC2fzHjBhRz9ysr2Nuo4RlFpPNR4T+Iz582iI3b9/CPuRqUXUQ6HxX+gzh5cCnDy7twx7TlNGl4RhHpZFT4D8LM+Pxpg1i5uZan39Kg7CLSuajwt+LM4T0ZWJbP7S8u1/CMItKpqPC3Ij3NuPGUSt7euIOX3olNt9AiImFQ4W/D+aN7U16Uw+0vanhGEek8VPjbkJWRxuSTB/LGqhpmrdSg7CLSOajwH8Klx1dQkp/Fb59/V239ItIpqPAfQm5WOp8/bRCvLtvMC0uqwo4jInLEVPij8Knx/agsy+dHT76tb/OKSNJT4Y9CZnoa3z13OKu21HH/ayvDjiMickRU+KN0ylFlnDG0O797YZlG6RKRpKbC3w7f/vjR1Dc2ceuzS8OOIiJy2FT422FgWQFXTxzA3+asY8G6bWHHERE5LCr87fTF0wdRkp/FD554W5d3ikhSUuFvp8KcTL5+5lDmrK7h8fkbwo4jItJuMSv8ZnafmVWZ2Vst5hWb2XNm9m7ws1usth9Lk8b04ZjeRfz0n0s0RKOIJJ1YHvFPBT52wLxvAM+7+2Dg+eB+0klLM7537jA27djDnS8tDzuOiEi7xKzwu/vLwIEd3JwPPBDcfgC4IFbbj7Wx/Ys5b1Q5d728grVb68KOIyIStXi38fdw940Awc/urS1oZpPNbLaZza6uTsxukb9x1lDM4GdPLwk7iohI1BL25K67T3H3se4+tqysLOw4B1XeNZcbTxnEUws3MnPFlrDjiIhEJd6F/z0z6wUQ/Ez6Xs+uP2Ugvbvm8oMn3tb4vCKSFOJd+B8HPhPc/gzw/+K8/Q6Xk5nOt84+msUbd/DwG2vCjiMickixvJzzIWAGMMTM1pnZtcDPgI+Y2bvAR4L7Se/sY3oybkAxtz67lO11DWHHERFpUyyv6rnc3Xu5e6a793H3e919i7uf4e6Dg5+dYlgrs8jlndt2N/Cb598NO46ISJsS9uRushleXsRlx1fwxxmrWFa1M+w4IiKtUuHvQF/96FHkZqXzwycXqx8fEUlYKvwdqKQgm5s+fBQvv1PNi0uT/oIlEemkVPg72KffH6ZxsYZpFJGEpMLfwTLT0/jOOcNYubmWqdM1TKOIJB4V/hg4dUh3Th/and8+v4zqnfVhxxER2Y8Kf4z8z8ePZk+DhmkUkcSjwh8jkWEa+/PXOWtZuG572HFERN6nwh9DXzxjcDBM4yJd3ikiCUOFP4a65GTytTOHMFvDNIpIAlHhj7FJY/oyoncXfva0hmkUkcSgwh9j6WnG984dzsbte7hz2oqw44iIqPDHw/H9izl3VDl3TVuuYRpFJHQq/HHyzbOGkpFmfO3R+TRrwBYRCdEhC7+ZTTSz58zsHTNbYWYrzUxtFu1U3jWX7503nJkrtnLPq3r5RCQ8GVEscy9wMzAHaIptnM7t4jF9eGFxFbc8u5SJg0oZXl4UdiQRSUHRNPVsd/en3b0qGEhli7trZPHDYGb85MJj6JaXxU0Pz2NPg/ajIhJ/0RT+F83sFjMbb2bH7ZtinqyTKs7P4paLR/Fu1S5+9vSSsOOISAqKpqnnhODn2BbzHDi94+OkhlOOKuOqCf2ZOn0Vpw3tzilHlYUdSURSyCELv7ufFo8gqeYbZw3ltWWb+erf5vPsTSdTnJ8VdiQRSRHRXNVTZGa/MrPZwfRLM9NZySOUk5nObZeNZlvdXr7194Xqy0dE4iaaNv77gJ3AJcG0A7j/SDZqZjeb2SIze8vMHjKznCNZX7IaXl7E184cwjOLNvG3OevCjiMiKSKawl/p7t9z9xXB9ANg4OFu0Mx6A/8FjHX3EUA6cNnhri/ZXXfSQMYPLOEHjy9i9ZbasOOISAqIpvDvNrOT9t0xs4nA7iPcbgaQa2YZQB6Qsl1XpqUZv7xkFGlpxs2PzKOxSeP0ikhsRVP4bwT+YGarzGw18HvghsPdoLuvB24F1gAbiXxP4F8HLmdmk/edV6iurj7czSWF8q65/O8njuHNNdu4/aXlYccRkU7ukIXf3ee5+yhgJHCMux/r7vMPd4Nm1g04HxgAlAP5ZnblQbY7xd3HuvvYsrLOf7njeaPKuWB0Ob95/l3mrd0WdhwR6cRaLfz7irGZfdnMvgxcB1zX4v7h+jCw0t2r3b0B+Dsw4QjW12n84PwR9OySw00Pz6W2Xn33i0hstHXEnx/8LDzIVHAE21wDnGhmeWZmwBnA4iNYX6dRlJvJLy8Zxeqtdfz4qbfDjiMinVSrX+By97uCm/9299daPhac4D0s7v66mT0KvAk0AnOBKYe7vs7mxIElXH9yJXdOW85pQ7rz0eE9w44kIp1MNCd3fxflvKgFl4cOdfcR7v4pd68/kvV1Nl/+yFEM69WFb/x9IVU794QdR0Q6mbba+Meb2VeAsn3t+sH0fSLX3kuMZGWk8dvLR1Nb38jXH12gb/WKSIdq64g/i0hbfgb7t+/vACbFPlpqG9S9kG9//GheWlrNn2auDjuOiHQibbXxTwOmmdlUd1flCcGnTuzH84ur+PFTixlfWcKg7oVhRxKRTiCaNv57zKzrvjtm1s3Mno1dJNnHzLjl4pHkZ2dw0yPz2Nuob/WKyJGLpvCXuvu2fXfcvQboHrNEsp/uhTn89MJjeGv9Dm779zthxxGRTiCawt9sZhX77phZPyIDsUicnDm8J5cd35c7pi1n1sqtYccRkSQXTeH/NvCqmT1oZg8CLwPfjG0sOdB3zhlGRXEeNz8yjx17GsKOIyJJLJq+ep4BjgMeAf4KjHF3tfHHWX52Br++dDSbduzhm48tpLlZH7pE5PBEc8QPkA1sBbYDw8zs5NhFktYcV9GNr585hKcWbuTXau8XkcN0yDF3zeznwKXAImDfZSVOpMlH4mzyyQNZUV3L715YRr+SfCaN6RN2JBFJMocs/MAFwBB1q5AYzIwff2IEa2vq+ObfF9CnWy4nDiwJO5aIJJFomnpWAJmxDiLRy0xP444rxlBRnMf1D85hRfWusCOJSBKJpvDXAfPM7C4z++2+KdbBpG1FeZncf9U40tOMa6a+QU3t3rAjiUiSiKbwPw78CJgOzGkxScgqSvK4+9Nj2LB9D9c/OIf6xqawI4lIEjhkG7+7PxCPIHJ4xvQr5taLR/FfD83lm48t5JeXjCIyvo2IyMFFc1XPSg7yTV13HxiTRNJu540qZ/XmWn753Dv0L83nv84YHHYkEUlg0VzVM7bF7RzgYqA4NnHkcH3h9EGs3FLLr557h34leZw/unfYkUQkQUXzzd0tLab17n4bcHrso0l7mBk/vfAYxg0o5mt/W8DsVerTR0QO7pCF38yOazGNNbMbiAzIIgkmOyOdu64cQ+9uuUx+cA6rt9SGHUlEElA0V/X8ssX0UyL99lwSy1By+LrlZ3HfVcfT7M41U99ge506dBOR/bU15u6XgpvfcffTgukj7j7Z3ZfGKZ8chgGl+dx15RjWbK3jxj/P0QAuIrKfto74rw5+dviXtcysq5k9amZLzGyxmY3v6G2kuhMGlvCzC0cyffkWvvOPtzRgu4i8r62rehab2SqgzMwWtJhvgLv7yCPY7m+AZ9x9kpllAXlHsC5pxUVj+rBqS6RDt/6l+dx4amXYkUQkAbQ12PrlZtYTeBY4r6M2aGZdgJOBq4Lt7AXU30CMfPkjR7FqSx0/f2YJ/UryOPuYXmFHEpGQtXkdv7tvAkZ18DYHAtXA/WY2ikj3D19yd12CEgNmxi2TRrK+po6bH5lHeddcRvftGnYsEQlRtAOxdKQMIlcG3eHuxwK1wDcOXMjMJpvZbDObXV1dHe+MnUpOZjp3f3os3btkc90Ds1lXUxd2JBEJURiFfx2wzt1fD+4/SmRHsB93n+LuY919bFlZWVwDdkYlBdncf9Xx1Dc2cc3UNzRur0gKi+YLXBdHMy9aQfPRWjMbEsw6A3j7cNcn0RvUvZA7rxzDiuparv/jHOr2NoYdSURCEM0R/zejnNceXwT+HFwtNBr4yRGuT6I0cVApt1w8ktdXbuEz981ip478RVJOqyd3zews4Gyg9wEDr3QBjuhQ0d3nsX/nbxJHnzi2Dxlpadz0yDw+de8sHrhmHEW5GmRNJFW0dcS/AZgN7GH/AVgeB86MfTSJpXNHlXP7FcexaMN2Pnn3TLZqBC+RlNFq4Xf3+cEgLIPc/YHg9uPAMneviVtCiZkzh/dkyqfH8m7VLi6fMpPqnfVhRxKROIimjf85M+tiZsXAfCLX3/8qxrkkTk4b0p37rzqeNVvruHTKDDZt3xN2JBGJsWgKf5G77wAuBO539zHAh2MbS+Jp4qBSHrhmHFU76rnkrhm6zl+kk4um8GeYWS8iXTE/GeM8EpJxA4p58Npx1NTt5dK7Zqovf5FOLJrC/0Mi/fUsd/c3zGwg8G5sY0kYjq3oxkOfPZG6vY1cctcMllXtCjuSiMRANEMv/s3dR7r7jcH9Fe5+UeyjSRhG9C7i4cnjaWp2Lpsyg6WbdoYdSUQ6WDTf3O1jZv9nZlVm9p6ZPWZmfeIRTsIxpGchD08eT3qacdmUGby1fnvYkUSkA0XT1HM/kcs4y4HewBPBPOnEBnUv4K/XjycvK4PL757J3DW6gleks4im8Je5+/3u3hhMUwH1mpYC+pXk88j1J1Kcn8WV97zOrJVbw44kIh0gmsK/2cyuNLP0YLoS2BLrYJIY+nTL45HJ4+lRlMNn7pvFa8s2hx1JRI5QNIX/GiKXcm4CNgKTgnmSInoW5fDI5PFUFOdx9dQ3eHFpVdiRROQIRHNVzxp3P8/dy9y9u7tf4O6r4xFOEkdZYTYPTT6Rwd0LmPzH2fxr0aawI4nIYYrmqp4HzKxri/vdzOy+mKaShFScn8VfPnsiw8uLuPHPb/LX2WvDjiQihyGapp6R7r5t352gg7ZjY5ZIElpRbiYPXjuOEwcW8/VHF/D9xxfR0NQcdiwRaYdoCn+amXXbdyforK3NQdqlcyvMyeSBq8dx3UkDmDp9FVfc8zqbd6lnT5FkEU3h/yUw3cx+ZGY/BKYDv4htLEl0Gelp/M85w7jt0tHMX7uN8373KgvWbQs7lohEIZqTu38ELgLeA6qBC939wVgHk+RwwbG9eezGCZgZk+6cwWNz1oUdSUQOIaomG3d/Gw2ILq0Y0buIx78wkS/8ZS5f+dt83tqwnW+dfTSZ6dF8oBSReNM7UzpESUE2D147jmsmDuD+11bxqXtfZ4va/UUSkgq/dJiM9DS+e+4wfnXJKOau2cZ5v39NHbyJJCAVfulwFx7Xh0dvmIC7c9Ed0/m/uWr3F0kkoRX+oN+fuWamUb06oWP6FPH4F09idN+u3PzIfH705Ns06np/kYQQ5hH/l4DFIW5fYqy0IJs/XXcCV03oz72vruTT981ia+3esGOJpLxQCn8wkMvHgXvC2L7ET2Z6Gt8/bzi3XjyK2atrOPd3r6rdXyRkYR3x3wZ8HWj1s7+ZTTaz2WY2u7q6Om7BJDYmjenDozeMp9mdSXdO5//NWx92JJGUFffCb2bnAFXuPqet5dx9iruPdfexZWUa96UzGNmnK49/4SRG9unKlx6ex/cfX8SehqawY4mknDCO+CcC55nZKuBh4HQz+1MIOSQEZYXZ/Pm6E7h6Yn+mTl/F2b95hTmrNbKXSDzFvfC7+zfdvY+79wcuA15w9yvjnUPCk5mexvfOHc6frzuB+sZmJt05gx8/+baO/kXiRNfxS2gmDirl2ZtP5pPjKrjn1ZU6+heJk1ALv7u/5O7nhJlBwlWQncH/fuIYHf2LxJGO+CUh6OhfJH5U+CVh6OhfJD5U+CXh6OhfJLZU+CUh6ehfJHZU+CWh6ehfpOOp8EvC09G/SMdS4ZekceDR/4d/NY0n5m/A3cOOJpJUVPglqew7+v/LZ0+gIDuDLz40lwvvmK7mH5F2UOGXpDShspSn/utD/OKikayv2c1Fd8zg839+kzVb6sKOJpLwLBk+Jo8dO9Znz54ddgxJULX1jUx5eQVTXl5BU7PzmQn9+MJpgynKyww7mkiozGyOu489cL6O+CXp5WdncPNHjuKlr53KBceWc8+rKznl1he579WV7G3UcI8iB1Lhl06jR5ccfjFpFE998UOMKC/ih0++zUd/PY1n3tqkE8AiLajwS6czrLwLD147jvuvOp7M9DRu+NMcLr1rJgvWbQs7mkhCUOGXTsnMOG1od57+0of430+MYMXmXZz3+9e46eG5rN+2O+x4IqHSyV1JCTv3NHDntOXc88pKHLhm4gAmnzyQ4vyssKOJxExrJ3dV+CWlbNi2m1ufXcrf564nNzOdy8b15boPDaR319ywo4l0OBV+kRaWVe3kzmkr+Mfc9QCcP7o3N5wykME9CkNOJtJxVPhFDmLDtt3c88pKHpq1ht0NTXx0WA9uPLWSYyu6hR1N5Iip8Iu0YWvtXh6Yvoqp01exfXcDJw4s5sZTB3Hy4FLMLOx4IodFhV8kCrX1jTw0aw33vLKSTTv2MLy8CzeeWslZI3qRnqYdgCQXFX6Rdtjb2Mw/5q7nzmnLWbG5lv4leVx/SiUXHteb7Iz0sOOJRCVhCr+Z9QX+CPQEmoEp7v6btp6jwi9haWp2nnt7E7e/tJwF67bTvTCbqycO4JKxfSgpyA47nkibEqnw9wJ6ufubZlYIzAEucPe3W3uOCr+Ezd2ZvnwLt7+0jNeWbSEz3ThzeE8+Oa6C8ZUlOg8gCam1wp8R7yDuvhHYGNzeaWaLgd5Aq4VfJGxmxsRBpUwcVMq77+3koVlreezNdTy5YCP9S/K4fFwFF43pQ6k+BUgSCLWN38z6Ay8DI9x9xwGPTQYmA1RUVIxZvXp1/AOKtGFPQxPPvLWJv7y+hlmrtpKZbnx036eAgSWk6WSwhCxhmnre37BZATAN+F93/3tby6qpRxLdsqoPPgVsq2ugX0kelx1fwaQxfSgr1KcACUdCFX4zywSeBJ51918dankVfkkWexqaeHZR5FPA6yuDTwHDenL5uAomVOpTgMRXwhR+i5wFewDY6u43RfMcFX5JRsuqdvHIG2t4dM46auoa6Fucy3mjyjlnZDlDexbqhLDEXCIV/pOAV4CFRC7nBPiWu/+zteeo8Esyq2+MnAt4dM46pi/fQlOzM6h7AeeOLOecUb2oLCsIO6J0UglT+A+HCr90Fpt31fP0W5t4cv4GZq3aijsM69WFc0b14tyR5fQtzgs7onQiKvwiCea9HXt4asFGnliwgblrtgEwqm9Xzh3Zi3NGltOzKCfcgJL0VPhFEtjarXU8tXAjT8zfwKINOzCD4/sVc+6oXnx0eE96dNFOQNpPhV8kSayo3sWTCyI7gXerdgEwoncXTh/agzOGdueY3kW6OkiiosIvkoTeeW8n/178Hi8sruLNNTU0O5QWZHP60DJOH9qDkwaXUpAd9y/gS5JQ4RdJcjW1e5n2TjXPL6nipaVV7NzTSGa6ceLAEk4f2p0zhvagokQnh+UDKvwinUhDUzNzVtfwwpIqnl/8HsurawEY1L2A04aUMWFQKeP6F5OvTwMpTYVfpBNbvaWWF5ZU8cKSKl5fsZW9Tc1kpBmj+3ZlQmUJ4ytLObaiKzmZGksglajwi6SI3XubmLO6hunLNzN9+RYWrNtGs0N2Rhpj+3djQmUpEypLOKZ3ERnpaWHHlRhKmG6ZRSS2crPSOWlwKScNLgVgx54GZq3YyvTlW5i+fDO3PLsUgILsDE4YUMwJA4sZ068bw8uL9IkgRajwi3RyXXIy+fCwHnx4WA8AtuyqZ+aKrby2fDMzlm/h+SVVAGSlpzGidxfG9i/muIpujOnXTT2LdlJq6hFJcdU763lzTQ1vrq5hzuoaFqzfzt7GSDdaFcV5jO3XjeP6deO4im4c1aNAzUNJRG38IhKV+sYmFm3YwZxVkR3B7NU1bN5VD0BOZhrDenVhZJ+ujOhdxMg+RVSWFZCuL5QlJBV+ETks7s66mt3MWV3DwvXbWbhuO29t2E7d3iYAcjPTGV7ehWP6FHFM78g0oDRfnwwSgAq/iHSYpmZnRfUuFq7fzoJ121m4fjuLNmxnT0OkiSgrI42jehQwtGcXhvYs5OheXRjSs1BjEseZCr+IxFRjUzPLq2tZuH47SzftYMmmnSzZtJPqnfXvL1NakM3RvQoZ0qOQQd0L3p+65mWFmLzz0uWcIhJTGelpDOlZyJCehfvN37yrnqWbdrJ4476dwQ4enLma+sbm95cpLciisuyDHUFlWQEDSvMp75qr8wcxoMIvIjFVWpBN6aBsJg4qfX9eU7OzvmY3y6p3sqxq1/vTE/M3sGNP4/vLZaWn0bc4lwGl+fQryad/aT4DSvKpKM6jV9ccMnUe4bCo8ItI3KWnGRUleVSU5HH60B7vz3d3qnfVs7yqltVbalm5pZZVm2tZvaWOV5dtfv8cAkCaQa+iXPp0y6VvcR59u+XRp1su5V1zKe+aQ8+iHLIz9IW0g1HhF5GEYWZ0L8yhe2EO4ytL9nusudl5b+ceVm2uY21NHeu21rFmax1ra3bz8jvVVLU4l7BPWWE25UU59CrKpWdRDt27ZNOzSw49gqmsMJsuORkpN/C9Cr+IJIW0NKNXUS69inIZT8l/PL6noYn123azcdseNmzfzYYWt5dV7+K15ZvZ2aIZaZ+sjDTKCrIpK8yONEsVZFFSkEVxfuR217wsivOy6JqXSbf8LPKz0pN+R6HCLyKdQk5mOpVlkRPDramtb6RqZz2btu+hauceqnfWfzDtqmddTR3z121ja+1empoPfsVjRppRlJtJUW4mXfZNORkU5mRSmJNBQfYHU152OvlZGeRmpZOXlU5uZjo5menkZkV+ZmekkZFmcd+RhFL4zexjwG+AdOAed/9ZGDlEJLXkZ2cwIDuDAaX5bS7X3Ozs2NPAltq91NTupaaugZq6yO3tuxvYtruB7bsb2BH8XLe1jh17GtlV37DfeYhomEVOYmdlpAU7gjQy0o2MNCMjPY2ffOIYxg0oPpJf+z/EvfCbWTrwB+AjwDrgDTN73N3fjncWEZGDSUszuuZFmnkoa99z9zY2U1vfSO3eRmrrm6jb20jd3iZ2721id0Nkqm9oYk9DM3samtjb1Ex9YzN7G5tpaGqmsclpaI78bGp28rM7/gR1GEf844Bl7r4CwMweBs4HVPhFJOllZaSRlZFFt/zE/VJaGBfB9gbWtri/Lpi3HzObbGazzWx2dXV13MKJiHR2YRT+g53F+I+zKO4+xd3HuvvYsrJ2ftYSEZFWhVH41wF9W9zvA2wIIYeISEoKo/C/AQw2swFmlgVcBjweQg4RkZQU95O77t5oZl8AniVyOed97r4o3jlERFJVKNfxu/s/gX+GsW0RkVSnru1ERFKMCr+ISIpJihG4zKwaqAU2h53lMJSSfLmVOX6SMXcyZobkzH2kmfu5+39cD58UhR/AzGYfbAixRJeMuZU5fpIxdzJmhuTMHavMauoREUkxKvwiIikmmQr/lLADHKZkzK3M8ZOMuZMxMyRn7phkTpo2fhER6RjJdMQvIiIdQIVfRCTFJFzhN7OPmdlSM1tmZt84yONmZr8NHl9gZseFkfOATIfKfEWQdYGZTTezUWHkPNChcrdY7ngzazKzSfHM10qWQ2Y2s1PNbJ6ZLTKzafHOeJA8h/r/KDKzJ8xsfpD56jByHpDpPjOrMrO3Wnk84d6HEFXuhHsvHipzi+U67n3o7gkzEem0bTkwEMgC5gPDDljmbOBpIv36nwi8ngSZJwDdgttnhZ052twtlnuBSN9KkxI9M9CVyGhuFcH97kmQ+VvAz4PbZcBWICvk3CcDxwFvtfJ4Qr0P25E7Ed+LbWZu8X/UYe/DRDvif39YRnffC+wblrGl84E/esRMoKuZ9Yp30BYOmdndp7t7TXB3JpExCMIWzWsN8EXgMaAqnuFaEU3mTwJ/d/c1AO4edu5oMjtQaGYGFBAp/I3xjXlAIPeXgxytSbT3IXDo3In4XozitYYOfh8mWuGPZljGqIZujKP25rmWyJFS2A6Z28x6A58A7oxjrrZE81ofBXQzs5fMbI6ZfTpu6Q4umsy/B44mMiDRQuBL7t4cn3iHLdHeh4cjUd6LbYrF+zCUbpnbEM2wjFEN3RhHUecxs9OI/LOdFNNE0Ykm923Af7t7U+RgNHTRZM4AxgBnALnADDOb6e7vxDpcK6LJfCYwDzgdqASeM7NX3H1HjLMdiUR7H7ZLgr0XD+U2Ovh9mGiFP5phGRNt6Mao8pjZSOAe4Cx33xKnbG2JJvdY4OHgn60UONvMGt39H3FJ+J+i/f/Y7O61QK2ZvQyMAsIq/NFkvhr4mUcac5eZ2UpgKDArPhEPS6K9D6OWgO/FQ+n492HYJzYOOIGRAawABvDBibDhByzzcfY/qTQrCTJXAMuACWG/xu3JfcDyUwn/5G40r/XRwPPBsnnAW8CIBM98B/D94HYPYD1QmgD/I/1p/SRpQr0P25E74d6Lh8p8wHId8j5MqCN+b2VYRjO7IXj8TiJntc8m8serI3K0FJooM38XKAFuD/bajR5yL4FR5k4o0WR298Vm9gywAGgG7nH3Ni+TCzsz8CNgqpktJFJI/9vdQ+0+2MweAk4FSs1sHfA9IBMS8324TxS5E+69GEXmjt9msBcREZEUkWhX9YiISIyp8IuIpBgVfhGRFKPCLyKSYlT4RURSjAq/pCwzKwl68ZxnZpvMbH1we5uZvd3OdV1gZsOiWC7bzB4JerV83cz6H/YvIHKYVPglZbn7Fncf7e6jifSD8uvg9mgi3wFojwuAQxZ+It0E1Lj7IODXwM/buR2RI6bCL3Jw6WZ2d9A//r/MLBfAzCrN7JmgA7hXzGyomU0AzgNuCT4xVJrZZ83sjaCP/cfMLC9Y7/nAA8HtR4EzLEE6QpLUocIvcnCDgT+4+3BgG3BRMH8K8EV3HwN8Fbjd3acDjwNfCz5BLCfSNfTx7j4KWEzkSB9a9Grp7o3AdiLfJBWJm4TqskEkgax093nB7TlAfzMrIDKQx99aHKRnt/L8EWb2YyIDwxQQ6bIBkrxXS+kcVPhFDq6+xe0mIl08pwHbgvMAhzIVuMDd55vZVUT6YoEPerVcZ2YZQBGHHoRDpEOpqUckSh7pH3+lmV0M7487u2/M1p1AYYvFC4GNZpYJXNFi/uPAZ4Lbk4AXXB1mSZyp8Iu0zxXAtWY2H1jEB8MoPgx8zczmmlkl8B3gdeA5YEmL598LlJjZMuDLQKuD3IvEinrnFBFJMTriFxFJMSr8IiIpRoVfRCTFqPCLiKQYFX4RkRSjwi8ikmJU+EVEUsz/B7fFzDv4VfrOAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(theta[:,0],costs);\n",
    "plt.xlabel(\"Theta0\");\n",
    "plt.ylabel(\"cost function\");\n",
    "plt.title(\"Cost function vs theta0\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2f1c88b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8168739114159269"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2_score(y,hypothesis[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a1a29d8",
   "metadata": {},
   "source": [
    "## **Multivariable Gradient Descent**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d51e1028",
   "metadata": {},
   "outputs": [],
   "source": [
    "def vectorized_gradient(x,y,learning_rate = 0.001,Epoch = 100):\n",
    "    theta = np.zeros((x.shape[1],1))\n",
    "    costs = []\n",
    "    thetas = []\n",
    "    hypothesis = []\n",
    "    for i in range(Epoch):\n",
    "        #calculate hypothesis\n",
    "        y_hat = np.dot(x,theta)\n",
    "        hypothesis.append(y_hat)\n",
    "        #calculate error\n",
    "        e = y_hat - y\n",
    "        #mean square error\n",
    "        mse = (np.dot(e.T,e)/(2*len(y)))[0][0]\n",
    "        thetas.append(theta.reshape(x.shape[1]))\n",
    "        costs.append(mse)\n",
    "        if(i>0 and np.abs(costs[i]-costs[i-1])<0.001):\n",
    "            print(\"stopped at iteration {}\".format(i))\n",
    "            break\n",
    "        #calculate gradients\n",
    "        gradient = np.dot(x.T,e)/len(y)\n",
    "        #update theta\n",
    "        theta = theta - learning_rate*gradient\n",
    "\n",
    "    return np.array(thetas), costs , hypothesis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "10e28108",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "stopped at iteration 91\n"
     ]
    }
   ],
   "source": [
    "theta,costs ,hypothesis = vectorized_gradient(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "05886e57",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEWCAYAAABhffzLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAp0ElEQVR4nO3deZwcdZ3/8dene67MZDKTzJE7mUlICCEBQoZAOJVDAREQFUFAFJHVHype66Lu6urqrqsui8e6yiLHyuEiyiEit4JcJpOQkEAIhNz3JJP7mqM/vz+qhjQhR2cy3TXd9X4+Uo/u+lZ1fT9dST5V/a2q79fcHRERiY9E1AGIiEhuKfGLiMSMEr+ISMwo8YuIxIwSv4hIzCjxi4jEjBK/5BUz+4yZrTGzrWZWk8N6v25mN+eqvt7OzBrMzM2sKOpY5OAp8Uu3mNlHzaw5TMCrzOxPZnbyIW5zsZmduZ/lxcANwHvcva+7rz+U+vZTz7vMbHl6mbv/q7tfnY36ekKYhLeFfx9d01ejjkt6Jx2t5aCZ2ZeA64FPA48CbcDZwAXAs1mseiBQBrySxTry2dHuviDqIKT30xm/HBQzqwK+A1zr7r93923u3u7uf3D3vw/XKTWzG81sZTjdaGal4bJaM3vIzDaaWauZ/dXMEmb2a2AE8Ie9na2a2Vhgfji70cye2ltzg5n9xcyuDt9/3MyeNbMfmdkGM1tkZuekrTvAzG4NY9xgZvebWQXwJ2BI2pnzEDP7ZzO7I+2z55vZK+H3+IuZHZG2bLGZfcXMXjazTWb2f2ZWtpd9WRp+fkJaWZ2Z7TCz+n3tq278nf2zmd0bxrHFzGaa2dFpy48Iv8PG8Dudn7asj5n9h5ktCb/Ls2bWJ23zl5nZUjNbZ2bfONjYJBpK/HKwphKcdd+3n3W+AZwAHAMcDUwB/jFc9mVgOVBHcAb/dcDd/QpgKfD+sBnnB+kbdPfXgSPD2Wp3Pz3DeI8nOGDUAj8AfmVmFi77NVAebrce+E933wacA6wM4+jr7ivTNxgehO4GvhB+j4cJDlglaatdTPArqBE4Cvj4noG5+y7g98Cle3zuaXdfyz72VYbfe08XAL8FBgB3AfebWXHYfPYH4LFwH3wOuNPMDg8/9yNgMnBi+NmvAqm07Z4MHA6cAXwz/QAovZcSvxysGmCdu3fsZ53LgO+4+1p3bwG+DVwRLmsHBgMjw18Kf/Xsdhi1xN3/x907gdvDugea2WCCBP9pd98QxvJ0htv8CPBHd3/c3dsJkmMfguTY5SfuvtLdWwkS6zH72NZdvD3xfzQsg4PfVzPDs/au6b1py2a4+71hvDcQHLxPCKe+wPfdvc3dnwIeAi4Nf11cBVzn7ivcvdPdnw8PWF2+7e473H02MJvgQC+9nBK/HKz1QO0B7uYYAixJm18SlgH8EFgAPGZmC83s+uyE+ZbVXW/cfXv4ti8wHGh19w3d2Obbvp+7p4BlwNC91QtsD+vcm6eAPmZ2vJmNJDhAdP2aOth9day7V6dNj6YtW7ZHvMvD7zEEWBaWdVkSfpdaggPEm/upM9PvKb2IEr8crBeAncCF+1lnJTAybX5EWIa7b3H3L7v7KOD9wJfM7IxwvYM9898WvpanlQ3K8LPLgAFmVr2XZQeK423fL2w6Gg6syLDu3RUFCfcegrP+jwIPufuWcNn+9tXBGp4WbwIYFn6PlcDwPa4djAi/yzqCv+vR3axTeiklfjko7r4J+CbwX2Z2oZmVh23F55hZV7v83cA/hhcqa8P17wAws/PM7LAwWW4GOsMJYA0w6iBiaSFIUJebWdLMriLDJOXuqwgu4v7czPqH3+HUtDhqLLiQvTf3AO8zszPCNvIvA7uA5zONfQ93ETQfXcbuZp4D7auDNdnMLgp/qX0hjPdF4G8EB9CvhvvgXQQHmd+EB6VbgBvCC9xJM5tq4YV6yV9K/HLQ3P0G4EsEF2xbCM6ePwvcH67yXaAZeBmYA8wMywDGAE8AWwl+Pfzc3f8SLvs3ggPGRjP7SobhfAr4e4ImqCM5uOR7BUE7+mvAWoKEiLu/RnDwWhjGMiT9Q+4+H7gc+CnBWfH7CS5Ktx1E3enb60q+QwgORl32t6/2Zra9/T7+G9OWPUBwcNlA8L0vCq8btAHnE1zvWAf8HPhYuA8AvkLwdzgdaAX+HeWNvGcaiEWksJnZPwOHufvlUccivYOO3CIiMaPELyISM2rqERGJGZ3xi4jETF500lZbW+sNDQ1RhyEikldmzJixzt3r9izPi8Tf0NBAc3Nz1GGIiOQVM1uyt3I19YiIxIwSv4hIzCjxi4jEjBK/iEjMKPGLiMSMEr+ISMwo8YuIxExBJ/4/z1/Lz/+yIOowRER6lYJO/M8vWMeNT7xBW0fqwCuLiMRE1hK/md1iZmvNbO4e5Z8zs/lm9kraiE1ZMWlEf9o6Ury6anM2qxERySvZPOO/DTg7vcDM3g1cABzl7kcCP8pi/Rw7oj8ALy3tznjaIiKFKWuJ392fIRiqLd1ngO+7+65wnbXZqh9gUFUZg6vKmLl0YzarERHJK7lu4x8LnGJmfzOzp83suH2taGbXmFmzmTW3tLR0u8JjR/TXGb+ISJpcJ/4ioD9wAsEA2feYme1tRXe/yd2b3L2pru4dvYpmbNKIapZv2MHaLTu7vQ0RkUKS68S/HPi9B6YBKaA2mxVOequdf2M2qxERyRu5Tvz3A6cDmNlYoARYl80KjxzSj+KkMVPNPSIiQBYHYjGzu4F3AbVmthz4FnALcEt4i2cbcKVnedDfsuIkRw6p0hm/iEgoa4nf3S/dx6LLs1XnvkwaUc3d05bS3pmiOFnQz6yJiBxQLLLgsSP6s7M9xfzVW6IORUQkcrFI/JNGVAOonV9EhJgk/qHVfaivLFU7v4gIMUn8ZsakEdU64xcRISaJH4J2/iXrt7N+666oQxERiVR8Ev/I4EEu9dsjInEXm8Q/cWgVJckE0xfv2W+ciEi8xCbxlxUnOXp4FdMWKfGLSLzFJvEDHNcwgLkrNrG9rSPqUEREIhOvxN84gI6U67ZOEYm1WCX+ySP7Y4aae0Qk1mKV+PuVFXPEoH66wCsisRarxA8wpXEALy3dSHtnKupQREQiEcvEv6O9k7krNkUdiohIJGKX+I9rGAConV9E4itrid/MbjGzteGgK3su+4qZuZllddjFvamrLKWxtkLt/CISW9k8478NOHvPQjMbDpwFLM1i3ft1XEN/pi/eQCqV1cG/RER6pawlfnd/BtjbafV/Al8FIsu6xzUMYNOOdt5YuzWqEEREIpPTNn4zOx9Y4e6zc1nvnqY0hu38au4RkRjKWeI3s3LgG8A3M1z/GjNrNrPmlpaWHo1lxIByBvYr5cWF63t0uyIi+SCXZ/yjgUZgtpktBoYBM81s0N5Wdveb3L3J3Zvq6up6NBAz48TRtbz45nrc1c4vIvGSs8Tv7nPcvd7dG9y9AVgOHOvuq3MVQ7qpo2tYv62N19eonV9E4iWbt3PeDbwAHG5my83sk9mqqztOHF0DwPNvros4EhGR3CrK1obd/dIDLG/IVt2ZGNa/nBEDynluwXo+cVJjlKGIiORU7J7cTXfSYTX8beF6OtRvj4jESKwT/9TRtWzZ1cErKzdHHYqISM7EO/GP6mrn122dIhIfsU78dZWljB3YVxd4RSRWYp34AU4cXcv0xa3s6uiMOhQRkZxQ4h9dw872FLM0Dq+IxETsE//xo2pImNr5RSQ+Yp/4q/oUM2FoFS8o8YtITMQ+8UPQzj9z6Qa27uqIOhQRkaxT4gdOHVtLR8p11i8isaDEDzSNHEB5SZKnX18bdSgiIlmnxA+UFCU4cXQNT7/eom6aRaTgKfGHThtbx7LWHSxevz3qUEREskqJP3Ta2HoAnp6v5h4RKWxK/KERNeU01lbw9Os9O8yjiEhvo8Sf5rSxdby4sJWd7eq+QUQKVzZH4LrFzNaa2dy0sh+a2Wtm9rKZ3Wdm1dmqvztOHVvLjvZOmhdviDoUEZGsyeYZ/23A2XuUPQ5McPejgNeBr2Wx/oN2wqgaSpIJ3dYpIgUta4nf3Z8BWvcoe8zdux6PfREYlq36u6O8pIgpjQPUzi8iBS3KNv6rgD/ta6GZXWNmzWbW3NKSu0R82tg6Xl+zlVWbduSsThGRXIok8ZvZN4AO4M59rePuN7l7k7s31dXV5Sy20w4P6vrzazrrF5HClPPEb2ZXAucBl3kvfEx2TH1fhg/ow5Pz1kQdiohIVuQ08ZvZ2cA/AOe7e698RNbMOPOIgTy7YB3b29Rbp4gUnmzeznk38AJwuJktN7NPAj8DKoHHzWyWmf0iW/UfirOOGMiujhTPvqGxeEWk8BRla8Pufulein+Vrfp60nGNA6gsK+LJeWt5z5GDog5HRKRH6cndvShOJnjX4fU8+doaUqledxlCROSQKPHvw5lH1LNuaxuzlm+MOhQRkR6lxL8P7xpbTzJhPPGq7u4RkcKixL8PVeXFTGkYwJPz1H2DiBQWJf79OOOIeuav2cJSDc4iIgVEiX8/zho/EIAn9DCXiBQQJf79GFlTwZj6vjz6yuqoQxER6TFK/AdwzsTBTF/cSsuWXVGHIiLSI5T4D+DciYNIOTz2qs76RaQwKPEfwOEDKxlVW8HDc1ZFHYqISI84YOI3s5PM7HEze93MFprZIjNbmIvgegMz45yJg3hxYSvrt6q5R0TyXyZn/L8CbgBOBo4DmsLX2Dh34mA6U87jephLRApAJol/k7v/yd3Xuvv6rinrkfUi4wf3Y2RNOQ/PVTu/iOS/TBL/n83sh2Y21cyO7ZqyHlkvYmacM2Ewzy9Yx8btbVGHIyJySDLplvn48LUprcyB03s+nN7r3ImD+MXTb/L4q2v4cNPwqMMREem2AyZ+d393LgLp7SYOrWJodR/+NHe1Er+I5LVM7uqpMrMbzKw5nP7DzKoy+NwtZrbWzOamlQ0I7xB6I3ztf6hfIFfMjHMnDuKvb7SouUdE8lombfy3AFuAi8NpM3BrBp+7DTh7j7LrgSfdfQzwZDifNy44Zijtnc7Dc3SRV0TyVyaJf7S7f8vdF4bTt4FRB/qQuz8DtO5RfAFwe/j+duDCgwk2akcO6cfougrun7Ui6lBERLotk8S/w8xO7poxs5OAHd2sb6C7rwIIX+v3taKZXdPVvNTS0tLN6nqWmXHhMUOZtqiVFRu7uwtERKKVSeL/DPBfZrbYzJYAPwM+nd2wwN1vcvcmd2+qq6vLdnUZu+CYoQA8OGtlxJGIiHTPARO/u89y96OBo4CJ7j7J3Wd3s741ZjYYIHzNu+GtRtSUc+yIau5/Sc09IpKf9pn4zezy8PVLZvYl4Grg6rT57ngQuDJ8fyXwQDe3E6kLJw1l/potzFu1OepQREQO2v7O+CvC18q9TH0PtGEzuxt4ATjczJab2SeB7wNnmdkbwFnhfN5538TBJBOmi7wikpf2+QCXu/8yfPuEuz+Xviy8wLtf7n7pPhadkXl4vVNN31JOG1vHH2at5B/eO45EwqIOSUQkY5lc3P1phmWxcsExQ1i5aSd/W7TnHasiIr3bPs/4zWwqcCJQt0ebfj8gme3Aerv3jB9E39IifjtjGVNH10QdjohIxvZ3xl9C0JZfxNvb9zcDH8p+aL1bn5Ik7z96CA/PWcXmne1RhyMikrH9tfE/DTxtZre5+5IcxpQ3PnLccO6etpSHZq/io8ePiDocEZGMZNLGf7OZVXfNmFl/M3s0eyHlj6OHVXH4wEr+r3lZ1KGIiGQsk8Rf6+4bu2bcfQP76WohTsyMDzcNY/ayjcxfvSXqcEREMpJJ4k+Z2VvtGGY2kmAgFgEuOnYYxUnjHp31i0ieyCTxfwN41sx+bWa/Bp4BvpbdsPLHgIoSzho/kPteWkFbRyrqcEREDiiTvnoeAY4F/g+4B5js7mrjT/PhpuG0bmvjyXlrog5FROSAMjnjBygl6Ft/EzDezE7NXkj559QxdQyuKuOuaUujDkVE5IAOOOaumf078BHgFaCrLcMJmnwESCaMS44bwX8+8TqL1m2jsbbiwB8SEYlIJmf8FwKHu/v73P394XR+luPKO5dOGU5RwrjjRT3yICK9WyaJfyFQnO1A8l19vzLOnjCI3zYvY0dbZ9ThiIjsUyaJfzswy8x+aWY/6ZqyHVg++tjUBjbv7ODB2equWUR6rwO28RMMnvJgtgMpBMc19GfcoEr+94UlXNw0HDN11ywivc8BE7+7397TlZrZFwlG9HJgDvAJd9/Z0/Xkmplx+Qkj+cf75zJz6UYmj+wfdUgiIu9wwKYeM1tkZgv3nLpboZkNBT4PNLn7BIIuni/p7vZ6mw9MGkplaRG/fmFx1KGIiOxVJm38TcBx4XQK8BPgjkOstwjoY2ZFQDmw8hC312tUlBbxwcnD+OOcVazdkvc/YkSkAGXy5O76tGmFu98InN7dCt19BfAjYCmwCtjk7o/tuZ6ZXWNmzWbW3NLS0t3qIvHxExvoSDn/+7xu7RSR3ieTpp5j06YmM/s0wYAs3WJm/YELgEZgCFBhZpfvuZ673+TuTe7eVFdX193qItFQW8F7xw/i1y8uYXtbR9ThiIi8TSZNPf+RNv0bQb89Fx9CnWcCi9y9xd3bgd8TDPFYUD516ig27Wjnt83Low5FRORt9pn4zey68O0/ufu7w+ksd7/G3ecfQp1LgRPMrNyC+x3PAOYdwvZ6pckj+3PsiGpufnYhnSn1Yi0ivcf+zvg/Eb726MNa7v434F5gJsGtnAngpp6so7e45tRRLGvdwaOvrI46FBGRt+zvPv55ZrYYqDOzl9PKDXB3P6q7lbr7t4Bvdffz+eKs8YMYWVPOTc8s5JwJg/RAl4j0CvsbbP1SMxsEPAqoU7ZuSCaMq09u5J8eeIXpizcwpXFA1CGJiOz/4q67r3b3o919yZ5TrgLMdx+aPJyaihJ+9ucFUYciIgJkPhCLdFOfkiRXnzKKZ15vYdayjVGHIyKixJ8LV0wdSXV5MT976o2oQxERyegBrg9nUib71re0iE+e1MgT89Yyd8WmqMMRkZjL5Iz/axmWyX5ceVIDlWVF/OwptfWLSLT2eVePmZ0DnAsM3WPglX6A+iE4SP3KivnEiQ385KkFvLZ6M+MG9Ys6JBGJqf2d8a8EmoGdwIy06UHgvdkPrfBcdXIjFSVJfvqkzvpFJDr7u49/NjDbzO4K+9Tp6mBtuLtvyFWAhaS6vISrTm7kp08t4DMrNjFhaFXUIYlIDGXSxv+4mfUzswHAbOBWM7shy3EVrE+dOorq8mJ+8OihdHckItJ9mST+KnffDFwE3Orukwl62JRu6FdWzLXvOoxnXm/hhTfXRx2OiMRQJom/yMwGE3TF/FCW44mFK6aOZHBVGT949DXc1XOniORWJon/OwT99bzp7tPNbBSgJ5EOQVlxkuvOGMNLSzfy+Ktrog5HRGImk6EXf+vuR7n7Z8L5he7+weyHVtg+NHkYo2or+OGj8+noTEUdjojESCZP7g4zs/vMbK2ZrTGz35nZsFwEV8iKkgn+/r2H88barfxm+rKowxGRGMmkqedWgnv3hwBDgT+EZXKIzp4wiCmNA7jh8dfZtKM96nBEJCYySfx17n6ru3eE023AIY1+bmbVZnavmb1mZvPMbOqhbC9fmRnfPG88G7a38ZMnddlERHIjk8S/zswuN7NkOF0OHOp9iD8GHnH3ccDRFOCYu5maMLSKjzQN5/bnF/Nmy9aowxGRGMgk8V9FcCvnamAV8KGwrFvMrB9wKvArAHdvc/eN3d1eIfjyew6nrDjJ9/4Y2+OfiORQJnf1LHX38929zt3r3f3CQxyBaxTQQvAE8EtmdrOZVey5kpldY2bNZtbc0tJyCNX1fnWVpXzu9MN46rW1/Hn+2qjDEZECl8ldPbebWXXafH8zu+UQ6iwCjgX+290nAduA6/dcyd1vcvcmd2+qqzukSwp54eMnNTCqroJvPfAKO9s7ow5HRApYJk09R6U3xYQdtE06hDqXA8vd/W/h/L0EB4JYKy1K8t0LJ7C0dTs/1UhdIpJFmST+RNgrJwBhZ2377NXzQNx9NbDMzA4Pi84AXu3u9grJiaNruejYodz0zELeWLMl6nBEpEBlkvj/A3jezP7FzL4DPA/84BDr/Rxwp5m9DBwD/Oshbq9gfOPcI6goLeLr980hlVI/PiLS8zK5uPu/wAeBNQQXZS9y918fSqXuPitsvz8qvFis/v1DNX1L+fo5RzB98QZ+O0NP9IpIz8uoycbdX0XNMTnz4aZh3DtzOd/94zxOG1vPoKqyqEMSkQKSSVOP5JiZ8YMPHkV7Z4rrf/+yum4WkR6lxN9LNdRWcP3Z4/jL/BbuaVaTj4j0HCX+XuxjUxs4YdQA/uWheazYuCPqcESkQCjx92KJhPHDDx2Nu/MP976su3xEpEco8fdywweU8433jefZBeu45blFUYcjIgVAiT8PXDplOO89ciD//shrzFm+KepwRCTPKfHnATPj3z94FHV9S/nc3TPZuqsj6pBEJI8p8eeJ6vISbrxkEktbt/NP98+NOhwRyWNK/HlkSuMArjtjLPe9tEK3eIpItynx55nPnn4YJ46u4R/vn8vcFWrvF5GDp8SfZ5IJ46eXTqK2ooS/+/UMWre1RR2SiOQZJf48VNO3lF9cMZmWrbv4/N0v0dGZijokEckjSvx56qhh1Xz3wgk8u2AdP3x0ftThiEge6faAKhK9i5uGM2f5Jn75zEJG1/Xl4uOGRx2SiOSByM74zSwZDrb+UFQxFIJvvn88p4yp5ev3zeG5BeuiDkdE8kCUTT3XAfMirL8gFCcT/NdlxzKqroJP3zGDBWs1ZKOI7F8kid/MhgHvA26Oov5C06+smFs+fhylRUk+cdt01m7ZGXVIItKLRXXGfyPwVWCft6OY2TVm1mxmzS0tLTkLLF8N61/Or65sYv3WNj72q2ls2tEedUgi0kvlPPGb2XnAWnefsb/13P2mcFzeprq6uhxFl9+OHl7NL6+YzJstW/nkbdPZ0dYZdUgi0gtFccZ/EnC+mS0GfgOcbmZ3RBBHQTplTB0/vmQSM5du4DN3zqCtQ/f4i8jb5Tzxu/vX3H2YuzcAlwBPufvluY6jkJ07cTDf+8BE/jK/hc/f/RLtesBLRNLoAa4CdemUEXzzvPE88spqPneXkr+I7BZp4nf3v7j7eVHGUMiuOrnxreT/2btmqtlHRACd8Re8q05u5FvvH8+jr6zh/905k53tuuArEndK/DHwiZMa+ZcLjuSJeWv4+K3T2LJTt3qKxJkSf0xcMbWBH19yDM2LN3DJTS+ybuuuqEMSkYgo8cfIBccM5X+ubOLNlq18+BcvsHjdtqhDEpEIKPHHzLsPr+fOq09g4/Y2PvDz55i2qDXqkEQkx5T4Y2jyyP7cf+1J9K8o4bKbX+R3M5ZHHZKI5JASf0yNrKngvs+cxHENA/jyb2fzb3+ap5G8RGJCiT/GqsqLuf2qKVx2/Ah++fRCPnbLNNbroq9IwVPij7niZILvfWAiP/jQUTQv2cD7f/oss5ZtjDosEckiJX4BgmEcf/fpEzEzPvyL5/mfZxaSSnnUYYlIFijxy1smDqvij58/mdPH1fO9h+dx5a3TNKiLSAFS4pe3qS4v4ReXT+Z7H5jAtEWtnHPjX3lk7uqowxKRHqTEL+9gZlx2/Ege+tzJDKoq49N3zOC637zEhm1tUYcmIj1AiV/2aczASu6/9iS+eOZY/vjyKs76z2d4eM4q3NX2L5LPlPhlv4qTCa47cwwPfvZkBvYr5f/dOZOrbpvOstbtUYcmIt2kxC8ZGT+kHw9cexL/dN54pi1q5cwbnuZnT72hbp5F8lAUg60PN7M/m9k8M3vFzK7LdQzSPUXJBJ88uZEnvnwap4+r50ePvc6ZNzzNH19W849IPonijL8D+LK7HwGcAFxrZuMjiEO6aXBVH/778sncdfXx9C0t4tq7ZnLxL19gxpINUYcmIhmIYrD1Ve4+M3y/BZgHDM11HHLoTjyslj9+/hT+7aKJLFq3jQ/+9/Ncfft0Xlu9OerQRGQ/LMqf6GbWADwDTHD3zXssuwa4BmDEiBGTlyxZkvsAJWPb2zq49bnF/OLpN9m6q4P3TRzMZ08/jHGD+kUdmkhsmdkMd296R3lUid/M+gJPA99z99/vb92mpiZvbm7OTWBySDZtb+eXz7zJ/76whK27OnjP+IFc++7DOHp4ddShicROr0r8ZlYMPAQ86u43HGh9Jf78s3F7G7c+t5hbn1vE5p0dTGkcwKdOGcUZ4+pJJCzq8ERiodckfjMz4Hag1d2/kMlnlPjz15ad7fzf9GXc+txiVmzcwajaCq6YOpKLjh1GVZ/iqMMTKWi9KfGfDPwVmAN0jfzxdXd/eF+fUeLPf+2dKf40dzW/enYRs5dtpE9xkgsnDeWy40dw5JB+BOcDItKTek3i7w4l/sIyZ/km7nhxCQ/MXsHO9hTjBlVycdNwLpw0lAEVJVGHJ1IwlPil19m0o50/zF7JPc3LeHn5JooSxmlj6zj/mCGcNX4g5SVFUYcokteU+KVXm7dqM/e9tIIHZ61k9eadlJckefe4es6dMJh3j6vTQUCkG5T4JS+kUs60xa08MGslj72ymvXb2igtSnDKmDrOPKKe04+op76yLOowRfKCEr/knc6UM21RK4/MXcXjr65h5aZgNLCjh1Vx2tg6Th1bxzHDqylKqq9Bkb1R4pe85u7MW7WFJ+et4c/z1zJr2UZSDpVlRRzfWMOJo2s48bAaxtZX6jkBkZASvxSUTdvbee7Ndfz1jRaef3M9S9YH4wP0Ly9m8sgBTGnsz+SRA5gwtB+lRcmIoxWJxr4Sv66YSV6qKi/m3ImDOXfiYACWb9jOC2+uZ9qiVpqXbOCJeWsAKEkmGD+kH5NGVHP0sGomDquisaZCvwok1nTGLwVp7ZadzFyygZeWbeSlpRt5eflGdrYHzwv2LS1i/JB+jB8cTEcM7seYgX0pK9YvAyksOuOXWKmvLOPsCYM5e0Lwi6CjM8WClq28vHwTc5Zv4tVVm7mneRnb24IRxBIGI2sqOHxgJYfV92V0fQWH1VUyqq6CilL9N5HCon/REgtFyQTjBvVj3KB+XNw0HAhuHV3Sup15qzbz+potzF+9hflrtvD4vDV0pnb/Eq6vLKWxtoLG2gpG1JQzckAFI2vKGd6/nH59itTdhOQdJX6JrUTC3kroXdcKANo6UixZv40Fa7eycN02Fq3bxuJ123j81TWs39b2tm1UlhYxtH8fhvXvw+CqPgyp7sOQ6jIG9StjUFUZA/uVqQlJeh0lfpE9lBQlGDOwkjEDK9+xbOuuDpau387S1m0s37CD5Rt2sKx1Oys27mT64g1s2tH+js9U9SmmvrKU+n6l1FeWUdu3hLrKUmr7llLTt5SaihJq+pbQv7xEBwnJCSV+kYPw1oXhIXsfWWzbrg5WbdrB6k27WLVpB2s272TN5l2s3RK8TlvXyrqtu9jVkdrr58tLkvQvL6F/RTHVfUqoLi+muryYqj67p35lxfQLXyvLiuhbVkRlWZFuW5WMKfGL9KCK0iIOq6/ksPp3/lro4u5s3dVBy5ZdtG5rY/22NtZvbWPD9jY2bGujNXzduKOdlRt3sGF7G5t3drztusPelCQTVJQmqSgtom9pERXh1Lc0SXlJEeUlSfqUJKkoKaJPcfA+/bW0OEGf4iRlxUlKixKUhe/LihOUFiVJ6hbYgqHEL5JjZkZlWTGVZcWMqsvsM10Hi0072tm8o4MtO9vZvDN43bqrgy07g2nbrg62htP2tmD9VRt3sL2tk+1tHWxr66RtH782DqQoYZQUJSgtSlDSNSUTlBQlg/JkguIioySZoDiZoLgoQXHCKE4mKEomKE52vTeKE8FrUcIoSiYoCtdLJozipJFMBGXJtOlt87b7fSJtPvFWOW+9T9judRIW7P+gPLjOkwjLE2ZY+JpIW7cQRZL4zexs4MdAErjZ3b8fRRwi+SL9YEH/Q9tWR2eKnR0ptrd1sKOtk53tKXa0d4bvw6kjKN/V3smujlTwviM4aOzqSNHWkaKtMxXOd9LW6bR1BNvYkuqgrSNFe2ewTken097ptHem6OhM0Z5yOjpTHOAHTK+RflCwrgMCuw8MZmAEBxFj9zqwe1nXAaVrOWGZhdvavV6wDmnz//qBiUxpHNCj3ynnid/MksB/AWcBy4HpZvagu7+a61hE4qgomaBvMkHfiJ9PSKWcjpTTkUrR3ul0hgeE9pTvXtaZotODZZ2p4ACSSpvvTDmdHqzfmepaxtvKOt1xd1IOqbA85by1voflneH7rnU7U44TzO9+T/iZrs8F87D7c87uZdC1zfBz4TYcJ/wTbOut8t3zhAfGitKev3YTxd/8FGCBuy8EMLPfABcASvwiMZJIGCUJowT1rpprUezxocCytPnlYZmIiORAFIl/b1dL3tHaZ2bXmFmzmTW3tLTkICwRkXiIIvEvB4anzQ8DVu65krvf5O5N7t5UV5fhrQ8iInJAUST+6cAYM2s0sxLgEuDBCOIQEYmlnF/cdfcOM/ss8CjB7Zy3uPsruY5DRCSuIrmfy90fBh6Oom4RkbjTfVQiIjGjxC8iEjN5MfSimbUAS7r58VpgXQ+Gk8+0L3bTvthN+2K3QtsXI939HbdF5kXiPxRm1ry3MSfjSPtiN+2L3bQvdovLvlBTj4hIzCjxi4jETBwS/01RB9CLaF/spn2xm/bFbrHYFwXfxi8iIm8XhzN+ERFJo8QvIhIzBZ34zexsM5tvZgvM7Pqo48kVMxtuZn82s3lm9oqZXReWDzCzx83sjfD1EAfxyx9mljSzl8zsoXA+lvvCzKrN7F4zey389zE1xvvii+H/j7lmdreZlcVlXxRs4k8b4vEcYDxwqZmNjzaqnOkAvuzuRwAnANeG3/164El3HwM8Gc7HxXXAvLT5uO6LHwOPuPs44GiCfRK7fWFmQ4HPA03uPoGgw8hLiMm+KNjET9oQj+7eBnQN8Vjw3H2Vu88M328h+M89lOD73x6udjtwYSQB5piZDQPeB9ycVhy7fWFm/YBTgV8BuHubu28khvsiVAT0MbMioJxgXJBY7ItCTvwa4hEwswZgEvA3YKC7r4Lg4ADURxhaLt0IfBVIpZXFcV+MAlqAW8Nmr5vNrIIY7gt3XwH8CFgKrAI2uftjxGRfFHLiz2iIx0JmZn2B3wFfcPfNUccTBTM7D1jr7jOijqUXKAKOBf7b3ScB2yjQpowDCdvuLwAagSFAhZldHm1UuVPIiT+jIR4LlZkVEyT9O93992HxGjMbHC4fDKyNKr4cOgk438wWEzT3nW5mdxDPfbEcWO7ufwvn7yU4EMRxX5wJLHL3FndvB34PnEhM9kUhJ/7YDvFoZkbQjjvP3W9IW/QgcGX4/krggVzHlmvu/jV3H+buDQT/Bp5y98uJ575YDSwzs8PDojOAV4nhviBo4jnBzMrD/y9nEFwLi8W+KOgnd83sXIL23a4hHr8XbUS5YWYnA38F5rC7XfvrBO389wAjCP7hf9jdWyMJMgJm9i7gK+5+npnVEMN9YWbHEFzkLgEWAp8gOAGM4774NvARgrvgXgKuBvoSg31R0IlfRETeqZCbekREZC+U+EVEYkaJX0QkZpT4RURiRolfRCRmlPglFsxsa/jaYGYf7eFtf32P+ed7cvsiPU2JX+KmATioxB/29Lo/b0v87n7iQcYkklNK/BI33wdOMbNZYX/sSTP7oZlNN7OXzezvIHjYKxzT4C6CB+Ews/vNbEbYh/s1Ydn3CXp4nGVmd4ZlXb8uLNz2XDObY2YfSdv2X9L6xb8zfHoUM/u+mb0axvKjnO8diYWiqAMQybHrCZ/eBQgT+CZ3P87MSoHnzOyxcN0pwAR3XxTOX+XurWbWB5huZr9z9+vN7LPufsxe6roIOIag3/va8DPPhMsmAUcS9B/1HHCSmb0KfAAY5+5uZtU9+9VFAjrjl7h7D/AxM5tF0KVFDTAmXDYtLekDfN7MZgMvEnQAOIb9Oxm429073X0N8DRwXNq2l7t7CphF0AS1GdgJ3GxmFwHbD/G7ieyVEr/EnQGfc/djwqkx7Jcdgm6Lg5WCfn7OBKa6+9EEfbuUZbDtfdmV9r4TKHL3DoJfGb8jGADkkYP4HiIZU+KXuNkCVKbNPwp8JuzGGjMbGw5OsqcqYIO7bzezcQRDWnZp7/r8Hp4BPhJeR6gjGP1q2r4CC8dPqHL3h4EvEDQTifQ4tfFL3LwMdIRNNrcRjEHbAMwML7C2sPfh9h4BPm1mLwPzCZp7utwEvGxmM939srTy+4CpwGyCQYC+6u6rwwPH3lQCD5hZGcGvhS926xuKHIB65xQRiRk19YiIxIwSv4hIzCjxi4jEjBK/iEjMKPGLiMSMEr+ISMwo8YuIxMz/ByeH80w9pZeKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(costs)\n",
    "plt.xlabel(\"Iterations\");\n",
    "plt.ylabel(\"cost function\");\n",
    "plt.title(\"Cost function vs Epoch\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f1047504",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEWCAYAAABhffzLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAtkElEQVR4nO3dd3xUdbrH8c+TTkIKhFBDlyJVIRSxrmXtbe2Ka8e+rmW9ut7d9e7dotfuuhYU0UXF3te1r4INCVJEqdKLJHQIBJLw3D9mcCOGMJDMnEnm+369ziszp/2+GZhnTs458/uZuyMiIokjKegAIiISWyr8IiIJRoVfRCTBqPCLiCQYFX4RkQSjwi8ikmBU+KVBMLPLzWyFmW00s/wYtvtbM3ssVu3VlZkdYmZLgs4h8U2FX3aLmZ1tZsXhArzczP5lZgfUcZ8LzOzwWpanAncDP3f3pu6+qi7t1dLOT4qmu//F3S+ORnv1wczczPaqp33t9oeGmXUys3+b2SYzm1nbv6PEDxV+iZiZXQfcC/wFaAV0AB4EToxy062ADOCbKLcju28sMBnIB24BXjSzgmAjyS65uyZNu5yAXGAjcFot66QT+mBYFp7uBdLDy1oAbwJrgdXAeEIHHmOAbcDm8P5v3GGf3YEywMPLPwQ6hZ+nVFvvI+Di8OPzgU+AO4E1wHzg6GrrNgdGhzOuAV4FssIZtoXb2Qi0BW4Fnqq27QmEPoDWhtvcu9qyBcANwDRgHfAckLGT12kt0KfavIJw+y139lrVsJ9x4dehLJz3DOAQYAlwPVACLAcu2KHtO4FFwArgYaBJLb//YODzcJblwANAWrV/my1AdrX9jwcuC/r/q6baJx3xS6T2I3TU/Uot69wCDAX2AfoTKhr/HV52PaGCVEDoCP63gLv7uYSK0PEeOo3zf9V36O6zgd7hp3nufmiEeYcAswgV0f8DRpmZhZeNATLD+20J3OPuZcDRwLJwjqbuvqz6Ds2sO6Ej3F+Hf4+3gDfMLK3aaqcDRwGdgX6EPoR+xN23AC8DZ+2w3cfuXsJOXqsa9nNQ+GH/cN7nws9bE/qgbgdcBPzdzJqFl91OqGDvA+wVXuf3tfz+VcC14ddxP+Aw4IrwvnoD89x9Q7VYU/nPv5fEKRV+iVQ+sNLdK2tZ5xzgj+5e4u6lwP8A54aXVQBtgI7uXuHu4909mh1FLXT3R929Cngy3HYrM2tDqMBd5u5rwlk+jnCfZwD/dPf33L2C0JFzE2BYtXXud/dl7r4aeINQga3JM/y48J8dngd1f60qCP07VLj7W4SO3nuEP/guAa5199Xhgv0X4Myd7cjdJ7n7F+5e6e4LgEeAg8OLmxL6y6a6dUD2bmSVAKjwS6RWAS3MLKWWddoCC6s9XxieB3AHMBd418zmmdlN0Yn5g++3P3D3TeGHTYH2wGp3X7MH+/zR7+fu24DFhI6af9IusCncZk0+BJqY2RAz60joA2L7X1N1fa1W7fABvT1HAaG/dCaZ2VozWwu8HZ5fIzPrbmZvmtn3Zrae0AdFi/DijUDODpvkABuQuKbCL5H6HCgHTqplnWVAx2rPO4Tn4e4b3P16d+8CHA9cZ2aHhdfb3SP/svDPzGrzWke47WKguZnl1bBsVzl+9PuFj6DbA0sjbPs/DYU+NJ4ndNR/NvDm9lMmu3it6mIlofP4vd09Lzzluvv2D6eafv+HgJlAN3fPIXTaafsps2+ALmZW/Qi/P7oIH/dU+CUi7r4O+D2h88UnmVmmmaWa2dFmtv28/Fjgv82swMxahNd/CsDMjjOzvcLFcj2hc8dV4e1WAF12I0spoWI73MySzexCoGuE2y4H/gU8aGbNwr/D9nPlK4B8M8vdyebPA8ea2WHhW0yvJ3Rx87NIs+/gGUKnj87hP6d5dvVa7Sji1y78YfMocI+ZtQy31c7Mjqy2rx1//+xwho1m1hO4vNr+ZgNTgD+YWYaZnUzousZLkeSR4KjwS8Tc/W7gOkIXbEsJHT1fReiuGIA/AcWE7mr5GvgqPA+gG/A+odMDnwMPuvtH4WV/JfSBsdbMbogwziXAbwidgurN7hXfcwmdB59J6M6XX4d/v5mEPrzmhbO0rb6Ru88ChgN/I3T0fDyhi9Jbd6Pt6vubQOivl7aEPoy2q+212tGtwJPhvKdH0Ox/ETqN9EX41M37QI9wnpp+/xsI/UWygdCHxnM77O9MoIjQ3VG3AaeGP5gljll0r6+JiEi80RG/iEiCUeEXEUkwKvwiIglGhV9EJMHU9mWcuNGiRQvv1KlT0DFERBqUSZMmrXT3n3xBr0EU/k6dOlFcXBx0DBGRBsXMFtY0X6d6REQSjAq/iEiCUeEXEUkwKvwiIglGhV9EJMGo8IuIJBgVfhGRBNOoC//kRWt46KPvgo4hIhJXGnXhf23KMm5/eyZvT18edBQRkbgRtcJvZo+bWYmZTd9h/tVmNsvMvqk2clNU3HxMT/q3z+OGF6Yxr3RjNJsSEWkwonnE/wRwVPUZZvYz4ESgn7v3Bu6MYvukpyTz4DkDSE02Ln/qKzZtrdz1RiIijVzUCr+7jwNW7zD7cuA2d98SXqckWu1v1y6vCfeduS+zSzZwyyvT0YhjIpLoYn2OvztwoJlNMLOPzWzQzlY0sxFmVmxmxaWldRvC86DuBfz6sO68MnkpT09YVKd9iYg0dLEu/ClAM2AooYGynzczq2lFdx/p7kXuXlRQ8JNeRXfb1YfuxSE9CvjjG98yZfHaOu9PRKShinXhXwK87CFfAtuAFrFoOCnJuOf0fSjITufKp79iddnWWDQrIhJ3Yl34XwUOBTCz7kAasDJWjTfLSuOh4QMo3bCFa56dTNU2ne8XkcQTzds5xwKfAz3MbImZXQQ8DnQJ3+L5LHCex/hqa7/CPG49oTfj56zk/g/mxLJpEZG4ELURuNz9rJ0sGh6tNiN11uD2TFq4hvs/nMM+HfL4WY+WQUcSEYmZRv3N3Z0xM/50Uh96tMrm2uemsGTNpqAjiYjETEIWfoAmack8PHwgVVXOFU9/xZbKqqAjiYjERMIWfoBOLbK48/T+TFuyjj++8W3QcUREYiKhCz/Akb1bc+nBXXh6wiJemrQk6DgiIlGX8IUf4Dc/78GQzs255dWvmbF8fdBxRESiSoUfSElO4m9n70tORiqXjpnE2k36cpeINF4q/GEtszN4aPhAlq/bzNVjJ1NZtS3oSCIiUaHCX83Ajs343xP7MH7OSu54Z1bQcUREoiJqX+BqqM4c3IHpy9bxyLh59Gqbw4n7tAs6kohIvdIRfw1+f1xvBndqzn+9NI3pS9cFHUdEpF6p8NcgLSWJv58zgGaZaVw6ZhKrNm4JOpKISL1R4d+Jgux0Hjl3IKUbt3DVM5Op0MVeEWkkVPhr0a8wj7+e3JfP563iL2/NCDqOiEi90MXdXThlYCHfLFvP45/Op3fbXE4dWBh0JBGROtERfwR+e0xPhnXN57evfM1UDdsoIg1cNAdiedzMSsKDruy47AYzczOLybCLdZWSnMQDZw+gZXY6l46ZROkGXewVkYYrmkf8TwBH7TjTzNoDRwCLoth2vWuelcbIc4tYu3krVzw9ia2VutgrIg1T1Aq/u48DVtew6B7gRqDBDXjbq20Od5zan4kL1vDHN78JOo6IyB6J6cVdMzsBWOruU80slk3Xm+P7t+WbZet5+OPv6N02l7MGdwg6kojIbolZ4TezTOAW4OcRrj8CGAHQoUN8FdffHNmDGcvX8/vXptOlRRZDuuQHHUlEJGKxvKunK9AZmGpmC4BC4Csza13Tyu4+0t2L3L2ooKAghjF3LTnJuP+sfenQPJPLnprEwlVlQUcSEYlYzAq/u3/t7i3dvZO7dwKWAAPc/ftYZahPuU1SGXXeIBy46Mli1pdXBB1JRCQi0bydcyzwOdDDzJaY2UXRaisonVpk8fDwgSxYWcaVT3+lPvxFpEGI5l09Z7l7G3dPdfdCdx+1w/JO7r4yWu3HytAu+fz55FAf/n/6p7p1EJH4py4b6sEZgzowt2Qjj46fT9eWTTl3aMegI4mI7JQKfz256ei9mVdaxq2vf0On/EwO7BZfF6RFRLZTXz31JDnJuO+sfenWsilXPP0Vc0s2Bh1JRKRGKvz1qGl6Co+dV0R6ShIXPTmRNWVbg44kIvITKvz1rLBZJo+cO5Dla8u5XH36iEgcUuGPgoEdm3P7qX35Yt5qfv/adNwbXLdEItKI6eJulJy8byFzSzby939/x14tm3LxgV2CjiQiAqjwR9X1R/RgXmkZf35rBp1bZHHY3q2CjiQiolM90ZSUZNx1en96t83h6rGTmb50XdCRRERU+KMtMy2FUecNIq9JKhc8MZGlazcHHUlEEpwKfwy0ysngiQsHU15RxQWjv2TdZnXoJiLBUeGPke6tsnlk+EDmryzjsjG6zVNEgqPCH0PD9mrB7af04/N5q7jppWm6zVNEAqG7emLsFwMKWbJmM3e/N5vCZk247uc9go4kIglGhT8AVx+6F0vWbOL+D+dS2CyT0we1DzqSiCQQFf4AmBl/Prkvy9eVc/MrX9M6N4ODuqs3TxGJjWiOwPW4mZWY2fRq8+4ws5lmNs3MXjGzvGi1H+9Sk5N48JwBP/Tm+e2y9UFHEpEEEc2Lu08AR+0w7z2gj7v3A2YDN0ex/biXnZHKExcMJjsjhQufmMjydbrHX0SiL5pDL44DVu8w7113rww//QIojFb7DUXr3AweP38QG7dUcsHoiRq0XUSiLsjbOS8E/rWzhWY2wsyKzay4tLQ0hrFib+82OTw0fABzSzZy5dNf6R5/EYmqQAq/md0CVAJP72wddx/p7kXuXlRQ0PgvfB7YrYC//qIv4+es5MYXp7Jtm+7xF5HoiPldPWZ2HnAccJjrG0w/clpRe0o2bOGOd2aR3zSd/z52b8ws6Fgi0sjEtPCb2VHAfwEHu/umWLbdUFxxSFdKN2xh1CfzKchO57KDuwYdSUQamagVfjMbCxwCtDCzJcAfCN3Fkw68Fz6S/cLdL4tWhobIzPj9cb1YXbaV2/41k/ysNE4r0he8RKT+RK3wu/tZNcweFa32GpOkJOPO0/qzZtNWbnr5a5plpnF4Lw3iIiL1Q520xam0lCQeGj6Q3m1zuPKZryhesHrXG4mIRECFP441TU9h9PmDaJfXhAufmMjsFRuCjiQijYAKf5zLb5rOkxcOpklaMr8c9aVG8BKROlPhbwDaN8/kyQsHs2lrJeeOmsDqsq1BRxKRBkyFv4Ho2TqHx84bxNI1m7ngiYmUbanc9UYiIjVQ4W9ABnduzgNnD+DrJWu5XF07iMgeUuFvYI7o1YrbftGPcbNLufa5KVSpawcR2U0aiKUBOn1Qe9aXV/Cnf84gMy2Z20/pR1KSunYQkcio8DdQFx/YhQ3lldz3wRyy0lP4w/G91K+PiEREhb8B+/Xh3SjbUsljn8ynaXoKNxypgdtFZNd2WfjNbH/gVqBjeH0D3N27RDea7IqZccuxe1O2tZIH/j2XrPQULj9EnbqJSO0iOeIfBVwLTAKqohtHdpeZ8aeT+lK2pYrb355J0/Rkzt2vU9CxRCSORVL417n7TkfKkuAlJxl3nd6fTVur+N1r35CZlsIpAxN+VEsR2YlIbuf8t5ndYWb7mdmA7VPUk8luSU1O4oGz92X/vfL5zYtTeXv68qAjiUiciuSIf0j4Z1G1eQ4cWv9xpC4yUpMZeW4R546awNVjJ/PYeSkc3L3xD1spIrtnl0f87v6zGiYV/TiVlZ7C6AsG061lNpeOKWbCvFVBRxKROLPLwm9muWZ2t5kVh6e7zCw3gu0eN7MSM5tebV5zM3vPzOaEfzar6y8gP5XbJJUxFw3+oTvnSQvXBB1JROJIJOf4Hwc2AKeHp/XA6Ai2ewI4aod5NwEfuHs34IPwc4mC/KbpPH3xUAqy0znv8S+ZvEjFX0RCIin8Xd39D+4+Lzz9D7DLe/jdfRyw47BRJwJPhh8/CZy0O2Fl97TOzWDsiKE0z0rjl49/ybQla4OOJCJxIJLCv9nMDtj+JPyFrj0dDaSVuy8HCP9subMVzWzE9tNLpaWle9ictMltwtgRQ8nLTGX4YxOYvnRd0JFEJGCRFP7Lgb+b2QIzWwg8AFwW3Vjg7iPdvcjdiwoKdGdKXbTLa8LYS4aSnZHKOY9N4JtlKv4iiSySu3qmuHt/oB/Q1933dfepe9jeCjNrAxD+WbKH+5HdVNgsk2dHDCUrLZnhj01gxvL1QUcSkYDstPCb2fDwz+vM7DrgYuDias/3xOvAeeHH5wGv7eF+ZA+0b57J2BFDSU9J5pzHJjDrew3eLpKIajvizwr/zK5harqrHZvZWOBzoIeZLTGzi4DbgCPMbA5wRPi5xFDH/CzGjhhKarJx9qNfMGeFir9IojH32kdwMrP93f3TXc2LpqKiIi8uLo5Vcwnhu9KNnDnyC9zh2RFD2avlLj/LRaSBMbNJ7l604/xILu7+LcJ50oB0LWjK2EtCvXGc/egXzC3ZGHAiEYmV2s7x72dm1wMF28/rh6dbgeSYJZSo2atlNmMvGcI2hzNHfq5z/iIJorYj/jRC5/JT+PH5/fXAqdGPJrHQrVU2z106lOQk48yRn+tWT5EEEMk5/o7uvjBGeWqkc/zRt2BlGWc/+gVlW6sYc9Fg+hXmBR1JROqoLuf4HzOzvGo7amZm79RnOAlepxZZPHfpfuQ0SeGcRyeoYzeRRiySwt/C3dduf+Lua6ilqwVpuNo3z+S5EfvRIjudX46aoC6dRRqpSAr/NjPrsP2JmXUkNBCLNEJt85rw3IihtM7N4PzRE/l07sqgI4lIPYuk8N8CfGJmY8xsDDAOuDm6sSRILXMyeHbEfnRonsmFT0zk49nqJE+kMYmkr563gQHAc8DzwEB31zn+Rq4gO52xI4bStaAplzxZzPvfrgg6kojUk0iO+AHSCfWtvw7oZWYHRS+SxIvmWWmMvWQoe7fJ5rKnJvH61GVBRxKRerDLwdbN7HbgDOAbYFt4thM65SONXG5mKk9dPISLnizmmmcns7G8krOHdNj1hiISt3ZZ+AmNktXD3bdEOYvEqeyMVP5x4WCuePorfvvK16zbXMHlh3QNOpaI7KFITvXMA1KjHUTiW0ZqMo+cO5AT+rfl9rdncvvbM9nVl/9EJD5FcsS/CZhiZh8APxz1u/uvopZK4lJqchL3nLEP2RkpPPTRd6zfXMEfT+xDcpIFHU1EdkMkhf/18CRCcpLxp5P6kNsklQc/+o715ZXcfXp/UpMjvU9ARIK2y8Lv7k/Wd6Nmdi2hEb0c+Bq4wN3L67sdiQ4z48ajepLTJJXb/jWTjeUVPHjOQJqkqdNWkYZgl4dpZjbfzObtOO1pg2bWDvgVUOTufQh18Xzmnu5PgnPZwV35y8l9+Wh2KeeN/pIN5RVBRxKRCERyqqd6z24ZwGlA83pot4mZVQCZgG4Qb6DOHtKB7IwUrn1uCmc88gVPXDCIljkZQccSkVpE8s3dVdWmpe5+L3Donjbo7kuBO4FFwHJgnbu/u+N6ZjbCzIrNrLi0VF0GxLPj+7dl1PmDWLCqjF889BnzSjWal0g8i+RUz4BqU5GZXUZoQJY9YmbNgBOBzkBbIMvMhu+4nruPdPcidy8qKCjY0+YkRg7uXsCzI4ayeWsVpz78OZMXqVtnkXgVya0Yd1Wb/kqo357T69Dm4cB8dy919wrgZWBYHfYncaJfYR4vXT6MpukpnP3oBD6cqf59ROJRbWPuXhN++Dt3/1l4OsLdR7j7rDq0uQgYamaZZmbAYcCMOuxP4kinFlm8dPkwurbM4pJ/TOL54sVBRxKRHdR2xH9B+Of99dmgu08AXgS+InQrZxIwsj7bkGAVZKfz7Ij9GNY1nxtfnMYDH87Rt3xF4khtd/XMMLMFQIGZTas23wB393572qi7/wH4w55uL/GvaXoKo84bxI0vTuXOd2ezYv0Wbj2ht77lKxIHdlr43f0sM2sNvAOcELtI0likpSRx9+n70Cong0fGzaNkQzn3nrGvvuglErBa7+N39++B/jHKIo1QUpJx8zF70yong//957ec+egXPPbLIgqy04OOJpKw1MGKxMSFB3Tm4eEDmfX9ek5+8FPmrNgQdCSRhKXCLzFzZO/WPH/pfpRXbOMXD33GZxrIXSQQkXyB67RI5olEol9hHq9eOYw2uRn88vEveUG3e4rEXCRH/DdHOE8kIoXNMnnx8mEM7ZLPb16cxl3vztLtniIxtNOLu2Z2NHAM0M7Mqt/LnwNURjuYNG45GamMvmAQ//3KdP724VwWrtrE/53aj4xU3fEjEm213dWzDCgmdCvnpGrzNwDXRjOUJIbU5CRuO6UvHVtk8n9vz2Lp2s08cu5AWjTVHT8i0WS7+hPbzFLDfeps72CtvbtPq3WjelZUVOTFxcWxbFJi7J/TlnP9C1PIz0rn0V8W0attTtCRRBo8M5vk7kU7zo/kHP97ZpZjZs2BqcBoM7u73hNKQju2XxteuHQYVducUx76jLenfx90JJFGK5LCn+vu64FfAKPdfSChHjZF6lXfwlxev2p/erTO5rKnJvG3D9THj0g0RFL4U8ysDaGumN+Mch5JcC1zMnh2xFBO3rcdd703m189O4XNW6uCjiXSqERS+P9IqL+e79x9opl1AeZEN5YksozUZO4+vT//dVRP3py2jNMf+Zzv15UHHUuk0djlxd14oIu7iev9b1dwzbOTyUpP4eFzBzKgQ7OgI4k0GHt8cdfMCs3sFTMrMbMVZvaSmRVGJ6bIjx3eqxUvX7E/6alJnPnIFzwzYVHQkUQavEhO9YwGXic0Pm474I3wPJGY6NE6mzeuOoChXfP57Stfc/PL09hSqfP+InsqksJf4O6j3b0yPD0B1Gn0czPLM7MXzWymmc0ws/3qsj9p/PIy0xh9/iCu/FlXxn65mDMe+YLl6zYHHUukQYqk8K80s+FmlhyehgOr6tjufcDb7t6TUH//GnNXdik5yfjNkT15ePgA5qzYwPF/+4QJ8+r6X1Ek8URS+C8kdCvn98By4NTwvD1iZjnAQcAoAHff6u5r93R/kniO6tOGV6/cn5yMVM55bAKjP52v+/1FdkPM7+oxs30IDa7+LaGj/UnANe5etsN6I4ARAB06dBi4cOHCmOaU+Le+vILrnpvK+zNWcNI+bfnLL/qSmVbroHIiCaUud/U8aWZ51Z43M7PH65AlBRgAPOTu+wJlwE07ruTuI929yN2LCgrqdElBGqmcjFRGnjuQ647ozmtTl3HiA58yt0Qje4nsSiSnevpVPxXj7muAfevQ5hJgibtPCD9/kdAHgchuS0oyfnVYN8ZcOITVZVs54YFPeW3K0qBjicS1SAp/UrhXTgDCnbXt8d/T4QHcF5tZj/Cswwid9hHZYwd0a8E/f3UgvdvmcM2zU7jlla8pr9AtnyI1iaSA3wV8ZmYvAk7oQu+f69ju1cDTZpYGzAMuqOP+RGidm8Ezlwzlzndm8ci4eUxdspYHzx5Ih/zMoKOJxJWILu6aWS/gUMCAD9w9pkfo6rJBdtd7367g+uen4MBdp/Xn571bBx1JJOZ2dnFXffVIo7V49SauePorvl66jvOHdeKmo3tqaEdJKHUZiEWkQWrfPJMXL9+PC/fvzBOfLeDkBz/ju9KNQccSCZwKvzRq6SnJ/P74Xjx+fhEr1pdz3P2f8HzxYn3hSxKaCr8khEN7tuJf1xzIPu3zuPHFafz6uSlsKK8IOpZIIFT4JWG0ysngqYuHcP0R3Xlj6jKO+9snTFm8NuhYIjGnwi8JJTnJuPqwbjx/6X5UVoUGdr//gzlUVm0LOppIzKjwS0Iq6tSct645kOP6teHu92Zz+iOfs3BV2a43FGkEVPglYeU2SeW+M/flvjP3YU7JRo65bzzPT9SFX2n8VPgl4Z24Tzve/vVB9C3M5caXpnHpmEmsLtsadCyRqFHhFwHa5TXhmYuHcssxe/PRrFKOvHcc73+7IuhYIlGhwi8SlpRkXHJQF169cn/ys9K4+B/FXP/8VNZt1m2f0rio8IvsoFfbHF6/6gCuPnQvXp2ylCPvGcdHs0qCjiVSb1T4RWqQlpLE9T/vwcuXDyM7I4XzR0/k5pensXFLZdDRROpMhV+kFv3b5/HG1Qdw2cFdeW7iYo68Zxzj55QGHUukTlT4RXYhIzWZm47uyQuXDSM9NYlzR33JDS9MZe0m3fkjDVNghd/Mks1sspm9GVQGkd0xsGMz3vrVgVz5s668Mnkph989jre+Xq77/qXBCfKI/xpgRoDti+y2jNRkfnNkT16/an9a56ZzxdNfcemYSaxYXx50NJGIBVL4zawQOBZ4LIj2Reqqd9tcXr1if24+uicfzy7l8Ls/5ukJC9m2TUf/Ev+COuK/F7gR2GnPWGY2wsyKzay4tFQX0yT+pCQncenBXXn71wfRp20ut7wynVMf/oyZ368POppIrWJe+M3sOKDE3SfVtp67j3T3IncvKigoiFE6kd3XuUUWz1wyhLtO68+CVZs49v5P+OtbM9i0Vbd+SnwK4oh/f+AEM1sAPAscamZPBZBDpN6YGacMLOSD6w7mtIGFPDJuHkfcPY4PZqjbB4k/MS/87n6zuxe6eyfgTOBDdx8e6xwi0dAsK43bTunHC5ftR1Z6Mhc9WczFTxazaNWmoKOJ/ED38YtEwaBOzXnz6gO56eiefPbdSg6/52PueW825RVVQUcTwRrCPchFRUVeXFwcdAyRPfL9unL+/NYM3pi6jMJmTfjdcb34ea9WmFnQ0aSRM7NJ7l6043wd8YtEWevcDP521r6MvWQomWnJXDpmEueNnsicFRuCjiYJSoVfJEb265rPP391IL87rheTF63hqPvG84fXprNGg75IjKnwi8RQanISFx3QmY9uOISzBrdnzBcLOeTOjxj96XwqNOC7xIgKv0gA8pum86eT+vKvaw6ib7tc/ueNbznq3nG89+0K9f0jUafCLxKgHq2zGXPRYEadV4Q7XPKPYs4Y+QVTFq8NOpo0Yir8IgEzMw7buxXvXHsQ/3tSH+aVbuSkv3/KVc98pfv/JSp0O6dInNm4pZKRH3/Ho+PnU7ltG2cP7sBVh3ajIDs96GjSwOzsdk4VfpE4tWJ9Ofe+P4fnixeTlpzEhQd0YsRBXcltkhp0NGkgVPhFGqj5K8u4+73ZvDF1GTkZKVx2SFfOH9aJzLSUoKNJnFPhF2ngvlm2jrvenc2HM0to0TSNyw7uyjlDOtIkLTnoaBKnVPhFGolJC1dzz3tz+GTuSlo0TefyQ7pyzpAOZKTqA0B+TIVfpJGZuGA197w3m8++W0XL7HRGHNSFs4d00Ckg+YEKv0gj9cW8Vdz3/hw+n7eK5llpXHRAZ87dryM5GboInOhU+EUauUkLV/PAh3P596xSsjNSOH9YJ84f1on8proNNFGp8IskiK+XrOOBf8/h3W9XkJacxBmD2nPJgV1o3zwz6GgSYyr8IglmbslGRo77jlcmL2Wbw7F923DJgV3oW5gbdDSJkbgp/GbWHvgH0BrYBox09/tq20aFX2TPrVhfzuOfzOfpCYvYuKWSwZ2bc/EBnTls71YkJ2kwmMYsngp/G6CNu39lZtnAJOAkd/92Z9uo8IvU3YbyCp6buJjRny5g6drNdMrP5PxhnThlYCHZuhDcKMVN4f9JALPXgAfc/b2draPCL1J/Kqu28c43K3jsk3lMXrSWpukpnDqwkF/u15EuBU2Djif1KC4Lv5l1AsYBfdx9/Q7LRgAjADp06DBw4cKFsQ8o0shNWbyWJz9bwJvTllFR5RzYrQXDh3bksJ4tSUlW570NXdwVfjNrCnwM/NndX65tXR3xi0RX6YYtjP1yEc9MWMT368tpk5vBmYM6cObg9rTKyQg6nuyhuCr8ZpYKvAm84+5372p9FX6R2Kis2sYHM0t46ouFjJ+zkuQk42c9WnLmoPYc0qNAfwU0MDsr/DH/breZGTAKmBFJ0ReR2ElJTuLI3q05sndrFq4q49mJi3mheAnvz1hBq5x0ThlQyKkDC3UtoIEL4q6eA4DxwNeEbucE+K27v7WzbXTELxKciqptfDizhOcmLuajWSVscxjYsRmnDizkmL5tND5AHIurUz27S4VfJD6UrC/nlclLeWHSEuaWbCQtJYnD927JyfsWcnD3AtJSdCoonqjwi0i9cXemLVnHK5OX8sbUZawq20peZipH92nN8f3bMqRzvr4cFgdU+EUkKiqqtjF+TimvT1nGu9+uYNPWKgqy0zm6T2uO7duGok7N9SEQEBV+EYm6zVur+GDmCt6cupx/zyphS+U2CrLTObJ3K47s3ZqhXfJJ1Z1BMaPCLyIxVbalkg9nlvDW18v5aFYpmyuqyMlI4dCeLTmiV2sO6t5CXUVEmQq/iASmvKKK8XNW8vb07/lw5grWbKogNdkY0jmfn/VsyaE9W9K5RVbQMRsdFX4RiQtV25yvFq3hvW9X8OHMEuaWbASgU34mB3Uv4ODuBQztkk9WuoaQrCsVfhGJS4tXb+LDmSV8PLuUz79bxeaKKlKTjX07NOOAvVowrGs+/dvn6drAHlDhF5G4t6WyiuIFaxg/ZyWfzl3J9GXrcIfMtGQGdmzG0C75DO3SnL7t8vSdgQjETZcNIiI7k56SzP57tWD/vVoAsKZsKxPmr+LTuauYMH8Vd7wzK7xeEvu0z6OoUzMGdmzGgA7NyMtMCzJ6g6IjfhFpMFZt3MLEBauZuGANExes5ptl66naFqphnVtksU/7PPoX5tKvfR692uSQkZoccOJg6VSPiDQ6m7ZWMm3JOiYtXMOUxWuZsngtpRu2AJCcZHRr2ZQ+7XLp3TaHvdvksHfrHHIzE+cWUp3qEZFGJzMtJXzePx8IdSXx/fpypi5ex/Sl6/h66To+mlXCi5OW/LBNm9wMerTOpnurbLq1bMpe4SmRvlOgwi8ijYaZ0Sa3CW1ym3BUn9Y/zC9ZX863y9czY/kGZn2/npnfb+CzuavYWrXth3UKstPp3CKLzvlZdGqRRaf8TNo3D02NrQdSFX4RafRa5mTQMieDQ3q0/GFeZdU2Fq3exNySjXxXWsZ3pRtZsLKMD2auYOXGrT/aPjsjhcJmmbTLywh9sORl0Dong1Y5GbTMTqdldgY5TVIIDTcS/1T4RSQhpSQn0aWgaY2Dymwor2Dhqk0sXr2JxWs2sWTNZpau2cySNZuZuGAN6zZX/GSbtOQk8pum0TzrP1OzzDRym6SS2ySVnCap5GSk0DQjhabpKWSlp5CVlkKTtGQy05Jj+j2FQAq/mR0F3AckA4+5+21B5BARqUl2Rip92uXSp11ujcs3ba3k+3XlrFi/hZIN5ZRu2ELpxi2s2riVVRu3sLpsKwtXbWLNpq1sKK+MqM3kJKNJajLpKUmkpSSRmhz6+eeT+jAkfA2jvgQx9GIy8HfgCGAJMNHMXnf3b2OdRURkT2Smpez0r4UdVVZtY0N5JevLK1i/uZINWyoo21JF2ZZKyrZWsnlrVWiqqKK8YhtbKqvYWrmNrVXbqKjaRtOM+i/TQRzxDwbmuvs8ADN7FjgRUOEXkUYnJTmJZllpNMuKny+YBfGd53bA4mrPl4TniYhIDARR+Gu67P2Tb5GZ2QgzKzaz4tLS0hjEEhFJDEEU/iVA+2rPC4FlO67k7iPdvcjdiwoKCmIWTkSksQui8E8EuplZZzNLA84EXg8gh4hIQor5xV13rzSzq4B3CN3O+bi7fxPrHCIiiSqQ+/jd/S3grSDaFhFJdBrJQEQkwajwi4gkmAbRH7+ZlQIL93DzFsDKeowTLcpZvxpKTmg4WZWzfsUiZ0d3/8ltkQ2i8NeFmRXXNBBBvFHO+tVQckLDyaqc9SvInDrVIyKSYFT4RUQSTCIU/pFBB4iQctavhpITGk5W5axfgeVs9Of4RUTkxxLhiF9ERKpR4RcRSTANuvCb2VFmNsvM5prZTTUsNzO7P7x8mpkNiHTbeMhpZu3N7N9mNsPMvjGza+IxZ7XlyWY22czejNecZpZnZi+a2czw67pfnOa8NvxvPt3MxppZRoA5e5rZ52a2xcxu2J1t4yFnHL6Pdvp6hpdH/33k7g1yItTB23dAFyANmAr02mGdY4B/ERoDYCgwIdJt4yRnG2BA+HE2MDsec1Zbfh3wDPBmPP67h5c9CVwcfpwG5MVbTkIDE80HmoSfPw+cH2DOlsAg4M/ADbuzbZzkjLf3UY05qy2P+vuoIR/x/zCEo7tvBbYP4VjdicA/POQLIM/M2kS4beA53X25u38F4O4bgBlEb7SyuryemFkhcCzwWJTy1TmnmeUABwGjANx9q7uvjbec4WUpQBMzSwEyqWHMiljldPcSd58IVOzutvGQM97eR7W8njF7HzXkwh/JEI47WyeWwz/WJecPzKwTsC8wof4jRpZhF+vcC9wIbItSvkgy7GqdLkApMDr8p/RjZpYVbzndfSlwJ7AIWA6sc/d3A8wZjW13V720FSfvo9rcSwzeRw258EcyhOPO1olo+Md6UpecoYVmTYGXgF+7+/p6zBZxhtrWMbPjgBJ3n1T/sX6iLq9nCjAAeMjd9wXKgGidl67L69mM0FFiZ6AtkGVmw+s5X60ZYrDt7qpzW3H0Pqp5wxi+jxpy4Y9kCMedrRPR8I/1pC45MbNUQv9Zn3b3l6OUsa459wdOMLMFhP60PdTMnorDnEuAJe6+/WjvRUIfBPGW83BgvruXunsF8DIwLMCc0dh2d9WprTh7H+1M7N5H0bp4EO2J0NHbPEJHRdsvovTeYZ1j+fHFsy8j3TZOchrwD+DeeH49d1jnEKJ7cbdOOYHxQI/w41uBO+ItJzAE+IbQuX0jdEH66qByVlv3Vn580TSu3ke15Iyr99HOcu6wLLrvo2i/EFF+kY8hdIX+O+CW8LzLgMuq/YP/Pbz8a6Cotm3jLSdwAKE/E6cBU8LTMfGWM5b/Yevh330foDj8mr4KNIvTnP8DzASmA2OA9ABztiZ0JLseWBt+nLOzbeMtZxy+j3b6elbbR1TfR+qyQUQkwTTkc/wiIrIHVPhFRBKMCr+ISIJR4RcRSTAq/CIiCUaFXxKWmeWb2ZTw9L2ZLQ0/Xmtm3+7mvk4ys14RrJduZs+Fe26cEO5CQCSmVPglYbn7Knffx933AR4G7gk/3ofd7yvlJGCXhR+4CFjj7nsB9wC372Y7InWmwi9Ss2QzezTcf/u7ZtYEwMy6mtnbZjbJzMaH+1YfBpwA3BH+i6GrmV1iZhPNbKqZvWRmmeH9nkjom7gQ6jLiMDOrqX8XkahR4RepWTfg7+7em9C3K08Jzx9JqPuEgcANwIPu/hnwOvCb8F8Q3wEvu/sgd+9PqBvgi8Lb/9B7o7tXAuuA/Bj9TiJAqF8JEfmp+e4+Jfx4EtAp3LvjMOCFagfp6TvZvo+Z/QnIA5oC74Tnx7JHS5EaqfCL1GxLtcdVQBNCfyGvDV8H2JUngJPcfaqZnU+o7xX4T++NS8KDrOQCq+snskhkdKpHJEIe6sN9vpmdBj+Mmds/vHgDoWH9tssGloe7Az6n2vzXgfPCj08FPnR1mCUxpsIvsnvOAS4ys6mEuk7ePqzes8BvwiN7dQV+R2iUp/cI9bK53Sgg38zmEhpbNaoDlIvURL1ziogkGB3xi4gkGBV+EZEEo8IvIpJgVPhFRBKMCr+ISIJR4RcRSTAq/CIiCeb/AVyojNzQL29pAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(theta[:,0],costs);\n",
    "plt.xlabel(\"Theta0\");\n",
    "plt.ylabel(\"cost function\");\n",
    "plt.title(\"Cost function vs theta0\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "021765d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7333176762950011"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2_score(y,hypothesis[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e95d22d",
   "metadata": {},
   "source": [
    "## **Sklearn**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "9af7db19",
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = LinearRegression().fit(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "98e001e7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8213156778363777"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r2_score(y,lr.predict(X))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6aa2bf73",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}