htru1_tutorial_channel.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This script follows [the standard CIFAR10 Pytorch example](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html). It extracts a single \"channel\" from the HTRU1 dataset and treats it as a greyscale image.\n",
    "\n",
    "The steps are:\n",
    "\n",
    "1. Load and normalizing the HTRU1 training and test datasets using torchvision\n",
    "2. Define a Convolutional Neural Network\n",
    "3. Define a loss function\n",
    "4. Train the network on the training data\n",
    "5. Test the network on the test data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First import some standard python libraries for plotting stuff and handling arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then import the pytorch and torchvision libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then import the pytorch neural network stuff:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then import the oprimization library from pytorch:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.optim as optim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally import the HTRU1 pytorch dataset class. This is not provided with pytorch, you need to [grab it from the HTRU1 github](\n",
    "https://raw.githubusercontent.com/as595/HTRU1/master/htru1.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from htru1 import HTRU1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define a torchvision transform to extract a single channel from the multi-channel dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def select_channel(x,c):\n",
    "    \n",
    "    from PIL import Image\n",
    "    \n",
    "    np_img = np.array(x, dtype=np.uint8)\n",
    "    ch_img = np_img[:,:,c]\n",
    "    img = Image.fromarray(ch_img, 'L')\n",
    "    \n",
    "    return img"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output of torchvision datasets are PILImage images of range [0, 1]. We transform them to Tensors of normalized range [-1, 1]. The first transform extracts Channel 0 (DM surface) from the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "transform = transforms.Compose(\n",
    "    [transforms.Lambda(lambda x: select_channel(x,0)),\n",
    "     transforms.ToTensor(),\n",
    "     transforms.Normalize([0.5],[0.5])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load in the training and test datasets. The first time you do this it will download the data to your working directory, but once the data is there it will just use it without repeating the download."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "0it [00:00, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading http://www.jb.man.ac.uk/research/ascaife/htru1-batches-py.tar.gz to ./data/htru1-batches-py.tar.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "183099392it [01:20, 2759732.06it/s]                               "
     ]
    }
   ],
   "source": [
    "trainset = HTRU1(root='./data', train=True, download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4, shuffle=True, num_workers=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "testset = HTRU1(root='./data', train=False, download=True, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=4, shuffle=False, num_workers=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are two classes in this dataset: pulsar and nonpulsar (i.e. RFI):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "classes = ('pulsar', 'nonpulsar')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A little function to display images nicely:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def imshow(img):\n",
    "    img = img / 2 + 0.5     # unnormalize\n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take a look at some randomly selected samples to see how they appear:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get some random training images\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztXWusZWV5fl65X5ThJjNnZmRASQ01RQ2xGJvGYE3RGvGHMVhjaUoyf2yqjUmL9Ycl6Q9NG61NWhuiVmyMaNEWNLbVUhrTH1IHNYggMA4MzDAXUG7euOjbH3s9Zz/n289Ze8/Zw9lnb98nmcw+6/J973dZa733NzIThUKhUJh/PG/WBBQKhULh6KBe6IVCobAgqBd6oVAoLAjqhV4oFAoLgnqhFwqFwoKgXuiFQqGwIKgXeqFQKCwIpnqhR8RlEXF3ROyOiKuPFlGFQqFQOHLEWgOLIuIYAPcAeD2AfQC+CeDtmXnn0SOvUCgUCpPi2CnufRWA3Zm5BwAi4noAlwNY9YV+8skn56ZNm6boslAoFH71cODAgUcy8+xx103zQt8K4EH5ex+A3+y7YdOmTdi5c+cUXRYKhcKvHq655pq9k1z3nBtFI2JnROyKiF0//elPn+vuCoVC4VcW07zQ9wPYLn9v646tQGZem5kXZ+bFJ5988hTdFQqFQqEP07zQvwnggog4LyKOB3AFgJuODlmFQqFQOFKsWYeemc9GxB8D+E8AxwD4ZGZ+70jb2bNnDwDgmWeeWT4WESPX/fKXvwQAPO95zxu5/vjjjwcAHHfccSP3Pf3008u/ec/WrVtHzj388MMAgJNOOmn5mOuLtPGcegmRjl/84he9bRxzzDEAgBe84AUjdPO6Y48dLs0jjzwy0teLX/ziFePcvn0oLFES4pwBwEMPPTRCx5NPPgkAeOqppwAAP//5z5fP/exnPxuhjePTdtke51Kv5zhPOOGE5WNUu+ncP//5z19Bj7bh9gKP/eQnPxlpl/du27Zt+dyhQ4dWXKNtXHTRRSPt33HHHQCAU089dfmYjpk48cQTAQzXRdeMe+CJJ54Y6VOv45xyHp999tmRfnQ/uTa4Z5XelkbOsY5F15ttsN2lpaWRNrhfgeGaartf+tKXVvSt9PA54HMGDNf7xz/+8fIx7js6T7BvYLhPeY32r8e4HqeccgqAlXuebXBvahu6P/g87d8/UDocPnx4pA3dpxyf7lc+h6Rjy5Yty+f4TGufXOdzzz0Xa8U0RlFk5lcAfGWaNgqFQqFwdDDVC/1ogF9K/Yq23DgAnHbaaQCAH/3oRwBWci38IisHyza0XX4xyRUpl8h79UvMY8qtEPw6a5/kGJWbc1IDz5MzUamA0PGRG1KurMXpp5++/NuNj2NXrqzlzHW+CeVkyH24teJ12iePqe2E66jzzDXlXOp8sE+dZ/7mOPUecn0HDx4cobFv/hS8TsfONrTPVmLRdedc6jFym9ou15nHdOxcHzenyrlyf7j1Y/tKB8en80HJgG2o9OMkT3LEk+5JjkU5WN6ra0s6OE59fti/rgHPK5fP9riv3fwp3e7Y448/DmA4t+SyW5oI7YPgXLINpZvj1Dl10tmRokL/C4VCYUFQL/RCoVBYEMxc5UKxRMURikoq9j366KMAhqLmWWedNXK9Gt+c4YcGiMceewzAStUB4dQ2FBeBoUjMe1UU4/UqVlKcUzGb4plTJ9CQpG1wHvpEMhWpSa/SRlWHttHSq+PkORXtKUKrIefBBwexZZwXFUfZHtUgwHDsTt3l0lCQNh0fxVS9nuIy95OqLrhWKt6qiD4J2K62ofuzpZHXqxqE9Oq+4zHeq/PCY9onj+k8sw0nxnP+dD/xOdE22nu1jbYfANi8efPImFvo/PQ5Ouh+Ih1u7PytKjwaT3Xvts+XM1bre6E1bgNDYzbvVRo5Lp0/Xten2lIDOVVWR0PNoigOvVAoFBYEM+fQnYGGX0rlau+//34AQ85c3aX0a0vwy6dfbnKs/NrS8AEMObpxXD5xxhlnABhyCABw5plnjvTJr7JyZRwrv/B6jr91fGzPGb0Ix3Fou84ww3adEbo1LAFDQ6ZyK7zHGYVa4y8wXBc9Rncu0qt0cx6Uk+G8KTfEtXQcD/eRntN5aMGx6Bq0YwKGHGBryAOG86dz2seNkXMcJ905jpv3uv3hDI+cX0cb+1LumntH14X7XyW4Fuqi6Iyt3H+OC3fSTEu/ttcnbei8s09n3NY93EqtOn+8TufPSWSce9LrpBnH0U+D4tALhUJhQVAv9EKhUFgQzFzl4sQQRqQ5308aE2gk1XOqLqFISKMdMBStqAYZ5+NKOLGPtNHYCIwauBRqyOF1fYYnFd1Ip/NXJ1SsdKImRWkVHTkGXuciGBXOoNQadl3Uq/bJNVKVC42mL3rRiwCsVFm1YqvS4YyLPKbqNGdE7VO5EEqHiwymuoFz4GIjxvkstz7vugbOl53X6bPB9XOqSmdcdAZe/ubzpfuPz4vOB69zxs6WLqWpT70CDPcHHRf0+WIb+uwTTl3BeXbzp9c74ynv5X5yalfdQ+6Z5hi4T3ROqbZ0vunToDj0QqFQWBDMnEMnJ6Vfeudixy8ar3MclhpoyBlpPomWq1Yujl9gdbHjV1fbpRGU1yu3wPaVe+JXX7ky3us4epenhNf1cUPKybB9pYPj0rHQiEtuSzk7Z+wix+G4VN6rnAylEueKp1GE5MoYMaprpvlDCM6Ncjxsg3uGtAJD46YzWjo4btJxca2r5DiXQ86Vcu1cI8d98pgznOkc8dlwHLrLM+NyH3EMnD89184tMNwzfRlUnbtln3uhtuvcVbnH9RilB2c0d27BvFdzvxw4cGCEDs6pk5I4l+55dI4FPKbOBGzPRQZPg+LQC4VCYUFQL/RCoVBYEMxc5eLESortavTat28fgKH47IwlKuI5VU6fyqIvfSkNrMBQfHKGSopUOhaKXWqwpdrmgQceGOmrjfrTvlS8VZERGIqNwDCZmNLN8TGNrt7DvlTNwrGoLzb7dMmOnKjO9vR65+PNY1w/VRXxnFMBOWM16VHxlfeqOsjdS7hkW/ztEkk5n32nTuP4dI7axGiqGnGRx+05Pe9Uc44Oqlx073Isbl6o4tKxUzWkEdur3QcMx67G8FY1on3wnItPcfvD+eBzTNq+U4URGsnZqi1VXdKXLE+fUe5B/u/WtiJFC4VCoWAxlkOPiE8CeBOAw5n5su7YGQA+B2AHgPsBvC0zR32JJgC/lI7rc5xJa6wAPAfLr6EaT8l90ECoHIpLbeoiKPnldtxqG3mp7SltpLc1RK3WJ48px9NyJDpO0qYpat19HDP/16hX3qtG4lYq0Hsdp9HHgen1bXEM7Yfjchy6K7ThjHTkCrXPSaLyXASvttGO2XHj47j8ti2V5Nx1bFclkJYz1/tcZKlbg5Y2Fw2s7fI8Ddnj0O55YLjudFcFhgUluO7OEOtSKTsO3RWz4G+93kWDtlHAzkisa8s9pvNBwz+v131No7bLaTQNJuHQPwXgsubY1QBuzswLANzc/V0oFAqFGWIsm5KZX4+IHc3hywG8tvt9HYD/AfDnayGAX/pxeTD4RePXzuUuUQ6W1yvn2uYbUW6IX0/lBJ3rFL/Ezh3RlY9rszO2v9txkqZx2e5aqBsb51K5LHKuLleNy/RHjkTpICfjJCGX98blEXF5R9qCFS7HiLbLY7reXD/Og46dUL19X2CRy0ZIScXZZNx+dcFlPK97jHPpAnVI4zg7UCtt6Hy3peW0DV3vNkhK59sVCOH8thknFc6W5LhgLRfI54t5YHQNnN7ZBUlxDziJmfPtApx0f3B8pEf3i+Py3TPUSnDap3PZ7LPrTIq16tDPyUxa4Q4COGdqSgqFQqEwFaY2iubgk7sq6xgROyNiV0TsclxToVAoFI4O1uq2eCgitmTmgYjYAmDU+tYhM68FcC0ALC0tjbz422rtALBjxw4AK8U5uuC5/CAUX9SdyUUYUuRpI+sAb9SjOOQqldNdS8VQ0qYitTNicayuzqL76PF6lxvD3Ucx24nDrqYo50pVKbxXRU2ugbbB63iv9snrdD5cgQv+dgUaXM1Zd6xVUen8cG7UjbPPhdW5pbmq7oRzDeS9bix9YrY+B1zHvuhDR5v26dIPc610vdso4HG1TUl33zyqEdAVUSFteozqkragDeDVH86Q3hp4da5o+Hc1SM85Z6hooHGWc6nzQbr1OWifaaWDx1Td2dZw1b6mwVo59JsAXNn9vhLAjVNTUigUCoWpMInb4mcxMICeFRH7AHwAwAcBfD4irgKwF8Db1kqAy3JILti5BPLL6lwO1TDocqIwcMAFM/UFhygXQunBGZacAYpfZW2jNY650mj65SYH3VfgwmXCc1KHywjojFM8pm6LzihFrrfNawIM50bng9e7wB8Gg6l0RUlI3cFcQQnOPcen3JArPqBBJC2cob4tjQaMGvrc+ugxtqfcZLsG4wyrhAu4cX26Y+xfuVSugTPuufHxuVLptQ8uFw65/L179y4f47q559E5KXBONdCw3QN6vQuS4vWHDh1aPsax8p2ie94ZjjkWpcNJaYQLuuuTdibFJF4ub1/l1Oum7r1QKBQKRw0VKVooFAoLgpnncqFIowZN1g/VyMU2Wk0NOhRzVGVAEd2lL6VI7dKdOjWFE9UpTqlo3+d33Zd+0/mGa59OxHOFEwiqr3TsLj9O25fSTZWIXs82HB2u6ISLxuT12lfr0+/qaup6P/LIIyuu13va/BmAj2Z1NVAJp6JpVTrAcKx9Kjztx61Lqz5yKhoFx6nPS1tcwUU1qo81VS2qjuQ9nCtn0NQ++/LMtGNTuAhNVf3wvJtTV2yijXYGRlVEuteoQnRxCKpebCOUdU6dkdipzHgd21A6uJ/1ean0uYVCoVBYxsw5dGdIJJSToTHNla3i19O5VbmIN5ds3yW+51dXucM2GlQ5FPavnJXj0FtuT7/SbEO5dtdGC6Wb8+E4ML2O/dJF7M4771w+x2Mu34fOMw2ZHJNyPi56lNKRrndbEtCV6lK3TCc9kHtsXSCB4Xqo22JfpKgr1+dyovA6d32fW5ozkDvpru1Hoc8G73WGWxf1yrnSOWiNoi4bpoti7csWOC7nipOE2r5c0RXdC30lGx0dbcQ5MNxPLm8R58q59KrEQilDpY123Vw+Ijcf06A49EKhUFgQ1Au9UCgUFgQzV7lQlHG1KNXHlWKlM0Y6UWXz5s0AfORiW8dRr1NR0xmDXHKptn31kXeqpNYQ5gyE7j4dcwtNlcvrVKSmWKkqF9JJerQNir66Ljym42ujb534rKBIqmNxxUIIGrdVPUY1jxZXYH1UV/CAY1Y1Rd9ccj50DVwRlXYP9BWd0DacSs7tJ6dedBGr/K17tx2LqkbobKB7oVX56J4nvc5o2KdycZGRzifb+dTzOj1HmnSfOOMzzzvfd6f24P5zNVZJh8YtcCzuXaHPizNSEz/84Q9HjvXtyUlRHHqhUCgsCGbOoZOrUY6gr1iBy3/CyEL9wrncEa0xSCus8yvqihQoJ3PuueeuuM7l2XDSgxrwnMGsbUO5Cle+S+dLrwGGXIJyTy984QtH2uCYeb1zp3PRuhqRy3ngmB0nrbRy7rVdzgM5QOVgXY4MzpFyjLyOtOn88V7likg311PBedCI1ZZz1DHwnIssdQVTXJpUV3TCGVGdQb+9zrk+OvddVxKNe51rp+ecu2qfIW9cbhu2wb0JDHPK8NlU7tblSnJpeVvnAZXkXN4d7l0dM4/xHaTvCu5hl37bpZZ2aXxdhHyfoX5SFIdeKBQKC4J6oRcKhcKCYOYqF4pRKoqxFt/555+/fIzitavazd8qUrsK8m0NT2dYckYb9V9uq9wrKDK56DZtt63M45JdqeGH4l5fRRNt34n2FGVdQiH27/x7xyX44lic/zTPOX9x0gMMxVV3PUVuFYepNnKRmWxDxVeKxtpnX1Sei4Jk+67aUKt6AYb7pK/2p8LVriTc9a7dvmpXatTj/OoccV+0sRo6Ft2TLrVvC7dfVQXl4jw4D64OrDMqk07du/ztann21ZLVeXb1SNvr1OGC+1PVkVTZcQwudubss89ePqbq0LWiOPRCoVBYEMycQ3fgl1W/ds5w18Jxk/r1bzlXl3fBcRDKmbRGJvcFd4YthaszStClTLknfv378mY47tAZSl2OGOc+6TgvN9Y2elSvcRKIM6a17l3OjVPXgNymzgfb4P/j8t70cWour48r/NDWU3Vcs3Nl1DlqXTv1eieNOsNnuxddG84NUcdC7tDtVxclOwlcpKi24erWtmMYF23qXDvbvty+dn3qujj32rYNvd5J560LravZ25eTaS0oDr1QKBQWBJMUuNgO4NMYFIJOANdm5kcj4gwAnwOwA8D9AN6WmY+u1s5qIBfuXOY0sIjXOf2w4zCpN9OvKLMyUsfmdLVOj9zH7blAE8el6teZfZCDcJygfumd6+MkULc7tqEuXAcOHFhBj46F3KYecwFZHBf7Ul0tx6CcYF/GQc6HFgngOdVXci8ot9eWBXPZFpVL7atvy3bHzXfrytjnOqdjcXvGBci4couO6+TvvvKFumbO7bO1gbj95wK/+vILOclW6XCujKTDPY9u7E6CY3vOdbS9T9t1wW4cn9PbuwIrzo7n8lW5oDvNLrtWTMKhPwvgvZl5IYBLALwrIi4EcDWAmzPzAgA3d38XCoVCYUYY+0LPzAOZ+a3u95MA7gKwFcDlAK7rLrsOwFueKyILhUKhMB5HZBSNiB0AXgHgVgDnZOaB7tRBDFQyRwzn1kfRR0VCis0UrcdFRlJMVOMi1QJt/hHAu6q1qU31d5tuFPA1NF1EKcfMc2r8pWrBqST6DMLOVVLHwja0Nievo3ipdJMOV//SFb1w6XNdKmDSpKofGqDYllZfZ34ZVcNQraNjZr+8zhVNGJfTg2gj/ACvEqHITbp17H0pdRVtOmaFK6TAdl2BC/csuT7bYhbaP/tyY1F1o9vrLVQNQndcfeZIp643z9PF1KWddu5/zg2VNGobLkqWc6QOFG2KZlWvtIUrAK/iJW3ck6qOmSTSdi2YuLWIOBXAFwC8JzNXVNjNwQyPOiIP7tsZEbsiYlef3rJQKBQK02EiDj0ijsPgZf6ZzPxid/hQRGzJzAMRsQXAYXdvZl4L4FoAWFpaGnnptwaMrj8AKznX1rijBj/HfTpjCb/Y7mvunP9dlXGWP+PXWb+wbcAQ4IMbSIcbO9twLmV9BijnqkZagSGHpLSRy3MFI9iG48pcEBEN2MpJO+MiuVnl2ttMic69S+l2QScc69atWwGs3B979uwZoc1lJiQonThJS6UetufcM12gWhv8pMecwa+vzJv21Vccw7noujxHbXEMJ2k52vpK+bnAJeVquQYqrVEq5/q50m8qnXDvOGO8k5JcgRVer3uG5xnU555pfeZIk653m8tI54pj12dD9+daMZZDj8HqfwLAXZn5YTl1E4Aru99XArhxamoKhUKhsGZMwqG/BsA7AXw3Ir7THfsLAB8E8PmIuArAXgBve25ILBQKhcIkGPtCz8z/BbBaiNjrpiWAopKKQBSpVdRkzgOKcZoDYffu3QC8ykDVDq6GJ8F2nd+r+kC359Sgw+tc+lIVb9u8GSpWUs3kiiD0GfKcUVTn1KmD2uIHOi+810XPKR2tiK6+tDRoOnWQzmmbd0fVCczroyorisaujqorKuAKfmhelxbOqOwi+1pfZRXtXXpUd6z1we9Lqay/dY7a9db5Jo1OHeRyGbnYCOd37Xy7W+gzyudKx+7UJaSXKkJVo7rU0k6Fx/lw6iCnTmOfakjn+LnXNBbF1W7luulY2B7VK7p32Ia+F7gnOS9rQUWKFgqFwoJg5rlcyGWpEZBfLf3q8mu3tLQEYKXBtC32AAy/lI67aQs7AMMvt7teryN3Sk7UuYU5dzoXAei44Pvuu2+EDo6vz5DnqqMrF8L5ZXQoMJxncsHOKOQ4JOWMOT4X2ec4O66pri3HpxkVW7qVa3E5TlqXNpUAuGe0DZXcWnAsKrE4oyXnwZWFcwUx2qIJwGjUsCt7Ny7HDtegr4iEM5i64iwuJw/pdhHCfUUZdM87l1uXYbKVUJ2LovbZ5/bJNnS/0gDryk+6CFQXrct7df7YhrbL54l7TWnkHtCxUKLpc34Yh+LQC4VCYUFQL/RCoVBYEMxc5eJ8OltRVs8/9NBDAFaK9vTfdKlyVR1DsY9qBBWRKSqpWMT+Vb3TRjWqWElVh4rqVCMobW2RDBXnKKY51YWqJLTohrapv9V46ZLns5bj5s2bV4wJ8Coll7CrNdhqNBzvVbHSqVfYLv9X1Y+LwCOdziec/VONpG2omkWN2S1ItxPjdY5IJ9dP6066tKsu0rZN7OWM4QqnmmnVQbon+5KEKR2tKsfV8lSVn6s92rbh9pw+B84QzH3P9dS9w7XVPrmfXFQ059L51Du/fL2OdPJdMS5WhAZNpYPnqY7R94hL88z92T7bR4Li0AuFQmFBMHMOvS9a0kWy8QurX2lySsq1kJtQDoWGUud2xC+yciYuQo/3MDJSOQ5+kfWrS05Nx9JyCe6cc8HU69qvuDNwOaOecq7kwmmMUZcrckou34fLedGmgdU+naufo43tK3fbV5BAOSpyjK1LqPalxjHHQRNcA5XuXF6VNg2zrg/p7nMTVTgjqivz5iJQ26ryrv0+AygwnA+2r+d4rz5zfeX0COcGq3unLdEGjJaPc5KCGuoJ52ZJKI2MRNX5IE0u7w7XQNvnM+EiZ5Vejo99KV1sT+eUtBWHXigUCoV6oRcKhcKiYOYqFxoKVE3hjA4Uf51PLNUImm53+/btAFYa2L761a8C8EmPnD+rEz8Jpzpw4j5FY5cC1VW553VObdMHV19Tx07Dp9JBUZ1irhq92J62QeOOqhbaRE+uSvu4SjcUYfuSHrnUtzpHHAPFW1UVsX8V1SfJ/OliB1zK1LYfpdepS1wksUvx6/Y6xXznV855c1GQzsDq6lnyOt1PTgXq6u22qg63F1SN5drlHuReO3To0PI5qgRV1cY+tW/Om6uj61R4fG84IzGvc8+gqks4H6pS4v7hXtO5cqmOXTzNkaI49EKhUFgQbBgO3VXQdsYxchV0uQOGRgTNI0IXJ40YbFPw9hUVUDiOm/Qol+i4fCcNtAYobd9FSzLCsc8ANa7Cel99Smc05JxqPg7OKXO0AKPpbdU1yxUZ4Xw5g5njSEmjy7Hj3NH6jK46Pic1EH3pmNWo3HKHOnZnmG7pGQfn1ur2Uxsp6gzCzuDnoh9X+7vtk+ddbV/CRbW6OpyusIqTWNpIb/3tpJg+o7JKD9wXLiKX+1X3H6VWfT+5+aAbK+lwqag1Za6Tuo4UxaEXCoXCgmDmHLrL5+D0XPxS8muq58g1KTfJ9h544IHlY30FHagbde5PyvHwC++y9bX5H4Ah1+Z0mK4wAuE4DtXPtdA+XQ4aBmQpZ0raOD53vUo4LiiD43JV3ck5aoAJOR2dU/ar+nqCEouzbSi9reujm49Jy31xjzm3VtenC95xOlLHmbcco3MvnBTOPc716a5rzzm3RZdXxdmGCB07ny+XD0b3dSsBqyTupB4+066IhQsQc3uB8+B03E73T7isj87tlM+Bjt3ZZI4GikMvFAqFBUG90AuFQmFBMFblEhEnAvg6gBO662/IzA9ExHkArgdwJoDbALwzM1cvMLgKqBJRdQnFIZePw9WpdMYSuj+poXT//v0AfGpOV8m7bV9pczlGnFGKIpiOhe5XzmDF61V0oxFS29D5avt0oGFVDYOkg+KiqldcilrS69aFUBG8z8XOpV3lveo+SdrUWO1UZq36yLkcau3KPqMo94z2yTGocaxVk2ibNJg50V5VDG3Vet1PThXG82qAbaF0cJ77UhNru23BDYXbY5OqDPhsuJqiOh9tfVR3ve5TzoOq9bgnnVsmx+dUoC5XEve6q1nq3h/u2XfGfjcfjt4jxSQc+lMALs3MiwC8HMBlEXEJgA8B+EhmvgTAowCumpqaQqFQKKwZk5SgSwD8/B3X/UsAlwL4/e74dQD+EsDHjpQAfvmUk3bcG7+sjvNx+U/45VZurzWYOQPUuFwuvJfuRmrQJB3OvUs5RtLhXKja+4DhV1+5kJZDd+XSxkkgHIvLvsffzr3QFRggd+OqozsuX2lzhR8Izpu24QI1WldQpYO0uXwcDs6AxzZ0nltu0p1TOtr5BoZz6QpoOAnHZVvsK63IveCKbzjJ0xm32+Axvd7lWiGc27HLTuqkGEolmsHSuVL2Fb1o6df+lZPmXLrcUZQGVCrg/nMl5VRy4j2cB92vPKe0TVPYgphIhx4Rx3QFog8D+BqAHwB4LDO5M/cB2LrKvTsjYldE7JokOq9QKBQKa8NEL/TM/EVmvhzANgCvAvDSSTvIzGsz8+LMvLjP7a5QKBQK0+GI/NAz87GIuAXAqwFsiohjOy59G4D9ayKgE3fUB5nGK6aoBUbrNqqh6ODBgyuuAYbinBY1oEjFe8cZKSiSaj6JNg+LtkFfVVfAwBWF4Fg0Wox9udwsfX7Jrs6nirwUNV3NSkpOKkG5tMatz7n25SLl2pTHekxpa1PwqujLtdL1dnARgwTFYN0fFOVdoQuXMpXzq3S3Khc1nLn0r30FQpy47cbiip1wDZy/uGuj7RsYNYbqOWcgdymoWz9/N1e6Fzg3el3rQ+72nzOKqsqFzxPXWOeDz7fSzfOqLiGdLm2ya8O9D1pDs6v1qsfWJVI0Is6OiE3d75MAvB7AXQBuAfDW7rIrAdw4NTWFQqFQWDMm4dC3ALguIo7B4APw+cz8ckTcCeD6iPgrAN8G8Im1EHD++ecDWMmRugyMbXV0vZ7nHnzwweVjrVEDGM3TohwFuWtnjHScPKEcWJtTQ+9VDrrN4qhff2aJdNGVfYY8Z8Ryrp1qJGa7zDanbbj8FmxXo+ZajlHHyVwoLheJKxdI6UQ5JWcwdoVEuM50z3TGXCcpOFCqcwUd1EhHlzZnLHaV4SkNuH1NjIvydBGlrRuu4/S0rT4u3BXmcJHbnEtdq1bKUCOgc/tktLVzn+QxjRTl/nfuhXv37h2K7wasAAAGW0lEQVShzRnICd2n7NNJG9zrLpeQm2dtl+8G9y5y0bpHw21xEi+X2wG8whzfg4E+vVAoFAobABUpWigUCguCmSfnouih4si9994LYKUIS3HE+fc6P3Tn99qmt1VjTJv8S6Htkg6qLrTKPX3DtQ2KqS5FqFODUMRTAxtFTTWe9tHojG+t+Af0GxqduO8MbG2tSFcMROFqZ7aips6fM+Y6X+I2gZmqdO655x4AKw3kLnUy0Uby6vU6B63vtrbp/O2XlpYA+KRzzgNs0nTJrdFS59MZXZ1ahe25SNHWIWE1OlroXDkjKp8D3ZOMR3G+7xyXzhWfF1eXlOuhKkKui+59/nYJyVxhC9LtVEouySBVmro/+G5zRvNpUBx6oVAoLAjiaLjKTIqlpaXcuXPnuvVXKBQKi4Brrrnmtsy8eNx1xaEXCoXCgqBe6IVCobAgqBd6oVAoLAjqhV4oFAoLgnU1ikbEwwB+AuCRcdducJyF+R7DvNMPzP8Y5p1+YP7HME/0n5uZZ4+7aF1f6AAQEbsmsdZuZMz7GOadfmD+xzDv9APzP4Z5p9+hVC6FQqGwIKgXeqFQKCwIZvFCv3YGfR5tzPsY5p1+YP7HMO/0A/M/hnmnfwTrrkMvFAqFwnODUrkUCoXCgmBdX+gRcVlE3B0RuyPi6vXsey2IiO0RcUtE3BkR34uId3fHz4iIr0XEvd3/p8+a1j50Rb6/HRFf7v4+LyJu7dbhcxFx/Lg2ZomI2BQRN0TE9yPiroh49RyuwZ92e+iOiPhsRJy4kdchIj4ZEYcj4g45Zuc8Bvi7bhy3R8QrZ0f5EKuM4a+7fXR7RPwrq7F1597XjeHuiPjd2VA9Hdbthd5VPPp7AG8AcCGAt0fEhevV/xrxLID3ZuaFAC4B8K6O5qsB3JyZFwC4uft7I+PdGJQNJD4E4COZ+RIAjwK4aiZUTY6PAviPzHwpgIswGMvcrEFEbAXwJwAuzsyXATgGwBXY2OvwKQCXNcdWm/M3ALig+7cTwMfWicZx+BRGx/A1AC/LzN8AcA+A9wFA91xfAeDXu3v+oXtnzRXWk0N/FYDdmbknM58GcD2Ay9ex/yNGZh7IzG91v5/E4EWyFQO6r+suuw7AW2ZD4XhExDYAvwfg493fAeBSADd0l2x0+k8D8NvoShxm5tOZ+RjmaA06HAvgpIg4FsDJAA5gA69DZn4dwI+aw6vN+eUAPp0DfAODAvJb1ofS1eHGkJlf7QrbA8A3MChwDwzGcH1mPpWZ9wHYjTmsyLaeL/StAB6Uv/d1x+YCEbEDg1J8twI4JzMPdKcOAhgtHb9x8LcA/gwAKwucCeAx2dQbfR3OA/AwgH/q1EYfj4hTMEdrkJn7AfwNgAcweJE/DuA2zNc6AKvP+bw+238E4N+73/M6hhUoo+gEiIhTAXwBwHsy8wk9lwM3oQ3pKhQRbwJwODNvmzUtU+BYAK8E8LHMfAUGqSNWqFc28hoAQKdrvhyDj9MSgFMwqgqYK2z0OR+HiHg/BirVz8yalqOJ9Xyh7wewXf7e1h3b0IiI4zB4mX8mM7/YHT5EkbL7//Cs6BuD1wB4c0Tcj4GK61IM9NGbOtEf2PjrsA/Avsy8tfv7Bgxe8POyBgDwOwDuy8yHM/MZAF/EYG3maR2A1ed8rp7tiPhDAG8C8I4c+m3P1RhWw3q+0L8J4ILOsn88BgaIm9ax/yNGp2/+BIC7MvPDcuomAFd2v68EcON60zYJMvN9mbktM3dgMN//nZnvAHALgLd2l21Y+gEgMw8CeDAifq079DoAd2JO1qDDAwAuiYiTuz3FMczNOnRYbc5vAvAHnbfLJQAeF9XMhkJEXIaBCvLNmflTOXUTgCsi4oSIOA8DA+//zYLGqZCZ6/YPwBsxsCz/AMD717PvNdL7WxiIlbcD+E73740Y6KFvBnAvgP8CcMasaZ1gLK8F8OXu9/kYbNbdAP4FwAmzpm8M7S8HsKtbh38DcPq8rQGAawB8H8AdAP4ZwAkbeR0AfBYDff8zGEhJV6025wACAw+2HwD4LgbePBt1DLsx0JXzef5Huf793RjuBvCGWdO/ln8VKVooFAoLgjKKFgqFwoKgXuiFQqGwIKgXeqFQKCwI6oVeKBQKC4J6oRcKhcKCoF7ohUKhsCCoF3qhUCgsCOqFXigUCguC/wfe5P8Z3ei2yAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nonpulsar nonpulsar nonpulsar nonpulsar\n"
     ]
    }
   ],
   "source": [
    "# show images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "# print labels\n",
    "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define a neural network that takes 3-channel images as input:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(1, 6, 5)\n",
    "        self.pool = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "net = Net()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll use Classification Cross-Entropy loss and SGD with momentum for optimization:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run a few epochs of training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  2000] loss: 0.205\n",
      "[1,  4000] loss: 0.087\n",
      "[1,  6000] loss: 0.046\n",
      "[1,  8000] loss: 0.038\n",
      "[1, 10000] loss: 0.040\n",
      "[1, 12000] loss: 0.038\n",
      "[2,  2000] loss: 0.030\n",
      "[2,  4000] loss: 0.026\n",
      "[2,  6000] loss: 0.030\n",
      "[2,  8000] loss: 0.032\n",
      "[2, 10000] loss: 0.030\n",
      "[2, 12000] loss: 0.028\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "nepoch = 2  # number of epochs\n",
    "\n",
    "for epoch in range(nepoch):  # loop over the dataset multiple times\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # get the inputs\n",
    "        inputs, labels = data\n",
    "\n",
    "        # zero the parameter gradients\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # forward + backward + optimize\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # print statistics\n",
    "        running_loss += loss.item()\n",
    "        if i % 2000 == 1999:    # print every 2000 mini-batches\n",
    "            print('[%d, %5d] loss: %.3f' %\n",
    "                  (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll try out a couple of test samples just for visual kicks. First load them up and take a look at the true labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAHNFJREFUeJztnWusXcV1x/8L4wfmYcfEsvxANVVQI4oaiCxKlFJF0KgmRXE+IASNUldF8pdUTapILZQPqaV+SNQqaSq1VFagOAUZU0KLhdIHdalQP4RikogQCIkhEIwumFCMzRvD6oez173rzv2fmdlnn3vOPYf1k67uPnvP3ntm9mu9Zo2oKoIgCILJ55RxVyAIgiAYDvFCD4IgmBLihR4EQTAlxAs9CIJgSogXehAEwZQQL/QgCIIpIV7oQRAEU0KnF7qIbBeRJ0TksIhcP6xKBUEQBO2RQQcWicgyAD8B8EkARwA8BOBaVX1seNULgiAIajm1w74XAzisqk8BgIjcAWAHgL4v9NWrV+vatWs7nDIIguD9x8zMzC9UdX2pXJcX+mYAz7rfRwD8em6HtWvXYteuXR1OGQRB8P5j9+7dz9SUW3SnqIjsEpFDInLo9ddfX+zTBUEQvG/p8kJ/DsA57veWZt08VHWPqm5T1W2rV6/ucLogCIIgR5cX+kMAzhORc0VkBYBrABwYTrWCIAiCtgxsQ1fVkyLyhwD+HcAyALeo6o/aHmf//v0AgHXr1s2uO+2003qVO3WuemauWb58+YJty5YtAwCccsrC75NfZxE9tm7FihULyosIXe7Hu+++O7t88uTJef8B4L333luwj9XD75ty7Nix2eV33nln3n8AuOqqq+aV37dv3+zyypUr5/0H5trq11kfWv/5PrU+8v3H+tmW2TFsmfWpX2f9Yf99n1lf+r6yZb/O9rE+8sdg5W350ksvRcptt93Wt+32HwDeeuutecfy95OVO/3002fX2X3tNdVVq1YBmOsru7/98ayMP55vnz0bb7755rx6+eW33357dt0bb7wBAHjttdcWHMP6zx/D6uvbnl4zYOE96fvb2uLLW518W6wf2P1n+/r+s+3+2bBydo/5QAy7n3w97Pxr1qxZUM7q6PvD+s/f6ydOnJhXnuHfC1bO95HVacOGDX2PUaKLUxSq+h0A3+lyjCAIgmA4dHqhDwP7Qh0/fnx23auvvgoAOPPMM2fXpV94/3X0Uo3hv9hGKlX4rzSTxtMvPcNLLf4LnG7357K6mXThv9ImZflzplLwINjx2o47YG3PSdyl8tYWvy7VYlh5punU1rdmG8OfM9f3g/Zt7lgeL6Xac+Dv75o2l66Lkevn3DPCYJqchz0b6b3u92Mabc2z4bexNjCpPddvdjz2HLD+y/WbL9/2/mTE0P8gCIIpIV7oQRAEU8LYTS4Gc4R5E0aqgnn1yFQxr54xFSh1inqYEybd5mGOTeYEZGYVqxMz6diyNymZes1MOul+JbqodTVqe8kBykwuBjOFpfv5ZV8u3adktmHX1GCmgH7n6XdOZoZhbc+ZrHKOaWZ+yJkJGKz/rI9K5pXctTK8Q9hMRf4eZvvafW/Oe18P29c7iZkJKjWJ+Hqwc5rD0z9z/Y4FzF0PX96cnMzUy64ZM/MMg5DQgyAIpoSxS+jsC2hfXRZGl/tiekmGSRppaB0LfSxJoTlJ0M7JnDBMOmQOFGu7OUc9bZ2ibaX2Wkk657grOXOZs8v61KQW5owsObBtHevbGuc2a0surJSdu5aSFtP1+F2OkeurtvXwwQos3Nj6118rFnJrpNJ7eo60nNXXS/R2Tqbh+7rZvnb/sTBYf+40BDjdJz1njYYzCCGhB0EQTAnxQg+CIJgSxm5yMfXFqypMbU5NIqU43Jxjra3qyNR9Zi5hph8r751BqVOPqeDMEdal3ozFUOVLqjozuVhf1prOcudqa15hDKoODztWnpljTM335qBhXMccudG9OZjJxWNmRX+9rZz999vsGWKjkT3p88JMleyZZiNh2bacs9rXx+qbc4Z7Ig49CIIgmGXsEjpzluTCBHOj25gU4L+sudwltSPqcuFdLB8HC6lMj+u3WZ4IL0HYcXN5ImqlyS4OvNz2nGbBnKK1zqM0xLPfuWok+WFrJDltre3xakcj50YpttVicqMah+Gs87lt7Jnw9bFwQn+uNFyRSdf++bK8Ll4DTvvDl2fXjGmNqWbNAijYaHW/LtU8S3S5j4yQ0IMgCKaEeKEHQRBMCWM3ubCYX6ZmpY6FkhrDRlym52KqGFM1WUwz+81UX3PeModLTuVl8bq1qtswqDVd1MQtl8wDaax5bkRsv3Pltg2a4KtU7xoTTpeYc2Z2MDOGT+dae/425WsDAWrbkhunwEwdudHAuRGm6fEAnkCPmUprR4kzk0tafw+LfWdtCKdoEARBMEtRQheRWwBcCeCoql7QrFsHYD+ArQCeBnC1qr7cpSIsNItJqWxEVu4rmpuMoe1ISn9+k1YGCVWz86dJ9P3x2CQZufp2cYq2dRq2dcgxR6l3mKUaC7tmLExvGCM0S+Vy6xYrXW66jaXP7ZJKuQYmNfu61tz3lgYbmJvcwx/DJtjw2kbqFGVhi/55sfK5yTdYHiVff5Z/xbaziSjScERfnknhLDcVGyU7DGok9FsBbE/WXQ/goKqeB+Bg8zsIgiAYI0UJXVUfEJGtyeodAD7RLO8F8N8A/nSQCrBcHewLmJKbSs0v58IWWT4RZh9m0op9uUshUbY9N0UW+/r78jUSeom24Wi1UmpN2KJfZ9fNS+jWPusHNmisdlBGbQjfsEMYu5TpV47Z0E3S9dPHDUoumyQL6W0rTfrrWDsFXWrjZhNc1NrymdRssPcNs7Wz6QtztvycJleqxzht6BtUdaZZfh7A4JPgBUEQBEOhs1NUe5+YvmKfiOwSkUMicsgmog2CIAiGz6Bhiy+IyEZVnRGRjQCO9iuoqnsA7AGATZs2LXjxm0PEmy5MFWOTA7RN68rUHOYUZaqmwVS8dLZ2DzOX+HVp+5iZwveHHcObKVJKIVGMtL+6mAfYNhaWZv3mJx1InUbeSTbMEMJacqGxpVC8muPmtrE2+XqYyaWtU7Sk7ueer7bhqoY3C5nJwjsG7Znwx7Brb/e6N3UwkwsLQ0zDVP05LX+MP246x6/fzhyrzAzD6paaWnLb0u2DMqiEfgDAzmZ5J4B7OtckCIIg6ERN2OI+9BygHxSRIwC+DOArAO4UkesAPAPg6kErYJIoc2j6xPQ5ySQnrTPnjsGcnez43gGbSivsq8vylHgJ3dqcOl78spfQ7bgsvIq1rcvUaW2pcUaynBd2PYG5fjCTXNuwyFI9Bg0vbDsoaNhhjExCZ89L2wFLjFQ6ZM8GI9dmH7aYCw1k0jWTak2z9ZqqLecCEXxfmQbABjL659zuRTuGn3Ampyl4bSCdZq70XA5DQq+Jcrm2z6bLO589CIIgGBoxUjQIgmBKGHsuF0t/6WHOQoM5IRm52G3m+GEJ9dk5TcVjMeTsnLlYWFPjWGpOr6KampgzuYybXD+zOHTvFLV2nThxAkB7c0JtfYYBG8Vacp4auQlNas85TKdojtqgg5yZgOWbYaMr2XZmpjCTizd/MOep3U8sB5Lt600jdi96M6DV3c5paa09bLQpG+HNRqey/C7DICT0IAiCKWHsEjqTcuwryiRoNqrL9mUZ11hoVjoruF9mo0cZJhn48vY1Z9PNedKvM3Oi+nVs4o6U2hFnbfOZlCTImtBHf11Y2KL1W26igcUKUcxRm91yGP2dg90LJUk3B9Mkc1oxk5bbOolzoXueVKpmxyjlZkmdln4b05jPPPNMAPw5y9W79K5I17V1TA9CSOhBEARTQrzQgyAIpoSxm1xYQiY2etQcFkwFMlXGx5Ey9SVNTM9MNEzd984SU8+snHfqvvxyL4Owj79lqmlqcqlVy0sTPxjDiLuuJadWpumKgbn+Wr9+/ew6czgdPXp03n5+uUuyrVoHX0rJ5JLWjc1r2RbWJv8c2P2XG19RS86cwZLOleqbUko5y0Z5puM7WIABc4CymHBrk28LS9+8Zs2aedtYu1gyL1+mxinKTEVsXRdCQg+CIJgSxi6hmzTrv9wmsfmvaPqFL4VQsckSUumepdZlX0kv+Z999tnz1vnQrDPOOAPA/K/uK6+8Mu/cvu5MO8mlz82Fqo0iNeegx2IjHb2EactsZvjFrlvuGCWnV1quizO3NpeLaTiLNcFFLmV029G3penjmIZakyqaPefsGCyXy7p16wDMH4V+1llnAZgbHQrkp6Vj4ZAsv0subLHtBDm1hIQeBEEwJcQLPQiCYEoYu8mFjZgy1ds7M1JTS0klNJiKZ+YMFnfKnE2mkgHAxo0bAcw5SmdmZma3WbpQ5lzJxd8yJ6DHjmEmHUaXEWd2/NpjtE3Py0w/3pzG0qi2Of4g5duaP3JtZs5ftt+gMfW18fBdyLUvZ27K7VdKL8ue/Zoka2yEJjNRGt7kYs+3D2awMRE+mMHGRrDjW7193ewcObMKS9w1DEe6JyT0IAiCKWHsEnrtfJm5tLU1xwfmSwzptpwU4sMWLcTJvvCWfwSYc4CyVMC1UhZz+g7bcdKPxRqBySRSL6H73ByLWY+25xjH6NQSw8790Y+2qYMZJQdozlHa73e/dZ70ncJS1LKw51xoMTt/acKK3LZc+S6EhB4EQTAl1ExwcQ6Ab6E3EbQC2KOq3xCRdQD2A9gK4GkAV6vqy60rQMKDUkm6qUff37kJILy0nH5Zmb2L5YgxexowF1LJQhTt+L48sx2mA0bYNh9WVTugKKV2WrpaaaGtNMHyZljf+OnJTELPZaasre8w/QDMtpuTOmvu20Hq4Wmbpa/t8dveC7W5XHJaZttwPmZDZ/VNp6ID5u41/5zbgED7D8w952xCDHYNakITl0q2xZMAvqSq5wO4BMDnReR8ANcDOKiq5wE42PwOgiAIxkTxha6qM6r6vWb5BIDHAWwGsAPA3qbYXgCfWaxKBkEQBGVaOUVFZCuAiwA8CGCDqlrM3vPomWQGphS2mI7kZGGLuYT5/hy5MDl2DO+0szBFK2eO0LQNRi7siYUuMbMDSxmco9YUkctXMSildKdmcvH9ZutqVdlROShLIXOp2aG2XrVpaJmZx0wAOTNc7fVn/VwbwlpjmmFppGsdpblgCX9cu3dyDmx/zWw0KLsn/SQW6QQX7P5jaa9Zn7LnIBey2YVqp6iInAHg2wC+qKrH/Tbt1YpeWRHZJSKHROSQH1obBEEQDJcqCV1ElqP3Mr9dVe9uVr8gIhtVdUZENgI4yvZV1T0A9gDApk2bFrz0mUTM8qqkX+JcjhZfjjkpWA4V+9p6h6adw+dysX2PHDmyoDxzjrH2pV9sJm35HDFWrnb6PUZbB2LbwSS50CzfH6btHD8+JxOkmeoGkQ7TdbUO4ZrzAPlBazUO0y6w56DksE0phQum23KZB/sdt9+xgLzDu1ZCZ05OlsUxPSeT0P3zZQOKvEXAjssk6dwgKSaF12riI5HQpXeWmwE8rqpfc5sOANjZLO8EcE/n2gRBEAQDUyPyfRzA5wD8UER+0Kz7MwBfAXCniFwH4BkAVy9OFYMgCIIaii90Vf0fAP10gcu7VsDUDJ9bwdQhH6ucOo98eaZympmEzb3IVDdb9mYYK/fSSy/NrjPHiZX3qpstM3OCN83kTAupM8aXz80tWjsKrYsposZsw9Rsr2raNfV9auXSeHS/PIxRdrXlaxyVwMJRvYOozLnxFQaLu/bUtKs02rhmNHLbkbOlNLfMvMOCHtJtudHfrG6+jJ2rZBpkZtyU2gkrmGm1doR8W2KkaBAEwZQw9lwuJpH6jIY5yci+bF6CtXXeacgcF/bVz02Hxb6wPl+LOVXYl545L9nkB2kIF5tog0ldvs0pbUeFlmjrFM3t59cxjSUNJy1JizknZFqmtC5HKbwwvScHcb7mNBxGro8Wa8SvUcp9lCvP6pvTRmrDk3PhkGxbup8/HguHzDk7S1pjjaN+bGGLQRAEwdImXuhBEARTwthNLpaa1jtAzQRhieeBuXSrzClksPSXTFWy4zPzSkkFsn2Z09VUNuYo9eqctSGdS9PXw8e+W5ysN/2Mg1wsdm6knG+7OZWZKs36iqmmNQ68xTI7tE0r29YJXXKKmuOYmd+6mNPsvs6ZcmpHVhts8obcPQQsTFjHnml27+SeW/885gIi/PsgDVwomVyYiTedH9j3xziTcwVBEAQTwNgldOa4YF80gzlGrFxpVFkaCpXLy+HxErR9WVm+GZM+fU4Ik6TY9FMmoTOnqJfQ7XjDkAQHCW/Mlc859Vjol11b30fWLubwyzkccyMRu4Q5DkNqGnR0JYNJgsOW7NqGLdbgpVrm7M85K2sdmjUSOtMQS3lY0ino2PPry7PRoGkep1rHahdCQg+CIJgS4oUeBEEwJYzd5JLGjPYjHZnJyrNZj7wKlMaOM3OMV4tMdcs5M7zpwBxW3gnDVLa0Ld5sY85flohrGHM69tveFXYspspa/zGnHjO5sGs1jARcNYmsuiTAYuuGkeSKOd9qTGy190Lt+IMaEw0b0ZmbQcwv5+pTim9P681GgLIRpswk0jYOnZVjdWSjXsPkEgRBEMyyZCR0NnlDzhlUcr6xUKS0fO0M5F7iTkc1MqmlNEdizpHIcrnY9i7pc42co4q1ve2kDayd/pxp/hO2b0kabzsys3bShn779VtXMxKQ0dY56/vD7o8aCdlTO7dpWwdyDn8P2/Pt61EjuTINp3QvMM063cYCKJiTM5ejpaQ5pdI90/RLqbbbEhJ6EATBlDB2Cd3sx15Ct2UvkVo5NuUUy5S4atWqBcdIc7mwJP5sMEQuvJFNtFEaQJBK+ay8r5u1PTftWIlhh7nVHDcnXec0p2HYxtvaexmlQTA10myXfmcSuvlpavuvLblwQSbVsnoYLNSPHS/XpyU/Buuj9Lllz6jH1nlfVnr+kuaZm8SCSePpoKP0HIMSEnoQBMGUEC/0IAiCKaFochGRVQAeALCyKX+Xqn5ZRM4FcAeAswE8DOBzqto/v2v/4wPg8/kx5yKb+5PNQcpyvlhIIDON5FRNb/6w47KZv1lIXm6kaNo2gE/yMGhazbahaiVzQo1DumReyZkx2HXPqbxtQ77aOlFL/T7oxBa5PmB1LIXCDWr6KaWhzdW3ps3MxMBGeTJHs+3LzsPCEHP1KQUT5CZRYf3Nwmtz+V1Yam72XhhV2OJbAC5T1Y8AuBDAdhG5BMBXAXxdVT8E4GUA13WuTRAEQTAwNVPQKYBXm5/Lmz8FcBmA323W7wXw5wBualsB+9oxCZ3BHCMmJTOnRs7RUAovTI+VLqflmaMjp23kwumYRGAaRhdqHVA5qS+3rjTtWK5/c2GfbbWN2voyaiXSQcMhu5Rn99OgtJ1Srq0mwiTYklaQSrpeomdZUmsolWdZQW3ZJGmvdTNpnEnh6bVi/cECIrpQZUMXkWXNBNFHAdwH4EkAx1TVanMEwOY+++4SkUMicshm+wmCIAiGT9ULXVXfVdULAWwBcDGAD9eeQFX3qOo2Vd3mJ3YOgiAIhkurOHRVPSYi9wP4GIC1InJqI6VvAfDcIBUwtcVPcGGqnTcxpCMMSzkhWAx5mgfGq5AsjjQ9N4M5/Lx6xlJtpqqVrwcbMWsml1K+G6PWOdZ2dOCgjtVS7o3UeTpILpc2dS3tO2guly6wa5uLqa+do7NtfWvT89Y4I9koSFbvXJw4e25KjtV0X/ZM58w8fl1t6ttczpf0WAB3io4kDl1E1ovI2mb5NACfBPA4gPsBXNUU2wngns61CYIgCAamRkLfCGCviCxD7wNwp6reKyKPAbhDRP4CwPcB3NylIqXsialEwMw3pVGb6cgxlp2RfSVZMnyWLZC1gX25czO3s4kAmNMmpXZEXRenaNtj5DLa5aTwYTtF2X41kmvbPDa1DGOCi5w0V6ud+Hqkob/+XmbOy5o21OYuyUnczHHLNPGc452FObLni9U9naSiX/lcOeZYZRL6MJyiNVEujwC4iKx/Cj17ehAEQbAEiJGiQRAEU8LYk3OxuT9ZDKotm/nDbzO1xc/DyUwuplYy9S9N3OVhahQbicpGpOXmGsw5Z9uaXDy1Ts5c+UFNF7XqflvTTy2lmP5Bj1UzAUXbuO7SOW3Z36csfW7b650jZ+oY1MEK8HvY7n//LLGkdykl81FurEituS41A9aaEplTlL0DmKN0ZHHoQRAEwdJHFisMi7Fp0ybdtWvXyM4XBEEwDezevfthVd1WKhcSehAEwZQQL/QgCIIpIV7oQRAEU0K80IMgCKaEkTpFReRFAK8B+MXITro4fBCT3YZJrz8w+W2Y9PoDk9+GSar/L6nq+lKhkb7QAUBEDtV4a5cyk96GSa8/MPltmPT6A5PfhkmvPyNMLkEQBFNCvNCDIAimhHG80PeM4ZzDZtLbMOn1Bya/DZNef2Dy2zDp9V/AyG3oQRAEweIQJpcgCIIpYaQvdBHZLiJPiMhhEbl+lOceBBE5R0TuF5HHRORHIvKFZv06EblPRH7a/P/AuOuao5nk+/sicm/z+1wRebC5DvtFZEXpGONERNaKyF0i8mMReVxEPjaB1+CPm3voURHZJyKrlvJ1EJFbROSoiDzq1tE+lx5/07TjERH56PhqPkefNvxlcx89IiL/bLOxNdtuaNrwhIj89nhq3Y2RvdCbGY/+FsAVAM4HcK2InD+q8w/ISQBfUtXzAVwC4PNNna8HcFBVzwNwsPm9lPkCetMGGl8F8HVV/RCAlwFcN5Za1fMNAP+mqh8G8BH02jIx10BENgP4IwDbVPUCAMsAXIOlfR1uBbA9Wdevz68AcF7ztwvATSOqY4lbsbAN9wG4QFV/DcBPANwAAM1zfQ2AX232+bvmnTVRjFJCvxjAYVV9SlXfBnAHgB0jPH9rVHVGVb/XLJ9A70WyGb16722K7QXwmfHUsIyIbAHwOwC+2fwWAJcBuKspstTrvwbAb6KZ4lBV31bVY5iga9BwKoDTRORUAKsBzGAJXwdVfQDA/yWr+/X5DgDf0h7fRW8C+Y2jqWl/WBtU9T+0N7E9AHwXvQnugV4b7lDVt1T1ZwAOYwJnZBvlC30zgGfd7yPNuolARLaiNxXfgwA2qOpMs+l5ABvGVK0a/hrAnwCwLP9nAzjmbuqlfh3OBfAigH9ozEbfFJHTMUHXQFWfA/BXAH6O3ov8FQAPY7KuA9C/zyf12f4DAP/aLE9qG+YRTtEKROQMAN8G8EVVPe63aS9MaEmGConIlQCOqurD465LB04F8FEAN6nqReiljphnXlnK1wAAGlvzDvQ+TpsAnI6FpoCJYqn3eQkRuRE9k+rt467LMBnlC/05AOe431uadUsaEVmO3sv8dlW9u1n9gqmUzf+j46pfgY8D+LSIPI2eiesy9OzRaxvVH1j61+EIgCOq+mDz+y70XvCTcg0A4LcA/ExVX1TVdwDcjd61maTrAPTv84l6tkXk9wFcCeCzOhe3PVFt6McoX+gPATiv8eyvQM8BcWCE529NY2++GcDjqvo1t+kAgJ3N8k4A94y6bjWo6g2qukVVt6LX3/+lqp8FcD+Aq5piS7b+AKCqzwN4VkR+pVl1OYDHMCHXoOHnAC4RkdXNPWVtmJjr0NCvzw8A+L0m2uUSAK8408ySQkS2o2eC/LSqvu42HQBwjYisFJFz0XPw/u846tgJVR3ZH4BPoedZfhLAjaM894D1/Q301MpHAPyg+fsUenbogwB+CuA/Aawbd10r2vIJAPc2y7+M3s16GMA/AVg57voV6n4hgEPNdfgXAB+YtGsAYDeAHwN4FMA/Ali5lK8DgH3o2fvfQU9Luq5fnwMQ9CLYngTwQ/SieZZqGw6jZyu35/nvXfkbmzY8AeCKcdd/kL8YKRoEQTAlhFM0CIJgSogXehAEwZQQL/QgCIIpIV7oQRAEU0K80IMgCKaEeKEHQRBMCfFCD4IgmBLihR4EQTAl/D9T3sBKp3U2ygAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GroundTruth:  pulsar pulsar pulsar pulsar\n"
     ]
    }
   ],
   "source": [
    "# print images\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then see what the network predicts that they are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = net(images)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted:  pulsar pulsar pulsar nonpulsar\n"
     ]
    }
   ],
   "source": [
    "_, predicted = torch.max(outputs, 1)\n",
    "\n",
    "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now calculate the overall accuracy of the network on **all** the test images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the network on the 10000 test images: 98 %\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a seriously imbalanced dataset, so let's take a look at the accuracy for individual classes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "class_correct = list(0. for i in range(10))\n",
    "class_total = list(0. for i in range(10))\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs, 1)\n",
    "        c = (predicted == labels).squeeze()\n",
    "        for i in range(4):\n",
    "            label = labels[i]\n",
    "            class_correct[label] += c[i].item()\n",
    "            class_total[label] += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of pulsar : 78 %\n",
      "Accuracy of nonpulsar : 99 %\n"
     ]
    }
   ],
   "source": [
    "for i in range(2):\n",
    "    print('Accuracy of %5s : %2d %%' % (classes[i], 100 * class_correct[i] / class_total[i]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}