t81_558_class8_kaggle.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# T81-558: Applications of Deep Neural Networks\n",
    "**Module 8: Kaggle Data Sets.**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), School of Engineering and Applied Science, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module Video Material\n",
    "\n",
    "Main video lecture:\n",
    "\n",
    "* [Part 8.1: Introduction to Kaggle](https://www.youtube.com/watch?v=XpGI4engRjQ&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN&index=24)\n",
    "* [Part 8.2: Simple Kaggle Solution for Keras](https://www.youtube.com/watch?v=AA3KFxjPxCo&index=25&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN)\n",
    "* [Part 8.3: Overview of this Semester's Kaggle Assignment](https://www.youtube.com/watch?v=suwZz0qLmww)\n",
    "\n",
    "Weekly video update:\n",
    "\n",
    "* [In Class Meeting #3, March 5, 2018](https://www.youtube.com/watch?v=onplZIDnmuc&t=10s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Helpful Functions\n",
    "\n",
    "You will see these at the top of every module.  These are simply a set of reusable functions that we will make use of.  Each of them will be explained as the semester progresses.  They are explained in greater detail as the course progresses.  Class 4 contains a complete overview of these functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import preprocessing\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import shutil\n",
    "import os\n",
    "import requests\n",
    "import base64\n",
    "\n",
    "\n",
    "# Encode text values to dummy variables(i.e. [1,0,0],[0,1,0],[0,0,1] for red,green,blue)\n",
    "def encode_text_dummy(df, name):\n",
    "    dummies = pd.get_dummies(df[name])\n",
    "    for x in dummies.columns:\n",
    "        dummy_name = \"{}-{}\".format(name, x)\n",
    "        df[dummy_name] = dummies[x]\n",
    "    df.drop(name, axis=1, inplace=True)\n",
    "\n",
    "\n",
    "# Encode text values to a single dummy variable.  The new columns (which do not replace the old) will have a 1\n",
    "# at every location where the original column (name) matches each of the target_values.  One column is added for\n",
    "# each target value.\n",
    "def encode_text_single_dummy(df, name, target_values):\n",
    "    for tv in target_values:\n",
    "        l = list(df[name].astype(str))\n",
    "        l = [1 if str(x) == str(tv) else 0 for x in l]\n",
    "        name2 = \"{}-{}\".format(name, tv)\n",
    "        df[name2] = l\n",
    "\n",
    "\n",
    "# Encode text values to indexes(i.e. [1],[2],[3] for red,green,blue).\n",
    "def encode_text_index(df, name):\n",
    "    le = preprocessing.LabelEncoder()\n",
    "    df[name] = le.fit_transform(df[name])\n",
    "    return le.classes_\n",
    "\n",
    "\n",
    "# Encode a numeric column as zscores\n",
    "def encode_numeric_zscore(df, name, mean=None, sd=None):\n",
    "    if mean is None:\n",
    "        mean = df[name].mean()\n",
    "\n",
    "    if sd is None:\n",
    "        sd = df[name].std()\n",
    "\n",
    "    df[name] = (df[name] - mean) / sd\n",
    "\n",
    "\n",
    "# Convert all missing values in the specified column to the median\n",
    "def missing_median(df, name):\n",
    "    med = df[name].median()\n",
    "    df[name] = df[name].fillna(med)\n",
    "\n",
    "\n",
    "# Convert all missing values in the specified column to the default\n",
    "def missing_default(df, name, default_value):\n",
    "    df[name] = df[name].fillna(default_value)\n",
    "\n",
    "\n",
    "# Convert a Pandas dataframe to the x,y inputs that TensorFlow needs\n",
    "def to_xy(df, target):\n",
    "    result = []\n",
    "    for x in df.columns:\n",
    "        if x != target:\n",
    "            result.append(x)\n",
    "    # find out the type of the target column.  Is it really this hard? :(\n",
    "    target_type = df[target].dtypes\n",
    "    target_type = target_type[0] if hasattr(target_type, '__iter__') else target_type\n",
    "    # Encode to int for classification, float otherwise. TensorFlow likes 32 bits.\n",
    "    if target_type in (np.int64, np.int32):\n",
    "        # Classification\n",
    "        dummies = pd.get_dummies(df[target])\n",
    "        return df.as_matrix(result).astype(np.float32), dummies.as_matrix().astype(np.float32)\n",
    "    else:\n",
    "        # Regression\n",
    "        return df.as_matrix(result).astype(np.float32), df.as_matrix([target]).astype(np.float32)\n",
    "\n",
    "# Nicely formatted time string\n",
    "def hms_string(sec_elapsed):\n",
    "    h = int(sec_elapsed / (60 * 60))\n",
    "    m = int((sec_elapsed % (60 * 60)) / 60)\n",
    "    s = sec_elapsed % 60\n",
    "    return \"{}:{:>02}:{:>05.2f}\".format(h, m, s)\n",
    "\n",
    "\n",
    "# Regression chart.\n",
    "def chart_regression(pred,y,sort=True):\n",
    "    t = pd.DataFrame({'pred' : pred, 'y' : y.flatten()})\n",
    "    if sort:\n",
    "        t.sort_values(by=['y'],inplace=True)\n",
    "    a = plt.plot(t['y'].tolist(),label='expected')\n",
    "    b = plt.plot(t['pred'].tolist(),label='prediction')\n",
    "    plt.ylabel('output')\n",
    "    plt.legend()\n",
    "    plt.show()\n",
    "\n",
    "# Remove all rows where the specified column is +/- sd standard deviations\n",
    "def remove_outliers(df, name, sd):\n",
    "    drop_rows = df.index[(np.abs(df[name] - df[name].mean()) >= (sd * df[name].std()))]\n",
    "    df.drop(drop_rows, axis=0, inplace=True)\n",
    "\n",
    "\n",
    "# Encode a column to a range between normalized_low and normalized_high.\n",
    "def encode_numeric_range(df, name, normalized_low=-1, normalized_high=1,\n",
    "                         data_low=None, data_high=None):\n",
    "    if data_low is None:\n",
    "        data_low = min(df[name])\n",
    "        data_high = max(df[name])\n",
    "\n",
    "    df[name] = ((df[name] - data_low) / (data_high - data_low)) \\\n",
    "               * (normalized_high - normalized_low) + normalized_low\n",
    "        \n",
    "# This function submits an assignment.  You can submit an assignment as much as you like, only the final\n",
    "# submission counts.  The paramaters are as follows:\n",
    "# data - Pandas dataframe output.\n",
    "# key - Your student key that was emailed to you.\n",
    "# no - The assignment class number, should be 1 through 1.\n",
    "# source_file - The full path to your Python or IPYNB file.  This must have \"_class1\" as part of its name.  \n",
    "# .             The number must match your assignment number.  For example \"_class2\" for class assignment #2.\n",
    "def submit(data,key,no,source_file=None):\n",
    "    if source_file is None and '__file__' not in globals(): raise Exception('Must specify a filename when a Jupyter notebook.')\n",
    "    if source_file is None: source_file = __file__\n",
    "    suffix = '_class{}'.format(no)\n",
    "    if suffix not in source_file: raise Exception('{} must be part of the filename.'.format(suffix))\n",
    "    with open(source_file, \"rb\") as image_file:\n",
    "        encoded_python = base64.b64encode(image_file.read()).decode('ascii')\n",
    "    ext = os.path.splitext(source_file)[-1].lower()\n",
    "    if ext not in ['.ipynb','.py']: raise Exception(\"Source file is {} must be .py or .ipynb\".format(ext))\n",
    "    r = requests.post(\"https://api.heatonresearch.com/assignment-submit\",\n",
    "        headers={'x-api-key':key}, json={'csv':base64.b64encode(data.to_csv(index=False).encode('ascii')).decode(\"ascii\"),\n",
    "        'assignment': no, 'ext':ext, 'py':encoded_python})\n",
    "    if r.status_code == 200:\n",
    "        print(\"Success: {}\".format(r.text))\n",
    "    else: print(\"Failure: {}\".format(r.text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# What is Kaggle?\n",
    "\n",
    "[Kaggle](http://www.kaggle.com) runs competitions in which data scientists compete in order to provide the best model to fit the data. The capstone project of this chapter features Kaggle’s [Titanic data set](https://www.kaggle.com/c/titanic-gettingStarted). Before we get started with the Titanic example, it’s important to be aware of some Kaggle guidelines. First, most competitions end on a specific date. Website organizers have currently scheduled the Titanic competition to end on December 31, 2016. However, they have already extended the deadline several times, and an extension beyond 2014 is also possible. Second, the Titanic data set is considered a tutorial data set. In other words, there is no prize, and your score in the competition does not count towards becoming a Kaggle Master. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kaggle Ranks\n",
    "\n",
    "Kaggle ranks are achieved by earning gold, silver and bronze medals.\n",
    "\n",
    "* [Kaggle Top Users](https://www.kaggle.com/rankings)\n",
    "* [Current Top Kaggle User's Profile Page](https://www.kaggle.com/stasg7)\n",
    "* [Jeff Heaton's (your instructor) Kaggle Profile](https://www.kaggle.com/jeffheaton)\n",
    "* [Current Kaggle Ranking System](https://www.kaggle.com/progression)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Typical Kaggle Competition\n",
    "\n",
    "A typical Kaggle competition will have several components.  Consider the Titanic tutorial:\n",
    "\n",
    "* [Competition Summary Page](https://www.kaggle.com/c/titanic)\n",
    "* [Data Page](https://www.kaggle.com/c/titanic/data)\n",
    "* [Evaluation Description Page](https://www.kaggle.com/c/titanic/details/evaluation)\n",
    "* [Leaderboard](https://www.kaggle.com/c/titanic/leaderboard)\n",
    "\n",
    "## How Kaggle Competitions are Scored\n",
    "\n",
    "Kaggle is provided with a data set by the competition sponsor.  This data set is divided up as follows:\n",
    "\n",
    "* **Complete Data Set** - This is the complete data set.\n",
    "    * **Training Data Set** - You are provided both the inputs and the outcomes for the training portion of the data set.\n",
    "    * **Test Data Set** - You are provided the complete test data set; however, you are not given the outcomes.  Your submission is  your predicted outcomes for this data set.\n",
    "        * **Public Leaderboard** - You are not told what part of the test data set contributes to the public leaderboard.  Your public score is calculated based on this part of the data set.\n",
    "        * **Private Leaderboard** - You are not told what part of the test data set contributes to the public leaderboard.  Your final score/rank is calculated based on this part.  You do not see your private leaderboard score until the end.\n",
    "\n",
    "![How Kaggle Competitions are Scored](https://raw.githubusercontent.com/jeffheaton/t81_558_deep_learning/master/images/class_3_kaggle.png \"How Kaggle Competitions are Scored\")\n",
    "\n",
    "## Preparing a Kaggle Submission\n",
    "\n",
    "Code need not be submitted to Kaggle.  For competitions, you are scored entirely on the accuracy of your sbmission file.  A Kaggle submission file is always a CSV file that contains the **Id** of the row you are predicting and the answer.  For the titanic competition, a submission file looks something like this:\n",
    "\n",
    "```\n",
    "PassengerId,Survived\n",
    "892,0\n",
    "893,1\n",
    "894,1\n",
    "895,0\n",
    "896,0\n",
    "897,1\n",
    "...\n",
    "```\n",
    "\n",
    "The above file states the prediction for each of various passengers.  You should only predict on ID's that are in the test file.  Likewise, you should render a prediction for every row in the test file.  Some competitions will have different formats for their answers.  For example, a multi-classification will usually have a column for each class and your predictions for each class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Select Kaggle Competitions\n",
    "\n",
    "There have been many interesting competitions on Kaggle, these are some of my favorites.\n",
    "\n",
    "## Predictive Modeling\n",
    "\n",
    "* [Otto Group Product Classification Challenge](https://www.kaggle.com/c/otto-group-product-classification-challenge)\n",
    "* [Galaxy Zoo - The Galaxy Challenge](https://www.kaggle.com/c/galaxy-zoo-the-galaxy-challenge)\n",
    "* [Practice Fusion Diabetes Classification](https://www.kaggle.com/c/pf2012-diabetes)\n",
    "* [Predicting a Biological Response](https://www.kaggle.com/c/bioresponse)\n",
    "\n",
    "## Computer Vision\n",
    "\n",
    "* [Diabetic Retinopathy Detection](https://www.kaggle.com/c/diabetic-retinopathy-detection)\n",
    "* [Cats vs Dogs](https://www.kaggle.com/c/dogs-vs-cats)\n",
    "* [State Farm Distracted Driver Detection](https://www.kaggle.com/c/state-farm-distracted-driver-detection)\n",
    "\n",
    "## Time Series\n",
    "\n",
    "* [The Marinexplore and Cornell University Whale Detection Challenge](https://www.kaggle.com/c/whale-detection-challenge)\n",
    "\n",
    "## Other\n",
    "\n",
    "* [Helping Santa's Helpers](https://www.kaggle.com/c/helping-santas-helpers)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Iris as a Kaggle Competition\n",
    "\n",
    "If the Iris data were used as a Kaggle, you would be given the following three files:\n",
    "\n",
    "* [kaggle_iris_test.csv](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/data/kaggle_iris_test.csv) - The data that Kaggle will evaluate you on.  Contains only input, you must provide answers.  (contains x)\n",
    "* [kaggle_iris_train.csv](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/data/kaggle_iris_train.csv) - The data that you will use to train. (contains x and y)\n",
    "* [kaggle_iris_sample.csv](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/data/kaggle_iris_sample.csv) - A sample submission for Kaggle. (contains x and y)\n",
    "\n",
    "Important features of the Kaggle iris files (that differ from how we've previously seen files):\n",
    "\n",
    "* The iris species is already index encoded.\n",
    "* Your training data is in a separate file.\n",
    "* You will load the test data to generate a submission file.\n",
    "\n",
    "The following program generates a submission file for \"Iris Kaggle\".  You can use it as a starting point for assignment 3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/anaconda3/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6\n",
      "  return f(*args, **kwds)\n",
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of classes: 3\n",
      "Epoch 00143: early stopping\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x1c1eae6a20>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "from tensorflow.contrib.learn.python.learn.metric_spec import MetricSpec\n",
    "from keras.models import Sequential\n",
    "from keras.layers.core import Dense, Activation\n",
    "from keras.callbacks import EarlyStopping\n",
    "\n",
    "path = \"./data/\"\n",
    "    \n",
    "filename_train = os.path.join(path,\"kaggle_iris_train.csv\")\n",
    "filename_test = os.path.join(path,\"kaggle_iris_test.csv\")\n",
    "filename_submit = os.path.join(path,\"kaggle_iris_submit.csv\")\n",
    "\n",
    "df_train = pd.read_csv(filename_train,na_values=['NA','?'])\n",
    "\n",
    "# Encode feature vector\n",
    "encode_numeric_zscore(df_train,'petal_w')\n",
    "encode_numeric_zscore(df_train,'petal_l')\n",
    "encode_numeric_zscore(df_train,'sepal_w')\n",
    "encode_numeric_zscore(df_train,'sepal_l')\n",
    "df_train.drop('id', axis=1, inplace=True)\n",
    "\n",
    "num_classes = len(df_train.groupby('species').species.nunique())\n",
    "\n",
    "print(\"Number of classes: {}\".format(num_classes))\n",
    "\n",
    "# Create x & y for training\n",
    "\n",
    "# Create the x-side (feature vectors) of the training\n",
    "x, y = to_xy(df_train,'species')\n",
    "    \n",
    "# Split into train/test\n",
    "x_train, x_test, y_train, y_test = train_test_split(    \n",
    "    x, y, test_size=0.25, random_state=45)\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Dense(20, input_dim=x.shape[1], activation='relu'))\n",
    "model.add(Dense(10))\n",
    "model.add(Dense(y.shape[1],activation='softmax'))\n",
    "model.compile(loss='categorical_crossentropy', optimizer='adam')\n",
    "monitor = EarlyStopping(monitor='val_loss', min_delta=1e-3, patience=5, verbose=1, mode='auto')\n",
    "\n",
    "model.fit(x_train,y_train,validation_data=(x_test,y_test),callbacks=[monitor],verbose=0,epochs=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log loss score: 0.07118869402445853\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "\n",
    "# Calculate multi log loss error\n",
    "pred = model.predict(x_test)\n",
    "score = metrics.log_loss(y_test, pred)\n",
    "print(\"Log loss score: {}\".format(score))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     id     species-0  species-1  species-2\n",
      "0   100  4.399644e-03   0.822177   0.173424\n",
      "1   101  2.584576e-05   0.116803   0.883171\n",
      "2   102  4.094144e-06   0.269512   0.730484\n",
      "3   103  9.898562e-01   0.010101   0.000043\n",
      "4   104  9.989003e-01   0.001095   0.000005\n",
      "5   105  9.905711e-01   0.009386   0.000043\n",
      "6   106  9.985886e-01   0.001395   0.000017\n",
      "7   107  7.393398e-05   0.197235   0.802691\n",
      "8   108  1.919765e-03   0.955195   0.042886\n",
      "9   109  2.431928e-06   0.104798   0.895199\n",
      "10  110  5.426903e-08   0.014084   0.985916\n",
      "11  111  9.815740e-01   0.018243   0.000183\n",
      "12  112  3.729802e-02   0.936112   0.026590\n",
      "13  113  1.957318e-07   0.003631   0.996369\n",
      "14  114  9.939096e-01   0.006051   0.000040\n",
      "15  115  2.922856e-05   0.297722   0.702249\n",
      "16  116  8.085333e-02   0.912950   0.006196\n",
      "17  117  9.757338e-01   0.024068   0.000198\n",
      "18  118  9.951767e-01   0.004785   0.000038\n",
      "19  119  9.974999e-01   0.002472   0.000028\n",
      "20  120  4.768867e-04   0.921012   0.078511\n",
      "21  121  1.161375e-03   0.936333   0.062506\n",
      "22  122  1.769423e-04   0.169756   0.830067\n",
      "23  123  4.958733e-07   0.001494   0.998505\n",
      "24  124  1.918341e-07   0.002948   0.997052\n",
      "25  125  9.975540e-01   0.002420   0.000026\n",
      "26  126  1.564197e-07   0.004913   0.995087\n",
      "27  127  1.481251e-05   0.091820   0.908165\n",
      "28  128  3.631131e-07   0.013427   0.986572\n",
      "29  129  9.964398e-01   0.003520   0.000041\n",
      "30  130  3.552327e-03   0.965142   0.031306\n",
      "31  131  9.971743e-01   0.002811   0.000015\n",
      "32  132  1.246822e-05   0.268193   0.731795\n",
      "33  133  1.056292e-02   0.928300   0.061137\n",
      "34  134  1.565125e-02   0.760090   0.224259\n",
      "35  135  9.936423e-01   0.006301   0.000056\n",
      "36  136  1.067219e-07   0.002918   0.997082\n",
      "37  137  1.866401e-05   0.670881   0.329100\n",
      "38  138  9.950300e-01   0.004933   0.000036\n",
      "39  139  6.827377e-03   0.382750   0.610423\n",
      "40  140  9.847964e-01   0.015144   0.000060\n",
      "41  141  1.846334e-07   0.004912   0.995088\n",
      "42  142  2.537448e-09   0.003800   0.996200\n",
      "43  143  2.938922e-05   0.017363   0.982608\n",
      "44  144  9.965025e-01   0.003447   0.000050\n",
      "45  145  3.250041e-03   0.988577   0.008173\n",
      "46  146  7.130954e-02   0.922629   0.006062\n",
      "47  147  4.425186e-02   0.900126   0.055622\n",
      "48  148  3.199087e-02   0.838535   0.129474\n",
      "49  149  9.965227e-01   0.003449   0.000028\n",
      "50  150  9.856280e-01   0.014242   0.000130\n"
     ]
    }
   ],
   "source": [
    "# Generate Kaggle submit file\n",
    "\n",
    "# Encode feature vector\n",
    "df_test = pd.read_csv(filename_test,na_values=['NA','?'])\n",
    "\n",
    "encode_numeric_zscore(df_test,'petal_w')\n",
    "encode_numeric_zscore(df_test,'petal_l')\n",
    "encode_numeric_zscore(df_test,'sepal_w')\n",
    "encode_numeric_zscore(df_test,'sepal_l')\n",
    "ids = df_test['id']\n",
    "df_test.drop('id', axis=1, inplace=True)\n",
    "\n",
    "x = df_test.as_matrix().astype(np.float32)\n",
    "\n",
    "# Generate predictions\n",
    "pred = model.predict(x)\n",
    "#pred\n",
    "\n",
    "# Create submission data set\n",
    "\n",
    "df_submit = pd.DataFrame(pred)\n",
    "df_submit.insert(0,'id',ids)\n",
    "df_submit.columns = ['id','species-0','species-1','species-2']\n",
    "\n",
    "df_submit.to_csv(filename_submit, index=False)\n",
    "\n",
    "print(df_submit)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Kaggle Project\n",
    "\n",
    "Kaggke competition site for current semester (Fall 2017):\n",
    "* [Spring 2018 Kaggle Assignment](https://www.kaggle.com/c/wustl-t81-558-washu-deep-learning-spring-2018)\n",
    "\n",
    "Previous Kaggle competition sites for this class (for your reference, do not use):\n",
    "* [Fall 2017 Kaggle Assignment](https://www.kaggle.com/c/wustl-t81-558-washu-deep-learning-fall-2017)\n",
    "* [Spring 2017 Kaggle Assignment](https://inclass.kaggle.com/c/applications-of-deep-learning-wustl-spring-2017)\n",
    "* [Fall 2016 Kaggle Assignment](https://inclass.kaggle.com/c/wustl-t81-558-washu-deep-learning-fall-2016)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Module 8 Assignment\n",
    "\n",
    "You can find the first assignmeht here: [assignment 8](https://github.com/jeffheaton/t81_558_deep_learning/blob/master/assignments/assignment_yourname_class1.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}