notebooks/07_tensorflow-keras_basic-mnist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Keras model serving with Konduit Serving\n",
    "\n",
    "This notebook illustrates a simple client-server interaction to perform inference on a TensorFlow model using the Python SDK for Konduit Serving.\n",
    "\n",
    "This tutorial is split into three parts: \n",
    "\n",
    "1. Freezing models \n",
    "2. Configuration \n",
    "2. Running the server"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requirements \n",
    "\n",
    "This tutorial requires TensorFlow to be installed. See the [TensorFlow installation documentation](https://www.tensorflow.org/install/pip) for details. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "{% hint style=\"info\" %} This tutorial is tested on TensorFlow 1.14, 1.15 and 2.00.  {% endhint %}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from konduit import ParallelInferenceConfig, ServingConfig, ModelConfigType, TensorFlowConfig\n",
    "from konduit import TensorDataTypesConfig, ModelStep, InferenceConfiguration\n",
    "from konduit.server import Server\n",
    "from konduit.client import Client\n",
    "import tensorflow as tf\n",
    "\n",
    "if tf.__version__[0] == \"2\": \n",
    "    from tensorflow import keras\n",
    "else: \n",
    "    print(\"No valid TensorFlow version detected\")\n",
    "    \n",
    "from keras.layers import Flatten, Dense, Dropout, Lambda\n",
    "from keras.models import Sequential\n",
    "from keras.datasets import mnist\n",
    "\n",
    "from PIL import Image\n",
    "import numpy as np\n",
    "import imageio\n",
    "import os\n",
    "import matplotlib.pyplot as plt \n",
    "import pandas as pd "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.0.0\n"
     ]
    }
   ],
   "source": [
    "tensorflow_version = tf.__version__\n",
    "print(tensorflow_version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Serving TensorFlow 2.0 models created using the Keras API \n",
    "\n",
    "The following code is adapted from `tf-import-examples` in the [`deeplearning4j-examples`](https://github.com/eclipse/deeplearning4j-examples/) repository."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following code, we build a model using TensorFlow's Keras API and save it as a TensorFlow Graph. The architecture is adapted from the following Kaggle kernel: https://inclass.kaggle.com/charel/learn-by-example-neural-networks-hello-world/notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples\n",
      "Epoch 1/8\n",
      "60000/60000 [==============================] - 3s 55us/sample - loss: 1.1179 - accuracy: 0.6525\n",
      "Epoch 2/8\n",
      "60000/60000 [==============================] - 2s 27us/sample - loss: 0.3438 - accuracy: 0.9022\n",
      "Epoch 3/8\n",
      "60000/60000 [==============================] - 2s 29us/sample - loss: 0.2634 - accuracy: 0.9242\n",
      "Epoch 4/8\n",
      "60000/60000 [==============================] - 2s 31us/sample - loss: 0.2195 - accuracy: 0.9363\n",
      "Epoch 5/8\n",
      "60000/60000 [==============================] - 2s 32us/sample - loss: 0.1893 - accuracy: 0.9452\n",
      "Epoch 6/8\n",
      "60000/60000 [==============================] - 2s 31us/sample - loss: 0.1661 - accuracy: 0.9524\n",
      "Epoch 7/8\n",
      "60000/60000 [==============================] - 2s 28us/sample - loss: 0.1488 - accuracy: 0.9573\n",
      "Epoch 8/8\n",
      "60000/60000 [==============================] - 2s 32us/sample - loss: 0.1341 - accuracy: 0.9610\n"
     ]
    }
   ],
   "source": [
    "# Load data\n",
    "train_data, test_data = mnist.load_data()\n",
    "x_train, y_train = train_data\n",
    "x_test, y_test = test_data\n",
    "\n",
    "# Normalize\n",
    "x_train = x_train / 255.0\n",
    "x_test  = x_test / 255.0\n",
    "\n",
    "weights = None\n",
    "\n",
    "inputs = keras.layers.Input(shape=(28, 28), name=\"input_layer\")\n",
    "x = keras.layers.Flatten()(inputs)\n",
    "x = keras.layers.Dense(200, activation=\"relu\")(x)\n",
    "x = keras.layers.Dense(100, activation=\"relu\")(x)\n",
    "x = keras.layers.Dense(60, activation=\"relu\")(x)\n",
    "x = keras.layers.Dense(30, activation=\"relu\")(x)\n",
    "outputs = keras.layers.Dense(10, activation=\"softmax\", name=\"output_layer\")(x)\n",
    "model = tf.keras.Model(inputs=inputs, outputs=outputs)\n",
    "model.compile(\n",
    "    optimizer='sgd', \n",
    "    loss='sparse_categorical_crossentropy', \n",
    "    metrics=['accuracy']\n",
    ")\n",
    "    \n",
    "history = model.fit(x_train, y_train,\n",
    "                    batch_size=64,\n",
    "                    epochs=8)\n",
    "\n",
    "model.save('../data/keras/mnist_keras.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Overview\n",
    "\n",
    "Konduit Serving works by defining a series of **steps**. These include operations such as \n",
    "1. Pre- or post-processing steps\n",
    "2. One or more machine learning models\n",
    "3. Transforming the output in a way that can be understood by humans\n",
    "\n",
    "If deploying your model does not require pre- nor post-processing, only one step - a machine learning model - is required. This configuration is defined using a single `ModelStep`. \n",
    "\n",
    "Before running this notebook, run the `build_jar.py` script and copy the JAR (`konduit.jar`) to this folder. Refer to the [Python SDK README](https://github.com/KonduitAI/konduit-serving/blob/master/python/README.md) for details. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Input and output nodes \n",
    "\n",
    "Konduit Serving requires input and output names to be specified. In TensorFlow, you can find the names of your input and output nodes by printing `model.inputs[0].op.name` and `model.inputs[1].op.name` respectively. For more details, please refer to this [StackOverflow answer](https://stackoverflow.com/a/49154874/12260518). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make note of hwo to obtain input_name and output_name\n",
    "input_data_types = {'input_layer': 'FLOAT'}\n",
    "input_names = list(input_data_types.keys())\n",
    "output_names = [\"output_layer/Identity\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configure the step \n",
    "\n",
    "Define the TensorFlow configuration as a `TensorFlowConfig` object. \n",
    "\n",
    "- `tensor_data_types_config`: The TensorFlowConfig object requires a dictionary `input_data_types`. Its keys should represent column names, and the values should represent data types as strings, e.g. `\"INT32\"`. See [here](https://github.com/KonduitAI/konduit-serving/blob/master/konduit-serving-api/src/main/java/ai/konduit/serving/model/TensorDataType.java) for a list of supported data types. \n",
    "- `model_config_type`: This argument requires a `ModelConfigType` object. Specify `model_type` as `TENSORFLOW`, and `model_loading_path` to point to the location of TensorFlow weights saved in the PB file format.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from konduit import ModelConfig \n",
    "\n",
    "tensorflow_config = ModelConfig(\n",
    "    model_config_type = ModelConfigType(\n",
    "        model_type='KERAS',\n",
    "        model_loading_path=os.path.abspath(\n",
    "            f'../data/keras/mnist_keras.h5'\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'@type': 'ModelConfig',\n",
       " 'modelConfigType': {'@type': 'ModelConfigType',\n",
       "  'modelType': 'KERAS',\n",
       "  'modelLoadingPath': 'C:\\\\Users\\\\Skymind AI Berhad\\\\Documents\\\\konduit-serving-examples\\\\data\\\\keras\\\\mnist_keras.h5'}}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensorflow_config.as_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have a `TensorFlowConfig` defined, we can define a `ModelStep`. The following parameters are specified: \n",
    "- `model_config`: pass the TensorFlowConfig object here \n",
    "- `parallel_inference_config`: specify the number of workers to run in parallel. Here, we specify `workers=1`.\n",
    "- `input_names`:  names for the input data  \n",
    "- `output_names`: names for the output data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf_step = ModelStep(\n",
    "    model_config=tensorflow_config,\n",
    "    parallel_inference_config=ParallelInferenceConfig(workers=1),\n",
    "    input_names=[\"input_layer\"], \n",
    "    output_names=[\"output_layer/Identity\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Configure the server\n",
    "\n",
    "Specify the following:\n",
    "- `http_port`: select a random port.\n",
    "- `input_data_type`, `output_data_type`: Specify input and output data types as strings. \n",
    "\n",
    "{% hint style=\"info\" %}\n",
    "Accepted input and output data types are as follows:\n",
    "\n",
    "*  Input: JSON, ARROW, IMAGE, ND4J \\(not yet implemented\\) and NUMPY.\n",
    "*  Output: NUMPY, JSON, ND4J \\(not yet implemented\\) and ARROW.\n",
    "{% endhint %}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "port = np.random.randint(1000, 65535)\n",
    "serving_config = ServingConfig(\n",
    "    http_port=port,\n",
    "    input_data_format='NUMPY',\n",
    "    output_data_format='NUMPY'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `ServingConfig` has to be passed to `Server` in addition to the steps as a Python list. In this case, there is a single step: `tf_step`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "server = Server(\n",
    "    serving_config=serving_config,\n",
    "    steps=[tf_step]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, `Server()` looks for the Konduit Serving JAR `konduit.jar` in the directory the script is run in. To change this default, use the `jar_path` argument."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run the server \n",
    "\n",
    "Start the server:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting server...\n",
      "\n",
      "Server has started successfully.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<subprocess.Popen at 0x291c9d2f5f8>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "server.start()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configure the client \n",
    "\n",
    "To configure the client, create a Client object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "client = Client(port=port)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We obtain test images from the test set defined by `keras.datasets`. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAANh0lEQVR4nO3df6zddX3H8dfL/sJeYFKwtSuVKqKxOsHlCppuSw3DAYYUo2w0GekSZskGCSxmG2ExkmxxjIiETWdSR2clCFOBQLRzksaNkLHKhZRSKFuRdVh71wvUrUXgtqXv/XG/LJdyz+dezvd7zve07+cjuTnnfN/ne77vfHtf/X7v+XzP+TgiBODY95a2GwDQH4QdSIKwA0kQdiAJwg4kMbufG5vreXGchvq5SSCVV/QLHYhxT1WrFXbb50u6RdIsSX8XETeUnn+chnSOz62zSQAFm2NTx1rXp/G2Z0n6qqQLJC2XtNr28m5fD0Bv1fmb/WxJT0fEMxFxQNKdklY10xaAptUJ+xJJP530eFe17HVsr7U9YnvkoMZrbA5AHXXCPtWbAG+49jYi1kXEcEQMz9G8GpsDUEedsO+StHTS41Ml7a7XDoBeqRP2hyWdYftdtudKulTSfc20BaBpXQ+9RcQh21dJ+idNDL2tj4gnGusMQKNqjbNHxEZJGxvqBUAPcbkskARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IIlaUzbb3ilpv6RXJR2KiOEmmgLQvFphr3w8Ip5v4HUA9BCn8UASdcMekn5o+xHba6d6gu21tkdsjxzUeM3NAehW3dP4FRGx2/ZCSffbfioiHpj8hIhYJ2mdJJ3oBVFzewC6VOvIHhG7q9sxSfdIOruJpgA0r+uw2x6yfcJr9yV9QtK2phoD0Kw6p/GLJN1j+7XX+VZE/KCRrgA0ruuwR8Qzks5ssBcAPcTQG5AEYQeSIOxAEoQdSIKwA0k08UGYFF747Mc61t552dPFdZ8aW1SsHxifU6wvuaNcn7/rxY61w1ueLK6LPDiyA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASjLPP0J/88bc61j499PPyyqfX3PjKcnnnoZc61m557uM1N370+vHYaR1rQzf9UnHd2Zseabqd1nFkB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkHNG/SVpO9II4x+f2bXtN+sVnzulYe/5D5f8zT9pe3sc/f7+L9bkf+p9i/cYP3t2xdt5bXy6u+/2Xji/WPzm/82fl63o5DhTrm8eHivWVxx3setvv+f4Vxfp71z7c9Wu3aXNs0r7YO+UvFEd2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCz7PP0NB3Nxdq9V77xHqr62/esbJj7S9WLCtv+1/K33l/48r3dNHRzMx++XCxPrR1tFg/+YG7ivVfmdv5+/bn7yx/F/+xaNoju+31tsdsb5u0bIHt+23vqG5P6m2bAOqayWn8NySdf8SyayVtiogzJG2qHgMYYNOGPSIekLT3iMWrJG2o7m+QdHHDfQFoWLdv0C2KiFFJqm4Xdnqi7bW2R2yPHNR4l5sDUFfP342PiHURMRwRw3M0r9ebA9BBt2HfY3uxJFW3Y821BKAXug37fZLWVPfXSLq3mXYA9Mq04+y279DEN5efYnuXpC9IukHSt21fLulZSZf0skmUHfrvPR1rQ3d1rknSq9O89tB3X+iio2bs+f2PFesfmFv+9f3S3vd1rC37+2eK6x4qVo9O04Y9IlZ3KB2d30IBJMXlskAShB1IgrADSRB2IAnCDiTBR1zRmtmnLS3Wv3LdV4r1OZ5VrH/nlt/sWDt59KHiuscijuxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATj7GjNU3+0pFj/yLzyVNZPHChPR73gyZfedE/HMo7sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE4+zoqfFPfqRj7dHP3DzN2uUZhP7g6quL9bf+64+nef1cOLIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKMs6Onnr2g8/HkeJfH0Vf/53nF+vwfPFasR7Gaz7RHdtvrbY/Z3jZp2fW2f2Z7S/VzYW/bBFDXTE7jvyHp/CmW3xwRZ1U/G5ttC0DTpg17RDwgaW8fegHQQ3XeoLvK9tbqNP+kTk+yvdb2iO2RgxqvsTkAdXQb9q9JOl3SWZJGJd3U6YkRsS4ihiNieM40H2wA0DtdhT0i9kTEqxFxWNLXJZ3dbFsAmtZV2G0vnvTwU5K2dXougMEw7Ti77TskrZR0iu1dkr4gaaXtszQxlLlT0hU97BED7C0nnFCsX/brD3as7Tv8SnHdsS++u1ifN/5wsY7XmzbsEbF6isW39qAXAD3E5bJAEoQdSIKwA0kQdiAJwg4kwUdcUcuO6z9QrH/vlL/tWFu149PFdedtZGitSRzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtlR9L+/+9Fifevv/HWx/pNDBzvWXvyrU4vrztNosY43hyM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBOHtys5f8crF+zef/oVif5/Kv0KWPXdax9vZ/5PPq/cSRHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSYJz9GOfZ5X/iM7+3q1i/5PgXivXb9y8s1hd9vvPx5HBxTTRt2iO77aW2f2R7u+0nbF9dLV9g+37bO6rbk3rfLoBuzeQ0/pCkz0XE+yV9VNKVtpdLulbSpog4Q9Km6jGAATVt2CNiNCIere7vl7Rd0hJJqyRtqJ62QdLFvWoSQH1v6g0628skfVjSZkmLImJUmvgPQdKUf7zZXmt7xPbIQY3X6xZA12YcdtvHS7pL0jURsW+m60XEuogYjojhOZrXTY8AGjCjsNueo4mg3x4Rd1eL99heXNUXSxrrTYsAmjDt0JttS7pV0vaI+PKk0n2S1ki6obq9tycdop4z31cs//nC22q9/Fe/eEmx/rbHHqr1+mjOTMbZV0i6TNLjtrdUy67TRMi/bftySc9KKv+rA2jVtGGPiAcluUP53GbbAdArXC4LJEHYgSQIO5AEYQeSIOxAEnzE9Rgwa/l7O9bW3lnv8ofl668s1pfd9m+1Xh/9w5EdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnP0Y8NQfdv5i34vmz/hLhaZ06j8fKD8hotbro384sgNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoyzHwVeuejsYn3TRTcVqvObbQZHLY7sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5DETOZnXyrpm5LeIemwpHURcYvt6yV9VtJz1VOvi4iNvWo0s90rZhXr75zd/Vj67fsXFutz9pU/z86n2Y8eM7mo5pCkz0XEo7ZPkPSI7fur2s0R8aXetQegKTOZn31U0mh1f7/t7ZKW9LoxAM16U3+z214m6cOSNleLrrK91fZ621N+N5LttbZHbI8c1HitZgF0b8Zht328pLskXRMR+yR9TdLpks7SxJF/ygu0I2JdRAxHxPAczWugZQDdmFHYbc/RRNBvj4i7JSki9kTEqxFxWNLXJZU/rQGgVdOG3bYl3Sppe0R8edLyxZOe9ilJ25pvD0BTZvJu/ApJl0l63PaWatl1klbbPksToy87JV3Rkw5Ry1++sLxYf+i3lhXrMfp4g92gTTN5N/5BSZ6ixJg6cBThCjogCcIOJEHYgSQIO5AEYQeSIOxAEo4+Trl7ohfEOT63b9sDstkcm7Qv9k41VM6RHciCsANJEHYgCcIOJEHYgSQIO5AEYQeS6Os4u+3nJP3XpEWnSHq+bw28OYPa26D2JdFbt5rs7bSIePtUhb6G/Q0bt0ciYri1BgoGtbdB7Uuit271qzdO44EkCDuQRNthX9fy9ksGtbdB7Uuit271pbdW/2YH0D9tH9kB9AlhB5JoJey2z7f977aftn1tGz10Ynun7cdtb7E90nIv622P2d42adkC2/fb3lHdTjnHXku9XW/7Z9W+22L7wpZ6W2r7R7a3237C9tXV8lb3XaGvvuy3vv/NbnuWpP+QdJ6kXZIelrQ6Ip7sayMd2N4paTgiWr8Aw/ZvSHpR0jcj4oPVshsl7Y2IG6r/KE+KiD8dkN6ul/Ri29N4V7MVLZ48zbikiyX9nlrcd4W+flt92G9tHNnPlvR0RDwTEQck3SlpVQt9DLyIeEDS3iMWr5K0obq/QRO/LH3XobeBEBGjEfFodX+/pNemGW913xX66os2wr5E0k8nPd6lwZrvPST90PYjtte23cwUFkXEqDTxyyNpYcv9HGnaabz76Yhpxgdm33Uz/XldbYR9qu/HGqTxvxUR8auSLpB0ZXW6ipmZ0TTe/TLFNOMDodvpz+tqI+y7JC2d9PhUSbtb6GNKEbG7uh2TdI8GbyrqPa/NoFvdjrXcz/8bpGm8p5pmXAOw79qc/ryNsD8s6Qzb77I9V9Klku5roY83sD1UvXEi20OSPqHBm4r6PklrqvtrJN3bYi+vMyjTeHeaZlwt77vWpz+PiL7/SLpQE+/I/0TSn7XRQ4e+3i3psernibZ7k3SHJk7rDmrijOhySSdL2iRpR3W7YIB6u03S45K2aiJYi1vq7dc08afhVklbqp8L2953hb76st+4XBZIgivogCQIO5AEYQeSIOxAEoQdSIKwA0kQdiCJ/wNGNvRIqiy+UgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: 0.0, 1: 0.0, 2: 0.0, 3: 0.002, 4: 0.0, 5: 0.0, 6: 0.0, 7: 0.998, 8: 0.0, 9: 0.0}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAANxUlEQVR4nO3de4xU93nG8ecBc7EwtqFgSjGygwOycSpDsiJx3YstN6nDH8GRckOJgyNHpGrcJhJSYrmV4igXWVVst1WjVCRGIZUvcn2JqWIlJsSR6wRhLy4BbJJAXOpgVmDEpuBWhd312z/2UG3wzpll5sycMe/3I41m5rxzznk18OyZmd+c+TkiBODsN6nuBgB0B2EHkiDsQBKEHUiCsANJnNPNnU31tJiuGd3cJZDK/+q/dTJOeLxaW2G3fYOkv5c0WdK3IuLOssdP1wy909e3s0sAJbbFloa1ll/G254s6euS3itpqaTVtpe2uj0AndXOe/YVkvZFxEsRcVLSg5JWVdMWgKq1E/YFkn495v6BYtlvsb3Wdr/t/iGdaGN3ANrRTtjH+xDgDd+9jYj1EdEXEX1TNK2N3QFoRzthPyBp4Zj7F0s62F47ADqlnbA/J2mx7bfYnirpI5I2VdMWgKq1PPQWEcO2b5X0A40OvW2IiBcq6wxApdoaZ4+IJyQ9UVEvADqIr8sCSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5BEV39KGq3Z/+WrS+sj0xtPzjn3yldL19161SMt9XTKZT/6RGl95rPnNqzN+4eftrVvnBmO7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBOPsPWDwe4tL67uX/WPH9j3UeIh+Qn5+3bdK6/f1zW9Ye2jzn5SuO7Jnb0s9YXwc2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZu6DZOPpPlj3YsX3/028Wldbv3vru0vqll5SfD//k0kdL6x+dOdCw9pWb55Suu+jzjLNXqa2w294v6bikEUnDEdFXRVMAqlfFkf26iDhSwXYAdBDv2YEk2g17SHrS9nbba8d7gO21tvtt9w/pRJu7A9Cqdl/GXxMRB21fJGmz7Z9HxNNjHxAR6yWtl6TzPbvN0y4AtKqtI3tEHCyuD0t6TNKKKpoCUL2Ww257hu2Zp25Leo+k3VU1BqBa7byMnyfpMduntnN/RHy/kq7eZIavf0dp/UdXfb3JFqaUVv9ucElp/akPl4x4Hjxcuu6Swf7S+qTp00vrX932+6X12+fsalgbnjVcui6q1XLYI+IlSVdV2AuADmLoDUiCsANJEHYgCcIOJEHYgSQ4xbUCry2YWlqf1ORvarOhtR+/r3x4a+SlX5TW27Hvi8tL6/fPvqvJFqY1rFz8fY413cSzDSRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM5egQu/s7W0/oH+j5XWPXistD48sP8MO6rOJ1f+sLR+3qTG4+joLRzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtm7YOTFX9bdQkP7v3J1af2WC7/WZAvlPzW9buBdDWszf7indN2RJnvGmeHIDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM5+lvvNTeXj6D/5ePk4+gWTysfRt56YXFrf8eXGvzt/7rFnS9dFtZoe2W1vsH3Y9u4xy2bb3mx7b3E9q7NtAmjXRF7Gf1vSDactu03SlohYLGlLcR9AD2sa9oh4WtLR0xavkrSxuL1R0o0V9wWgYq1+QDcvIgYkqbi+qNEDba+13W+7f0gnWtwdgHZ1/NP4iFgfEX0R0TelZJI/AJ3VatgP2Z4vScX14epaAtAJrYZ9k6Q1xe01kh6vph0AndJ0nN32A5KulTTH9gFJX5B0p6SHbN8i6WVJH+xkk2jdkbdHab3ZOHoza378ydL6ku8ylt4rmoY9IlY3KF1fcS8AOoivywJJEHYgCcIOJEHYgSQIO5AEp7ieBU5uvqRhbevldzVZu3zo7aqta0rrV6z7VWmdn4PuHRzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtnfBM5ZdGlp/Utv/ZeGtVlNTmHd3uSXwi75UvlI+cjgYPkG0DM4sgNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoyzvwlc9tArpfXlU1v/m716y5+X1pf87LmWt43ewpEdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnL0HDK65urT+xXnNfvt9WsPKmv1/WrrmFZ/bV1rnd9/PHk2P7LY32D5se/eYZXfYfsX2juKysrNtAmjXRF7Gf1vSDeMsvycilhWXJ6ptC0DVmoY9Ip6WdLQLvQDooHY+oLvV9s7iZf6sRg+yvdZ2v+3+ITX5wTMAHdNq2L8h6TJJyyQNSGr4CVJErI+Ivojom1LyQRKAzmop7BFxKCJGIuJ1Sd+UtKLatgBUraWw254/5u77Je1u9FgAvaHpOLvtByRdK2mO7QOSviDpWtvLJIWk/ZI+1cEe3/TOWfB7pfU/+qttpfXzJrX+9mfri28trS8Z5Hz1LJqGPSJWj7P43g70AqCD+LoskARhB5Ig7EAShB1IgrADSXCKaxfsuX1haf27v/uvbW3/ul0fbFjjFFacwpEdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnL0Ltr/vniaPaO8XfC74i9cb1oYHB9vaNs4eHNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnG2c8CQ/MuaFibcnJBFzt5o5FXjzSsxYny6cA8rfz7B5PnzmmpJ0kamXthaX3vuqktb3siYsQNa5f/ZZPfIDh2rKV9cmQHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZz8LfO/hDXW30NAf/Pt4kwCPOnLo/NJ1Z809Xlrf9o77W+qp1y39m1tL64s+t7Wl7TY9stteaPsp23tsv2D7M8Xy2bY3295bXM9qqQMAXTGRl/HDktZFxBWS3iXp07aXSrpN0paIWCxpS3EfQI9qGvaIGIiI54vbxyXtkbRA0ipJG4uHbZR0Y6eaBNC+M/qAzvalkpZL2iZpXkQMSKN/ECRd1GCdtbb7bfcPqfy70AA6Z8Jht32epEckfTYiJvxN/IhYHxF9EdE3pc0fVgTQugmF3fYUjQb9voh4tFh8yPb8oj5f0uHOtAigCk2H3mxb0r2S9kTE3WNKmyStkXRncf14Rzo8C6x68aOl9S1ve7hLnXTfT5c/UNu+/ydONqwNReOf356IlTtvLq3/147WT79d8Mxwy+uWmcg4+zWSbpK0y/aOYtntGg35Q7ZvkfSypMaThAOoXdOwR8QzkhqdaX99te0A6BS+LgskQdiBJAg7kARhB5Ig7EASnOLaBef+2X+U1q/8avkpjdHBf6WZlx8trXfyNNIr/+0TpfV4eUZb21/08GuNi8/uamvbs7S3rXodOLIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKOiK7t7HzPjneaE+WATtkWW3Qsjo57lipHdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiiadhtL7T9lO09tl+w/Zli+R22X7G9o7is7Hy7AFo1kekHhiWti4jnbc+UtN325qJ2T0R8rXPtAajKROZnH5A0UNw+bnuPpAWdbgxAtc7oPbvtSyUtl7StWHSr7Z22N9ie1WCdtbb7bfcP6URbzQJo3YTDbvs8SY9I+mxEHJP0DUmXSVqm0SP/XeOtFxHrI6IvIvqmaFoFLQNoxYTCbnuKRoN+X0Q8KkkRcSgiRiLidUnflLSic20CaNdEPo23pHsl7YmIu8csnz/mYe+XtLv69gBUZSKfxl8j6SZJu2zvKJbdLmm17WWSQtJ+SZ/qSIcAKjGRT+OfkTTe71A/UX07ADqFb9ABSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeScER0b2f2q5L+c8yiOZKOdK2BM9OrvfVqXxK9tarK3i6JiLnjFboa9jfs3O6PiL7aGijRq731al8SvbWqW73xMh5IgrADSdQd9vU1779Mr/bWq31J9NaqrvRW63t2AN1T95EdQJcQdiCJWsJu+wbbv7C9z/ZtdfTQiO39tncV01D319zLBtuHbe8es2y27c229xbX486xV1NvPTGNd8k047U+d3VPf9719+y2J0v6paR3Szog6TlJqyPixa420oDt/ZL6IqL2L2DY/mNJr0n6TkS8rVj2t5KORsSdxR/KWRHx+R7p7Q5Jr9U9jXcxW9H8sdOMS7pR0s2q8bkr6etD6sLzVseRfYWkfRHxUkSclPSgpFU19NHzIuJpSUdPW7xK0sbi9kaN/mfpuga99YSIGIiI54vbxyWdmma81ueupK+uqCPsCyT9esz9A+qt+d5D0pO2t9teW3cz45gXEQPS6H8eSRfV3M/pmk7j3U2nTTPeM89dK9Oft6uOsI83lVQvjf9dExFvl/ReSZ8uXq5iYiY0jXe3jDPNeE9odfrzdtUR9gOSFo65f7GkgzX0Ma6IOFhcH5b0mHpvKupDp2bQLa4P19zP/+ulabzHm2ZcPfDc1Tn9eR1hf07SYttvsT1V0kckbaqhjzewPaP44ES2Z0h6j3pvKupNktYUt9dIerzGXn5Lr0zj3WiacdX83NU+/XlEdP0iaaVGP5H/laS/rqOHBn0tkvSz4vJC3b1JekCjL+uGNPqK6BZJvyNpi6S9xfXsHurtnyXtkrRTo8GaX1Nvf6jRt4Y7Je0oLivrfu5K+urK88bXZYEk+AYdkARhB5Ig7EAShB1IgrADSRB2IAnCDiTxfy43Cn7d/BIFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: 0.0, 1: 0.001, 2: 0.949, 3: 0.004, 4: 0.0, 5: 0.044, 6: 0.001, 7: 0.0, 8: 0.001, 9: 0.0}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAMR0lEQVR4nO3dXawcdRnH8d/P0hYtEFugpJQGQRtiAa16KCQ1CiIEiFq8wNALUhNiSRSjCRcQvZBgQohRiPEFU2xDfYOogG0ivjSNphoNcoq1tBQskgqHNj2SqlSE0pfHizOYQzk7e7ozs7Pt8/0km92dZ2fnyfb8OrPzn3P+jggBOPa9qe0GAPQHYQeSIOxAEoQdSIKwA0kc18+NTfP0OF4z+rlJIJVX9JJejX2eqFYp7LavkPR1SVMkfTci7ih7/fGaoQt9aZVNAijxSKzvWOv5MN72FEnfknSlpAWSltpe0Ov7AWhWle/siyQ9HRHPRMSrku6XtKSetgDUrUrY50p6btzzkWLZ69hebnvY9vB+7auwOQBVVAn7RCcB3nDtbUSsiIihiBiaqukVNgegiiphH5E0b9zzMyTtrNYOgKZUCfujkubbPsv2NEnXSlpbT1sA6tbz0FtEHLB9o6RfaWzobVVEbK2tMwC1qjTOHhEPS3q4pl4ANIjLZYEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1Ioq9TNiMfv+/cjrWfr/1+6brnf+fG0vq8L/+hp56yYs8OJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kwzo5GjV5wUsfaAR0sXfctO6PudlKrFHbbOyTtlXRQ0oGIGKqjKQD1q2PPfklEvFDD+wBoEN/ZgSSqhj0k/dr2RtvLJ3qB7eW2h20P79e+ipsD0Kuqh/GLI2Kn7dmS1tl+MiI2jH9BRKyQtEKSTvIszrgALam0Z4+IncX9qKSHJC2qoykA9es57LZn2D7xtceSLpe0pa7GANSrymH8aZIesv3a+/woIn5ZS1c4ZvzzXZ3H0kcOlJ/DOXnlH+tuJ7Wewx4Rz0h6d429AGgQQ29AEoQdSIKwA0kQdiAJwg4kwa+4opJYvLC0/ruP3Nmx9sENny1d9x36c089YWLs2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZUcmeBW8urc+Z8paOtbk/nVp3OyjBnh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCcHZVc+unyP/f8s5fe2rF2wm+fKl23fEJnHCn27EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBOPsKDXl3HNK67fPvq+0vvLFMzrWDv7r3z31hN503bPbXmV71PaWcctm2V5ne3txP7PZNgFUNZnD+HslXXHYslskrY+I+ZLWF88BDLCuYY+IDZL2HLZ4iaTVxePVkq6uuS8ANev1BN1pEbFLkor72Z1eaHu57WHbw/u1r8fNAaiq8bPxEbEiIoYiYmiqpje9OQAd9Br23bbnSFJxP1pfSwCa0GvY10paVjxeJmlNPe0AaErXcXbb90m6WNIptkckfUnSHZJ+bPt6Sc9KuqbJJtGe5y87udL6G/eeWVJ9udJ748h0DXtELO1QurTmXgA0iMtlgSQIO5AEYQeSIOxAEoQdSIJfcUWpFxfsr7T+pm8u7Fh7q8r/DDXqxZ4dSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnD25fVdeUFpfc/k3Suu3vfC+0vqsBzZ3rB0qXRN1Y88OJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kwzp7cyIfKfwTeNe340vqyHeeX1me/9OQR94RmsGcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZ0/u1PNGS+sHo/y3zo9bM7POdtCgrnt226tsj9reMm7Zrbaft72puF3VbJsAqprMYfy9kq6YYPldEbGwuD1cb1sA6tY17BGxQdKePvQCoEFVTtDdaHtzcZjf8Yub7eW2h20P79e+CpsDUEWvYb9b0tslLZS0S9LXOr0wIlZExFBEDE3V9B43B6CqnsIeEbsj4mBEHJJ0j6RF9bYFoG49hd32nHFPPy5pS6fXAhgMXcfZbd8n6WJJp9gekfQlSRfbXigpJO2QdEODPaKC4846s7T+1XN+Ulq/59/zSuuzVjHH+tGia9gjYukEi1c20AuABnG5LJAEYQeSIOxAEoQdSIKwA0nwK67HuO03nF5av6jLRY2feuyS0vo8LrE4arBnB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkGGc/xh2a90ql9V/+V/mUzTh6sGcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZz/GffvCH1Raf+4vptTUCdrGnh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCc/RjwykcXday9//g/dVmbH4Esuu7Zbc+z/Rvb22xvtf25Yvks2+tsby/uZzbfLoBeTeYw/oCkmyLinZIukvQZ2wsk3SJpfUTMl7S+eA5gQHUNe0TsiojHisd7JW2TNFfSEkmri5etlnR1U00CqO6ITtDZfpuk90h6RNJpEbFLGvsPQdLsDusstz1se3i/9lXrFkDPJh122ydIekDS5yPixcmuFxErImIoIoamqsssggAaM6mw256qsaD/MCIeLBbvtj2nqM+RNNpMiwDq0HXcxbYlrZS0LSLuHFdaK2mZpDuK+zWNdIiunv1YdKxNd/k/8W0vnF9aP2HNxtJ65y1j0ExmkHWxpOskPW57U7HsCxoL+Y9tXy/pWUnXNNMigDp0DXtE/F6SO5QvrbcdAE3hclkgCcIOJEHYgSQIO5AEYQeS4PcbjwJTTjqptH7z4od7fu8f/eIDpfWzD/yx5/fGYGHPDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM5+FDi0r/zPeT3x39M71j78/FDpuvNv31paP1haxdGEPTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJME4+1EguoyzP1UylD5Nfy9dl3H0PNizA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASXcNue57t39jeZnur7c8Vy2+1/bztTcXtqubbBdCryVxUc0DSTRHxmO0TJW20va6o3RURX22uPQB1mcz87Lsk7Soe77W9TdLcphsDUK8j+s5u+22S3iPpkWLRjbY3215le2aHdZbbHrY9vF/ll30CaM6kw277BEkPSPp8RLwo6W5Jb5e0UGN7/q9NtF5ErIiIoYgYmqrpNbQMoBeTCrvtqRoL+g8j4kFJiojdEXEwIg5JukfSoubaBFDVZM7GW9JKSdsi4s5xy+eMe9nHJW2pvz0AdZnM2fjFkq6T9LjtTcWyL0haanuhpJC0Q9INjXQIoBaTORv/e0meoNT7pOAA+o4r6IAkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0k4Ivq3Mfsf0uvmED5F0gt9a+DIDGpvg9qXRG+9qrO3MyPi1IkKfQ37GzZuD0dEyezi7RnU3ga1L4neetWv3jiMB5Ig7EASbYd9RcvbLzOovQ1qXxK99aovvbX6nR1A/7S9ZwfQJ4QdSKKVsNu+wvZTtp+2fUsbPXRie4ftx4tpqIdb7mWV7VHbW8Ytm2V7ne3txf2Ec+y11NtATONdMs14q59d29Of9/07u+0pkv4q6TJJI5IelbQ0Ip7oayMd2N4haSgiWr8Aw/YHJP1H0vci4rxi2Vck7YmIO4r/KGdGxM0D0tutkv7T9jTexWxFc8ZPMy7pakmfVIufXUlfn1AfPrc29uyLJD0dEc9ExKuS7pe0pIU+Bl5EbJC057DFSyStLh6v1tgPS9916G0gRMSuiHiseLxX0mvTjLf62ZX01RdthH2upOfGPR/RYM33HpJ+bXuj7eVtNzOB0yJilzT2wyNpdsv9HK7rNN79dNg04wPz2fUy/XlVbYR9oqmkBmn8b3FEvFfSlZI+UxyuYnImNY13v0wwzfhA6HX686raCPuIpHnjnp8haWcLfUwoInYW96OSHtLgTUW9+7UZdIv70Zb7+b9BmsZ7omnGNQCfXZvTn7cR9kclzbd9lu1pkq6VtLaFPt7A9ozixIlsz5B0uQZvKuq1kpYVj5dJWtNiL68zKNN4d5pmXC1/dq1Pfx4Rfb9JukpjZ+T/JumLbfTQoa+zJf2luG1tuzdJ92nssG6/xo6Irpd0sqT1krYX97MGqLfvS3pc0maNBWtOS729X2NfDTdL2lTcrmr7syvpqy+fG5fLAklwBR2QBGEHkiDsQBKEHUiCsANJEHYgCcIOJPE/O0elww7G9HEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: 0.0, 1: 0.99, 2: 0.001, 3: 0.003, 4: 0.001, 5: 0.001, 6: 0.001, 7: 0.004, 8: 0.001, 9: 0.0}\n"
     ]
    }
   ],
   "source": [
    "for img in x_test[0:3]: \n",
    "    plt.imshow(img)\n",
    "    predicted = client.predict(\n",
    "        data_input={'input_layer': np.expand_dims(img.reshape(28, 28), axis=0)}\n",
    "    )\n",
    "    plt.show()\n",
    "    print(dict(zip(np.arange(10), predicted[0].round(3))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch prediction\n",
    "\n",
    "To predict in batches, the `data_input` dictionary has to be specified differently for client images in NDARRAY format. To input a batch of observations, ensure that your inputs are in the NCHW format: number of observations, channels (optional if single channel), height and width. An example is as follows: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "predicted = client.predict(\n",
    "    data_input={'input_layer': x_test[0:3].reshape(3, 28, 28)}\n",
    ")\n",
    "\n",
    "server.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We compare the predicted probabilities and the corresponding labels: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.002</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.998</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.949</td>\n",
       "      <td>0.004</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.044</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.990</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.003</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.004</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0      1      2      3      4      5      6      7      8    9\n",
       "0  0.0  0.000  0.000  0.002  0.000  0.000  0.000  0.998  0.000  0.0\n",
       "1  0.0  0.001  0.949  0.004  0.000  0.044  0.001  0.000  0.001  0.0\n",
       "2  0.0  0.990  0.001  0.003  0.001  0.001  0.001  0.004  0.001  0.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(predicted).round(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7, 2, 1], dtype=uint8)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test[0:3]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}