notebooks/01_tensorflow-basic-mnist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow model serving with Konduit Serving\n",
    "\n",
    "This notebook illustrates a simple client-server interaction to perform inference on a TensorFlow model using the Python SDK for Konduit Serving.\n",
    "\n",
    "This tutorial is split into three parts: \n",
    "\n",
    "1. Freezing models \n",
    "2. Configuration \n",
    "2. Running the server"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requirements \n",
    "\n",
    "This tutorial requires TensorFlow to be installed. See the [TensorFlow installation documentation](https://www.tensorflow.org/install/pip) for details. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "{% hint style=\"info\" %} This tutorial is tested on TensorFlow 1.14, 1.15 and 2.00.  {% endhint %}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from konduit import ParallelInferenceConfig, ServingConfig, ModelConfigType, TensorFlowConfig\n",
    "from konduit import TensorDataTypesConfig, ModelStep, InferenceConfiguration\n",
    "from konduit.server import Server\n",
    "from konduit.client import Client\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "if tf.__version__[0] == \"1\":     \n",
    "    from tensorflow import keras\n",
    "elif tf.__version__[0] == \"2\": \n",
    "    import tensorflow.compat.v1 as tf\n",
    "    from tensorflow.compat.v1 import keras\n",
    "else: \n",
    "    print(\"No valid TensorFlow version detected\")\n",
    "    \n",
    "from keras.layers import Flatten, Dense, Dropout, Lambda\n",
    "from keras.models import Sequential\n",
    "from keras.datasets import mnist\n",
    "\n",
    "from PIL import Image\n",
    "import numpy as np\n",
    "import imageio\n",
    "import os\n",
    "import matplotlib.pyplot as plt \n",
    "import pandas as pd "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.0.0\n"
     ]
    }
   ],
   "source": [
    "tensorflow_version = tf.__version__\n",
    "print(tensorflow_version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating frozen models (Tensorflow 1.x)\n",
    "\n",
    "In TensorFlow 1.x, \"frozen\" models can be exported in the TensorFlow Graph format. For deployment, we only need information on the graph and checkpoint variables. Freezing a model allows you to discard information that is not required for deploying your model. \n",
    "\n",
    "{% hint style=\"warning\" %} TensorFlow 2.0 introduces the <a href=\"https://www.tensorflow.org/guide/saved_model\">SavedModel format</a> as the universal format for saving models. Even though the deployable protobuff (PB) files have the same file extension as frozen TensorFlow Graph files, SavedModel protobuff files are not currently supported in Konduit Serving. A workaround for TensorFlow 2.0 is to adapt the code from this tutorial for your use case to create TensorFlow Graph protobuffs. \n",
    "{% endhint %}\n",
    "\n",
    "The following code is adapted from `tf-import-examples` in the [`deeplearning4j-examples`](https://github.com/eclipse/deeplearning4j-examples/) repository."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following code, we build a model using TensorFlow's Keras API and save it as a TensorFlow Graph. The architecture is adapted from the following Kaggle kernel: https://inclass.kaggle.com/charel/learn-by-example-neural-networks-hello-world/notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\Skymind AI Berhad\\AppData\\Local\\Continuum\\miniconda3\\lib\\site-packages\\tensorflow_core\\python\\ops\\resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n",
      "input_layer\n",
      "output_layer/Softmax\n",
      "Train on 60000 samples\n",
      "Epoch 1/8\n",
      "60000/60000 [==============================] - 3s 46us/sample - loss: 0.7233 - accuracy: 0.7890\n",
      "Epoch 2/8\n",
      "60000/60000 [==============================] - 3s 50us/sample - loss: 0.2496 - accuracy: 0.9271\n",
      "Epoch 3/8\n",
      "60000/60000 [==============================] - 3s 45us/sample - loss: 0.1837 - accuracy: 0.9461\n",
      "Epoch 4/8\n",
      "60000/60000 [==============================] - 3s 45us/sample - loss: 0.1466 - accuracy: 0.9575\n",
      "Epoch 5/8\n",
      "60000/60000 [==============================] - 3s 46us/sample - loss: 0.1224 - accuracy: 0.9640\n",
      "Epoch 6/8\n",
      "60000/60000 [==============================] - 3s 45us/sample - loss: 0.1046 - accuracy: 0.9695\n",
      "Epoch 7/8\n",
      "60000/60000 [==============================] - 3s 46us/sample - loss: 0.0900 - accuracy: 0.9738\n",
      "Epoch 8/8\n",
      "60000/60000 [==============================] - 3s 55us/sample - loss: 0.0784 - accuracy: 0.9775\n",
      "10000/10000 [==============================] - 0s 26us/sample - loss: 0.1049 - accuracy: 0.9682\n",
      "WARNING:tensorflow:From <ipython-input-3-3e44ee2b8acb>:54: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use `tf.compat.v1.graph_util.convert_variables_to_constants`\n",
      "WARNING:tensorflow:From C:\\Users\\Skymind AI Berhad\\AppData\\Local\\Continuum\\miniconda3\\lib\\site-packages\\tensorflow_core\\python\\framework\\graph_util_impl.py:275: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use `tf.compat.v1.graph_util.extract_sub_graph`\n",
      "INFO:tensorflow:Froze 10 variables.\n",
      "INFO:tensorflow:Converted 10 variables to const ops.\n"
     ]
    }
   ],
   "source": [
    "# Load data\n",
    "train_data, test_data = mnist.load_data()\n",
    "x_train, y_train = train_data\n",
    "x_test, y_test = test_data\n",
    "\n",
    "# Normalize\n",
    "x_train = x_train / 255.0\n",
    "x_test  = x_test / 255.0\n",
    "\n",
    "weights = None\n",
    "\n",
    "def get_model(training=False): \n",
    "    inputs = keras.layers.Input(shape=(28, 28), name=\"input_layer\")\n",
    "    x = keras.layers.Flatten()(inputs)\n",
    "    x = keras.layers.Dense(200, activation=\"relu\")(x)\n",
    "    x = keras.layers.Dense(100, activation=\"relu\")(x)\n",
    "    x = keras.layers.Dense(60, activation=\"relu\")(x)\n",
    "    x = keras.layers.Dense(30, activation=\"relu\")(x)\n",
    "    outputs = keras.layers.Dense(10, activation=\"softmax\", name=\"output_layer\")(x)\n",
    "    model = tf.keras.Model(inputs=inputs, outputs=outputs)\n",
    "    model.compile(\n",
    "        optimizer='sgd', \n",
    "        loss='sparse_categorical_crossentropy', \n",
    "        metrics=['accuracy']\n",
    "    )\n",
    "    \n",
    "    if training: \n",
    "        print(model.inputs[0].op.name)\n",
    "        print(model.outputs[0].op.name)\n",
    "        \n",
    "    return model\n",
    "    \n",
    "    \n",
    "def train():\n",
    "    with tf.Session() as sess:\n",
    "        keras.backend.set_session(sess)\n",
    "        model = get_model(True)\n",
    "        model.fit(x_train, y_train, epochs=8)\n",
    "        weights = model.get_weights()\n",
    "    return weights\n",
    "\n",
    "def save(weights):\n",
    "    # save model to a protobuff\n",
    "    keras.backend.clear_session()\n",
    "    with tf.Session() as sess:\n",
    "        keras.backend.set_session(sess)\n",
    "        model = get_model(False)\n",
    "        model.set_weights(weights)\n",
    "        model.evaluate(x_test, y_test)\n",
    "        output_node_name = model.output.name.split(':')[0]\n",
    "        output_graph_def = tf.graph_util.convert_variables_to_constants(\n",
    "            sess, \n",
    "            sess.graph.as_graph_def(), \n",
    "            [output_node_name]\n",
    "        )\n",
    "        \n",
    "        with tf.gfile.GFile(\n",
    "            name=f\"../data/mnist/mnist_{tensorflow_version}.pb\", \n",
    "            mode=\"wb\"\n",
    "        ) as f:\n",
    "            f.write(output_graph_def.SerializeToString())\n",
    "\n",
    "weights = train()\n",
    "save(weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Overview\n",
    "\n",
    "Konduit Serving works by defining a series of **steps**. These include operations such as \n",
    "1. Pre- or post-processing steps\n",
    "2. One or more machine learning models\n",
    "3. Transforming the output in a way that can be understood by humans\n",
    "\n",
    "If deploying your model does not require pre- nor post-processing, only one step - a machine learning model - is required. This configuration is defined using a single `ModelStep`. \n",
    "\n",
    "Before running this notebook, run the `build_jar.py` script and copy the JAR (`konduit.jar`) to this folder. Refer to the [Python SDK README](https://github.com/KonduitAI/konduit-serving/blob/master/python/README.md) for details. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Input and output nodes \n",
    "\n",
    "Konduit Serving requires input and output names to be specified. In TensorFlow, you can find the names of your input and output nodes by printing `model.inputs[0].op.name` and `model.inputs[1].op.name` respectively. For more details, please refer to this [StackOverflow answer](https://stackoverflow.com/a/49154874/12260518). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make note of hwo to obtain input_name and output_name\n",
    "input_data_types = {'input_layer': 'FLOAT'}\n",
    "input_names = list(input_data_types.keys())\n",
    "output_names = [\"output_layer/Softmax\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configure the step \n",
    "\n",
    "Define the TensorFlow configuration as a `TensorFlowConfig` object. \n",
    "\n",
    "- `tensor_data_types_config`: The TensorFlowConfig object requires a dictionary `input_data_types`. Its keys should represent column names, and the values should represent data types as strings, e.g. `\"INT32\"`. See [here](https://github.com/KonduitAI/konduit-serving/blob/master/konduit-serving-api/src/main/java/ai/konduit/serving/model/TensorDataType.java) for a list of supported data types. \n",
    "- `model_config_type`: This argument requires a `ModelConfigType` object. Specify `model_type` as `TENSORFLOW`, and `model_loading_path` to point to the location of TensorFlow weights saved in the PB file format.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "tensorflow_config = TensorFlowConfig(\n",
    "    tensor_data_types_config = TensorDataTypesConfig(\n",
    "        input_data_types=input_data_types\n",
    "        ),\n",
    "    model_config_type = ModelConfigType(\n",
    "        model_type='TENSORFLOW',\n",
    "        model_loading_path=os.path.abspath(\n",
    "            f'../data/mnist/mnist_{tensorflow_version}.pb'\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'@type': 'TensorFlowConfig',\n",
       " 'tensorDataTypesConfig': {'@type': 'TensorDataTypesConfig',\n",
       "  'inputDataTypes': {'input_layer': 'FLOAT'}},\n",
       " 'modelConfigType': {'@type': 'ModelConfigType',\n",
       "  'modelType': 'TENSORFLOW',\n",
       "  'modelLoadingPath': 'C:\\\\Users\\\\Skymind AI Berhad\\\\Documents\\\\konduit-serving-examples\\\\data\\\\mnist\\\\mnist_2.0.0.pb'}}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensorflow_config.as_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have a `TensorFlowConfig` defined, we can define a `ModelStep`. The following parameters are specified: \n",
    "- `model_config`: pass the TensorFlowConfig object here \n",
    "- `parallel_inference_config`: specify the number of workers to run in parallel. Here, we specify `workers=1`.\n",
    "- `input_names`:  names for the input data  \n",
    "- `output_names`: names for the output data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf_step = ModelStep(\n",
    "    model_config=tensorflow_config,\n",
    "    parallel_inference_config=ParallelInferenceConfig(workers=1),\n",
    "    input_names=input_names,\n",
    "    output_names=output_names\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Configure the server\n",
    "\n",
    "Specify the following:\n",
    "- `http_port`: select a random port.\n",
    "- `input_data_type`, `output_data_type`: Specify input and output data types as strings. \n",
    "\n",
    "{% hint style=\"info\" %}\n",
    "Accepted input and output data types are as follows:\n",
    "\n",
    "*  Input: JSON, ARROW, IMAGE, ND4J \\(not yet implemented\\) and NUMPY.\n",
    "*  Output: NUMPY, JSON, ND4J \\(not yet implemented\\) and ARROW.\n",
    "{% endhint %}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "port = np.random.randint(1000, 65535)\n",
    "serving_config = ServingConfig(\n",
    "    http_port=port,\n",
    "    input_data_format='NUMPY',\n",
    "    output_data_format='NUMPY'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `ServingConfig` has to be passed to `Server` in addition to the steps as a Python list. In this case, there is a single step: `tf_step`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "server = Server(\n",
    "    serving_config=serving_config,\n",
    "    steps=[tf_step]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, `Server()` looks for the Konduit Serving JAR `konduit.jar` in the directory the script is run in. To change this default, use the `jar_path` argument."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run the server \n",
    "\n",
    "Start the server:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting server..\n",
      "\n",
      "Server has started successfully.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<subprocess.Popen at 0x1f97946c4e0>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "server.start()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configure the client \n",
    "\n",
    "To configure the client, create a Client object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "client = Client(port=port)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We obtain test images from the test set defined by `keras.datasets`. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAANh0lEQVR4nO3df6zddX3H8dfL/sJeYFKwtSuVKqKxOsHlCppuSw3DAYYUo2w0GekSZskGCSxmG2ExkmxxjIiETWdSR2clCFOBQLRzksaNkLHKhZRSKFuRdVh71wvUrUXgtqXv/XG/LJdyz+dezvd7zve07+cjuTnnfN/ne77vfHtf/X7v+XzP+TgiBODY95a2GwDQH4QdSIKwA0kQdiAJwg4kMbufG5vreXGchvq5SSCVV/QLHYhxT1WrFXbb50u6RdIsSX8XETeUnn+chnSOz62zSQAFm2NTx1rXp/G2Z0n6qqQLJC2XtNr28m5fD0Bv1fmb/WxJT0fEMxFxQNKdklY10xaAptUJ+xJJP530eFe17HVsr7U9YnvkoMZrbA5AHXXCPtWbAG+49jYi1kXEcEQMz9G8GpsDUEedsO+StHTS41Ml7a7XDoBeqRP2hyWdYftdtudKulTSfc20BaBpXQ+9RcQh21dJ+idNDL2tj4gnGusMQKNqjbNHxEZJGxvqBUAPcbkskARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IIlaUzbb3ilpv6RXJR2KiOEmmgLQvFphr3w8Ip5v4HUA9BCn8UASdcMekn5o+xHba6d6gu21tkdsjxzUeM3NAehW3dP4FRGx2/ZCSffbfioiHpj8hIhYJ2mdJJ3oBVFzewC6VOvIHhG7q9sxSfdIOruJpgA0r+uw2x6yfcJr9yV9QtK2phoD0Kw6p/GLJN1j+7XX+VZE/KCRrgA0ruuwR8Qzks5ssBcAPcTQG5AEYQeSIOxAEoQdSIKwA0k08UGYFF747Mc61t552dPFdZ8aW1SsHxifU6wvuaNcn7/rxY61w1ueLK6LPDiyA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASjLPP0J/88bc61j499PPyyqfX3PjKcnnnoZc61m557uM1N370+vHYaR1rQzf9UnHd2Zseabqd1nFkB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkHNG/SVpO9II4x+f2bXtN+sVnzulYe/5D5f8zT9pe3sc/f7+L9bkf+p9i/cYP3t2xdt5bXy6u+/2Xji/WPzm/82fl63o5DhTrm8eHivWVxx3setvv+f4Vxfp71z7c9Wu3aXNs0r7YO+UvFEd2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCz7PP0NB3Nxdq9V77xHqr62/esbJj7S9WLCtv+1/K33l/48r3dNHRzMx++XCxPrR1tFg/+YG7ivVfmdv5+/bn7yx/F/+xaNoju+31tsdsb5u0bIHt+23vqG5P6m2bAOqayWn8NySdf8SyayVtiogzJG2qHgMYYNOGPSIekLT3iMWrJG2o7m+QdHHDfQFoWLdv0C2KiFFJqm4Xdnqi7bW2R2yPHNR4l5sDUFfP342PiHURMRwRw3M0r9ebA9BBt2HfY3uxJFW3Y821BKAXug37fZLWVPfXSLq3mXYA9Mq04+y279DEN5efYnuXpC9IukHSt21fLulZSZf0skmUHfrvPR1rQ3d1rknSq9O89tB3X+iio2bs+f2PFesfmFv+9f3S3vd1rC37+2eK6x4qVo9O04Y9IlZ3KB2d30IBJMXlskAShB1IgrADSRB2IAnCDiTBR1zRmtmnLS3Wv3LdV4r1OZ5VrH/nlt/sWDt59KHiuscijuxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATj7GjNU3+0pFj/yLzyVNZPHChPR73gyZfedE/HMo7sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AE4+zoqfFPfqRj7dHP3DzN2uUZhP7g6quL9bf+64+nef1cOLIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKMs6Onnr2g8/HkeJfH0Vf/53nF+vwfPFasR7Gaz7RHdtvrbY/Z3jZp2fW2f2Z7S/VzYW/bBFDXTE7jvyHp/CmW3xwRZ1U/G5ttC0DTpg17RDwgaW8fegHQQ3XeoLvK9tbqNP+kTk+yvdb2iO2RgxqvsTkAdXQb9q9JOl3SWZJGJd3U6YkRsS4ihiNieM40H2wA0DtdhT0i9kTEqxFxWNLXJZ3dbFsAmtZV2G0vnvTwU5K2dXougMEw7Ti77TskrZR0iu1dkr4gaaXtszQxlLlT0hU97BED7C0nnFCsX/brD3as7Tv8SnHdsS++u1ifN/5wsY7XmzbsEbF6isW39qAXAD3E5bJAEoQdSIKwA0kQdiAJwg4kwUdcUcuO6z9QrH/vlL/tWFu149PFdedtZGitSRzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtlR9L+/+9Fifevv/HWx/pNDBzvWXvyrU4vrztNosY43hyM7kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBOHtys5f8crF+zef/oVif5/Kv0KWPXdax9vZ/5PPq/cSRHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSYJz9GOfZ5X/iM7+3q1i/5PgXivXb9y8s1hd9vvPx5HBxTTRt2iO77aW2f2R7u+0nbF9dLV9g+37bO6rbk3rfLoBuzeQ0/pCkz0XE+yV9VNKVtpdLulbSpog4Q9Km6jGAATVt2CNiNCIere7vl7Rd0hJJqyRtqJ62QdLFvWoSQH1v6g0628skfVjSZkmLImJUmvgPQdKUf7zZXmt7xPbIQY3X6xZA12YcdtvHS7pL0jURsW+m60XEuogYjojhOZrXTY8AGjCjsNueo4mg3x4Rd1eL99heXNUXSxrrTYsAmjDt0JttS7pV0vaI+PKk0n2S1ki6obq9tycdop4z31cs//nC22q9/Fe/eEmx/rbHHqr1+mjOTMbZV0i6TNLjtrdUy67TRMi/bftySc9KKv+rA2jVtGGPiAcluUP53GbbAdArXC4LJEHYgSQIO5AEYQeSIOxAEnzE9Rgwa/l7O9bW3lnv8ofl668s1pfd9m+1Xh/9w5EdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnP0Y8NQfdv5i34vmz/hLhaZ06j8fKD8hotbro384sgNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoyzHwVeuejsYn3TRTcVqvObbQZHLY7sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5DETOZnXyrpm5LeIemwpHURcYvt6yV9VtJz1VOvi4iNvWo0s90rZhXr75zd/Vj67fsXFutz9pU/z86n2Y8eM7mo5pCkz0XEo7ZPkPSI7fur2s0R8aXetQegKTOZn31U0mh1f7/t7ZKW9LoxAM16U3+z214m6cOSNleLrrK91fZ621N+N5LttbZHbI8c1HitZgF0b8Zht328pLskXRMR+yR9TdLpks7SxJF/ygu0I2JdRAxHxPAczWugZQDdmFHYbc/RRNBvj4i7JSki9kTEqxFxWNLXJZU/rQGgVdOG3bYl3Sppe0R8edLyxZOe9ilJ25pvD0BTZvJu/ApJl0l63PaWatl1klbbPksToy87JV3Rkw5Ry1++sLxYf+i3lhXrMfp4g92gTTN5N/5BSZ6ixJg6cBThCjogCcIOJEHYgSQIO5AEYQeSIOxAEo4+Trl7ohfEOT63b9sDstkcm7Qv9k41VM6RHciCsANJEHYgCcIOJEHYgSQIO5AEYQeS6Os4u+3nJP3XpEWnSHq+bw28OYPa26D2JdFbt5rs7bSIePtUhb6G/Q0bt0ciYri1BgoGtbdB7Uuit271qzdO44EkCDuQRNthX9fy9ksGtbdB7Uuit271pbdW/2YH0D9tH9kB9AlhB5JoJey2z7f977aftn1tGz10Ynun7cdtb7E90nIv622P2d42adkC2/fb3lHdTjnHXku9XW/7Z9W+22L7wpZ6W2r7R7a3237C9tXV8lb3XaGvvuy3vv/NbnuWpP+QdJ6kXZIelrQ6Ip7sayMd2N4paTgiWr8Aw/ZvSHpR0jcj4oPVshsl7Y2IG6r/KE+KiD8dkN6ul/Ri29N4V7MVLZ48zbikiyX9nlrcd4W+flt92G9tHNnPlvR0RDwTEQck3SlpVQt9DLyIeEDS3iMWr5K0obq/QRO/LH3XobeBEBGjEfFodX+/pNemGW913xX66os2wr5E0k8nPd6lwZrvPST90PYjtte23cwUFkXEqDTxyyNpYcv9HGnaabz76Yhpxgdm33Uz/XldbYR9qu/HGqTxvxUR8auSLpB0ZXW6ipmZ0TTe/TLFNOMDodvpz+tqI+y7JC2d9PhUSbtb6GNKEbG7uh2TdI8GbyrqPa/NoFvdjrXcz/8bpGm8p5pmXAOw79qc/ryNsD8s6Qzb77I9V9Klku5roY83sD1UvXEi20OSPqHBm4r6PklrqvtrJN3bYi+vMyjTeHeaZlwt77vWpz+PiL7/SLpQE+/I/0TSn7XRQ4e+3i3psernibZ7k3SHJk7rDmrijOhySSdL2iRpR3W7YIB6u03S45K2aiJYi1vq7dc08afhVklbqp8L2953hb76st+4XBZIgivogCQIO5AEYQeSIOxAEoQdSIKwA0kQdiCJ/wNGNvRIqiy+UgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: 0.0, 1: 0.0, 2: 0.001, 3: 0.008, 4: 0.0, 5: 0.0, 6: 0.0, 7: 0.992, 8: 0.0, 9: 0.0}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAANxUlEQVR4nO3de4xU93nG8ecBc7EwtqFgSjGygwOycSpDsiJx3YstN6nDH8GRckOJgyNHpGrcJhJSYrmV4igXWVVst1WjVCRGIZUvcn2JqWIlJsSR6wRhLy4BbJJAXOpgVmDEpuBWhd312z/2UG3wzpll5sycMe/3I41m5rxzznk18OyZmd+c+TkiBODsN6nuBgB0B2EHkiDsQBKEHUiCsANJnNPNnU31tJiuGd3cJZDK/+q/dTJOeLxaW2G3fYOkv5c0WdK3IuLOssdP1wy909e3s0sAJbbFloa1ll/G254s6euS3itpqaTVtpe2uj0AndXOe/YVkvZFxEsRcVLSg5JWVdMWgKq1E/YFkn495v6BYtlvsb3Wdr/t/iGdaGN3ANrRTtjH+xDgDd+9jYj1EdEXEX1TNK2N3QFoRzthPyBp4Zj7F0s62F47ADqlnbA/J2mx7bfYnirpI5I2VdMWgKq1PPQWEcO2b5X0A40OvW2IiBcq6wxApdoaZ4+IJyQ9UVEvADqIr8sCSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5BEV39KGq3Z/+WrS+sj0xtPzjn3yldL19161SMt9XTKZT/6RGl95rPnNqzN+4eftrVvnBmO7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBOPsPWDwe4tL67uX/WPH9j3UeIh+Qn5+3bdK6/f1zW9Ye2jzn5SuO7Jnb0s9YXwc2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZu6DZOPpPlj3YsX3/028Wldbv3vru0vqll5SfD//k0kdL6x+dOdCw9pWb55Suu+jzjLNXqa2w294v6bikEUnDEdFXRVMAqlfFkf26iDhSwXYAdBDv2YEk2g17SHrS9nbba8d7gO21tvtt9w/pRJu7A9Cqdl/GXxMRB21fJGmz7Z9HxNNjHxAR6yWtl6TzPbvN0y4AtKqtI3tEHCyuD0t6TNKKKpoCUL2Ww257hu2Zp25Leo+k3VU1BqBa7byMnyfpMduntnN/RHy/kq7eZIavf0dp/UdXfb3JFqaUVv9ucElp/akPl4x4Hjxcuu6Swf7S+qTp00vrX932+6X12+fsalgbnjVcui6q1XLYI+IlSVdV2AuADmLoDUiCsANJEHYgCcIOJEHYgSQ4xbUCry2YWlqf1ORvarOhtR+/r3x4a+SlX5TW27Hvi8tL6/fPvqvJFqY1rFz8fY413cSzDSRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM5egQu/s7W0/oH+j5XWPXistD48sP8MO6rOJ1f+sLR+3qTG4+joLRzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtm7YOTFX9bdQkP7v3J1af2WC7/WZAvlPzW9buBdDWszf7indN2RJnvGmeHIDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM5+lvvNTeXj6D/5ePk4+gWTysfRt56YXFrf8eXGvzt/7rFnS9dFtZoe2W1vsH3Y9u4xy2bb3mx7b3E9q7NtAmjXRF7Gf1vSDactu03SlohYLGlLcR9AD2sa9oh4WtLR0xavkrSxuL1R0o0V9wWgYq1+QDcvIgYkqbi+qNEDba+13W+7f0gnWtwdgHZ1/NP4iFgfEX0R0TelZJI/AJ3VatgP2Z4vScX14epaAtAJrYZ9k6Q1xe01kh6vph0AndJ0nN32A5KulTTH9gFJX5B0p6SHbN8i6WVJH+xkk2jdkbdHab3ZOHoza378ydL6ku8ylt4rmoY9IlY3KF1fcS8AOoivywJJEHYgCcIOJEHYgSQIO5AEp7ieBU5uvqRhbevldzVZu3zo7aqta0rrV6z7VWmdn4PuHRzZgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJxtnfBM5ZdGlp/Utv/ZeGtVlNTmHd3uSXwi75UvlI+cjgYPkG0DM4sgNJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoyzvwlc9tArpfXlU1v/m716y5+X1pf87LmWt43ewpEdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnL0HDK65urT+xXnNfvt9WsPKmv1/WrrmFZ/bV1rnd9/PHk2P7LY32D5se/eYZXfYfsX2juKysrNtAmjXRF7Gf1vSDeMsvycilhWXJ6ptC0DVmoY9Ip6WdLQLvQDooHY+oLvV9s7iZf6sRg+yvdZ2v+3+ITX5wTMAHdNq2L8h6TJJyyQNSGr4CVJErI+Ivojom1LyQRKAzmop7BFxKCJGIuJ1Sd+UtKLatgBUraWw254/5u77Je1u9FgAvaHpOLvtByRdK2mO7QOSviDpWtvLJIWk/ZI+1cEe3/TOWfB7pfU/+qttpfXzJrX+9mfri28trS8Z5Hz1LJqGPSJWj7P43g70AqCD+LoskARhB5Ig7EAShB1IgrADSXCKaxfsuX1haf27v/uvbW3/ul0fbFjjFFacwpEdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnL0Ltr/vniaPaO8XfC74i9cb1oYHB9vaNs4eHNmBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnG2c8CQ/MuaFibcnJBFzt5o5FXjzSsxYny6cA8rfz7B5PnzmmpJ0kamXthaX3vuqktb3siYsQNa5f/ZZPfIDh2rKV9cmQHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZz8LfO/hDXW30NAf/Pt4kwCPOnLo/NJ1Z809Xlrf9o77W+qp1y39m1tL64s+t7Wl7TY9stteaPsp23tsv2D7M8Xy2bY3295bXM9qqQMAXTGRl/HDktZFxBWS3iXp07aXSrpN0paIWCxpS3EfQI9qGvaIGIiI54vbxyXtkbRA0ipJG4uHbZR0Y6eaBNC+M/qAzvalkpZL2iZpXkQMSKN/ECRd1GCdtbb7bfcPqfy70AA6Z8Jht32epEckfTYiJvxN/IhYHxF9EdE3pc0fVgTQugmF3fYUjQb9voh4tFh8yPb8oj5f0uHOtAigCk2H3mxb0r2S9kTE3WNKmyStkXRncf14Rzo8C6x68aOl9S1ve7hLnXTfT5c/UNu+/ydONqwNReOf356IlTtvLq3/147WT79d8Mxwy+uWmcg4+zWSbpK0y/aOYtntGg35Q7ZvkfSypMaThAOoXdOwR8QzkhqdaX99te0A6BS+LgskQdiBJAg7kARhB5Ig7EASnOLaBef+2X+U1q/8avkpjdHBf6WZlx8trXfyNNIr/+0TpfV4eUZb21/08GuNi8/uamvbs7S3rXodOLIDSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKOiK7t7HzPjneaE+WATtkWW3Qsjo57lipHdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiiadhtL7T9lO09tl+w/Zli+R22X7G9o7is7Hy7AFo1kekHhiWti4jnbc+UtN325qJ2T0R8rXPtAajKROZnH5A0UNw+bnuPpAWdbgxAtc7oPbvtSyUtl7StWHSr7Z22N9ie1WCdtbb7bfcP6URbzQJo3YTDbvs8SY9I+mxEHJP0DUmXSVqm0SP/XeOtFxHrI6IvIvqmaFoFLQNoxYTCbnuKRoN+X0Q8KkkRcSgiRiLidUnflLSic20CaNdEPo23pHsl7YmIu8csnz/mYe+XtLv69gBUZSKfxl8j6SZJu2zvKJbdLmm17WWSQtJ+SZ/qSIcAKjGRT+OfkTTe71A/UX07ADqFb9ABSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeScER0b2f2q5L+c8yiOZKOdK2BM9OrvfVqXxK9tarK3i6JiLnjFboa9jfs3O6PiL7aGijRq731al8SvbWqW73xMh5IgrADSdQd9vU1779Mr/bWq31J9NaqrvRW63t2AN1T95EdQJcQdiCJWsJu+wbbv7C9z/ZtdfTQiO39tncV01D319zLBtuHbe8es2y27c229xbX486xV1NvPTGNd8k047U+d3VPf9719+y2J0v6paR3Szog6TlJqyPixa420oDt/ZL6IqL2L2DY/mNJr0n6TkS8rVj2t5KORsSdxR/KWRHx+R7p7Q5Jr9U9jXcxW9H8sdOMS7pR0s2q8bkr6etD6sLzVseRfYWkfRHxUkSclPSgpFU19NHzIuJpSUdPW7xK0sbi9kaN/mfpuga99YSIGIiI54vbxyWdmma81ueupK+uqCPsCyT9esz9A+qt+d5D0pO2t9teW3cz45gXEQPS6H8eSRfV3M/pmk7j3U2nTTPeM89dK9Oft6uOsI83lVQvjf9dExFvl/ReSZ8uXq5iYiY0jXe3jDPNeE9odfrzdtUR9gOSFo65f7GkgzX0Ma6IOFhcH5b0mHpvKupDp2bQLa4P19zP/+ulabzHm2ZcPfDc1Tn9eR1hf07SYttvsT1V0kckbaqhjzewPaP44ES2Z0h6j3pvKupNktYUt9dIerzGXn5Lr0zj3WiacdX83NU+/XlEdP0iaaVGP5H/laS/rqOHBn0tkvSz4vJC3b1JekCjL+uGNPqK6BZJvyNpi6S9xfXsHurtnyXtkrRTo8GaX1Nvf6jRt4Y7Je0oLivrfu5K+urK88bXZYEk+AYdkARhB5Ig7EAShB1IgrADSRB2IAnCDiTxfy43Cn7d/BIFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: 0.0, 1: 0.0, 2: 0.999, 3: 0.001, 4: 0.0, 5: 0.0, 6: 0.0, 7: 0.0, 8: 0.0, 9: 0.0}\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAMR0lEQVR4nO3dXawcdRnH8d/P0hYtEFugpJQGQRtiAa16KCQ1CiIEiFq8wNALUhNiSRSjCRcQvZBgQohRiPEFU2xDfYOogG0ivjSNphoNcoq1tBQskgqHNj2SqlSE0pfHizOYQzk7e7ozs7Pt8/0km92dZ2fnyfb8OrPzn3P+jggBOPa9qe0GAPQHYQeSIOxAEoQdSIKwA0kc18+NTfP0OF4z+rlJIJVX9JJejX2eqFYp7LavkPR1SVMkfTci7ih7/fGaoQt9aZVNAijxSKzvWOv5MN72FEnfknSlpAWSltpe0Ov7AWhWle/siyQ9HRHPRMSrku6XtKSetgDUrUrY50p6btzzkWLZ69hebnvY9vB+7auwOQBVVAn7RCcB3nDtbUSsiIihiBiaqukVNgegiiphH5E0b9zzMyTtrNYOgKZUCfujkubbPsv2NEnXSlpbT1sA6tbz0FtEHLB9o6RfaWzobVVEbK2tMwC1qjTOHhEPS3q4pl4ANIjLZYEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1Ioq9TNiMfv+/cjrWfr/1+6brnf+fG0vq8L/+hp56yYs8OJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kwzo5GjV5wUsfaAR0sXfctO6PudlKrFHbbOyTtlXRQ0oGIGKqjKQD1q2PPfklEvFDD+wBoEN/ZgSSqhj0k/dr2RtvLJ3qB7eW2h20P79e+ipsD0Kuqh/GLI2Kn7dmS1tl+MiI2jH9BRKyQtEKSTvIszrgALam0Z4+IncX9qKSHJC2qoykA9es57LZn2D7xtceSLpe0pa7GANSrymH8aZIesv3a+/woIn5ZS1c4ZvzzXZ3H0kcOlJ/DOXnlH+tuJ7Wewx4Rz0h6d429AGgQQ29AEoQdSIKwA0kQdiAJwg4kwa+4opJYvLC0/ruP3Nmx9sENny1d9x36c089YWLs2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZUcmeBW8urc+Z8paOtbk/nVp3OyjBnh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCcHZVc+unyP/f8s5fe2rF2wm+fKl23fEJnHCn27EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBOPsKDXl3HNK67fPvq+0vvLFMzrWDv7r3z31hN503bPbXmV71PaWcctm2V5ne3txP7PZNgFUNZnD+HslXXHYslskrY+I+ZLWF88BDLCuYY+IDZL2HLZ4iaTVxePVkq6uuS8ANev1BN1pEbFLkor72Z1eaHu57WHbw/u1r8fNAaiq8bPxEbEiIoYiYmiqpje9OQAd9Br23bbnSFJxP1pfSwCa0GvY10paVjxeJmlNPe0AaErXcXbb90m6WNIptkckfUnSHZJ+bPt6Sc9KuqbJJtGe5y87udL6G/eeWVJ9udJ748h0DXtELO1QurTmXgA0iMtlgSQIO5AEYQeSIOxAEoQdSIJfcUWpFxfsr7T+pm8u7Fh7q8r/DDXqxZ4dSIKwA0kQdiAJwg4kQdiBJAg7kARhB5JgnD25fVdeUFpfc/k3Suu3vfC+0vqsBzZ3rB0qXRN1Y88OJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kwzp7cyIfKfwTeNe340vqyHeeX1me/9OQR94RmsGcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZ0/u1PNGS+sHo/y3zo9bM7POdtCgrnt226tsj9reMm7Zrbaft72puF3VbJsAqprMYfy9kq6YYPldEbGwuD1cb1sA6tY17BGxQdKePvQCoEFVTtDdaHtzcZjf8Yub7eW2h20P79e+CpsDUEWvYb9b0tslLZS0S9LXOr0wIlZExFBEDE3V9B43B6CqnsIeEbsj4mBEHJJ0j6RF9bYFoG49hd32nHFPPy5pS6fXAhgMXcfZbd8n6WJJp9gekfQlSRfbXigpJO2QdEODPaKC4846s7T+1XN+Ulq/59/zSuuzVjHH+tGia9gjYukEi1c20AuABnG5LJAEYQeSIOxAEoQdSIKwA0nwK67HuO03nF5av6jLRY2feuyS0vo8LrE4arBnB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkGGc/xh2a90ql9V/+V/mUzTh6sGcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZz/GffvCH1Raf+4vptTUCdrGnh1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkmCc/RjwykcXday9//g/dVmbH4Esuu7Zbc+z/Rvb22xvtf25Yvks2+tsby/uZzbfLoBeTeYw/oCkmyLinZIukvQZ2wsk3SJpfUTMl7S+eA5gQHUNe0TsiojHisd7JW2TNFfSEkmri5etlnR1U00CqO6ITtDZfpuk90h6RNJpEbFLGvsPQdLsDusstz1se3i/9lXrFkDPJh122ydIekDS5yPixcmuFxErImIoIoamqsssggAaM6mw256qsaD/MCIeLBbvtj2nqM+RNNpMiwDq0HXcxbYlrZS0LSLuHFdaK2mZpDuK+zWNdIiunv1YdKxNd/k/8W0vnF9aP2HNxtJ65y1j0ExmkHWxpOskPW57U7HsCxoL+Y9tXy/pWUnXNNMigDp0DXtE/F6SO5QvrbcdAE3hclkgCcIOJEHYgSQIO5AEYQeS4PcbjwJTTjqptH7z4od7fu8f/eIDpfWzD/yx5/fGYGHPDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJMM5+FDi0r/zPeT3x39M71j78/FDpuvNv31paP1haxdGEPTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJME4+1EguoyzP1UylD5Nfy9dl3H0PNizA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EASXcNue57t39jeZnur7c8Vy2+1/bztTcXtqubbBdCryVxUc0DSTRHxmO0TJW20va6o3RURX22uPQB1mcz87Lsk7Soe77W9TdLcphsDUK8j+s5u+22S3iPpkWLRjbY3215le2aHdZbbHrY9vF/ll30CaM6kw277BEkPSPp8RLwo6W5Jb5e0UGN7/q9NtF5ErIiIoYgYmqrpNbQMoBeTCrvtqRoL+g8j4kFJiojdEXEwIg5JukfSoubaBFDVZM7GW9JKSdsi4s5xy+eMe9nHJW2pvz0AdZnM2fjFkq6T9LjtTcWyL0haanuhpJC0Q9INjXQIoBaTORv/e0meoNT7pOAA+o4r6IAkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0k4Ivq3Mfsf0uvmED5F0gt9a+DIDGpvg9qXRG+9qrO3MyPi1IkKfQ37GzZuD0dEyezi7RnU3ga1L4neetWv3jiMB5Ig7EASbYd9RcvbLzOovQ1qXxK99aovvbX6nR1A/7S9ZwfQJ4QdSKKVsNu+wvZTtp+2fUsbPXRie4ftx4tpqIdb7mWV7VHbW8Ytm2V7ne3txf2Ec+y11NtATONdMs14q59d29Of9/07u+0pkv4q6TJJI5IelbQ0Ip7oayMd2N4haSgiWr8Aw/YHJP1H0vci4rxi2Vck7YmIO4r/KGdGxM0D0tutkv7T9jTexWxFc8ZPMy7pakmfVIufXUlfn1AfPrc29uyLJD0dEc9ExKuS7pe0pIU+Bl5EbJC057DFSyStLh6v1tgPS9916G0gRMSuiHiseLxX0mvTjLf62ZX01RdthH2upOfGPR/RYM33HpJ+bXuj7eVtNzOB0yJilzT2wyNpdsv9HK7rNN79dNg04wPz2fUy/XlVbYR9oqmkBmn8b3FEvFfSlZI+UxyuYnImNY13v0wwzfhA6HX686raCPuIpHnjnp8haWcLfUwoInYW96OSHtLgTUW9+7UZdIv70Zb7+b9BmsZ7omnGNQCfXZvTn7cR9kclzbd9lu1pkq6VtLaFPt7A9ozixIlsz5B0uQZvKuq1kpYVj5dJWtNiL68zKNN4d5pmXC1/dq1Pfx4Rfb9JukpjZ+T/JumLbfTQoa+zJf2luG1tuzdJ92nssG6/xo6Irpd0sqT1krYX97MGqLfvS3pc0maNBWtOS729X2NfDTdL2lTcrmr7syvpqy+fG5fLAklwBR2QBGEHkiDsQBKEHUiCsANJEHYgCcIOJPE/O0elww7G9HEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{0: 0.0, 1: 0.996, 2: 0.001, 3: 0.0, 4: 0.0, 5: 0.0, 6: 0.0, 7: 0.001, 8: 0.001, 9: 0.0}\n"
     ]
    }
   ],
   "source": [
    "for img in x_test[0:3]: \n",
    "    plt.imshow(img)\n",
    "    predicted = client.predict(\n",
    "        data_input={'input_layer': np.expand_dims(img.reshape(28, 28), axis=0)}\n",
    "    )\n",
    "    plt.show()\n",
    "    print(dict(zip(np.arange(10), predicted[0].round(3))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch prediction\n",
    "\n",
    "To predict in batches, the `data_input` dictionary has to be specified differently for client images in NDARRAY format. To input a batch of observations, ensure that your inputs are in the NCHW format: number of observations, channels (optional if single channel), height and width. An example is as follows: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "predicted = client.predict(\n",
    "    data_input={'input_layer': x_test[0:3].reshape(3, 28, 28)}\n",
    ")\n",
    "\n",
    "server.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We compare the predicted probabilities and the corresponding labels: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.008</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.992</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.999</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.996</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.001</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     0      1      2      3    4    5    6      7      8    9\n",
       "0  0.0  0.000  0.001  0.008  0.0  0.0  0.0  0.992  0.000  0.0\n",
       "1  0.0  0.000  0.999  0.001  0.0  0.0  0.0  0.000  0.000  0.0\n",
       "2  0.0  0.996  0.001  0.000  0.0  0.0  0.0  0.001  0.001  0.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(predicted).round(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7, 2, 1], dtype=uint8)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test[0:3]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}