TGNet-training.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Seoul City taxi request forecast\n",
    "\n",
    "------------\n",
    "\n",
    "<b>[ Objective ]</b>\n",
    "\n",
    "- We want predict the future taxi-demand within 30-min of all locations.\n",
    "\n",
    "![motivation](./output_file/graph/goal_setting.png)\n",
    "\n",
    "<br>\n",
    "\n",
    "\n",
    "<b>[ Summary ]</b>\n",
    "\n",
    "- TGNet(temporal guided network) applied Seoul City & NYC taxi datasets.\n",
    "- It outperforms previous models. \n",
    "- Research Paper - accepted NIPS 2018 workshop [(link)](https://openreview.net/pdf?id=BygF00DuiX)\n",
    "\n",
    "<br>\n",
    "\n",
    "<b>[ Intro ]</b>\n",
    "\n",
    "- On-demand taxi request service is very convenient in city life.\n",
    "- Taxi request data shows variety pattern depends on the time and location\n",
    "\n",
    "<br>\n",
    "\n",
    "![SEO_data_stats](./output_file/graph/seo_data_stats.png)\n",
    "```\n",
    "Seoul Ride Hailing Requests:\n",
    "(left) the ride hailing requests in residential and commercial area,\n",
    "(middle) different patterns in residential area on holiday and\n",
    "(right) average requests in each day of week\n",
    "```\n",
    "\n",
    "<br>\n",
    "\n",
    "<b>[ Data Processing ]</b>\n",
    "\n",
    "- To capture the Spatial / Temporal information in the same time\n",
    "- We use Unet based the fully convolutional network\n",
    "- To do this, we split the Seoul city to 50x50 grid (approx. 800m x 700m ) and the data aggregated in 30 mins.\n",
    "- We used the Requests data on the spot and drop-off data on the spot.\n",
    "- Two data type have different patterns, below shows the results.\n",
    "\n",
    "\n",
    "\n",
    "<br>\n",
    "<b>[ Results ]</b>\n",
    "\n",
    "- We applied the TGNet model to Seoul City datasets.\n",
    "\n",
    "\n",
    "| Method | MAPE | RMSE |\n",
    "|:--------|:--------:|:--------:|\n",
    "| ARIMA | 23.05% | 12.53 |\n",
    "| XGBoost | 19.68% | 8.08 |\n",
    "| Baseline | 18.87% | 7.06 |\n",
    "| Baseline + Drop-off | 18.62% | 6.73 |\n",
    "| Baseline + Temporal Information | 17.97% | 6.68 |\n",
    "| <b>TGNet</b> | <b>17.72</b> | <b>6.32</b> |\n",
    "\n",
    "\n",
    "\n",
    "<br><br><br>\n",
    "<b>[ Contribution ]</b>\n",
    "\n",
    "- Our contribution is introduced temporal data directly to model output, then increase the performance.\n",
    "- Another one is using drop-off data. In some locations, drop-off data may represent the future demand.\n",
    "- We located those spot that held festival, concert and events irregularly.\n",
    "\n",
    "![Event_spots](./output_file/graph/event_graph.png)\n",
    "\n",
    "```\n",
    "- We select 4 Area which has stadium or complex hall\n",
    "- That place held the event irregularly such as Concert, Festival, Convention\n",
    "  - (1) Gocheok Skydome [37.498555, 126.867300]\n",
    "  - (2) Jangchung Sports Complex [37.558160, 127.006782]\n",
    "  - (3) Seoul worldcup stadium - (Sangam) [37.568130, 126.897210]\n",
    "  - (4) Seoul olympic stadium - (Jamsil) [37.515686, 127.072793]\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (1) Data Preparation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "pd.options.display.max_columns = 100\n",
    "\n",
    "from data_utils import*\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Original Taxi Data\n",
    " - more than 1M data daily "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(864444, 47)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>passenger_id</th>\n",
       "      <th>driver_id</th>\n",
       "      <th>taxi_id</th>\n",
       "      <th>taxi_kind</th>\n",
       "      <th>parent_call_id</th>\n",
       "      <th>status</th>\n",
       "      <th>origin_name</th>\n",
       "      <th>origin_type</th>\n",
       "      <th>origin_name1</th>\n",
       "      <th>origin_name2</th>\n",
       "      <th>origin_name3</th>\n",
       "      <th>destination_name</th>\n",
       "      <th>destination_type</th>\n",
       "      <th>destination_name1</th>\n",
       "      <th>destination_name2</th>\n",
       "      <th>destination_name3</th>\n",
       "      <th>passenger_name1</th>\n",
       "      <th>passenger_name2</th>\n",
       "      <th>passenger_name3</th>\n",
       "      <th>return_taxi</th>\n",
       "      <th>retry_count</th>\n",
       "      <th>referrer</th>\n",
       "      <th>dispatch_status</th>\n",
       "      <th>distance_to_start</th>\n",
       "      <th>seconds_to_start</th>\n",
       "      <th>distance_to_end</th>\n",
       "      <th>seconds_to_end</th>\n",
       "      <th>business_type</th>\n",
       "      <th>created_at</th>\n",
       "      <th>dispatched_at</th>\n",
       "      <th>pickup_at</th>\n",
       "      <th>dropoff_at</th>\n",
       "      <th>origin_x</th>\n",
       "      <th>origin_y</th>\n",
       "      <th>destination_x</th>\n",
       "      <th>destination_y</th>\n",
       "      <th>passenger_x</th>\n",
       "      <th>passenger_y</th>\n",
       "      <th>taxi_x</th>\n",
       "      <th>taxi_y</th>\n",
       "      <th>pickup_x</th>\n",
       "      <th>pickup_y</th>\n",
       "      <th>dropoff_x</th>\n",
       "      <th>dropoff_y</th>\n",
       "      <th>origin_confirmid</th>\n",
       "      <th>destination_confirmid</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>+2Z74TjpX66vzTTqldT17Q==</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>medium</td>\n",
       "      <td>\\N</td>\n",
       "      <td>70</td>\n",
       "      <td>강남역 2호선 11번출구</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>강남구</td>\n",
       "      <td>역삼1동</td>\n",
       "      <td>가락고등학교</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>송파구</td>\n",
       "      <td>송파2동</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>\\N</td>\n",
       "      <td>30</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>10281</td>\n",
       "      <td>1159</td>\n",
       "      <td>\\N</td>\n",
       "      <td>20170223005941</td>\n",
       "      <td>20170223010039</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>127.027466881,202428.811013</td>\n",
       "      <td>37.4987688776,444367.83926199999</td>\n",
       "      <td>127.11741790799999,210382.5</td>\n",
       "      <td>37.501679474299998,444697.00000100001</td>\n",
       "      <td>0,0</td>\n",
       "      <td>0,0</td>\n",
       "      <td>0,0</td>\n",
       "      <td>0,0</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>rgaSUQKMizfM2kLXM8UH9g==</td>\n",
       "      <td>176700</td>\n",
       "      <td>270837</td>\n",
       "      <td>medium</td>\n",
       "      <td>\\N</td>\n",
       "      <td>50</td>\n",
       "      <td>청원경찰서</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>충청북도</td>\n",
       "      <td>청주시 청원구</td>\n",
       "      <td>우암동</td>\n",
       "      <td>MBC 충북</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>충청북도</td>\n",
       "      <td>청주시 흥덕구</td>\n",
       "      <td>가경동</td>\n",
       "      <td>충청북도</td>\n",
       "      <td>청주시 청원구</td>\n",
       "      <td>우암동</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>\\N</td>\n",
       "      <td>10</td>\n",
       "      <td>267</td>\n",
       "      <td>77</td>\n",
       "      <td>6886</td>\n",
       "      <td>911</td>\n",
       "      <td>1</td>\n",
       "      <td>20170223054502</td>\n",
       "      <td>20170223054514</td>\n",
       "      <td>20170223054648</td>\n",
       "      <td>20170223060234</td>\n",
       "      <td>127.485305129,243394.75</td>\n",
       "      <td>36.650728711699998,350362.50000200002</td>\n",
       "      <td>127.43486283999999,238900.275077</td>\n",
       "      <td>36.6189478943,346814.106616</td>\n",
       "      <td>127.4856856,243428.19744799999</td>\n",
       "      <td>36.651750100000001,350476.01787099999</td>\n",
       "      <td>127.4877274,243611.08940600001</td>\n",
       "      <td>36.651181700000002,350413.86703999998</td>\n",
       "      <td>127.48529859999999,243394.262001</td>\n",
       "      <td>36.650557999999997,350343.55278600001</td>\n",
       "      <td>127.4342894,238849.05478400001</td>\n",
       "      <td>36.618795300000002,346796.94092600001</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>kdR5jWv5k9B0TeKh8pXqCQ==</td>\n",
       "      <td>84473</td>\n",
       "      <td>370325</td>\n",
       "      <td>medium</td>\n",
       "      <td>\\N</td>\n",
       "      <td>50</td>\n",
       "      <td>GS25 화곡봉제산점</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>강서구</td>\n",
       "      <td>화곡8동</td>\n",
       "      <td>현대백화점 목동점</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>양천구</td>\n",
       "      <td>목1동</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>강서구</td>\n",
       "      <td>화곡본동</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>\\N</td>\n",
       "      <td>10</td>\n",
       "      <td>481</td>\n",
       "      <td>166</td>\n",
       "      <td>3949</td>\n",
       "      <td>1077</td>\n",
       "      <td>0</td>\n",
       "      <td>20170223092528</td>\n",
       "      <td>20170223092535</td>\n",
       "      <td>20170223092843</td>\n",
       "      <td>20170223093959</td>\n",
       "      <td>126.850067139,186748.70643399999</td>\n",
       "      <td>37.537258148200003,448649.87177000003</td>\n",
       "      <td>126.875354544,188982</td>\n",
       "      <td>37.526063358899997,447404.12500100001</td>\n",
       "      <td>126.84861650000001,186620.879705</td>\n",
       "      <td>37.539401699999999,448887.98540000001</td>\n",
       "      <td>126.8462892,186414.436013</td>\n",
       "      <td>37.535217099999997,448423.879342</td>\n",
       "      <td>126.8502605,186765.80568699999</td>\n",
       "      <td>37.537312900000003,448655.921325</td>\n",
       "      <td>126.8740275,188864.84660799999</td>\n",
       "      <td>37.527074800000001,447516.53869800002</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>96k73WFpaDf6lpy5r/twXQ==</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>medium</td>\n",
       "      <td>\\N</td>\n",
       "      <td>60</td>\n",
       "      <td>401레스토랑</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>마포구</td>\n",
       "      <td>서교동</td>\n",
       "      <td>발산역 5호선 1번출구</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>강서구</td>\n",
       "      <td>가양1동</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>마포구</td>\n",
       "      <td>서교동</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>20170223003223</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>126.920590159,192982.89906200001</td>\n",
       "      <td>37.550697047299998,450133.826397</td>\n",
       "      <td>126.83753244,185645.02771900001</td>\n",
       "      <td>37.558935652700001,451057.65605200001</td>\n",
       "      <td>126.920252033,192953.025226</td>\n",
       "      <td>37.550748931299999,450139.61018999998</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>YAv/3FRD4rxVgzaNOfcsUg==</td>\n",
       "      <td>237302</td>\n",
       "      <td>575378</td>\n",
       "      <td>medium</td>\n",
       "      <td>\\N</td>\n",
       "      <td>50</td>\n",
       "      <td>공덕삼성래미안2차아파트 108동</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>마포구</td>\n",
       "      <td>공덕동</td>\n",
       "      <td>KBS연구관앞주차장</td>\n",
       "      <td>PLACE</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>영등포구</td>\n",
       "      <td>여의동</td>\n",
       "      <td>서울특별시</td>\n",
       "      <td>마포구</td>\n",
       "      <td>공덕동</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>\\N</td>\n",
       "      <td>10</td>\n",
       "      <td>282</td>\n",
       "      <td>59</td>\n",
       "      <td>6170</td>\n",
       "      <td>859</td>\n",
       "      <td>1</td>\n",
       "      <td>20170223095554</td>\n",
       "      <td>20170223095559</td>\n",
       "      <td>20170223100045</td>\n",
       "      <td>20170223101322</td>\n",
       "      <td>126.95990770900001,196457.22771800001</td>\n",
       "      <td>37.550870074000002,450150.82206899999</td>\n",
       "      <td>126.91543579099999,192525.05688300001</td>\n",
       "      <td>37.526920318599998,447495.29682300001</td>\n",
       "      <td>126.9592286,196397.244125</td>\n",
       "      <td>37.551413699999998,450211.18384999997</td>\n",
       "      <td>126.9626564,196700.14724200001</td>\n",
       "      <td>37.551556699999999,450226.92929200002</td>\n",
       "      <td>126.95952560000001,196423.45127600001</td>\n",
       "      <td>37.550635399999997,450124.79054399999</td>\n",
       "      <td>126.9153972,192521.637812</td>\n",
       "      <td>37.526841500000003,447486.552004</td>\n",
       "      <td>\\N</td>\n",
       "      <td>\\N</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0              passenger_id driver_id taxi_id taxi_kind  \\\n",
       "0           0  +2Z74TjpX66vzTTqldT17Q==        \\N      \\N    medium   \n",
       "1           1  rgaSUQKMizfM2kLXM8UH9g==    176700  270837    medium   \n",
       "2           2  kdR5jWv5k9B0TeKh8pXqCQ==     84473  370325    medium   \n",
       "3           3  96k73WFpaDf6lpy5r/twXQ==        \\N      \\N    medium   \n",
       "4           4  YAv/3FRD4rxVgzaNOfcsUg==    237302  575378    medium   \n",
       "\n",
       "  parent_call_id  status        origin_name origin_type origin_name1  \\\n",
       "0             \\N      70      강남역 2호선 11번출구       PLACE        서울특별시   \n",
       "1             \\N      50              청원경찰서       PLACE         충청북도   \n",
       "2             \\N      50        GS25 화곡봉제산점       PLACE        서울특별시   \n",
       "3             \\N      60            401레스토랑       PLACE        서울특별시   \n",
       "4             \\N      50  공덕삼성래미안2차아파트 108동       PLACE        서울특별시   \n",
       "\n",
       "  origin_name2 origin_name3 destination_name destination_type  \\\n",
       "0          강남구         역삼1동           가락고등학교            PLACE   \n",
       "1      청주시 청원구          우암동           MBC 충북            PLACE   \n",
       "2          강서구         화곡8동        현대백화점 목동점            PLACE   \n",
       "3          마포구          서교동     발산역 5호선 1번출구            PLACE   \n",
       "4          마포구          공덕동       KBS연구관앞주차장            PLACE   \n",
       "\n",
       "  destination_name1 destination_name2 destination_name3 passenger_name1  \\\n",
       "0             서울특별시               송파구              송파2동              \\N   \n",
       "1              충청북도           청주시 흥덕구               가경동            충청북도   \n",
       "2             서울특별시               양천구               목1동           서울특별시   \n",
       "3             서울특별시               강서구              가양1동           서울특별시   \n",
       "4             서울특별시              영등포구               여의동           서울특별시   \n",
       "\n",
       "  passenger_name2 passenger_name3  return_taxi  retry_count referrer  \\\n",
       "0              \\N              \\N            0            1       \\N   \n",
       "1         청주시 청원구             우암동            0            0       \\N   \n",
       "2             강서구            화곡본동            0            1       \\N   \n",
       "3             마포구             서교동            0            0       \\N   \n",
       "4             마포구             공덕동            0            0       \\N   \n",
       "\n",
       "  dispatch_status distance_to_start seconds_to_start distance_to_end  \\\n",
       "0              30                \\N               \\N           10281   \n",
       "1              10               267               77            6886   \n",
       "2              10               481              166            3949   \n",
       "3              \\N                \\N               \\N              \\N   \n",
       "4              10               282               59            6170   \n",
       "\n",
       "  seconds_to_end business_type      created_at   dispatched_at  \\\n",
       "0           1159            \\N  20170223005941  20170223010039   \n",
       "1            911             1  20170223054502  20170223054514   \n",
       "2           1077             0  20170223092528  20170223092535   \n",
       "3             \\N            \\N  20170223003223              \\N   \n",
       "4            859             1  20170223095554  20170223095559   \n",
       "\n",
       "        pickup_at      dropoff_at                               origin_x  \\\n",
       "0              \\N              \\N            127.027466881,202428.811013   \n",
       "1  20170223054648  20170223060234                127.485305129,243394.75   \n",
       "2  20170223092843  20170223093959       126.850067139,186748.70643399999   \n",
       "3              \\N              \\N       126.920590159,192982.89906200001   \n",
       "4  20170223100045  20170223101322  126.95990770900001,196457.22771800001   \n",
       "\n",
       "                                origin_y  \\\n",
       "0       37.4987688776,444367.83926199999   \n",
       "1  36.650728711699998,350362.50000200002   \n",
       "2  37.537258148200003,448649.87177000003   \n",
       "3       37.550697047299998,450133.826397   \n",
       "4  37.550870074000002,450150.82206899999   \n",
       "\n",
       "                           destination_x  \\\n",
       "0            127.11741790799999,210382.5   \n",
       "1       127.43486283999999,238900.275077   \n",
       "2                   126.875354544,188982   \n",
       "3        126.83753244,185645.02771900001   \n",
       "4  126.91543579099999,192525.05688300001   \n",
       "\n",
       "                           destination_y                       passenger_x  \\\n",
       "0  37.501679474299998,444697.00000100001                               0,0   \n",
       "1            36.6189478943,346814.106616    127.4856856,243428.19744799999   \n",
       "2  37.526063358899997,447404.12500100001  126.84861650000001,186620.879705   \n",
       "3  37.558935652700001,451057.65605200001       126.920252033,192953.025226   \n",
       "4  37.526920318599998,447495.29682300001         126.9592286,196397.244125   \n",
       "\n",
       "                             passenger_y                          taxi_x  \\\n",
       "0                                    0,0                             0,0   \n",
       "1  36.651750100000001,350476.01787099999  127.4877274,243611.08940600001   \n",
       "2  37.539401699999999,448887.98540000001       126.8462892,186414.436013   \n",
       "3  37.550748931299999,450139.61018999998                              \\N   \n",
       "4  37.551413699999998,450211.18384999997  126.9626564,196700.14724200001   \n",
       "\n",
       "                                  taxi_y  \\\n",
       "0                                    0,0   \n",
       "1  36.651181700000002,350413.86703999998   \n",
       "2       37.535217099999997,448423.879342   \n",
       "3                                     \\N   \n",
       "4  37.551556699999999,450226.92929200002   \n",
       "\n",
       "                                pickup_x  \\\n",
       "0                                     \\N   \n",
       "1       127.48529859999999,243394.262001   \n",
       "2         126.8502605,186765.80568699999   \n",
       "3                                     \\N   \n",
       "4  126.95952560000001,196423.45127600001   \n",
       "\n",
       "                                pickup_y                       dropoff_x  \\\n",
       "0                                     \\N                              \\N   \n",
       "1  36.650557999999997,350343.55278600001  127.4342894,238849.05478400001   \n",
       "2       37.537312900000003,448655.921325  126.8740275,188864.84660799999   \n",
       "3                                     \\N                              \\N   \n",
       "4  37.550635399999997,450124.79054399999       126.9153972,192521.637812   \n",
       "\n",
       "                               dropoff_y origin_confirmid  \\\n",
       "0                                     \\N               \\N   \n",
       "1  36.618795300000002,346796.94092600001               \\N   \n",
       "2  37.527074800000001,447516.53869800002               \\N   \n",
       "3                                     \\N               \\N   \n",
       "4       37.526841500000003,447486.552004               \\N   \n",
       "\n",
       "  destination_confirmid  \n",
       "0                    \\N  \n",
       "1                    \\N  \n",
       "2                    \\N  \n",
       "3                    \\N  \n",
       "4                    \\N  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "df_csv = pd.read_csv('/data/project/rw/kakao_dataset_csv/20170223.csv')\n",
    "print (df_csv.shape)\n",
    "df_csv.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data preperation before making numpy "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>year</th>\n",
       "      <th>month</th>\n",
       "      <th>day</th>\n",
       "      <th>hour</th>\n",
       "      <th>minute</th>\n",
       "      <th>start_x</th>\n",
       "      <th>start_y</th>\n",
       "      <th>end_x</th>\n",
       "      <th>end_y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2017</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>126.901143</td>\n",
       "      <td>37.574250</td>\n",
       "      <td>126.904912</td>\n",
       "      <td>37.562310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2017</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>126.917085</td>\n",
       "      <td>37.481712</td>\n",
       "      <td>126.899039</td>\n",
       "      <td>37.485608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2017</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>127.005671</td>\n",
       "      <td>37.599246</td>\n",
       "      <td>124.848488</td>\n",
       "      <td>33.474969</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>2017</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>33.0</td>\n",
       "      <td>127.038195</td>\n",
       "      <td>37.524708</td>\n",
       "      <td>127.023412</td>\n",
       "      <td>37.567072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>2017</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>127.025147</td>\n",
       "      <td>37.604080</td>\n",
       "      <td>127.073415</td>\n",
       "      <td>37.660624</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  year  month  day  hour  minute     start_x    start_y  \\\n",
       "0           0  2017    1.0  1.0  21.0    41.0  126.901143  37.574250   \n",
       "1           1  2017    1.0  1.0  21.0    12.0  126.917085  37.481712   \n",
       "2           2  2017    1.0  1.0  18.0    19.0  127.005671  37.599246   \n",
       "3           3  2017    1.0  1.0   7.0    33.0  127.038195  37.524708   \n",
       "4           4  2017    1.0  1.0   7.0    10.0  127.025147  37.604080   \n",
       "\n",
       "        end_x      end_y  \n",
       "0  126.904912  37.562310  \n",
       "1  126.899039  37.485608  \n",
       "2  124.848488  33.474969  \n",
       "3  127.023412  37.567072  \n",
       "4  127.073415  37.660624  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_csv = pd.read_csv('./data/data_2017_selected.csv')\n",
    "data_csv.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making Numpy data frome the table \n",
    "- select the data within boundary \n",
    "\n",
    "\n",
    "#### Seoul City boundary \n",
    "```\n",
    "bound_left = 126.764467 \n",
    "bound_right = 127.183756\n",
    "bound_top = 37.701454\n",
    "bound_bottom = 37.428415\n",
    "```\n",
    "\n",
    "#### Data aggregation \n",
    "```\n",
    "50 x 50 grid within in 30min \n",
    "```\n",
    "\n",
    "#### Start & End (destination) data size \n",
    "```\n",
    "( total days to train * 24 * ( 60 / 30 ) )  x 50 x 50\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./data/np_start_data.npz Loaded\n",
      "./data/np_end_data.npz Loaded\n",
      "4491.0 6128.0\n",
      "(8688, 50, 50) (8688, 50, 50)\n"
     ]
    }
   ],
   "source": [
    "st_data = load_np_data('./data/np_start_data.npz')\n",
    "ed_data = load_np_data('./data/np_end_data.npz')\n",
    "print (st_data.max(), ed_data.max())\n",
    "print (st_data.shape, ed_data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data visualization "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'End : 08h')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAykAAAMoCAYAAADLNq/PAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzs3Xl81eWZ///rMiwBAgQIixDWYYksokNQaMGWGmqjdaAyDEP5VerYHzPW9qdtZ6Yz+rCt7bczXcaqte20tg5Fy5dSagV1ZFQso6aAslhEkX0H2WQJYU+4f3/kMJPCuS6ST06Sm+T1fDx8SD7v3Ofc+Zzlzp2TvI+GEAQAAAAAYnFFQ08AAAAAAKpikwIAAAAgKmxSAAAAAESFTQoAAACAqLBJAQAAABAVNikAAAAAosImBWhgqvpLVf0/DT0PAEBcVPUbqvqrBrjez6pqSX1fL1AVmxRET1XHqOoSVT2qqodU9Q+qOjKV1fqJVFX7qGpQ1Wa1uAxV1e+q6gep/76rqlolv0ZVV6rqidT/r6nNnAEAcVLVbap6UlXLqvz3owaay6dVdbuqHlfV+arasUrWUVWfSWXbVfXTDTFHwMImBVFT1XYi8ryIPCYiHUWkh4g8KCKnM3T5iTcmF5ghIhNFZLiIXC0it4rI36auo4WILBCRX4lIBxGZJSILUscBAI3PrSGEnCr/faG+J6CqQ0TkZyLyGRHpKiInROQnVT7lxyJyJpVNE5F/T40BosAmBbEbKCISQpgTQqgIIZwMIbwUQnhbVa8SkZ+KyOjUT6qOiIio6i2q+paqlqrqTlX9xvkLq/KqyZ2qukNEfi8ir6XiI6nLGZ1gntNF5KEQwq4Qwm4ReUhEPpvKPioizUTkkRDC6RDCD0VEReRjVcZ3UNX/VNVjqvqGqv5ZgjkAACJ2/tV/Vf03VT2sqltVtbhK3ldVX02tBS+LSF4trm6aiDwXQngthFAmIg+IyG2q2lZV24jIJBF5IIRQFkIoEZFnpXJDU3W+aecJ1Ac2KYjdBhGpUNVZqlqsqh3OByGE90Tk70RkaeonVbmp6LiI3C4iuSJyi4jcpaoTL7jcj4jIVSJyk4jckDqWm7qcpRdOIvUrZ0eceQ4RkdVVPl6dOnY+ezuEEKrkb1fJRUT+WipfIeogIptE5NvOdQEALl/Xi8h6qdyAfE9Enqjy68H/V0RWprJvSeUPwEyqekRVxxjxn6xLIYTNUvnKycDUf+UhhA1VPr/qunWpeQJ1jk0KohZCKBWRMSISROTnInJAVZ9V1a7OmP8OIawJIZwLIbwtInOkclNS1TdCCMdDCCerOY+SKpugdHJE5GiVj4+KSE7qCf3C7HzetsrHz4QQ3gwhlIvIbBHhb1YA4PI1P7WBOP/f/1sl2x5C+HkIoUIqf/33ShHpqqq9RGSkVL66cTqE8JqIPOddSQghN/UqSDre2pMjIqVG5s7Tmw+QSWxSEL0QwnshhM+GEPJFZKiIdBeRR6zPV9XrVXWxqh5Q1aNS+WrLhS+Z78zwNMtEpF2Vj9uJSFnq1ZMLs/P5sSof763y7xNSuYAAAC5PE1MbiPP//bxK9j/P9yGEE6l/5kjl2nY4hHC8yudur8UcvLWnRuvSBfME6gWbFFxWQgjrROSXUrlZEal8heVC/1cqf7e2ZwihvVT+3cqFL1EH499JvSuVfzR/3vDUsfPZ1Re8TH51lRwAgPel8u8T21Q51qsWl/cn65Kq9hORllL5a9QbRKSZqg6o8vlV1y2gwbFJQdRUtUBVv6Kq+amPe4rIVBFZlvqUfSKSf0FTVlsRORRCOKWq14nIpWoVD4jIORHpV4upPikiX1bVHqraXUS+IpWbKRGR/xaRChH5/1S1paqeb3n5fS2uDwDQiIQQtovIChF5UFVbpP7W5NZaXORsEblVVcemNj7fFJHfhRCOpV6t+Z2IfFNV26jqh0Vkgog8VcsvA8gYNimI3TGp/OO9N1T1uFRuTt6Ryk2ASOU3+u+KyF5VPZg69nmpfOI9JiJfE5HfeFeQehn72yLyh9TvDo+68HNST/JlzsX8TCp/d3hNan7/mTomIYQzUllPfLuIHBGRv5HKXwU4c4mvHQBweXrugvdJeaaa4z4tlWveIRH5ulT+AMyUuuyx6bIQwrtS+evOs0Vkv1T+AO/zVT7l8yLSKpXNEZG7UmOAKOifFg4BAAAAQMPilRQAAAAAUWGTAgAAACAqbFIAAAAARIVNCgAAAICosEkBAAAAEJVmtRmsqp8QkUdFJEtEfhFC+M4lPp8qMQCoWwdDCJ0behKxqOk61aZNm5Cbm5s2KyuzW8hLS0trMcuG17p160Tjzp49myhLqnnz5omyEydOmFl98ubonS9vnMe7TO8298ZlZWWZ2alTp8wsOzs77fF27S584/v/1aJFCzPzrsubo/e1VVRUmFlOTo6ZHTt2zMzKy8vNrFWrVmZ2/PhxM/O+9kzzvm7vObFjx45mdujQoWqtU4k3KaqaJSI/FpHxIrJLRJar6rMhhLVJLxMAUGvbG3oCsUiyTuXm5spdd92VNluyZIl5XQsXLqzdZBvY4MGDE43bs2dPoiypzp3t72u6d+9uZitWrMj4XJLw5u+dL2+cx7tM7zb3xnmbinXr1plZnz590h6/6aabzDH5+flmtnHjRjNr27atme3du9fMvG+6R48ebWavvvqqmR04cMDMvNvgzTffNDPvPGfaNddcY2YlJSVmVlxcbGazZ8+u1jpVm1/3uk5ENoUQtqTelO7XUvlupQAAxIB1CgAuU7XZpPQQkZ1VPt6VOvYnVHWGqq5Q1Th+jAEAaCpqvE55v2IBAKg/df6H8yGEx0MIhSGEwrq+LgAAaqrqOtWmTZuGng4AQGq3SdktIj2rfJyfOgYAQAxYpwDgMlWbTcpyERmgqn1VtYWI/LWIPJuZaQEAUGusUwBwmdIQkrcCq+rNIvKIVFY7/kcI4duX+HwqiAGgbq3k12v/V03Xqby8vHDLLbekzZ588klznNcsVRctV6h73KZAcpd4/FRrnarV+6SEEF4QkRdqcxkAANQV1ikAuDzxjvMAAAAAosImBQAAAEBU2KQAAAAAiAqbFAAAAABRYZMCAAAAICq1avcCcPkaN26cmS1evLgeZwLE4+zZs3LgwIG0WWGh3Zi5YsWKuppS1Lya0Xbt2pnZunXr6mI6GUXNcMObMGGCma1fv97MLof7Fy6NV1IAAAAARIVNCgAAAICosEkBAAAAEBU2KQAAAACiwiYFAAAAQFTYpAAAAACIChXEQBNFzTBwsfLycrOCuHfv3ua4srIyM0tah+rV+1rquzbXu76YKny9cxnTPFF9eXl5DT0FODLxmOOVFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKJCBTHQiOXn55tZbm6umfXs2TPR9S1cuDDROCAW2dnZMmjQoLTZ9u3bzXE5OTkZn4tX4blixYqMX19jRs3w5alZM/vb1IKCAjPbsmWLmXFfqB+ZOM+8kgIAAAAgKmxSAAAAAESFTQoAAACAqLBJAQAAABAVNikAAAAAosImBQAAAEBUqCAGGrFdu3aZ2YABA8zMqxL2ao0948aNM7PFixcnukwg01RVsrKy0mYHDx5MdJleVapXXUzNMBoT63GQl5dnjikvLzezbdu2mVm/fv3MzKvGLSoqMrNFixaZGS5GBTEAAACARodNCgAAAICosEkBAAAAEBU2KQAAAACiwiYFAAAAQFTYpAAAAACIChXEQCM2atQoM2vfvr2ZDR06NNH15ebmJhoHxCKEIBUVFWmzdevWZfz6unfvnvHLvNzdcccdZjZz5sx6nAkyyaoaHjZsmDnGeiyKiLRs2dLM9u/fb2YTJkwws/Xr15tZXSgsLDQzr4LcqkqOqSZ50qRJZvb0009X6zJ4JQUAAABAVNikAAAAAIgKmxQAAAAAUWGTAgAAACAqbFIAAAAARIV2LyAiXhvXsmXLzCw/Pz/t8V27diW6PM/EiRPN7OjRo2a2cePGRNcH1KczZ87I9u3b02YFBQWJLjMnJ8fMvAafTEvaalQXrWYenivi5jVSlZWVmVl2dnba44cOHTLHeA1eW7ZsMbN+/fqZWbdu3cysvtu9vMf/3XffbWbe2l6fvOfENm3a1PryeSUFAAAAQFTYpAAAAACICpsUAAAAAFFhkwIAAAAgKmxSAAAAAESFTQoAAACAqFBBDETEqwVOUk/sjZk8ebKZvfzyy2bWt29fM9u7d6+ZLV682MyAWFxxxRXStm3btFlJSYk5rnv37mbmVRDXJ69etbS0tB5n4leXetWyRUVFZrZo0aJazQnV49UMe+tDnz590h4/deqUOaaiosLM2rdvb2atWrUyM+++Pnr0aDOr7yrus2fPmtmCBQvSHveeh/bs2VPrOV3IOyfjx4+v9eXzSgoAAACAqLBJAQAAABAVNikAAAAAosImBQAAAEBU2KQAAAAAiAqbFAAAAABRoYIYiEhxcbGZ7dy508zy8/PTHu/WrZs55uGHH67+xKrJq1qdOHGimc2fPz/jcwGSCCGY1Z9eba5XxenVgtan+q5Q9c6XVwM7cuRIM7OqV+uC93y8cOHCeptHbAYNGpRonFWB26tXL3PMjh07zOzMmTNmdvLkSTPLysoys/rm3cdee+01MyssLEx73KsZnjJlipm9/vrrZpa0unjTpk2JxlXFKykAAAAAosImBQAAAEBU2KQAAAAAiAqbFAAAAABRYZMCAAAAICpsUgAAAABE5ZIVxKr6HyLySRHZH0IYmjrWUUTmikgfEdkmIn8VQjhcd9PE5cqqxs3NzTXHvPPOO3U1nSiMGjXKzLya4SNHjpjZsGHD0h4/evRo9SdWTZ07dzYzr0py1qxZZmbdT0REdu3aVb2JocnK5DoVQpBTp07VeA5elejWrVtrfHlJeXXHgwcPNrNFixZlfC5e5XFRUZGZeVXm9VkLXJ+3W2zGjBljZh988IGZ3XTTTWa2ZMmStMe9NWX//v2J5uE9ht98800zGzJkiJnVBavuXMR/HFjfQ61YscIcUxc1w56OHTvW+jKq80rKL0XkExcc+ycReSWEMEBEXkl9DABAQ/ilsE4BQKNyyU1KCOE1ETl0weEJInL+x6KzRMR+lzYAAOoQ6xQAND5J33G+awjh/dS/94pIV+sTVXWGiMxIeD0AACSRaJ1q2bJlPUwNAHAptf7D+RBCEJHg5I+HEApDCIW1vS4AAGqqJutU8+bN63FmAABL0k3KPlW9UkQk9X/7L5sAAKh/rFMAcBlLukl5VkSmp/49XUQWZGY6AABkBOsUAFzGqlNBPEdEPioieaq6S0S+LiLfEZHfqOqdIrJdRP6qLieJy9fkyZPTHn/44YfNMY29jnbZsmWJxs2YUfM/7XrhhRcSXZfn5MmTZrZ+/Xoza+y3KxpOJteprKws6dSpU9qspKQk0fxKS0sTjUvCqyBeu3Ztvc3jUuqi8jipadOmpT0+e/Zsc0xBQYGZedXLlwvvvu597Xv37jWzW2+9Ne3xVatWmWPmzp1rZh7rMSzi1yTPmzcv0fUl5T0OvPOcpDK4LmqGPStXrqz1ZVxykxJCmGpEN9b62gEAqCXWKQBofHjHeQAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQlUu2ewG1kaTOjzra9Hbu3FnjMQMGDDCzpOe5VatWZnbgwIGMXx9Qn86dOyfHjx+v8bhYamc7d+5sZr179zazp59+OuNzsap9Rfx63/q2b9++tMfvuusuc8yCBU33bXfy8vLM7NixY2b2yiuv1Pi6xowZY2YHDx40s/bt2ycaV981vd7X563fM2fOrIvpZJT3tgPVfb7klRQAAAAAUWGTAgAAACAqbFIAAAAARIVNCgAAAICosEkBAAAAEBU2KQAAAACiQgUx6pRVO+tV0+Xm5prZO++8U+s5xcw7L14Fcc+ePdMe9yqBi4uLzWzNmjVmltTUqVMTXV9jv82BJIqKitIe79atmzmmoqLCzAoLC81sxYoV1Z9YFYsXLzaz7t27J7rMpBWxBQUFZta8efO0x73npfquqo1Jp06dzCxJffeJEyfMrEePHomyLVu2mJn12BHxK4FLSkrMLOnj59SpU2aW6XXYe8zVxf157dq1tb4MXkkBAAAAEBU2KQAAAACiwiYFAAAAQFTYpAAAAACICpsUAAAAAFFhkwIAAAAgKlQQo05Nnjw57fH33nvPHNOUK2etymYRv5rZqhb0aouT1htmZ2ebWVO+7dA4hBDcWtBYWDWwpaWl5hivnth7fkkqppper1r2zJkzaY+3bt26rqYTPa+udvTo0Wb2wx/+0Mys6t+nn37aHDNt2jQzKy8vNzPPW2+9ZWa9e/c2s+uvv97MHnrooURzSVrvnUR9Px4zcX28kgIAAAAgKmxSAAAAAESFTQoAAACAqLBJAQAAABAVNikAAAAAosImBQAAAEBUqCBGnfrQhz6U9rhXQWzVFouIzJs3r9Zzamj333+/mW3YsMHMDh48aGZbtmxJe/zIkSPmmMLCQjM7ffq0meXk5JjZuHHjzCwvL8/Mli5dmugyFy9ebGZDhw41M6qSYSkvL3cfa7H44IMP0h73HvPt2rUzszZt2phZQUGBma1bt87MYrJ7924zs6pxn3nmGXPMHXfcYWYzZ86s/sSqqO/z7K0B11xzjZlZ642IyE033WRmJ06cSHvcO5d79+41s61bt5pZ3759zaxjx45m5q1TXuWxV9nsZV5Nb5IK37q4DyWdfybwSgoAAACAqLBJAQAAABAVNikAAAAAosImBQAAAEBU2KQAAAAAiAqbFAAAAABRoYIYteZVvT777LNpj19//fXmmBdffLHWc7pQfn6+me3atcvMktbYetf37W9/28ymT59uZhs3bjSzAQMGmJll/vz5ZjZq1CgzW7NmTY2vS8SvdvRug9GjR5uZd56pGUZjtmjRohqPyc7ONjOvdrm0tLTG1yUiUlxcbGYLFy5MdJmeMWPGmNngwYPNzKrEv/nmm80xv/vd76o/sSq8OlePVZN8Kd79xKuPLSsrM7MJEyaYWYsWLczMuo+dOnXKHON9r9C6dWsz2759u5kl5VVLe+dkwYIFGZ+LVTU8fPhwc4z39gHefWHs2LFm9vrrr5uZp7rVxbySAgAAACAqbFIAAAAARIVNCgAAAICosEkBAAAAEBU2KQAAAACiwiYFAAAAQFSoIEa1eFW8nTt3NrPFixenPX7gwAFzzLBhw8zMq6qtC0lrbL15erWc+/fvNzOvbteqIG7fvr05xqv2nTdvXqJ5eF/30qVLzcyzc+fOjM8FsOTk5JhVtuvWravn2WRWSUlJvV5fXdQMe7wq9m3btpnZkCFD0h73bu/c3FwzmzZtmpnNnj3bzDxr1641s+rWudZknFfZvGnTJjPzaq6tc+adS+95vLy83My870u6detmZl26dDEzrz564MCBZmbVBYskf07p27dv2uNenbN324wbN87MDh06VP2JVZH0flkVr6QAAAAAiAqbFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFSqIUS1eFa9XXedlljVr1piZV4Xcv39/M5s/f36N51Eb3jy9Wk6vnrhnz55m1qJFi7THvarII0eOmJk3/5ycHDPz6iK9etCxY8eamXe7vvzyy2ZGPTGSUFVp2bJlQ0+jUfAqWz1Jq0tbt25tZllZWWaWl5eX9vj69evNMc2bNzczq3pfROSee+4xsz/84Q9mtmLFCjNLyrt9vOdI720CvBr9s2fPpj1+7Ngxc8zx48fNzJu/N48zZ86Ymfd1f+YznzGzlStXmtnw4cPNrLS01My8x4G15nvny6sg3rdvn5l5tcaZqBn28EoKAAAAgKiwSQEAAAAQFTYpAAAAAKLCJgUAAABAVNikAAAAAIgK7V6oUydPnkx7fNmyZYkur7Cw0Mzqu8HL4zVneXbu3Glmn/70p83s2WefTXvca+LyGttyc3PNzOM1aiUdN3PmTDOjpQuZVlFRkfjxiz/ltS/VRVvV6dOnzayiosLM3njjjbTHd+/ebY45ePCgmXltha1atTKzHj16mFlZWZmZrVu3zsw8EyZMMDOv0cnLhgwZYmY/+tGP0h7v16+fOWbLli1mNnr0aDPr1auXmXltYl6bpHd/8Bq85syZY2ZJ27GsFq82bdqYYz744AMz85q/SkpKqj+xKgoKCsysuvdZXkkBAAAAEBU2KQAAAACiwiYFAAAAQFTYpAAAAACICpsUAAAAAFFhkwIAAAAgKpesIFbVniLypIh0FZEgIo+HEB5V1Y4iMldE+ojINhH5qxDC4bqbKqrLq3P1Mq8W2BvnVf3t2LHDzBozr4Jy7969ZuZV9m3evNnMrr766rTHN27caI6ZOHGimbVv397MkvLm4l3fzTffbGYvvPCCmVFP3HRkcp06ceKErFy5sm4n3EQkrRkuKioys0WLFpmZV++7dOlSM7vtttvSHs/LyzPHWBWwIiKdOnUys9WrV5vZwoULzawu7N+/38y8WuDnnnvOzO6++24za9euXdrjXbt2Ncd453LAgAFm5tUMe/W3XoVv3759zcx7+4Bp06aZ2fe//30z81jz9N6mwfu66+I5b/z48YnmUlV1XkkpF5GvhBAGi8goEblbVQeLyD+JyCshhAEi8krqYwAA6hvrFAA0MpfcpIQQ3g8hrEr9+5iIvCciPURkgojMSn3aLBGxfywLAEAdYZ0CgManRn+Toqp9RORaEXlDRLqGEN5PRXul8mV2AAAaDOsUADQOl/yblPNUNUdEnhaRe0MIpar6P1kIIahqMMbNEJEZtZ0oAACeTKxTzZpVe1kEANShar2SoqrNpfKJf3YI4Xepw/tU9cpUfqWIpP0LrBDC4yGEwhCC/dc8AADUQqbWKTYpABCHS25StPJHUU+IyHshhB9UiZ4Vkempf08XkQWZnx4AAD7WKQBofKrzI6MPi8hnRGSNqv4xdew+EfmOiPxGVe8Uke0i8ld1M0XUlFfLd8stt5jZ9ddfb2aPPvqomXnVdUeOHDEzy9ChQ81s06ZNZubVJHt1tN64G2+80cy8r7tLly5m9hd/8RdmtmrVKjM7efKkmfXu3Tvtca9i8vTp02bm3QazZs0yM0/S28CrMaVmGCkZW6cqKiqktLS0ziaKSlOmTDEz7/x7Ne2tW7c2M2992717d9rj3vpVVlZmZi1btjSzzp07m5nHq7Fdv369mXnPn9u3bzezPn36mNlNN91kZhs2bDCzG264Ie1xrzraU/XXOS/04x//2My8imtvXT982G4vX7t2rZklreL2LFiQ/uct1n35UqpbCVwThw4dqvVlXHKTEkIoERHrnmB/BwcAQD1gnQKAxod3nAcAAAAQFTYpAAAAAKLCJgUAAABAVNikAAAAAIgKmxQAAAAAUbns37Uqae3snXfeaWbvvvtu2uPLli2r/sQa0OLFi83MqzH0ahO/9a1vmZlXvbd169a0x4cNG2aOWbNmjZnVReXsgw8+aGZLliwxs8985jNm5lUOejXDc+bMMbObb77ZzE6cOFHjeXiX583xnXfeMTPvceWN69Chg5ldLo87NA6tW7eWkSNHps2s2s+6MmHCBDNbvnx52uN79uwxxxQW2u+pXBc1qZ7NmzebmXX+RUQGDhxoZh988IGZHTt2zMystc+rO549e7aZefcT7zI9Xs2wV4fs3Ye8y/TWonHjxplZz549zcyq0W/RooU55rnnnjOz0aNHm9nnPvc5Mztz5oyZHT9+3Mysqn8R/3FXn5I+jr37ZdJ64tzc3ETjquKVFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKJy2VcQe5W0o0aNMrPS0tK6mE70HnnkETM7cuSImXlVcl7loJUtXbrUHJO0Vnr69Olm5tUre3WEX/7yl83s3LlzZpaTk2Nm3vnyanrbt29vZlZF4NixY80xjz/+uJkVFxebmXf7eBWgQ4cONbOdO3eaGVCfzp49K7t376636+vevXuicVb9qvfc6lXV1kUFqcerSs3Ozk6UjRgxwsyaN29uZldckf7ntd7z8Y033mhm3vnKy8szs3/+5382M2+d8tZF7zl5zJgxZnb48GEza9eunZkdPHjQzKx12LvPejXDHu/+7NUMb9iwwcy2bNliZosWLarexCJVF49x63FVo8vIwDwAAAAAIGPYpAAAAACICpsUAAAAAFFhkwIAAAAgKmxSAAAAAESFTQoAAACAqFz2FcQer5avQ4cOZmbVNH7rW98yxzzwwANm9qUvfcnM5s2bZ2be/JPyKhW9ithhw4aZmVdd3LVr17THCwsLzTHz5883M8+sWbPM7Atf+IKZhRDMrGPHjmZmfW0iIkuWLDEz73x5Ncpe/eG2bdvMzDJ16lQz8yqnvbrgLl261HgeIslv88bMezx696G6eN5oSlq3bi1//ud/njbzanOT2rNnj5l5tbOdO3dOe3zSpEnmmMcee8zMZsyYYWZebW5JSYmZJeVd5oQJE8ysvLzczM6ePWtmBw4cSHu8WTP7WyTv+bioqMjMvMend5mnTp0yM68ye9y4cWbmVf96VcIf//jHzWz//v1mZn0NI0eOrPEYEb8K2asS7tu3b6LL9B6ruJhXeV5dvJICAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKLCJgUAAABAVC77CmKvpvOWW24xs3PnzpnZmjVrajzm+9//vpm9+eabZjZ58mQz8+oBly1bZmZJefXEH/rQh8ysdevWZjZgwIC0x71axEGDBpnZ+vXrzax9+/Zm5s0/OzvbzLwKUO/2KSgoMLO9e/eamXe7etfXs2fPtMenTZtmjjl06JCZfe1rXzMz6zYV8auXrTmKiOTn55vZ5V6p6z1HebyaYa8i+nI/Xw3t3Llzcvz48Xq7vu7du5uZ99xq1fSOGDHCHDNmzBgz86pex44da2Y9evQws7lz55pZUsuXLzez/v37m1lFRYWZWXXy3nOWV7XrPVd7t4FVhSzif20PkLtFAAAgAElEQVRWHbWIyOLFi83MW2uvvfZaM/Pqfb01c9OmTWmPe7XZXu3v1q1bzcx7XHn1yjk5OWbmrVNJefOsz8pj73sF7/suby3q169freYkwispAAAAACLDJgUAAABAVNikAAAAAIgKmxQAAAAAUWGTAgAAACAqbFIAAAAAROWyryD2anMzbeHChWbm1R2XlZWZmVdHe/311ye6zKTnxKvXGz58uJl5FXpWhePp06fNMV4V3oIFC8xs3759ZvbKK68kuj6v7vJXv/qVmXm3q1f76J3LIUOGmNkbb7yR9rhXYfjSSy+ZmVdjm7Ti9sYbbzSzw4cPZ/z6krIqg5NWAnuVll7l9NSpU81szpw5ZobaqaiocJ9fM817jB49etTMrOeDlStXmmPGjx9vZm+99ZaZXXXVVWbWrVs3M7v99tvN7MknnzQzz+DBg83siivsn7t6z7tW5fSJEyfMMV26dDEzb+325tirV69E406ePGlmXsX+7NmzzayoqMjMhg0bZmZeHbJ1zt5//31zzI4dO8xs5syZZua56667zMyr2PceW8XFxWbmfe+YtGbYepsDb5313kbDW8O8+0lhYaGZbd++3cyqi1dSAAAAAESFTQoAAACAqLBJAQAAABAVNikAAAAAosImBQAAAEBU2KQAAAAAiMplX0HsSVrFO27cuLTHvSq/n/3sZ2bmVaj+8Ic/NLP33nvPzLyKQ68qNWm17KpVq8zMqmwVEenatWva40899ZQ5xjN37lwz+9SnPmVmFRUVZrZp0yYz27x5s5nNnz/fzDxWLbOIyP79+83s3XffNbNJkyalPT5v3jxzzHXXXWdm99xzj5k9+uijZubdF6yaZBG/Uturxq6LemLr9vFuN69m+OzZs4nm4dWKou6cO3fOrKStb4sWLTKzCRMmpD0+YsQIc4xXD+utb179bV5enpm1bt3azKz5i/hV89458b72jh07mlmbNm3SHl+9erU5xjsnHu95ZOPGjWZWXl5uZknrYz3e9xFnzpwxM1U1M+t53qsg3rJli5klrbj27pcvv/yymY0cOdLMkrKqhEVE1q1bZ2bWWyf88Y9/NMd4b3Hg1W174zzbtm1LNK4qXkkBAAAAEBU2KQAAAACiwiYFAAAAQFTYpAAAAACICpsUAAAAAFFhkwIAAAAgKhpCqL8rU62/KxO/DjVJPbFVTSwi0r59ezNbsWKFmXkVqt78+/fvb2anT582s4ULF5pZUj/+8Y/N7Oqrr057/He/+505xqotFvGrCocMGWJmc+bMMbOPfexjZnbo0CEz86r+Fi9ebGYTJ040s1atWplZ27ZtzWzJkiVpj3v385///Odm5t0Gy5cvNzNv/vfdd5+ZJX2seo9J7zbwao2tzHusepfnnROvctRzierllSGEwkQXDOnYsWP4+Mc/njbzKtBjYdWRi4gcPXrUzK699loz27Nnj5m1a9fOzE6dOmVmXiXw3r17zSxppe6YMWPMbPTo0WmPezXDXm26V5tbXFxsZuvXr090md5tMHz4cDPznpO/+c1vJro+r7Z56dKlaY97Vbve/blZM/tdNLz10qsSfumll8zMq4H2eN+vPfTQQ2bm3Vesr8Grrj98+LCZ/eIXvzCzKVOmmJn3Ng2eFStWVGud4pUUAAAAAFFhkwIAAAAgKmxSAAAAAESFTQoAAACAqLBJAQAAABAVNikAAAAAomL3tzUCSWqGPV7NsFdB6o2bNWuWmeXk5GT8MpNUr4qILFu2zMzuvvtuM0tyXYWFdivdZz/7WTPbvn27mR04cMDMvPP8wAMPmNn9999vZt7X4NWAepW0Xi1nkvu6d5t+5CMfMbPS0lIzGzFihJl5NcPXX3+9mXm8muGkrLkMGjTIHONVh9bFHFF3srKy3OdXS0FBgZl5FauZ5tWkdurUycyys7PNzKuc9SrJi4qKzOzs2bNm5j2X33PPPWb26KOPmllJSUmizOJVGnu19l6ds6dHjx5m9sILL5iZ97z7jW98w8y2bt1qZidOnDAzT5LHQZ8+fRJd19q1a81s1KhRZtalSxcza9GihZl5jxGvPtr7XsGr2LceI978k0pave7VWHtvzVEVr6QAAAAAiAqbFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKhccpOiqtmq+qaqrlbVd1X1wdTxvqr6hqpuUtW5qmrXHgAAUEdYpwCg8alOBfFpEflYCKFMVZuLSImqLhSRL4vIwyGEX6vqT0XkThH59zqca4PzqmM3bdpkZj179jQzrwqvQ4cOZrZy5UozmzFjhpl5VYVe3d0//uM/mtltt91mZpZdu3YlyrzzfOONN5rZpEmTzKxVq1Zm5hkwYICZeZW6Xg3jJz7xCTPzzrN123n1t0888YSZZWVlmdmSJUvM7O233zazI0eOmJlXy9m/f38zS1ozPmzYMDOrqKhIe7x169bmmKT3IWRMxtaps2fPyr59+2o8gby8vATTznx1cefOnc3s5MmTZuY9dgcOHGhmXl2597j2nj/79etnZhs2bDCzH/zgB2b25S9/2cyS8GqLvfl7X/fOnTsTjfPO86uvvmpm06dPNzOvtt+ryvfOi1W369XRevP37uve4+qtt94ys/3795uZ9zjwqvlnz55tZt73ay+99JKZXXvttWmPezXJ9V2Hf+zYsVpfxiVfSQmVylIfNk/9F0TkYyLy29TxWSIysdazAQCghlinAKDxqdbfpKhqlqr+UUT2i8jLIrJZRI6EEM6/a9QuEbHfaQgAgDrEOgUAjUu1NikhhIoQwjUiki8i14mI/TraBVR1hqquUNXqvb0kAAA1lKl16syZM3U2RwBA9dWo3SuEcEREFovIaBHJVdXzf9OSLyK7jTGPhxAKQwjpfxkRAIAMqe065f1ONwCg/lSn3auzquam/t1KRMaLyHtSuQj8ZerTpovIgrqaJAAAFtYpAGh8qtPudaWIzFLVLKnc1PwmhPC8qq4VkV+r6v8RkbdExK4KAgCg7rBOAUAjc8lNSgjhbRG5qOsshLBFKn/vt8nw6tu8+t7Dhw+bmVdd6o37yle+YmZebZ1X75ubm2tmu3en/S0JERH56le/amaTJ09Oe/zQoUPmmL/5m78xM69y1qt6vvXWW83s6quvNjPPv/3bv5nZlClTzMyr+ly6dKmZefcxq8r6i1/8ojnm3XffNbMHHnjAzDzFxcVmlrR6M2nNcH5+vpl5jy3rMbJ3715zzLJly6o/MWRcJtepli1bSp8+fWo8h65du5pZUVGRmXl1tV6tsXV927ZtM8c0b97czLyadq/y3qte9dZMrzbX452vsrIyM/Oek9u3b5/2+GuvvWaO8eqhn3zySTPzqnGvu86+qyapxRYRWbRokZl16tTJzDZv3mxmSW+77OzstMcfe+wxc8zcuXPNrGPHjmb28MMPm5m3TnmX+a//+q9mltTx48fNzFszrTXMq+j2noe8GuikvOrl6uId5wEAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKLCJgUAAABAVKpTQYxq2Lhxo5l5jVrTp083s+7du5vZli1bzGzw4MFmNmPGDDOzGk5ERA4cOGBmXrvL9u3b0x4fOXKkOcY7X56FCxeaWcuWLc3s/fffN7N77rnHzB599FEz+9u//Vsz8xq8vK/daxPr1atX2uOjR482xwwaNMjMBg4caGZeS453G9QFr8HLa0bxWofuvPPOtMe9di80HuXl5XLw4MEaj/Me1+3atTOzIUOGmFnv3r3NzGrc8i7v6NGjZrZ69Woz85of+/fvb2beGuadY+/6vGzTpk1m5j3f/eIXv0h73Jt/YaH9/tRJm5LatGljZl4LVFJeg9fYsWPNzGvOuuOOO8zMaqQ7cuSIOWb8+PFmtn//fjPzbrv6XqfGjBljZllZWWbmtahZ53LBAvutoJYvX25mdSFpC1xVvJICAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKLCJgUAAABAVKggrgfjxo0zM69Cb8eOHWbmVQJ37tzZzKyqWhGRLl26mNnw4cPNzKoZFhFZtmxZ2uNXX321OeZf/uVfzOy+++4zM8/8+fPNzKvlvOaaa8xs1KhRZubdrkkrlr25nD17Nu3xd955xxyzdetWM/PGTZw40cy86k3v6/aqhL1xkydPNrOcnBwz8yqIn3jiCTND46eqZi2oV2t60003Jbq+P/zhD2bmPZdb9++nn37aHFNUVGRm5eXlZlZWVmZmr7/+upl5dfhe9eoHH3xgZmvWrDGzCRMmmJm3ZlprdMeOHc0xXt2xZ926dYmyadOmmZn31gLeufRuc69m2KvUveGGG8xs/fr1aY/v3LnTHONVR6uqmXn1+95jxON93adOnTKzYcOGmZlXqf3kk0+amVf1bElaCew973mXWVBQYGbefb0qXkkBAAAAEBU2KQAAAACiwiYFAAAAQFTYpAAAAACICpsUAAAAAFFhkwIAAAAgKhpCqL8rU62/K6sDXk2qVyPnVaF6rPreS/GqcVu1amVmXj2xp1kzu8m6Xbt2Nb48r+7Oq8L0agW9St2YeHXVjzzyiJm1adMm7fHNmzebY7z784kTJ8ystLTUzO6//34z+/u//3sz8yoovdt87NixZnbmzBkzmz17tplZ9dELFy40xyStUK4jK0MIhfV9pY1Fz549w7333ps2+/Wvf22O8+q3PV5t7oIFC2p8ecXFxWbm1Y7n5eWZmfc4e+aZZ8zMq673nkd69OhhZi+88IKZ9evXz8y8GlhrDTt9+rQ5xqtQ9tY977n1scceM7P65tVVjxw50sw6dOhgZi1atEh7fPny5eYY760RrMsT8Stud+/ebWbe93JDhgwxM+/+3KdPHzO78sorzew3v/lNjefy4osvmmO8OXpVwkkriC+hWusUr6QAAAAAiAqbFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFSqI68HQoUPN7MiRI2bmVeF5lbpeHerPfvYzMzt69KiZPfXUU2Z2+PBhM7PqV0ePHm2OKSgoMLOSkhIzO3DggJmNGDHCzF555RUzS1of69VA33777Wa2b98+M/v85z9vZseOHUt7/Ior7J9DfPDBB2bmVYB64x544AEzu/baa83Mm6dXQezxKoi9KtZNmzalPX651FgLFcS10rt373DfffelzX7729+a47zncq+msxYVnjU2ZcoUM/Me11bFuYjI8ePHzaxr165mVl5ebmZeHb5XlexVBluPaxG7ytarXv39739vZl6VvPcc37NnTzPzvlf72te+ZmaeGTNmmJlXm+tV+Ho10FbmPXa8282rgfZq7desWWNmXr1ydna2mXnfrzVv3tzMvLdwsNZ1EZHFixenPe7NcdGiRWZWFy5RXUwFMQAAAIDLD5sUAAAAAFFhkwIAAAAgKmxSAAAAAESFTQoAAACAqLBJAQAAABAVu/sMGdO/f38z82p/N27caGZexa1X71tRUWFmmzdvNjOvXs+rILbm0q5dO3PMM888Y2adO3c2s6uuusrMktYMe1WSVgWgiEiHDh3MzKsV9Go53377bTO78cYb0x4vLS01x3gOHjxoZi+88IKZrVixwszmz59vZvfcc4+ZeZWjXj2lV+145ZVXmplV4e3Vgy5cuNDMcHkJIcjZs2fTZl696qpVq8ysPmuGPXPnzjWzSZMmmZlXR7t69Woz2759u5kNHjzYzLxq2ZMnT5qZV7Hq1dBbz5Pt27c3x3z4wx82M6vSWMSvbG7btq2Z3XvvvWbm8W7X1157zcy829xbM63Hjoh9Xrz666TP/9447/sZ7/uusWPHmpn3fZ63Lu7fv9/MDh06ZGa9e/dOe9x77EyYMMHMPAsWLEg0LhPPe7ySAgAAACAqbFIAAAAARIVNCgAAAICosEkBAAAAEBU2KQAAAACiwiYFAAAAQFSoIK4Bq5pURGTAgAFm1qVLFzPzalknT55sZvPmzTMzz969e82sV69eZuZV45aVlZmZVWV79dVXm2O2bNliZl49oFcX6VUmerya4X/4h38ws4EDB5pZSUmJmXmVx+vWrTOzoqKitMe9uuOXX37ZzLxaaa822zN06FAze/TRRxNdpsd7vD744INm9l//9V9pj3uVnN51FRYWmplXTZn0PovaCSHImTNn0mbPP/+8Oa579+5mVlBQYGbe49pTXFyc9viBAwfMMd7zi1ex7dXTe5X3GzZsMDOvdrZTp05m5j2neRXRXiW5tS42b97cHNO1a1cz27lzp5kNGzbMzL73ve+ZWVLeOenWrZuZPfnkk2Z2ww03mNm+ffvMbMyYMWmPe5XNO3bsMDPPuXPnzMy7r3vfY3z96183s8997nNm5t2POnbsaGbemnPq1Km0x711w3peE/GroxsSr6QAAAAAiAqbFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFSqI05g4cWLa47179zbHvPHGG2bm1blaNZIifrWvp2/fvmaWm5trZl5Fo1eh6dXkWZV3Xu2eV/tr3TYifp2zx6vGfeedd8zMq+L9z//8TzMbPny4mY0YMcLMfvWrX5nZq6++mva4VwfpnS+vNnv//v1m5tXteufrrrvuMrPXX3/dzLzbx6v69M6L9Th47rnnzDHe/dKrhKRmOD7nzp2TEydOpM28muHOnTubmVfT61UQT5kyxcxWr16d9viQIUPMMd5a5GVe7a9XH1taWmpm/fv3NzNv7duzZ4+ZWbebiMjp06fNzKrp9WqGV65caWajRo0yM+88e2874J3LiooKM/Oed73nwUGDBplZVlaWmXnfD1j1vl4Vu3f/8qr+vedrz+c//3kz+/CHP2xm3/3ud83MeouAS/Gqnq2aa69W2jtf3m3QkHglBQAAAEBU2KQAAAAAiAqbFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKhQQZxGkipbrzpw3rx5Zpa0/jYpqxJYRGT8+PFm1q5dOzN7+eWXzcyqyfPqYb36Rq8a15Ofn29mH/3oR83sgQceMLNly5aZmVc5+tWvftXMpk6dama9evUys23btqU97lVo3nbbbWb20ksvmZl1m4qIHDlyxMy8ul2vNtWrKvVun5/+9KdmduWVV5rZ5s2b0x735v+jH/3IzHB5UVWzhtqrGfY8/fTTZubVdg8ePNjMTp06lfb40qVLzTFe/fnZs2fNzHu+3rJli5l5vHXjxhtvNDOvNvfWW281s5YtW5qZVe/rzaNt27ZmtmHDBjPz1vwdO3aYWXZ2tpl5z2ff+973zMz7vmXv3r1m5lUee5XB1nNokvu5iP/9jLfme8/l3/nOd8zMqwSfNGmSmXXo0MHMdu/ebWbe/aFTp05pj8+dO9cc49V3x4pXUgAAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKLCJgUAAABAVKq9SVHVLFV9S1WfT33cV1XfUNVNqjpXVe1KBwAA6hjrFAA0HjWpIL5HRN4TkfNdtN8VkYdDCL9W1Z+KyJ0i8u8Znt9lw6sZHjdunJktXrw443O57rrrzKxv375mtmTJEjPLysoys6eeesrMrrrqqrTHn3nmGXOMV0frVS3+5Cc/MbM33njDzLzKaa/G0KqtFPHrgr/+9a+b2Ztvvmlmv/3tb83Muh95Fb1WzaqIX2Pq8aodPUkrfJPUhYuI3HzzzWbWvXv3tMe9+0Jubq6Z1UWVOEy1XqcqKirMx/bq1avNcV6N6tixY83Mu0yvBrZ9+/ZmZmndurWZec8HAwcONDOvJtW7vnXr1plZcXGxmRUVFZmZV4Hu1cC+9tpraY+fOHHCHPPee++Z2YQJE8ysY8eOZuY9Vxw6dMjMrrjC/nnzyJEjzezkyZNmVl5ebmarVq0yM6/m1sq829Sr0ffeGsGrLk563/Pqfa11Q8SvIPd4a/7atWtrPI+/+7u/M7Ovfe1r1Z9YNXlzqW4dcrVeSVHVfBG5RUR+kfpYReRjInL+DM4SkYnVukYAADKMdQoAGpfq/rrXIyLyjyJyLvVxJxE5EkI4v9XeJSI9Mjw3AACqi3UKABqRS25SVPWTIrI/hLAyyRWo6gxVXaGqK5KMBwDAk8l1yvsVHwBA/anO36R8WET+QlVvFpFsqfxd30dFJFdVm6V+SpUvIrvTDQ4hPC4ij4uIqGrIyKwBAPhfGVunrrzyStYpAIjAJV9JCSH8cwghP4TQR0T+WkR+H0KYJiKLReQvU582XUQW1NksAQAwsE4BQONTm/dJ+aqIfFlVN0nl7/4+kZkpAQCQEaxTAHCZ0hDq75Vtft2rfkyfPt3Mrr32WjPbunWrmb399ttm5tUoDx06NO1xr2oxaWWzN+6GG24wM68e8N577zWzF1980cy8Sur65J0Tr6r61VdfNbNly5bVak6ZNHXqVDN7/fXXE12mVSeck5NjjmnVqpWZ1UXN+CWsDCEU1veVNhYtWrQInTt3Tpt5tZlejeqiRYvM7Etf+pKZHT9+3Mwef/xxM0uioKDAzD71qU+ZmVeT7NUyP/GEvV8cNGiQmRUW2ndtb13xng++8pWvpD3u/X2Sd77++Mc/mpk3R+8+tGXLFjOzqv5F/PuJd3/26mNXrLD/xNi7fXr37p32+L59+8wx3n3IOyfeY64ueI9j6/lEROTdd981s9mzZ9d4Hvfff7+ZZWdnm5lX3/3QQw/VeB7VUK11inecBwAAABAVNikAAAAAosImBQAAAEBU2KQAAAAAiAqbFAAAAABRYZMCAAAAICrVecd5XGZmzZplZs2a2Td5u3btzKx9+/aJ5mLVLRYXF5tjevbsaWYbN240swMHDpiZVzPs+dznPmdmkydPTnSZdcGqeu7WrZs5xqs3re+a4fz8/ERZixYtzGzXrl2J5nLHHXekPf6tb30r0eXh8tKyZUvp169f2syrbPUqT70612PHjiXKrMv05uhZt26dmXmVut4cvXpir9J7+fLlZnbTTTeZmXW7iYjMnTvXzLp06ZL2uFdb7D0vnTp1yswWLLDfT3T9+vVmNmTIEDN7+OGHzSyppPcjr57YukyvKt+rJ167dm31J5YB06ZNMzPv/uxVJSetqLfWKe92877H82r0GxKvpAAAAACICpsUAAAAAFFhkwIAAAAgKmxSAAAAAESFTQoAAACAqLBJAQAAABAVKoibGK+SdtWqVWa2c+dOM/OqGK0a2MOHD5tj1qxZY2aeYcOGJbpMqyb5UubNm5doXFJebfPChQvTHr/77rvNMS+++GKt55QpXl1wbm6umXmVnUlRNdy0tWrVynwuKSkpSXSZXi2oV+/buXNnM7Pqdr0619mzZ5uZZ/Xq1WbmfW3e849Vmy4ikpWVZWabN282s3PnzplZYWGhmb399ttpjzdv3twc41XteufLk5eXZ2ZJa/S9r9v7GpJ68MEHzcw6z16VvFfZXN+SPn4mTZpkZkmrni0zZ85MNM6rSfcUFBSYmVdrXl28kgIAAAAgKmxSAAAAAESFTQoAAACAqLBJAQAAABAVNikAAAAAokK7VxPz7W9/28y8VhiP18w0atSotMeXLVuW6Lq8hqvhw4eb2Zw5cxJdX32bOHGimSVpYrnrrrtqM520vPvJ4sWLM359SdvXgMtBphuWkjaQeZI2ED3zzDNmNnr0aDMbPHiwmZ08edLMvMZF72uwGgStBjURkccff9zMkqqL264uGrxuv/12M9u2bZuZLV26NO3xTDdcXUpRUZGZLVq0KOPXV15envHLTNqCamnXrp2ZfelLXzIz73s52r0AAAAANDpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKLCJgUAAABAVKggxv/Izs42sxEjRiS6zKRVw5aFCxea2eHDhzN6XQ1h/vz5DT2FS6qLmmEgFqdOnZKNGzc29DQuGxMmTDCzBQsWmNnTTz9tZl4d6syZM83sO9/5jpktX77czE6cOJH2eF3UDMdkzJgxZubVIT/55JN1MZ2M+uIXv2hmjz32mJlNmTLFzDZv3mxmXtWz9zhIyvt+zVJQUGBmXiW4N/8k86gJXkkBAAAAEBU2KQAAAACiwiYFAAAAQFTYpAAAAACICpsUAAAAAFFhkwIAAAAgKhpCqL8rU62/K0NG5efnm9muXbtqPM4bg4Y3depUM5szZ049zgQJrAwhFDb0JC5XnTp1CsXFxWmz2bNnm+O6d+9uZuPGjTMz7zIbM68OtW/fvmbWokULM2vWzH5XBatmWERk7NixaY/fd9995pimbNKkSWbmVUtnWtL66/rmPTfs2bMn0WVa9dHXX3+9OWbTpk1m5lUJz50718y8ymZvnFRzneKVFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKJCBTEANC5UENdCbm5u+OhHP5o282pNrUpQEZGDBw+aWU5OjpmtWLHCzJoqr8518ODBiS5z0aJFSacD1JnCQvtp3KoaPnLkSKLrWr9+vZn92Z/9mZm9/vrrZvbJT37SzB5//HEqiAEAAABcftikAAAAAIgKmxQAAAAAUWGTAgAAACAqbFIAAAAARIVNCgAAAICoNGvoCeDykJ+fb2a7du2qx5kAQN05ffq0W8dpKSkpMTOvSrSp1gwXFRWZWfPmzc1s4cKFZubVEzfV8+zxbgOvypZzWT+881xWVpb2+Pjx480xS5curfHliYi0bdvWzPbs2WNmzz//vJlVF6+kAAAAAIgKmxQAAAAAUWGTAgAAACAqbFIAAAAARIVNCgAAAICosEkBAAAAEBUNIdTflanW35UBQNO0MoRgd97CVVhYGKzqzx49epjj+vXrZ2YHDx40syFDhphZeXm5mfXv3z/t8by8PHPMsmXLzMzTrJn9bgVjxowxs//+7/9OdH3Z2dmJ5uLdBl26dDGzEydOpD3eunVrc3jJZU4AACAASURBVMzZs2fNzJOTk2Nmp0+fNrOsrCwz8+6XS5YsMbMWLVqYmXVOREQ2bdpkZt5t161bt7THDx8+bI7x6m+98+U9Dtq0aWNmLVu2NLOjR4+amXe7tmrVysy8t3c4efKkmV1xRfrXGLy3hNi/f7+ZeQ4cOGBmixYtMrMvfOELZnbfffdVa53ilRQAAAAAUWGTAgAAACAqbFIAAAAARIVNCgAAAICosEkBAAAAEBU2KQAAAACiUt8VxAdEZHvqwzwRsXsZmybOSXqcl4txTi7GOanUO4TQuaEncblinaoWzsvFOCcX45xcjHNSqVrrVL1uUv7kilVX0OX/pzgn6XFeLsY5uRjnBJnGfSo9zsvFOCcX45xcjHNSM/y6FwAAAICosEkBAAAAEJWG3KQ83oDXHSvOSXqcl4txTi7GOUGmcZ9Kj/NyMc7JxTgnF+Oc1ECD/U0KAAAAAKTDr3sBAAAAiEqDbFJU9ROqul5VN6nqPzXEHBqaqv6Hqu5X1XeqHOuoqi+r6sbU/zs05Bzrm6r2VNXFqrpWVd9V1XtSx5vseVHVbFV9U1VXp87Jg6njfVX1jdRjaK6qtmjoudY3Vc1S1bdU9fnUx03+nCBzWKdYp9JhnboY65SNdap26n2ToqpZIvJjESkWkcEiMlVVB9f3PCLwSxH5xAXH/klEXgkhDBCRV1IfNyXlIvKVEMJgERklInen7htN+bycFpGPhRCGi8g1IvIJVR0lIt8VkYdDCP1F5LCI3NmAc2wo94jIe1U+5pwgI1in/scvhXXqQqxTF2OdsrFO1UJDvJJynYhsCiFsCSGcEZFfi8iEBphHgwohvCYihy44PEFEZqX+PUtEJtbrpBpYCOH9EMKq1L+PSeUDu4c04fMSKpWlPmye+i+IyMdE5Lep403qnIiIqGq+iNwiIr9IfazSxM8JMop1Slin0mGduhjrVHqsU7XXEJuUHiKys8rHu1LHINI1hPB+6t97RaRrQ06mIalqHxG5VkTekCZ+XlIvF/9RRPaLyMsisllEjoQQylOf0hQfQ4+IyD+KyLnUx52Ec4LMYZ2yNenn46pYp/4X61RarFO1xB/ORypU1q41yeo1Vc0RkadF5N4QQmnVrCmelxBCRQjhGhHJl8qf8BY08JQalKp+UkT2hxBWNvRcgKasKT4fn8c69adYp/4U61RmNGuA69wtIj2rfJyfOgaRfap6ZQjhfVW9Uip/ItGkqGpzqXzinx1C+F3qcJM/LyIiIYQjqrpYREaLSK6qNkv9RKapPYY+LCJ/oao3i0i2iLQTkUelaZ8TZBbrlK3JPx+zTtlYp/4H61QGNMQrKctFZECq4aCFiPy1iDzbAPOI0bMiMj317+kisqAB51LvUr+v+YSIvBdC+EGVqMmeF1XtrKq5qX+3EpHxUvk70ItF5C9Tn9akzkkI4Z9DCPkhhD5S+fzx+xDCNGnC5wQZxzpla7LPxyKsU+mwTl2MdSozGuTNHFM7y0dEJEtE/iOE8O16n0QDU9U5IvJREckTkX0i8nURmS8ivxGRXiKyXUT+KoRw4R8tNlqqOkZEXheRNfK/v8N5n1T+vm+TPC+qerVU/nFdllT+UOE3IYRvqmo/qfxj3o4i8paI/D8hhNMNN9OGoaofFZG/DyF8knOCTGKdYp1Kh3XqYqxTPtap5HjHeQAAAABR4Q/nAQAAAESFTQoAAACAqLBJAQAAABAVNikAAAAAosImBQAAAEBU2KQAAAAAiAqbFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKLCJgUAAABAVNikAAAAAIgKmxQAAAAAUWGTAgAAACAqbFIAAAAARIVNCgAAAICosEkBAAAAEBU2KQAAAACiwiYFAAAAQFTYpAAAAACICpsUAAAAAFFhkwIAAAAgKmxSAAAAAESFTQoAAACAqLBJAQAAABAVNikAAAAAosImBQAAAEBU2KQAAAAAiAqbFAAAAABRYZMCAAAAICpsUgAAAABEhU0KAAAAgKiwSQEAAAAQFTYpAAAAAKLCJgUAAABAVNikAA1MVT+rqiUNPQ8AQFxU9Ruq+qsGuN4+qhpUtVl9XzdwHpsURE9Vx6jqElU9qqqHVPUPqjoyldX6G/xMPBlrpe+q6gep/76rqlol/5iqrlLVUlXdoqozajNnAECcVHWbqp5U1bIq//2ogebyaVXdrqrHVXW+qnaskvVR1RdU9bCq7lXVH7EpQUzYpCBqqtpORJ4XkcdEpKOI9BCRB0XkdIYuP1NPyDNEZKKIDBeRq0XkVhH529R1NBeRZ0TkZyLSXkSmiMgPVHV4hq4bABCXW0MIOVX++0J9T0BVh0jluvMZEekqIidE5CdVPuUnIrJfRK4UkWtE5CMi8vl6niZgYpOC2A0UEQkhzAkhVIQQToYQXgohvK2qV4nIT0VkdOonVUdERFT1FlV9K/WqxU5V/cb5C6vyqsmdqrpDRH4vIq+l4iOpyxmdYJ7TReShEMKuEMJuEXlIRD6byjqKSDsReSpUWi4i74nI4KoXoKr/lvqJ1lZVLU4wBwBAxM6/+m8936tqX1V9VVWPqerLIpJXi6ubJiLPhRBeCyGUicgDInKbqrZN5X1F5DchhFMhhL0i8l8iMuTCy1DVHap6UFXvr8VcgBpjk4LYbRCRClWdparFqtrhfBBCeE9E/k5ElqZ+UpWbio6LyO0ikivy/7d37+FVVnca93+rQDhDOBVQ0LQCIgooh4InwFMFpFJaqkgd8LUMU6fDWOtLq+0lKtQL+1q0HabFodQanbZUKVRrpWoRwVFCEasiqICKAgJC5IycZL1/JM4g7PtH8mTv5Eny/VyXl/DcWXuv/eTZe2Wxkzt2hZndEEL46jG3O9DMzjCzy81sQOmx/NLbWXLsJEq/5WyHM88zzezVo/7+aukxizFuMbPfm9n/E0KoU7oJOtXMjv42tX5m9paVLEj/n5n9+uhvFwMA1Bje6/3vzGx5aTbFSv4BTAoh7AghXCDiz6xLMca3zeyglf7jn5n9zMxGhRAahRBONrMhVrJROdoFZna6mV1iZpNK/3EQqBRsUpBqMcZdVvIiGc3sV2a2NYTweAihrTPmuRjjihjjkRjja1ayQRh4zIfdEWPcG2P8uIzz+J+jNkGZNDGznUf9faeZNTlq4fm9mU2ykm9Te97MfhRjXH/Ux78XY/xVjPETMyu0krff5WMEAKTan0o3EJ/+989HZRlf70MIp5hZXzO7LcZ4IMa42Mz+7N1JjDE/xqh+LvPYdclK//7pOymLrWQjs8vMNpjZS2b2p2M+/s7S72B41Uo2PHybMioNmxSkXozxjRjjdTHGDmZ2lpmdZCX/ApRRCKFfCGFhCGFrCGGnlbzbcuxb5uszDK2IPVbyLV2famZme2KMMYTQ1cxmW8m7O3lWsih8P4RwxVEfv/nTP8QY95X+sUmW5wgAqBxfLd1AfPrfr47K1Ov9SWa2Pca496iPfa8Cczh2XbLSv+8OIXzOSt41mWtmja1kjWxhZj855uM3H/Xnfca6hErEJgXVSozxTTN70Eo2K2Yl77Ac63dm9riZdYwxNreSn1s59lunovhzUivts//C1LP0mFnJXFfHGJ8qfXfnLTP7i5W8tQ4AgJnZJjNrEUJofNSxUypwe59Zl0IIXzSz+lbybdQtS2/7P0vftSk2s9+Y2dAK3B+QVWxSkGohhK4hhJtDCB1K/97RzK4xs6LSD9liZh1CCHlHDWtqZh/FGPeHEL5kZqNPcDdbzeyImX2xAlN9yMy+F0I4OYRwkpndbCWbKTOzf5hZ59Ia4hBCOM3MhpnZaxW4PwBADRJjfM9KvuXqzhBCXunPmnylAjf5WzP7SgjhwtKNz2Qzmxtj3B1j3GZm71rJz2zWDSHkW8nPv7AuITXYpCDtdlvJDxkuDSHstZLNyetWsgkwK2nnWmlmm0MI20qP/auZTQ4h7LaSnwN5xLuD0rfb7zKzF0q/d7j/sR9T+iK/x7mZ/7KS7x1eUTq/v5Qe+/SHFa83s/+wku/9XWRmfzSzWf5DBwBUU38+5vekzCvjuNFWsuZ9ZGa3W8k/gEmlt31hpizGuNJKvt35t1ZSNdzUPlsx/DUzG2wl/1C31swOmdlNZZwnkHMhxmx8pwsAAAAAZAfvpAAAAABIFTYpAAAAAFKFTQoAAACAVGGTAgAAACBV2KQAAAAASJW6FRkcQhhsZj83szpmNivGePcJPp4qMQDIrW0xxjZVPYm0KO861bBhw9i8efOM2e7du+W4ffv2ySypvLy8E3/QMQ4ePJj1eeSC99iqy2PItpp+TtTjy8/Pl2MOHTokswMHDsjs8OHDMqtbN9mXvt5tNm7cWGZ79+6Vmfd5re7XQ5MmTWS2Z8+eMq1TiTcpIYQ6ZvYLM7vMzDaY2bIQwuMxxlVJbxMAUGHvVfUE0iLJOtW8eXMbM2ZMxmzBggXyvl5++eWKTTaD9u3bl3vMe+9Vj0+/99iqy2PItpp+TtTju/LKK+WYrVu3ymz16tUyKy4ullmrVq1k5vFu89xzz5XZkiVLZOZ9Xqv79dC7d2+ZLVq0qEwPoCLf7vUlM1sbY3wnxnjQzGab2fAK3B4AANnEOgUA1VRFNiknm9n6o/6+ofTYZ4QQxocQXgohvFSB+wIAoLzKvU59/PHHlTY5AICW8x+cjzHOjDH2iTH2yfV9AQBQXkevUw0bNqzq6QAArGKblI1m1vGov3coPQYAQBqwTgFANVWRTcoyM+scQvhCCCHPzEaZ2ePZmRYAABXGOgUA1VSIMXkrcAhhqJn9zEqqHR+IMd51go+nghgAcms53177f1inACB1yrROVWiTUl68+ANAzrFJqQDWKQDIuTKtU/zGeQAAAACpwiYFAAAAQKqwSQEAAACQKmxSAAAAAKQKmxQAAAAAqVK3qicAoGr0799fZkVFRZU4EyA9mjRpYr17986YLVq0SI479dRTZfbee+9VeF4APmvgwIEy856rqBzZeE3knRQAAAAAqcImBQAAAECqsEkBAAAAkCpsUgAAAACkCpsUAAAAAKnCJgUAAABAqlBBDNRSHTp0kNnNN98ss2nTpuViOkAqfO5zn7NGjRplzHr16iXHvfzyy1mfC7XGtUtlX1/VgfccoGY43bLxGsU7KQAAAABShU0KAAAAgFRhkwIAAAAgVdikAAAAAEgVNikAAAAAUoVNCgAAAIBUoYIYqKX27t0rs61bt1biTID02LVrl82fPz9j5tWh5gI1w7VLba0Z9gwdOlRmhw4dktmsWbNyMR1UMt5JAQAAAJAqbFIAAAAApAqbFAAAAACpwiYFAAAAQKqwSQEAAACQKmxSAAAAAKQKFcRAJRszZozMHnrooUS32b9//4zHe/ToIcesX79eZqNHj5bZvn37ZDZnzhyZTZgwQWbLli2TWVFRkcyAbGvUqJF17do1Y0ZFLJAezzzzTNZv06sZT1MluJpndZijWdnnyTspAAAAAFKFTQoAAACAVGGTAgAAACBV2KQAAAAASBU2KQAAAABShU0KAAAAgFShghioZElrhj0tWrTIeLxjx46Jbm/dunUy27Bhg8weeOABmT355JMyo2YYadG4cWPr169fxsyrIO7Vq5fMqC4GtBEjRsjMW4u6desms4KCApktWrQo0W3mot7Xe90oLi6u1LlkWzbmyDspAAAAAFKFTQoAAACAVGGTAgAAACBV2KQAAAAASBU2KQAAAABShXYvIEXGjBkjs+bNm5f79urW1U/xVq1ayexvf/ubzLp06SIzr6WrZcuWMgPS4vDhw7Z9+/aMWXVv8Bo4cKDMvMajynbqqafKrDq0GtV0XhuX9zwYNGhQxuONGjWSY1atWiUzr/nLa/e65ZZbZLZkyRKZ5ULS1w31HKlpzw/eSQEAAACQKmxSAAAAAKQKmxQAAAAAqcImBQAAAECqsEkBAAAAkCpsUgAAAACkChXEQIqcccYZMvvggw9kduDAgYzHly9fLse0b99eZnv37k2U9e3bV2a/+93vZAakxUknnWR33HFHxuzWW2+V46pDBfHu3btllqZ65ZpWo1qbdOvWTWaFhYUZj48dO1aO6dGjR6Js5MiRMvvZz34mM+85UtmqexV3NirPeScFAAAAQKqwSQEAAACQKmxSAAAAAKQKmxQAAAAAqcImBQAAAECqsEkBAAAAkCpUEKNK9O/fX2ZFRUWVOJPqo0WLFjLbvHlzxuNeXfApp5ySaB55eXkyW7VqlcxGjx6daNycOXPKNjEgC9asWWOXX355Vm8zLVWiSauEk84/6TivutSriB0xYoTMbrvtNpklmUdZK1Rrov3798ts/vz5MhsyZEjG43369JFjnnzySZl560bjxo1l5lFzNKseVdzec85TUFAgs6TX+rp16xKNOxrvpAAAAABIFTYpAAAAAFKFTQoAAACAVGGTAgAAACBV2KQAAAAASBU2KQAAAABS5YQVxCGEB8xsmJl9GGM8q/RYSzP7g5kVmNk6M7sqxrg9d9NEdTVp0qSMx5ctW1bJM0kPr365devWMnvqqadkpqqGvTpFryry7LPPltnGjRtl1rNnT5m9+OKLMtu+PdnLR6dOnWS2du3aRLeJ6qey1imvEjTpcy3bevXqJbOkFapJa5Jzcb68muFZs2aVbWJl5FWvpqVWOle8x+d9fkaNGiWzNm3aZDyetGZ40KBBMvPq97t37y4zr+I6Ke9cduvWTWZbtmyRWXFxccbjSSvBc1GpnY3nQVneSXnQzAYfc+wWM1sQY+xsZgtK/w4AQFV40FinAKBGOeEmJca42Mw+OubwcDMrLP1zoZl9NcvzAgCgTFinAKDmSfozKW1jjJtK/7zZzNpmaT4AAGQD6xQAVGMn/JmUE4kxxhBCVHkIYbyZja/o/QAAkER51qk6depU2rwAAFrSd1K2hBDam5mV/v9D9YExxpkxxj4xxj4J7wsAgPJKtE6xSQGAdEi6SXnczMaW/nmsmT2WnekAAJAVrFMAUI2VpYL492Y2yMxahxA2mNntZna3mT0SQviWmb1nZlflcpKovlQNoFdhePPNN8ts2rRpFZ5TVSsqKpLZ5MmTZebVB+bn52c8fuDAATlmzZo1MvN49cQffij/sdo6duwos6QVrdQMwyy761SDBg1kLahXqenVhVYmr2Y4F/XESSV9zs+bNy/LMzG76aabMh6fO3euHFMTaoY93uMbOHCgzHbu3Cmzq6++OuNx73Xcm0dhYaHMPN7XEd4anNRll10mM+95p2qGzZJdf9Xxmj3hJiXGeI2ILsnyXAAAKDfWKQCoefiN8wAAAABShU0KAAAAgFRhkwIAAAAgVdikAAAAAEgVNikAAAAAUiXEKH8Jb/bvzPmNv6iZHn300YzHvWq9n/70pzKr6ZWzXv1y586dZfbiiy9mPH7ttdfKMcuXL5fZ4cOHZbZ+/XqZ1a9fX2bTp0+XGbJqOb88NznWqezxatPTVIfqVeoqixYtysFMqgevynrYsGEyW7ZsWcbj+/btk2PWrVtX5nkdTdWIn+j+vHEzZsxINBdPtp8jQ4YMkVnS2u+kTlB5XqZ1indSAAAAAKQKmxQAAAAAqcImBQAAAECqsEkBAAAAkCpsUgAAAACkCpsUAAAAAKlCBTFyasyYMeUes3r1apkVFRVVZDqpd+utt8qsXr16Mmvbtm3G4wcOHJBjPvnkE5ktXbpUZtdff73Mnn/+eZl5Fi5cKLNt27bJrKZXUidEBXEF1K9fP7Zv3z5jlqba3LQ4Qc1oJc4EaeDVOV9wwQUZj69atUqOadasmcy8NXHjxo0y865Zb13ctGmTzFasWCEz73lQi58/VBADAAAAqH7YpAAAAABIFTYpAAAAAFKFTQoAAACAVGGTAgAAACBV2KQAAAAASJW6VT0BVH9TpkyRWZs2bTIe3717txyzb9++Cs+puvLqFhcvXiyzAQMGZDzuVRp7mWflypUy86qEr7jiCplddNFFMps6dWrZJgZUoVNPPVVmlVldnLTSNOn8c1GTmota1nHjxslMPfZ58+ZlfR41gXetJFnbvfM8ZMgQmW3ZskVm3bt3l5lXedyuXTuZPffcczJL+hxPy3WUltevY/FOCgAAAIBUYZMCAAAAIFXYpAAAAABIFTYpAAAAAFKFTQoAAACAVGGTAgAAACBVQoyx8u4shMq7M2RVp06dZPaDH/xAZuvXr894/Omnn5ZjOnToILM5c+bILE282kRVF2xm1qVLF5lt3LhRZqqeuGfPnnLMmjVrZFZQUCCz/Px8manKaTOzDRs2yKx+/foy27t3r8x27NghM682sbpcRwktjzH2qepJVFdNmzaNvXv3zpgtWrSokmejqcrQpHWhXgWppyrrSY81YsQImXk1t3fffXfG417F7dy5c2WWpnOSVNIa6AkTJsisQYMGGY8vWLBAjikuLpbZoEGDZNaoUSOZrVu3TmZdu3aVmVev7J0T7zF4klxHaaoSPsFcyrRO8U4KAAAAgFRhkwIAAAAgVdikAAAAAEgVNikAAAAAUoVNCgAAAIBUYZMCAAAAIFXqVvUEUD20bt1aZtu2bSv3uL59+8oxO3fuLPvEjuLVJHu8x3bRRRfJbOrUqTKbP3++zBo3biyz9u3by+zss8+WmfocqAroE1m1apXMOnfuLDOvJvngwYMymz59usymTJkiM2+e3ucAUPbs2ZOqqmEl23Wiuagnrew6VK8GduzYsTJbu3ZtxuPemuLV3xYWFsosF7zH5s3Fq8r33HDDDTJ77bXXZHbttddmPP7666/LMV59r7deejXD3rrhfW3i1U5feeWVMvPWt2xr1aqVzCq7gjgb98c7KQAAAABShU0KAAAAgFRhkwIAAAAgVdikAAAAAEgVNikAAAAAUoVNCgAAAIBUoYIYZTJ8+HCZ1a9fX2YtWrTIeLxr165yzMMPPyyz/v37y6xDhw4yGzZsmMy2bt0qs4kTJ8rM49XmrlmzRmbt2rWTmVebqOoWu3fvLse8//77MmvQoIHMvErI/fv3y6ygoEBmSeujgWzLz8+X1ePz5s2r5NlUb5VdeTp06FCZeZW0F154YcbjM2bMkGN69epV5nkdzatl9upjvTVs8uTJieaStKbde+z9+vWT2ebNmzMeHzhwYKJ5/P73v5dZ0s/P9u3bZeZdz96a6c3FW0+9+1PnzLvOczGPXOOdFAAAAACpwiYFAAAAQKqwSQEAAACQKmxSAAAAAKQKmxQAAAAAqcImBQAAAECqUEGMMvn4449ltm3bNpl169Yt43GvJq+oqEhmN998s8zy8vJk9tZbb8ls6tSpMhsyZIjMvPrGv/zlLzJT9aZmZvXq1ZPZBx98ILPOnTtnPO7VMj/77LMy8zRu3FhmPXv2lNmhQ4dk5lU279y5U2bq+jLz5zlnzhyZoXbbsWMHVcPlkLTq9eWXX87yTPzKYK/6V9W5XnvttXLMhg0bEt3XNddcI7O7775bZrk4X948vdrZSy65RGZr166Vmfr8JK003rJli8w++ugjmX3ta1+T2ZtvvimzpLzz7H1evXEtW7Ys1/ET3VdV1gx7eCcFAAAAQKqwSQEAAACQKmxSAAAAAKQKmxQAAAAAqcImBQAAAECq0O6FMnn66adldtVVV8ls69atGY977VcjR46U2eWXXy6z119/XWYvvviizDp16iQzr8Grf//+MuvRo4fMFi5cKDOv+athw4YyO3z4cMbj6vyb+Y9tzJgxMmvTpo3MPJs3b5aZ1/zl3d+qVatk5j0+QMnLy7P27dtnzLwGnFGjRsls9erVMstFa1Nl8uaftPkrF1q1aiWzRx55JOPx4uJiOca7FrxWSG8tVS1jZmaLFi2SWVLeYxg3bpzMvLV237595Z6H1+DlufPOO2XmNThu2rRJZieffLLMbrrpJpll+5yY+Z8f1WzpfT2TtMHLG5e0Ia6seCcFAAAAQKqwSQEAAACQKmxSAAAAAKQKmxQAAAAAqcImBQAAAECqsEkBAAAAkConrCAOIXQ0s4fMrK2ZRTObGWP8eQihpZn9wcwKzGydmV0VY9yeu6mirLwKX68ubtq0aTLzKnXr168vsyZNmmQ8Pnz4cDlm8eLFMvvv//5vmXlVtV4doVfZ5ykqKpLZ6NGjZXbeeefJzKsMbtGihcxUxXJBQYEc410nXmWiV9G4ceNGmfXs2VNm9913n8y+8Y1vyKxx48YyQ+2RzXXq4MGDiaozlyxZUu4xNV3SeuVc1Jp69b6qknb27NlyjFc57VUQjx07NtE475x49crdu3eX2a5du2TmVd57a4DnW9/6Vsbja9eulWOWLVsmM28NVjXiZmY7d+6Umcf7WqdBgwYy86ryk1IV+9514kn6vPKuvWzcX1neSTlsZjfHGLuZWX8z+04IoZuZ3WJmC2KMnc1sQenfAQCobKxTAFDDnHCTEmPcFGN8ufTPu83sDTM72cyGm1lh6YcVmtlXczVJAAAU1ikAqHnK9TMpIYQCMzvHzJaaWdsY46fvkW62krfZAQCoMqxTAFAznPBnUj4VQmhiZn80s+/GGHeFEP43izHGEEIU48ab2fiKThQAAA/rFADUHGV6JyWEUM9KXvh/G2OcW3p4SwihfWne3sw+zDQ2xjgzxtgnxtgnGxMGAOBYrFMAULOccJMSSv4p6tdm9kaM8d6josfN7NOairFm9lj2pwcAgI91CgBqnhBjxne//+8DQrjAzJ43sxVmdqT08A+t5Pt9HzGzU8zsPSupdvzoBLfl3xmy4t5775XZK6+8IrPVq1fLrEuXLjLr3LmzzM4+++yMxwsLCzMeN/Pn6FX7ejV/jRo1kplXHfjss8/KLMnjNjOrV6+ezLy6yB07dshMPYY1a9bIMZs3b5ZZu3btZPb000/L7KqrrpLZm2++KTOvxtC7HlQNYy23vLa9I8A6Vf14ValPPfWUr+QW0QAAIABJREFUzJ555hmZ/eY3v5FZr169ZKaq072qXW8eSetcPV6Fsre+nXXWWTLzqn+9ul2vmnnChAkyU7XAzz//vBzTvHlzmZ122mky8+qvu3btKrNFixbJzPs6yKsgz8X1oJ4/ubivHCnTOnXCn0mJMf6PmQURX1LeWQEAkE2sUwBQ8/Ab5wEAAACkCpsUAAAAAKnCJgUAAABAqrBJAQAAAJAqbFIAAAAApEqZf+N8TXPzzTfLbPfu3RmPz5w5U47p37+/zIqKiso+sSx45JFHZNa3b1+ZefV6BQUFMqtbV19Ge/bsyXjcqylcsGCBzCZPniyzKVOmyMyr7/VqJr0KYq/e18vOO+88mXm2b98usxYtWpR7zPr162XmXSfe5+7IkSMyO3DggMwOHz4sszZt2sgMqExeja1XeerxqnjPPfdcmanK0+pSQeqtKd6vRrj88stl9s4778jMq5ZVnwNVTWzmXwu5+BxceumlMlu5cqXMPvww4+8vNTOz7373uzJ74YUXZLZz506ZbdiwQWaqgthbb7xfEeBVVXufb+/a857H3uc86fXgPf+9cdm+xpLOw5ON10veSQEAAACQKmxSAAAAAKQKmxQAAAAAqcImBQAAAECqsEkBAAAAkCpsUgAAAACkSo2uIB45cqTMVM2wmVmrVq0yHh8yZIgc41XOetW4hYWFMlu7dq3MPEkrj88//3yZeVW2Z5xxhsxUPfGvf/1rOWbfvn0y83g1w2eeeabMVE2ymVnnzp1l1q1bN5l5lX0PPfSQzLwq6yuuuEJm6pxt2rRJjrnuuutk5tU+7tq1S2aHDh2S2Ze+9CWZ/ehHP5JZZVd4o3bLy8uTValJa4Y93uuI99rao0ePjMeff/55OWb+/Pky89Y3b1xSXkXs4MGDZebVmqq128ysX79+MlOvn97ruHd7Hm/+TZs2lZlXlX/jjTfKbM6cOTL76KOPZHbaaafJbOvWrTJr2bKlzFRFdKdOneQY72s17768St127drJLGltbtLXhrRUhudiHsXFxRW+Dd5JAQAAAJAqbFIAAAAApAqbFAAAAACpwiYFAAAAQKqwSQEAAACQKmxSAAAAAKRKja4gfuWVV2T2r//6rzJ79913Mx6/9tpr5ZhvfvObMvOqCjt27CgzryI2Ka/Oddu2bTK7/fbbZXbKKafI7K233sp43Dsnr776qsy8il6vqrB58+YyW7lypcy8aum+ffvKzONVY3uVil27di33fXnz987JqlWrZObVTw4bNkxmc+fOlVmXLl1kVlsriB999FGZedf69ddfn4vp1BoHDx6s1FrQs846S2br16+XmXpuexWqXpWwV2PryUU9sXf+vcyrUfaq7VXltFfL7/Hqbz1eLbP3qwxijDIbOnSozB588EGZeXXy3nn+7W9/KzOvvl7xqn3Hjh1b7tszM6tXr57MvM9dWuqC08Q7X14leFnPJe+kAAAAAEgVNikAAAAAUoVNCgAAAIBUYZMCAAAAIFXYpAAAAABIFTYpAAAAAFKlRlcQe/V0Xj3xQw89lPH4zp075RivHvDw4cMyO/PMM2X2hz/8QWZXX321zJLyapk/9zm9n/XqalWlbv369eWY1157TWYtWrSQ2SWXXCKzxx57TGZ9+vSRmVe1uHfvXpl5vHFenV9eXp7MDh48mPG4Vz/59ttvy+yRRx6R2fDhw2X27LPPysw7z5s2bZKZej7miqq59qqQvWps75odP368zLZs2SIzVZmKdPLWIq/m1qukXbduXcbj3towe/ZsmS1YsEBmo0ePltngwYNllrTWOCmvVr1BgwYyU5X4ixcvlmO85+7JJ58ss6Q1w23atJGZ91rx/vvvy8z7FQje/c2bN09mK1askJmqqO/du7cc471+evXE3tcz3mu5V43btm1bmVV33mvUc889JzPvfBUUFFRgRiV4JwUAAABAqrBJAQAAAJAqbFIAAAAApAqbFAAAAACpwiYFAAAAQKqwSQEAAACQKiHGWHl3FkLl3dkJTJo0SWbdu3fPeNyrSfWqCjt37iyza665RmZeZeL69etl9i//8i8yW7t2rcy8ut3p06fL7NVXX5WZqjFcunSpHHPKKafIbPv27TLr0aOHzDZu3Cgzr8bQG7d69WqZdejQQWYbNmyQ2Y9+9COZqTpnM7N9+/ZlPL5nzx45ZsSIETJLyqvQ/OIXvyizunV1G7qXff3rXy/bxMph5MiRGY97lZbedeJds//+7/8us1tvvVVmCxculFlRUdHyGKPue4YrTetUklrQVq1aJbq9HTt2yMyroL/wwgtlNmjQIJkl5dW0e499woQJMlNVvG+88Uai+/Lq3WfNmiUzr87VW7u92v6f//znMjv33HNl5tWc5+fny8yruVZVyV/+8pflGK9Gf8CAATLzeGufqkk2M5szZ47M5s+fn2guEydOlNk999wjM/UYevXqJcesXLlSZgcOHJCZ9zWSd816z9X33nuvTOsU76QAAAAASBU2KQAAAABShU0KAAAAgFRhkwIAAAAgVdikAAAAAEgVNikAAAAAUkV3e9ZwkydPllmnTp0yHh89erQcc+mll8osLy9PZl61W/PmzRNlXq2pl3kVek8++aTMzj//fJmpx+fVN6qaQjOzw4cPy8w7J169ZqNGjWR22WWXyeyhhx6Smcerlq1Tp47MQggy27ZtW8bj3ufUq+H2eM+dQ4cOyeyDDz6QWf369WXmVRyOGTNGZqr++kS32a9fv4zHvevSqxlWnxszXXdsZlavXj2ZtWjRQmaomLy8PFm/6l03ubBixQqZXXXVVRmPe+vNunXrZLZo0SKZebWs999/v8y8SvJ58+bJLGnlqad169YyU89Rb71ZsmSJzF544QWZeTXQ3murV3/bsGFDme3evVtmd999t8xuuukmmXnXSnFxscyuvPLKcs/Du2a9X5vgrX379++Xmfc5UPM38+t2PV61tEc9/71fazFq1CiZeXXO3nP1l7/8pcy86vJvf/vbMvvMbZTpowAAAACgkrBJAQAAAJAqbFIAAAAApAqbFAAAAACpwiYFAAAAQKqwSQEAAACQKiHGWHl3FkLl3VkOeHWhF110kcx69+4tsxdffFFmX/nKV2Tm1fJ98sknMhs8eLDMkho/frzMOnfunPF4QUGBHNO4cWOZeVWvXtVigwYNZObV5nr1sV42bdo0mT366KMy6969u8z27t0rs6lTp2Y8vmHDBjnmBz/4gcy8z4F3m978f/KTn8jMq2/cunWrzFRdsJnZO++8I7ODBw/KTF2zXqXxwoULZebVXe7bt09m3nkuKiqSmZktjzH28T4AWmWvU96179XtqorY++67T44ZMWKEzJo1ayazwsJCmQ0cOFBmXr37hRdeKDOvbterJ/Z4dbVdu3bNeLxly5ZyjPd63KtXL5l5deuvvPKKzPbs2SMzr8bWq4/1eNeKd122bdtWZl4tsOJV3Obn58vMq+xPeg1559k7J02bNpWZV+fsUa8b3bp1k2O86/Kuu+7K6jzM/F/hMGvWrDKtU7yTAgAAACBV2KQAAAAASBU2KQAAAABShU0KAAAAgFRhkwIAAAAgVdikAAAAAEiVulU9geqkZ8+eMvNqf9evXy8zVSNp5tc+Dho0SGZLliyRmaqqNfOr64YPHy6zmTNnyiwJb46tW7eWmVf1/Nxzz8nMe9z//M//LDNvnr/4xS9ktmnTJpl51b/Lly+XmVfZqbRr105mXkWjV1XqzXHAgAEy8+obvRrDzZs3y8yrgZ40aZLMdu7cmfG497gvv/xymc2aNUtmSSo5kVt5eXnWvn37jJl3nSaV9DZV1fCoUaPkmNmzZ8vMW4s8Xh2+Z8uWLTLzKlsfeughmY0ZM0Zm3nMtyfPQO8+nn366zLxK8jfeeENmffv2ldk999wjM68i9o477pDZU089JbPi4mKZeRXESaxYsUJmXo2197zyqnj//ve/y8yrj+7SpUuicV4F8e9+9zuZPf/88xmPN2/eXI5JWr2ctCY9G3gnBQAAAECqsEkBAAAAkCpsUgAAAACkCpsUAAAAAKnCJgUAAABAqpxwkxJCaBBC+HsI4dUQwsoQwp2lx78QQlgaQlgbQvhDCCEv99MFAOCzWKcAoOYpSwXxATO7OMa4J4RQz8z+J4Qw38y+Z2b3xRhnhxDuN7NvmdmMHM61ynm1dYcPH5bZvn37ZPb666/L7LzzzpPZ1q1bZfa5z+m9p1dxqGo3zczuvfdemf3yl7+U2dq1a2Wm3HrrrTJ78MEHZbZ7926ZrVmzRmbdu3eX2ZAhQ2Tmfc69quS9e/fKbPLkyTIrKiqSmare9Oo6zz33XJndf//9MmvSpEmi2/Qe94IFC2S2fft2mU2cOFFmEyZMSHSbqip5x44dcox3LVAzXCmytk41atRIvtZ7dZvjxo2TmVdDnW3e89Nbw+bOnSuzG264IdFcktYTe2vf5z//eZl5la0//elPZZakmtWrc/YqZ706/P/6r/+S2dKlS2XmfV69zFufhw4dKjOvbtdbw5KYMUM/Xb3Mq81t1apVorl8+ctflplXEf31r3890f09/vjjMmvTpk3G496vMUgqac3woUOHKnzfJ3wnJZbYU/rXeqX/RTO72Mw+/eUMhWb21QrPBgCAcmKdAoCap0w/kxJCqBNCeMXMPjSzZ8zsbTPbEWP89J8PN5jZybmZIgAAPtYpAKhZyrRJiTF+EmM828w6mNmXzKxrWe8ghDA+hPBSCOGlhHMEAMCVrXXqwIEDOZsjAKDsytXuFWPcYWYLzexcM8sPIXz6My0dzGyjGDMzxtgnxtinQjMFAOAEKrpOeT9PAACoPGVp92oTQsgv/XNDM7vMzN6wkkVgZOmHjTWzx3I1SQAAFNYpAKh5ytLu1d7MCkMIdaxkU/NIjPGJEMIqM5sdQvixmf3DzH6dw3kCAKCwTgFADXPCTUqM8TUzOyfD8Xes5Pt+a42//vWvMuvRo4fM+vXrJ7ODBw/K7MiRIzJbvXq1zIqLixONu+KKK2R29dVXy8z7Hu5333034/HXXntNjtm2bZvMrrvuOplNmjRJZuvXr5eZd068+liv1viHP/yhzJo3by4z77H3799fZqqK9+KLL5ZjPM8++6zMvFrjqVOnykxV+5rp68TMbPr06TJL6gtf+ILMVIV3fn6+HOPVQyP3srlOhRBkhaxXSf7MM8+U527KxKuPbdu2bcbjXt2xV9ndokULmXnVuF/72tdkdumll8rMWzN/8IMfyOykk06SmbcW3XjjjTIbNWpUxuPeWurVrRcWFsosaZ2z91p+++23y8yrV1aP28xf+7x6X88tt9yS8bj39cVPfvITmXlrt/e4vefVRx99JLOnn35aZnfffbfMPPfcc4/MvGt98+bNGY97z4GuXfWP6SWtyveuhRUrViS6zaPxG+cBAAAApAqbFAAAAACpwiYFAAAAQKqwSQEAAACQKmxSAAAAAKRKiDFW3p2FUHl3lgMjR46UWcuWLWXmNUk0btxYZtu3b5fZzp07ZbZjxw6ZeU0sXqOT11B26NAhmf3mN7+RmdK5c2eZee1RXktGo0aNZFZQUCCzKVOmyMwzevRomS1evFhmc+bMkdl//Md/yKxhw4YZjz/11FNyzNChQ2X28ccfy+zhhx+WWWW3XE2YMEFmXlvR5MmTZfaHP/wh43GvgSZllvPLc5Nr2bJlvPzyyzNmS5YskePee++9RPfnvcbcdtttiW5T8drJvNdBr5GwW7duMvvggw9k5rVXdu/eXWbLli2Tmdc01KRJE5mppjSvgcxb85P6+c9/LjOvrerDDz+UmdfaNHDgwLJN7BiLFi2SmdcYpprN2rdvL8fUratLaDdt2iSz888/X2aVbdy4cTLzvq4cPHiwzH75y19mPP7+++/LMUkbyHKkTOsU76QAAAAASBU2KQAAAABShU0KAAAAgFRhkwIAAAAgVdikAAAAAEgVNikAAAAAUoUK4nLwquK8OlfPunXrZOZV+3pVi15dpFczuXbtWpl5dch5eXkyW7NmTbnHeNWOEydOlFlSDzzwgMzOOeccmf31r3+V2WOPPSYzr6b3V7/6lcy8z922bdsyHveuBa/+2qu03Lt3r8yuu+46mY0fP15mTZs2lZlX7erVn6paZjOzW2+9VWadOnXKePziiy+WY2bOnCmzKkAFcQV4FcRe5enu3btl5tXHerzXeSVpFbJXHbt69WqZeTW23vPaq7G98cYbZea9/px++ukye+KJJ2SmqpK//e1vyzH79u2T2YABA2SWlKrvNTN78sknZeatG97nIKlnnnlGZocPH854/ODBg3LMnj17ZOatU14V73333SezpEaMGCGzf/u3f5PZW2+9JTNVh2+m67FfeOEFOSYp79dozJo1K+nNUkEMAAAAoPphkwIAAAAgVdikAAAAAEgVNikAAAAAUoVNCgAAAIBUYZMCAAAAIFXqVvUEqpOePXsmGrdx40aZde/eXWZe1aJXp9ioUSOZeTWqP/7xj2V2++23y2zXrl0yGzx4cMbjb7zxhhxTp04dmU2ZMkVmnttuu01m119/faLbTOree++VWe/evWXm1UAfOXKk3PNQtZtmZgcOHJDZihUryn1fZmYXXnihzBo0aCCz/Px8mXkVlKNHj5aZqhk207XgmzdvlmNQcxw6dMitGla8ul2PVzOcpE54yJAhMtuyZYvMZs+eLTOvXtWrV/aqS1WFqplfqVuvXj2Zea8HSc7l9OnTZdahQweZTZs2TWa//e1vZeadyxkzZsjMk7SS2jNu3DiZeZ8D9bXJO++8I8d4r/9NmjSRmVdj7T1Xva8/mjVrJrO6dfWX015V/p/+9CeZtW/fPtFclKRV6N64bL9+HYt3UgAAAACkCpsUAAAAAKnCJgUAAABAqrBJAQAAAJAqbFIAAAAApAqbFAAAAACpQgVxOXg1tl6l6dSpU2X25z//WWbf+MY3ZObVDJ933nky86r3vIrDQ4cOyUxVtpqZbdu2LeNxr+K2TZs2Mjv77LNl5tXw/vGPf5SZV397xRVXyMzj1Qx7VYVeXbVXOXjOOedkPP7222/LMd7j/tzn9L9fHDx4UGYjR46UmVfZuW/fPpmdf/75Mnv//fdl5p3Lxo0by0zxHjdqjiNHjtju3bvLPS5pvadX0+nVCc+fP79cx83MbrjhBpl589+/f7/MFi1aJDPvPHbp0kVmffv2lVnr1q1l9tJLL8mssLBQZqpGtaCgINHteQYOHCiz3/zmNzJ78MEHZeZ9DjxeRfRdd90ls5kzZ8rMq7ZXa5g3j7feektml112mcxatGghM+/rp+LiYpnt2LFDZt617q1FHq9iuV27dhmPe8+rpK9RnlxUXB+Nd1IAAAAApAqbFAAAAACpwiYFAAAAQKqwSQEAAACQKmxSAAAAAKQKmxQAAAAAqUIFcTl4NcNe7a9X59qvXz+ZJa2te+yxx2Q2btw4mZ1yyikye/zxx2V21llnyWznzp0Zj9evX1+O8SoH69SpIzOv4tarJ/Zqhr1K3fbt28vMq1hWdcFmZieddJLMvOvh1VdfzXjce9xeDaN3LQwYMEBmDzzwgMyKiopkNnToUJl5laNejbJXjaiuS88FF1wgM68qcs+ePTKbPn16ueeB3MrPz7fhw4dnzJ599lk5buLEiTJbsGCBzLxaUK9OOIknn3xSZqqG18x/nnm8ynuvWtx73XrllVdk9sILL8jspptukpmqhW/YsKEc061bN5l5tfxbtmyR2R133CGzpFXVns9//vMy8+rdvV8F4K3fo0ePznj89NNPl2Py8/Nl9ve//11m3mPzzqX3qxi8a8hbn+vVqyczb+3wnnfec0TxnuOeXNcMe3gnBQAAAECqsEkBAAAAkCpsUgAAAACkCpsUAAAAAKnCJgUAAABAqrBJAQAAAJAqVBBnoGpnv/71r8sxmzZtktnhw4dltmvXLpl9/PHHMvOq6VR9ppnZ7t27ZeZVYXrVv17V3wcffJDxeM+ePeWYZs2aycyrbPZqZb36W49Xdzlp0iSZrVmzRmZr166VmVddXFxcLLOOHTtmPL5161Y55uDBgzLz6ifbtGkjM6+a0nseTJs2TWb79u2TmXete/WUgwYNklnz5s0zHl+5cqUc41U7eufZq7ieM2eOzJA7u3btklXDXm33+vXrZdarVy+Zea+7SXjXolclOmrUKJnNnj1bZgMHDpTZxRdfLLMJEybI7L777pPZm2++KTPvNdJb+1TV8Pbt2+UY7/PmZbfccovMvOpi73O3atUqmV111VUya9WqlczeffddmXn1y9u2bZOZuja9NdF73Dt27JCZV/2+efNmmXnmzp0rM29N8aq/CwoKZPa9731PZkuWLMl4vEWLFnLMlVdeKTPv10wkfU3JBt5JAQAAAJAqbFIAAAAApAqbFAAAAACpwiYFAAAAQKqwSQEAAACQKmxSAAAAAKQKFcQZqOrPli1byjFe/eSAAQNk9sUvflFmXv3tzJkzZfbwww/LbMOGDTKrW1dfDl5949KlS2XWtm3bjMcXL14sx3j1vT169JDZO++8IzPvPE+dOlVmw4YNk9mHH34os/3798ts2bJlMsvPz5fZkSNHZKY+P97teee5adOmMnv99ddl1rlzZ5l59ZrnnXeezH7605/KrHfv3jI76aSTZOZVUH7yyScZj6tr2cw/X179NTXD6XPkyBH5fPKu71mzZuVqShmpymCvIvy1116TmVcz7PGe16rK2cyvZW7Xrp3Mhg4dWqZ5HeuMM86Qmao5/8Y3viHHjB49WmYjRoyQ2e9//3uZeXWu3vnyqpe9Kt4FCxbIbNy4cTLz6uS9rzHUryTwvobwvi7x6o696npvfb7hhhtk5n0dMXHiRJl559L7/Hz00Ucye/rppzMe99Yi72tY7/qaN2+ezDzZqC7mnRQAAAAAqcImBQAAAECqsEkBAAAAkCpsUgAAAACkCpsUAAAAAKnCJgUAAABAqlBBXA5eneLw4cNl1rp1a5l59XpeNa7HqwD8+OOPZeZVtnoVgRdddJHMLrzwwozHv/CFL8gxffv2lZlXw+s9toYNG8rsggsukNn7778vM6+G0asq9OqcGzduLLO9e/fKTFUVnnzyyYnu6/TTT5fZ448/LjPvnHi1v2eeeabMxo4dK7NmzZrJbNeuXTLzHruay4QJE+QYr2YY1UteXp6szvRqhr3rtLCwUGZe9WeXLl1ktmTJkozHvWrPIUOGyMyTtErUe2wjR46U2WOPPSazu+66S2Ze5bpXlfy3v/0t4/FWrVrJMd5j27hxo8wKCgpkltQ111wjs1dffVVmAwcOlJn3awK8+7v66qtltmrVqozHva896tSpIzNVaWxmNnfuXJl5n9ctW7bI7Morr5SZ9xzx1o433nhDZqeccorM+vXrl/H4jBkz5JjqiHdSAAAAAKQKmxQAAAAAqcImBQAAAECqsEkBAAAAkCpsUgAAAACkCpsUAAAAAKkSYoxl+8AQ6pjZS2a2McY4LITwBTObbWatzGy5mf1TjPHgCW6jbHdWDXl1il413bRp07I+lylTpshszZo1MvvKV74is/3798usUaNGMlOVg5dccokc88c//lFm/fv3l1nXrl1l9sILL8jMOydf/epXZebxqhGbN28us61bt8rsn/7pn2Smrr8BAwbIMU2aNJHZGWecIbO33npLZtddd53MvBrGTZs2yaxz584y82q6vefk9OnTZaaqxhctWiTHdOzYUWa33XabzHJkeYyxT2XfaRqkdZ3yqn/nz5+f6DZHjRqV8fjs2bMT3Z63TnmVrUkf23e+8x2ZedXLXtW89xx9++23ZbZz506ZKd5jGzdunMy8x/2tb32r3PMwMzv//PNl5tXCt2vXTmYHD+qnifdrFbZv3y6zJ554IuNxrwq5R48eMrvjjjtk5klam11cXJzoNpO65557yj1m4sSJMlOvGWbJXzcqoEzrVHneSbnRzI4udP6Jmd0XY+xkZtvNLNmzCwCA7GCdAoAaokyblBBCBzO7wsxmlf49mNnFZjan9EMKzSzZPzkDAFBBrFMAULOU9Z2Un5nZ983s01/33crMdsQYP30fcYOZZfzV1iGE8SGEl0IIL1VopgAAaKxTAFCDnHCTEkIYZmYfxhiXJ7mDGOPMGGOf2vo90gCA3GKdAoCap24ZPuZ8M7syhDDUzBqYWTMz+7mZ5YcQ6pb+K1UHM9uYu2kCACCxTgFADXPCd1JijLfGGDvEGAvMbJSZPRtj/KaZLTSzT+tzxprZYzmbJQAAAusUANQ8ZXknRfmBmc0OIfzYzP5hZr/OzpSqpzlz5sisU6dOlTgTs8LCQpldfPHFMmvatKnMvKrkiy66SGaXXnppxuP333+/HPPjH/9YZt78R48eLbNzzjlHZl597K5du2S2dOlSmS1btkxmffv2ldnkyZNl5lXqDh06NOPxLVu2yDFe5fT69etl5lUQe7zaX8/48eNl9swzz8hMVQmbmbVv377c8/Dqjm+99dZy3x4qRSrWKVXFXhHZrgz1KlS97OWXX5aZV0/80kv6x3+aNWsms6eeekpm3uuB9xjGjh2b8bi3lnr1sP/5n/8ps4YNG8psxIgRMps3b57MPv/5zyca51UXe2vfXXfdJTOPV++rHDhwQGbdunWTWdJqb+96TsqrWPZqs9955x2ZzZgxo9zz8M5lLmrSs6Fcm5QY43Nm9lzpn98xsy9lf0oAACTDOgUANQO/cR4AAABAqrBJAQAAAJAqbFIAAAAApAqbFAAAAACpwiYFAAAAQKpUpIIYZbR27dpqcX+XXHKJzLza3IULF8pMVQ62adNGjtm7d6/MfvjDH8rs+uuvl9mUKVNk1rt3b5l5tblJa/mSjmvUqJHMVq5cmfF448aN5ZidO3fK7Pvf/77MioqKZJbUr371K5nVq1cv0Vy869IzadKkjMepGUbRLyNDAAAKPElEQVRSXv2t59RTT5XZueeem/F40mpi776Szt97rfPqdr368CuuuEJmSetj161bV+4xXs2wd762bt0qM6/213tsXjZx4kSZeTXKuVBcXFzuMV7lfS6qvZPyrmfv8+M975I8Pu/29u/fL7OqrBn28E4KAAAAgFRhkwIAAAAgVdikAAAAAEgVNikAAAAAUoVNCgAAAIBUYZMCAAAAIFVCjLHy7iyEyrszlNvUqVMTjTvnnHNkpuoWvTrd1atXy8yrge3fv7/MOnToILM5c+bILE28GmX12J9//nk55tChQzJLei3kwpAhQ2Q2bNgwmX3nO9/JxXSqg+Uxxj5VPYnqqnnz5vH888/PmKW1pvNouagSTirpXB5++GGZzZ07V2b169eXmVfNPHDgwIzHvXXKuxaSVvs+8cQTMlu0aFGi26zs6+GGG24o95jBgwfL7N5775XZ7t27ZZa09reynyNJeY8hiSp43GVap3gnBQAAAECqsEkBAAAAkCpsUgAAAACkCpsUAAAAAKnCJgUAAABAqtDuVcuMHz8+0bhWrVrJrLi4WGaXXXZZxuPTpk2TY4qKiso+sRpm0qRJMisoKJDZo48+Wu778tppOnXqJLPhw4fLzPu85sLIkSNlVl1a23KAdq8KqF+/fmzfvn3GrLo0/9Rk9913n8wWL14sM+9z5zVBZdsDDzwgs+uvv77S5lERqg3NzKxly5YymzdvXsbjvXr1kmMq83NzImlqBRs3blzG47Nmzcr6feXocdPuBQAAAKD6YZMCAAAAIFXYpAAAAABIFTYpAAAAAFKFTQoAAACAVGGTAgAAACBV6lb1BFC5Zs6cmWjcokWLZPbuu+/KrHHjxhmPb9u2LdE8arrJkydX2n0NGTJEZtu3b5fZmWeemYvpJFKLa4aRI3l5ebLumwriyjFixIhE47yadq/KVlXqeuteUnfeeWfWbzMXvFrgbJ+XXNQMJ63N9dZFr7a/sq1Zs6bS7sv7FRS5fk3knRQAAAAAqcImBQAAAECqsEkBAAAAkCpsUgAAAACkCpsUAAAAAKnCJgUAAABAqlBBjDJZvXq1zJYtWyaz8ePHZzy+du3aCs8JFePVKY4cOVJm119/fS6mA6RC8+bN7fLLL8+YedWrXmVrLipWazKv8nTp0qUy27Rpk8zOPfdcmc2ePbtsE8uCqqxzLY8klc1mualtVryq6nnz5iW6zVzUDCetQ/asW7cu4Wwy8+ZYla9fvJMCAAAAIFXYpAAAAABIFTYpAAAAAFKFTQoAAACAVGGTAgAAACBV2KQAAAAASJUQY6y8Owuh8u4MWdWpUyeZde7cWWa5qPMD4FoeY+xT1ZOorlq0aBEvuuiijFnSWlMcL2mN7dixY2VWWFhYoTkBtZVXoV5cXCwzr0L5BNXLZVqneCcFAAAAQKqwSQEAAACQKmxSAAAAAKQKmxQAAAAAqcImBQAAAECqsEkBAAAAkCpUEKNMxowZI7OHHnqoEmcC4ASoIK4A1qmql7RmOGmtMY7nVdK+/PLLlTgTZKKu9XXr1skxXl1wLowaNUpms2fPpoIYAAAAQPXDJgUAAABAqrBJAQAAAJAqbFIAAAAApAqbFAAAAACpwiYFAAAAQKrUreoJoHqgZhhAbdC7d2976aWXMmYhhES3eeqpp8rMqwVNMs4b46nselJPhw4dEo3z6lfHjRsnM1Wp27RpUzmmR48eMjv55JNl9sorr8jstNNOk5k3l40bN8rs7LPPlpm6zs3MZsyYIbNsX8/dunWTYxo0aCAz7zp54YUXZNa2bVuZeZ+7Q4cOyez000+X2WWXXSazZs2ayWz58uUye/PNN2WmeM+BF198sdy3Z2Y2f/58mbVo0SLRbR6Nd1IAAAAApAqbFAAAAACpwiYFAAAAQKqwSQEAAACQKmxSAAAAAKQKmxQAAAAAqRJijJV3ZyFsNbNP++lam9m2Srvz6oFzkhnn5Xick+NxTkqcGmNsU9WTqK5Yp8qE83I8zsnxOCfH45yUKNM6VamblM/ccQgvxRj7VMmdpxTnJDPOy/E4J8fjnCDbuKYy47wcj3NyPM7J8Tgn5cO3ewEAAABIFTYpAAAAAFKlKjcpM6vwvtOKc5IZ5+V4nJPjcU6QbVxTmXFejsc5OR7n5Hick3Kosp9JAQAAAIBM+HYvAAAAAKlSJZuUEMLgEMJbIYS1IYRbqmIOVS2E8EAI4cMQwutHHWsZQngmhLCm9P8tqnKOlS2E0DGEsDCEsCqEsDKEcGPp8Vp7XkIIDUIIfw8hvFp6Tu4sPf6FEMLS0ufQH0IIeVU918oWQqgTQvhHCOGJ0r/X+nOC7GGdYp3KhHXqeKxTGutUxVT6JiWEUMfMfmFmQ8ysm5ldE0LoVtnzSIEHzWzwMcduMbMFMcbOZrag9O+1yWEzuznG2M3M+pvZd0qvjdp8Xg6Y2cUxxp5mdraZDQ4h9Dezn5jZfTHGTma23cy+VYVzrCo3mtkbR/2dc4KsYJ36Xw8a69SxWKeOxzqlsU5VQFW8k/IlM1sbY3wnxnjQzGab2fAqmEeVijEuNrOPjjk83MwKS/9caGZfrdRJVbEY46YY48ulf95tJU/sk60Wn5dYYk/pX+uV/hfN7GIzm1N6vFadEzOzEEIHM7vCzGaV/j1YLT8nyCrWKWOdyoR16nisU5mxTlVcVWxSTjaz9Uf9fUPpMZi1jTFuKv3zZjNrW5WTqUohhAIzO8fMllotPy+lbxe/YmYfmtkzZva2me2IMR4u/ZDa+Bz6mZl938yOlP69lXFOkD2sU1qtfj0+GuvU/2Gdyoh1qoL4wfmUiiW1a7Wyei2E0MTM/mhm340x7jo6q43nJcb4SYzxbDPrYCX/wtu1iqdUpUIIw8zswxjj8qqeC1Cb1cbX40+xTn0W69RnsU5lR90quM+NZtbxqL93KD0Gsy0hhPYxxk0hhPZW8i8StUoIoZ6VvPD/NsY4t/RwrT8vZmYxxh0hhIVmdq6Z5YcQ6pb+i0xtew6db2ZXhhCGmlkDM2tmZj+32n1OkF2sU1qtfz1mndJYp/4X61QWVMU7KcvMrHNpw0GemY0ys8erYB5p9LiZjS3981gze6wK51LpSr9f89dm9kaM8d6jolp7XkIIbUII+aV/bmhml1nJ90AvNLORpR9Wq85JjPHWGGOHGGOBlbx+PBtj/KbV4nOCrGOd0mrt67EZ61QmrFPHY53Kjir5ZY6lO8ufmVkdM3sgxnhXpU+iioUQfm9mg8ystZltMbPbzexPZvaImZ1iZu+Z2VUxxmN/aLHGCiFcYGbPm9kK+7/v4fyhlXy/b608LyGEHlbyw3V1rOQfFR6JMU4OIXzRSn6Yt6WZ/cPMro0xHqi6mVaNEMIgM/t/Y4zDOCfIJtYp1qlMWKeOxzrlY51Kjt84DwAAACBV+MF5AAAAAKnCJgUAAABAqrBJAQAAAJAqbFIAAAAApAqbFAAAAACpwiYFAAAAQKqwSQEAAACQKmxSAAAAAKTK/w+KMHe6Fam9vQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1008x1008 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "f, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2,2, figsize=(14, 14))\n",
    "idx = 480\n",
    "ax1.imshow(np.log(st_data[idx, :, :]+1),  cmap='gray')\n",
    "ax1.set_title(\"Start : 00h\")\n",
    "idx = 480\n",
    "ax2.imshow(np.log(ed_data[idx, :, :]+1),  cmap='gray')\n",
    "ax2.set_title(\"End : 00h\")\n",
    "\n",
    "idx = 496\n",
    "ax3.imshow(np.log(st_data[idx, :, :]+1),  cmap='gray')\n",
    "ax3.set_title(\"Start : 08h\")\n",
    "idx = 496\n",
    "ax4.imshow(np.log(ed_data[idx, :, :]+1),  cmap='gray')\n",
    "ax4.set_title(\"End : 08h\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (2) Model Training \n",
    "\n",
    "------------\n",
    "\n",
    "<b>[ Overview ]</b>\n",
    "\n",
    "- Model Preview.\n",
    "\n",
    "![objectives](./output_file/graph/tgnet_preview.png)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "\n",
    "################################################################################\n",
    "## Import Module\n",
    "################################################################################\n",
    "\n",
    "from utils import *\n",
    "import os\n",
    "\n",
    "import tensorflow as tf\n",
    "import keras\n",
    "import keras.backend as K\n",
    "\n",
    "from keras import layers\n",
    "from keras.layers import Input, Dense, Conv2D, MaxPooling2D, AveragePooling2D, UpSampling2D, Flatten, Reshape, Dropout, Conv2DTranspose\n",
    "from keras.layers import Concatenate, BatchNormalization, Add\n",
    "from keras.models import Model, Sequential\n",
    "from keras.layers import InputLayer\n",
    "from keras.utils.training_utils import multi_gpu_model\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "################################################################################\n",
    "## Hyperparam  - args = parser.parse_args()\n",
    "################################################################################\n",
    "\n",
    "LRATE = 1e-2 # [ 1e-5, 1e-4, 1e-3, 5e-2, 1e-2, 1e-1 ]\n",
    "BATCH_SIZE = 128 # [ 512, 256, 128, 64 ]\n",
    "EPOCHS = 3000\n",
    "SCALE = 'log' # or 'max' or 'log'\n",
    "NGPU = 4\n",
    "NPTS = 100 # early stopping patience iteration\n",
    "\n",
    "## Folder / File Name\n",
    "DATA_FOLDER = '/data/public/rw/prj-mobility/datasets/2018_ori/'\n",
    "\n",
    "## Baseline Model Folder \n",
    "BASE_MODEL_NAME = 'BASE_MODEL'\n",
    "MODEL_VER = '1201_baseline_2ch_base_v1_end_bn_ba128'\n",
    "\n",
    "RANDOM_SEED = 42\n",
    "tf.set_random_seed(RANDOM_SEED)\n",
    "np.random.seed(RANDOM_SEED)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model Name : BASE_MODEL_1201_baseline_2ch_base_v1_end_bn_ba128\n",
      "Output Folder : ../output_file/BASE_MODEL/\n",
      "Tfgraph Folder : ../tfgraph/BASE_MODEL/1201_baseline_2ch_base_v1_end_bn_ba128\n",
      "Save Folder : ../saved_model/BASE_MODEL/\n"
     ]
    }
   ],
   "source": [
    "################################################################################\n",
    "## Working Directory Setting \n",
    "################################################################################\n",
    "\n",
    "OUTPUT_FOLDER = '../output_file/'\n",
    "TF_FOLDER = '../tfgraph/'\n",
    "MODEL_SAVE_FOLDER = '../saved_model/'\n",
    "\n",
    "MODEL_NAME_ = BASE_MODEL_NAME+'_'+MODEL_VER\n",
    "OUTPUT_FOLDER = OUTPUT_FOLDER+BASE_MODEL_NAME+'/'\n",
    "TF_FOLDER = TF_FOLDER+BASE_MODEL_NAME+'/'+MODEL_VER\n",
    "MODEL_SAVE_FOLDER = MODEL_SAVE_FOLDER+BASE_MODEL_NAME+'/'\n",
    "\n",
    "print ('Model Name :',MODEL_NAME_)\n",
    "\n",
    "if os.path.isdir(OUTPUT_FOLDER) == False:\n",
    "    os.makedirs(OUTPUT_FOLDER)\n",
    "    print ('Output Folder :',OUTPUT_FOLDER, 'created')\n",
    "else :\n",
    "    print ('Output Folder :',OUTPUT_FOLDER)\n",
    "\n",
    "if os.path.isdir(TF_FOLDER) == False:\n",
    "    os.makedirs(TF_FOLDER)\n",
    "    print ('Tfgraph Folder :',TF_FOLDER, 'created')\n",
    "else :\n",
    "    print ('Tfgraph Folder :',TF_FOLDER)\n",
    "    \n",
    "if os.path.isdir(MODEL_SAVE_FOLDER) == False:\n",
    "    os.makedirs(MODEL_SAVE_FOLDER)\n",
    "    print ('Save Folder :',MODEL_SAVE_FOLDER, 'created')\n",
    "else :\n",
    "    print ('Save Folder :',MODEL_SAVE_FOLDER)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/public/rw/prj-mobility/datasets/2018_ori/X_train_data_st_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/X_train_data_ed_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/Y_train_data_st_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/week_train_data_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/time_train_data_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/holi_train_data_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/prev_train_data_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/X_test_data_st_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/X_test_data_ed_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/Y_test_data_st_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/week_test_data_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/time_test_data_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/holi_test_data_30min_16ch_ori.npz Loaded\n",
      "/data/public/rw/prj-mobility/datasets/2018_ori/prev_test_data_30min_16ch_ori.npz Loaded\n",
      "4491.0 6128.0\n"
     ]
    }
   ],
   "source": [
    "################################################################################\n",
    "## Data Load\n",
    "################################################################################\n",
    "## Train period = 2018/01/01 ~ 2018/04/30 ( 80% / 20% validation split)\n",
    "## Test period = 2018/05/01 ~ 2018/06/30 \n",
    "\n",
    "x_st_train = load_np_data(DATA_FOLDER+'X_train_data_st_30min_16ch_ori.npz')\n",
    "x_ed_train = load_np_data(DATA_FOLDER+'X_train_data_ed_30min_16ch_ori.npz')\n",
    "y_train = load_np_data(DATA_FOLDER+'Y_train_data_st_30min_16ch_ori.npz')\n",
    "x_week_train = load_np_data(DATA_FOLDER+'week_train_data_30min_16ch_ori.npz')\n",
    "x_time_train = load_np_data(DATA_FOLDER+'time_train_data_30min_16ch_ori.npz')\n",
    "x_holi_train = load_np_data(DATA_FOLDER+'holi_train_data_30min_16ch_ori.npz')\n",
    "x_prev_train = load_np_data(DATA_FOLDER+'prev_train_data_30min_16ch_ori.npz')\n",
    "\n",
    "x_st_test = load_np_data(DATA_FOLDER+'X_test_data_st_30min_16ch_ori.npz')\n",
    "x_ed_test = load_np_data(DATA_FOLDER+'X_test_data_ed_30min_16ch_ori.npz')\n",
    "y_test = load_np_data(DATA_FOLDER+'Y_test_data_st_30min_16ch_ori.npz')\n",
    "x_week_test = load_np_data(DATA_FOLDER+'week_test_data_30min_16ch_ori.npz')\n",
    "x_time_test = load_np_data(DATA_FOLDER+'time_test_data_30min_16ch_ori.npz')\n",
    "x_holi_test = load_np_data(DATA_FOLDER+'holi_test_data_30min_16ch_ori.npz')\n",
    "x_prev_test = load_np_data(DATA_FOLDER+'prev_test_data_30min_16ch_ori.npz')\n",
    "\n",
    "print (x_st_train.max(), x_ed_train.max())\n",
    "\n",
    "\n",
    "### Data normalization in logscale \n",
    "\n",
    "x_st_train = logscale(x_st_train) \n",
    "x_ed_train = logscale(x_ed_train)\n",
    "y_train = logscale(y_train)\n",
    "\n",
    "x_st_test = logscale(x_st_test)\n",
    "x_ed_test = logscale(x_ed_test)\n",
    "y_test = logscale(y_test)\n",
    "\n",
    "x_temporal_train = make_temporal_concat(x_week_train, x_time_train, x_holi_train, x_prev_train)\n",
    "x_temporal_test = make_temporal_concat(x_week_test, x_time_test, x_holi_test, x_prev_test)\n",
    "\n",
    "### Train / Valid Split\n",
    "train_data_size = int(len(x_st_train)*0.8)\n",
    "x_st_valid = x_st_train[train_data_size:]\n",
    "x_ed_valid = x_ed_train[train_data_size:]\n",
    "y_valid = y_train[train_data_size:]\n",
    "x_temporal_valid = x_temporal_train[train_data_size:]\n",
    "\n",
    "x_st_train = x_st_train[:train_data_size]\n",
    "x_ed_train = x_ed_train[:train_data_size]\n",
    "y_train = y_train[:train_data_size]\n",
    "x_temporal_train = x_temporal_train[:train_data_size]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Session Reseted\n",
      "BASE_MODEL_1201_baseline_2ch_base_v1_end_bn_ba128 Model Created\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_3 (InputLayer)            (None, 50, 50, 59)   0                                            \n",
      "__________________________________________________________________________________________________\n",
      "input_1 (InputLayer)            (None, 50, 50, 10)   0                                            \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_1 (Conv2D)               (None, 50, 50, 16)   960         input_3[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "lambda_2 (Lambda)               (None, 50, 50, 8)    0           input_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_2 (Conv2D)               (None, 50, 50, 16)   272         conv2d_1[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_1 (Concatenate)     (None, 50, 50, 24)   0           lambda_2[0][0]                   \n",
      "                                                                 conv2d_2[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_3 (Conv2D)               (None, 48, 48, 64)   13888       concatenate_1[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_1 (Activation)       (None, 48, 48, 64)   0           conv2d_3[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "batch_normalization_1 (BatchNor (None, 48, 48, 64)   256         activation_1[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "average_pooling2d_1 (AveragePoo (None, 12, 12, 64)   0           batch_normalization_1[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_4 (Conv2D)               (None, 12, 12, 256)  147712      average_pooling2d_1[0][0]        \n",
      "__________________________________________________________________________________________________\n",
      "activation_2 (Activation)       (None, 12, 12, 256)  0           conv2d_4[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "batch_normalization_2 (BatchNor (None, 12, 12, 256)  1024        activation_2[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_5 (Conv2D)               (None, 12, 12, 256)  590080      batch_normalization_2[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "activation_3 (Activation)       (None, 12, 12, 256)  0           conv2d_5[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "batch_normalization_3 (BatchNor (None, 12, 12, 256)  1024        activation_3[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_2 (Concatenate)     (None, 12, 12, 512)  0           batch_normalization_2[0][0]      \n",
      "                                                                 batch_normalization_3[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_6 (Conv2D)               (None, 12, 12, 256)  1179904     concatenate_2[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_4 (Activation)       (None, 12, 12, 256)  0           conv2d_6[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "batch_normalization_4 (BatchNor (None, 12, 12, 256)  1024        activation_4[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_3 (Concatenate)     (None, 12, 12, 768)  0           batch_normalization_2[0][0]      \n",
      "                                                                 batch_normalization_3[0][0]      \n",
      "                                                                 batch_normalization_4[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_transpose_1 (Conv2DTrans (None, 48, 48, 256)  3145984     concatenate_3[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_5 (Activation)       (None, 48, 48, 256)  0           conv2d_transpose_1[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "batch_normalization_5 (BatchNor (None, 48, 48, 256)  1024        activation_5[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "dropout_1 (Dropout)             (None, 48, 48, 256)  0           batch_normalization_5[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_4 (Concatenate)     (None, 48, 48, 320)  0           dropout_1[0][0]                  \n",
      "                                                                 batch_normalization_1[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_transpose_2 (Conv2DTrans (None, 50, 50, 512)  1475072     concatenate_4[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_6 (Activation)       (None, 50, 50, 512)  0           conv2d_transpose_2[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "input_2 (InputLayer)            (None, 50, 50, 16)   0                                            \n",
      "__________________________________________________________________________________________________\n",
      "batch_normalization_6 (BatchNor (None, 50, 50, 512)  2048        activation_6[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_7 (Conv2D)               (None, 50, 50, 64)   9280        input_2[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "dropout_2 (Dropout)             (None, 50, 50, 512)  0           batch_normalization_6[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_8 (Conv2D)               (None, 50, 50, 64)   36928       conv2d_7[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "lambda_1 (Lambda)               (None, 50, 50, 2)    0           input_1[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "concatenate_5 (Concatenate)     (None, 50, 50, 618)  0           dropout_2[0][0]                  \n",
      "                                                                 lambda_2[0][0]                   \n",
      "                                                                 conv2d_8[0][0]                   \n",
      "                                                                 input_2[0][0]                    \n",
      "                                                                 conv2d_2[0][0]                   \n",
      "                                                                 lambda_1[0][0]                   \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_transpose_3 (Conv2DTrans (None, 50, 50, 512)  316928      concatenate_5[0][0]              \n",
      "__________________________________________________________________________________________________\n",
      "activation_7 (Activation)       (None, 50, 50, 512)  0           conv2d_transpose_3[0][0]         \n",
      "__________________________________________________________________________________________________\n",
      "batch_normalization_7 (BatchNor (None, 50, 50, 512)  2048        activation_7[0][0]               \n",
      "__________________________________________________________________________________________________\n",
      "conv2d_9 (Conv2D)               (None, 50, 50, 1)    513         batch_normalization_7[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "activation_8 (Activation)       (None, 50, 50, 1)    0           conv2d_9[0][0]                   \n",
      "==================================================================================================\n",
      "Total params: 6,925,969\n",
      "Trainable params: 6,921,745\n",
      "Non-trainable params: 4,224\n",
      "__________________________________________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "################################################################################\n",
    "## Model\n",
    "################################################################################\n",
    "\n",
    "K.clear_session()\n",
    "print ('Session Reseted')\n",
    "\n",
    "model_input_train_data = [x_st_train, x_ed_train, x_temporal_train]\n",
    "model_input_valid_data = [x_st_valid, x_ed_valid, x_temporal_valid]\n",
    "model_input_test_data = [x_st_test, x_ed_test, x_temporal_test]\n",
    "\n",
    "def make_temporal_model(st_data, ed_data, tmp_data):\n",
    "\n",
    "        im_h,im_w,im_c = st_data[0].shape\n",
    "        \n",
    "        start_input = Input(shape=(st_data[0].shape))\n",
    "        end_input = Input(shape=(ed_data[0].shape))\n",
    "        temporal_input = Input(shape=(tmp_data[0].shape))\n",
    "        \n",
    "        input_tensors = [start_input, end_input, temporal_input]\n",
    "        \n",
    "        ## Coord input has x-y coord in integer number \n",
    "        coord_input = keras.layers.Lambda(lambda xin: xin[:,:,:,-2:] )(start_input)\n",
    "        start_input = keras.layers.Lambda(lambda xin: xin[:,:,:,:-2] )(start_input)\n",
    "    \n",
    "        # Conv2D default Parameters \n",
    "        # - use_bias=True \n",
    "        # - kernel_initializer='glorot_uniform' \n",
    "        #  ( Xavier uniform initializer , range +/- sqrt(6 / (input_unit + output_unit)) )\n",
    "        # - bias_initializer='zeros'\n",
    "\n",
    "        ## Temporal Data Embedding \n",
    "        net_temp = Conv2D(16, kernel_size=(1,1), strides=(1,1))(temporal_input)\n",
    "        net_temp = Conv2D(16, kernel_size=(1,1), strides=(1,1))(net_temp)\n",
    "        # outlayer Shape - [ 50 x 50 x 16 ]\n",
    "        \n",
    "        ## Embedded Temporal data concat \n",
    "        net1 = layers.concatenate([start_input, net_temp], axis=-1)\n",
    "\n",
    "        net1 = Conv2D(64, kernel_size=(3,3), strides=(1,1), activation=None, padding='valid')(net1)\n",
    "        net1 = layers.Activation('relu')(net1)\n",
    "        net1 = BatchNormalization()(net1)\n",
    "        # outlayer Shape - [ 48 x 48 x 64 ]\n",
    "        \n",
    "        ## Skip connection Layer\n",
    "        net11 = AveragePooling2D(pool_size=(4,4), strides=4)(net1)\n",
    "        # outlayer Shape - [ 12 x 12 x 64 ]\n",
    "\n",
    "        # Dense Block Start \n",
    "        net2 = Conv2D(256, kernel_size=(3,3), strides=(1,1), activation=None, padding='same')(net11)\n",
    "        net2 = layers.Activation('relu')(net2)\n",
    "        net2 = BatchNormalization()(net2)\n",
    "        # outlayer Shape - [ 12 x 12 x 256 ]\n",
    "        \n",
    "        net3 = Conv2D(256, kernel_size=(3,3), strides=(1,1), activation=None, padding='same')(net2)\n",
    "        net3 = layers.Activation('relu')(net3)\n",
    "        net3 = BatchNormalization()(net3)\n",
    "        # outlayer Shape - [ 12 x 12 x 256 ]\n",
    "        \n",
    "        ## Skip connection Layer\n",
    "        net33 = layers.concatenate([net2, net3], axis=-1)\n",
    "        \n",
    "        net4 = Conv2D(256, kernel_size=(3,3), strides=(1,1), activation=None, padding='same')(net33)\n",
    "        net4 = layers.Activation('relu')(net4)\n",
    "        net4 = BatchNormalization()(net4)\n",
    "        # outlayer Shape - [ 12 x 12 x 256 ]\n",
    "        \n",
    "        ## Skip connection Layer\n",
    "        net44 = layers.concatenate([net2, net3, net4], axis=-1)\n",
    "\n",
    "        ## Upsampling Layer \n",
    "        net5 = Conv2DTranspose(256, kernel_size=(4,4), strides=(4,4), padding='same')(net44)\n",
    "        net5 = layers.Activation('relu')(net5)\n",
    "        net5 = BatchNormalization()(net5)\n",
    "        net5 = Dropout(0.5)(net5)\n",
    "        # outlayer Shape - [ 48 x 48 x 256 ]\n",
    "        \n",
    "        ## Skip connection Layer\n",
    "        net55 = layers.concatenate([net5, net1], axis=-1)\n",
    "\n",
    "        net6 = Conv2DTranspose(512, kernel_size=(3,3), strides=(1,1), padding='valid')(net55)\n",
    "        net6 = layers.Activation('relu')(net6)\n",
    "        net6 = BatchNormalization()(net6)\n",
    "        net6 = Dropout(0.5)(net6)\n",
    "        # outlayer Shape - [ 50 x 50 x 512 ]\n",
    "        \n",
    "        # End point data Embedding Layer \n",
    "        net_end = Conv2D(64, kernel_size=(3,3), strides=(1,1), padding='same')(end_input)\n",
    "        net_end = Conv2D(64, kernel_size=(3,3), strides=(1,1), padding='same')(net_end)\n",
    "        # outlayer Shape - [ 50 x 50 x 64 ]\n",
    "        \n",
    "        net7 = layers.concatenate([net6, start_input, net_end, end_input, net_temp, coord_input], axis=-1)\n",
    "        net7 = Conv2DTranspose(512, kernel_size=(1,1), strides=(1,1), padding='same')(net7)\n",
    "        net7 = layers.Activation('relu')(net7)\n",
    "        net7 = BatchNormalization()(net7)\n",
    "        # outlayer Shape - [ 50 x 50 x 512 ]\n",
    "\n",
    "        output = Conv2D(1, kernel_size=(1,1), strides=(1,1), padding='same')(net7)\n",
    "        output = layers.Activation('relu')(output)\n",
    "        # outlayer Shape - [ 50 x 50 x 1 ]\n",
    "\n",
    "        model = Model(inputs=input_tensors, outputs=output)\n",
    "        return model\n",
    "\n",
    "b_model = make_temporal_model(x_st_train,x_ed_train, x_temporal_train)\n",
    "model = multi_gpu_model(b_model, gpus=NGPU)\n",
    "print (MODEL_NAME_, 'Model Created')\n",
    "print (b_model.summary())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "################################################################################\n",
    "## Callback \n",
    "################################################################################\n",
    "\n",
    "\n",
    "from keras.callbacks import ModelCheckpoint\n",
    "from keras.optimizers import SGD, Adam\n",
    "\n",
    "from keras.callbacks import Callback\n",
    "\n",
    "class SGDLearningRateTracker(Callback):\n",
    "    def on_epoch_end(self, epoch, logs={}):\n",
    "        optimizer = self.model.optimizer\n",
    "        lr = K.eval(optimizer.lr) # K.eval(optimizer.lr * (1. / (1. + optimizer.decay * optimizer.iterations)))\n",
    "        print('LR: {:.6f}'.format(lr))\n",
    "\n",
    "tb_hist = keras.callbacks.TensorBoard(log_dir=TF_FOLDER, histogram_freq=0, write_graph=True, write_images=True)\n",
    "early_stopping = keras.callbacks.EarlyStopping(monitor='loss', min_delta=0, patience=NPTS, verbose=0, mode='min')\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 4595 samples, validate on 1149 samples\n",
      "Epoch 1/5\n",
      "4595/4595 [==============================] - 30s 7ms/step - loss: 0.5858 - mean_absolute_error: 0.3853 - invlog_rmse: 1293.7412 - invlog_mape: 1.2481 - val_loss: 4.2832 - val_mean_absolute_error: 1.3402 - val_invlog_rmse: 1986.6659 - val_invlog_mape: 13.9895\n",
      "LR: 0.010000\n",
      "Epoch 2/5\n",
      "4595/4595 [==============================] - 25s 5ms/step - loss: 0.1732 - mean_absolute_error: 0.2191 - invlog_rmse: 10.9425 - invlog_mape: 0.2471 - val_loss: 0.3021 - val_mean_absolute_error: 0.3070 - val_invlog_rmse: 11.8417 - val_invlog_mape: 0.5313\n",
      "LR: 0.010000\n",
      "Epoch 3/5\n",
      "4595/4595 [==============================] - 24s 5ms/step - loss: 0.1611 - mean_absolute_error: 0.2120 - invlog_rmse: 10.3322 - invlog_mape: 0.2380 - val_loss: 0.2873 - val_mean_absolute_error: 0.2930 - val_invlog_rmse: 11.3708 - val_invlog_mape: 0.4923\n",
      "LR: 0.010000\n",
      "Epoch 4/5\n",
      "4595/4595 [==============================] - 24s 5ms/step - loss: 0.1600 - mean_absolute_error: 0.2123 - invlog_rmse: 281.8170 - invlog_mape: 0.7186 - val_loss: 0.1848 - val_mean_absolute_error: 0.2260 - val_invlog_rmse: 11.5506 - val_invlog_mape: 0.1983\n",
      "LR: 0.010000\n",
      "Epoch 5/5\n",
      "4595/4595 [==============================] - 24s 5ms/step - loss: 0.1570 - mean_absolute_error: 0.2101 - invlog_rmse: 10.8131 - invlog_mape: 0.2364 - val_loss: 0.1710 - val_mean_absolute_error: 0.2191 - val_invlog_rmse: 11.5232 - val_invlog_mape: 0.2096\n",
      "LR: 0.010000\n",
      "../output_file/BASE_MODEL/BASE_MODEL_1201_baseline_2ch_base_v1_end_bn_ba128step_1_loss_metric_invmape_0.210.csv  Val loss Saved\n",
      "--- Train Time : 0.04 hour  ---\n",
      "--- # of Epochs: 5  ---\n"
     ]
    }
   ],
   "source": [
    "################################################################################\n",
    "## Training  \n",
    "################################################################################\n",
    "\n",
    "EPOCHS=5\n",
    "\n",
    "import time\n",
    "start_time = time.time() \n",
    "\n",
    "\n",
    "model.compile(loss=['mse'], optimizer=Adam(lr=LRATE,  decay=0.01), metrics=['mae', invlog_rmse, invlog_mape])\n",
    "    \n",
    "history = model.fit(model_input_train_data , y_train,\n",
    "                  batch_size=BATCH_SIZE,\n",
    "                  epochs=EPOCHS,\n",
    "                  callbacks=[tb_hist, early_stopping, SGDLearningRateTracker()],\n",
    "                  shuffle = True,\n",
    "                  verbose=1\n",
    "                  ,validation_data=(model_input_valid_data, y_valid)\n",
    "                 )\n",
    "\n",
    "val_loss = history.history['val_loss']\n",
    "val_metric = history.history['val_invlog_mape']\n",
    "\n",
    "history_save(val_loss, val_metric, model_name =  MODEL_NAME_+'step_1', output_folder=OUTPUT_FOLDER)\n",
    "\n",
    "#model.compile(loss=['mae'], optimizer=Adam(lr=LRATE,  decay=0.01), metrics=['mae', invlog_rmse, invlog_mape])\n",
    "\n",
    "#history = model.fit(model_input_train_data , y_train,\n",
    "#                  batch_size=BATCH_SIZE,\n",
    "#                  epochs=EPOCHS,\n",
    "#                  callbacks=[tb_hist, early_stopping, SGDLearningRateTracker()],\n",
    "#                  shuffle = False,\n",
    "#                  verbose=1\n",
    "#                  ,validation_data=(model_input_valid_data, y_valid)\n",
    "#                 )\n",
    "    \n",
    "#val_loss = history.history['val_loss']\n",
    "#val_metric = history.history['val_invlog_mape']\n",
    "\n",
    "end_time = time.time()\n",
    "n_epochs = len(val_metric)\n",
    "\n",
    "print(\"--- Train Time : %0.2f hour  ---\" %(  (end_time - start_time)/3600  ))\n",
    "print(\"--- # of Epochs: %0.f  ---\" %( n_epochs ) )\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model Saved ../saved_model/BASE_MODEL/BASE_MODEL_1201_baseline_2ch_base_v1_end_bn_ba128st2.h5\n",
      "../output_file/BASE_MODEL/BASE_MODEL_1201_baseline_2ch_base_v1_end_bn_ba128step_2_loss_metric_invmape_0.210.csv  Val loss Saved\n",
      "Test True saved :  ../output_file/BASE_MODEL/BASE_MODEL_1201_baseline_2ch_base_v1_end_bn_ba128_gt.csv\n",
      "Test Pred saved :  ../output_file/BASE_MODEL/BASE_MODEL_1201_baseline_2ch_base_v1_end_bn_ba128_pred.csv\n",
      "\n"
     ]
    }
   ],
   "source": [
    "\n",
    "################################################################################\n",
    "## Model & Output Save  \n",
    "################################################################################\n",
    "\n",
    "save_model_name = MODEL_SAVE_FOLDER+MODEL_NAME_+'st2.h5'\n",
    "b_model.save(save_model_name)\n",
    "print ('Model Saved', save_model_name)\n",
    "\n",
    "history_save(val_loss, val_metric, model_name =  MODEL_NAME_+'step_2', output_folder=OUTPUT_FOLDER)\n",
    "make_test_ouput(model, model_input_test_data , y_test, model_name=MODEL_NAME_, norm=SCALE, output_folder=OUTPUT_FOLDER)\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}