feature_selection.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature Selection w/ a Random Forrest Classifier\n",
    "\n",
    "I want to use some patient info as input to a CNN that will classify ECG rhythm types. This is the process I use to find the best features from our given data.\n",
    "I am using the kaggle dataset [PTB-XL - Atrial Fibrillation Detection](https://www.kaggle.com/datasets/arjunascagnetto/ptbxl-atrial-fibrillation-detection)\n",
    "\n",
    "The description of this dataset states that it is intended to be used for detection of three ecg rhythms: Normal, Atrial Fibrillation (AF), all other arrhythmia. However, the dataset contains mutliple rhythm types, I will attempt to classify all of them.\n",
    "\n",
    "Rhythm types and support:\n",
    "\n",
    "|Rhythm|# examples|\n",
    "|:-----|---------:|\n",
    "|Atrial Fibrillation| 1514|\n",
    "|Atrial Flutter| 73|\n",
    "|Bigeminal Pattern (Unknown Origin, SV or Ventricular)| 82|\n",
    "|Normal Functioning Artificial Pacemaker| 296|\n",
    "|Paroxysmal Supraventricular Tachycardia| 24|\n",
    "|Sinus Arrhythmia| 772|\n",
    "|Sinus Bradycardia| 637|\n",
    "|Sinus Rhythm| 2000|\n",
    "|Sinus Tachycardia| 826|\n",
    "|Supraventricular Arrhythmia| 157|\n",
    "|Supraventricular Tachycardia| 27|\n",
    "|Trigeminal Pattern (Unknown Origin, SV or Ventricular)| 20|\n",
    "\n",
    "In order to simplify things a little, I will consider sinus rhythm (SR), sinus bradycardia (SB), and sinus tachycardia (ST) as normal sinus rhythm (NSR). My thought is that with a good beat detector the heart rate can be found separately and a brady or tachy determination can be made based on user provided threshholds.\n",
    "\n",
    "## Import the Data\n",
    "Read the csv from PTB-XL as a pandas dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>diagnosi</th>\n",
       "      <th>ecg_id</th>\n",
       "      <th>ritmi</th>\n",
       "      <th>patient_id</th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>height</th>\n",
       "      <th>weight</th>\n",
       "      <th>nurse</th>\n",
       "      <th>site</th>\n",
       "      <th>...</th>\n",
       "      <th>validated_by_human</th>\n",
       "      <th>baseline_drift</th>\n",
       "      <th>static_noise</th>\n",
       "      <th>burst_noise</th>\n",
       "      <th>electrodes_problems</th>\n",
       "      <th>extra_beats</th>\n",
       "      <th>pacemaker</th>\n",
       "      <th>strat_fold</th>\n",
       "      <th>filename_lr</th>\n",
       "      <th>filename_hr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>STACH</td>\n",
       "      <td>10900</td>\n",
       "      <td>VA</td>\n",
       "      <td>15654.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6</td>\n",
       "      <td>records100/10000/10900_lr</td>\n",
       "      <td>records500/10000/10900_hr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AFLT</td>\n",
       "      <td>10900</td>\n",
       "      <td>AF</td>\n",
       "      <td>15654.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6</td>\n",
       "      <td>records100/10000/10900_lr</td>\n",
       "      <td>records500/10000/10900_hr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>SR</td>\n",
       "      <td>8209</td>\n",
       "      <td>SR</td>\n",
       "      <td>12281.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>records100/08000/08209_lr</td>\n",
       "      <td>records500/08000/08209_hr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>STACH</td>\n",
       "      <td>17620</td>\n",
       "      <td>VA</td>\n",
       "      <td>2007.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>1</td>\n",
       "      <td>164.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>records100/17000/17620_lr</td>\n",
       "      <td>records500/17000/17620_hr</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>SBRAD</td>\n",
       "      <td>12967</td>\n",
       "      <td>VA</td>\n",
       "      <td>8685.0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>, I-AVR,</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>records100/12000/12967_lr</td>\n",
       "      <td>records500/12000/12967_hr</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 30 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  diagnosi  ecg_id ritmi  patient_id   age  sex  height  weight  nurse  site  \\\n",
       "0    STACH   10900    VA     15654.0  54.0    0     NaN     NaN    0.0   0.0   \n",
       "1     AFLT   10900    AF     15654.0  54.0    0     NaN     NaN    0.0   0.0   \n",
       "2       SR    8209    SR     12281.0  55.0    0     NaN     NaN    1.0   2.0   \n",
       "3    STACH   17620    VA      2007.0  29.0    1   164.0    56.0    7.0   1.0   \n",
       "4    SBRAD   12967    VA      8685.0  57.0    0     NaN     NaN    0.0   0.0   \n",
       "\n",
       "   ... validated_by_human baseline_drift static_noise burst_noise  \\\n",
       "0  ...              False            NaN          NaN         NaN   \n",
       "1  ...              False            NaN          NaN         NaN   \n",
       "2  ...               True            NaN          NaN         NaN   \n",
       "3  ...               True            NaN          NaN         NaN   \n",
       "4  ...              False            NaN   , I-AVR,           NaN   \n",
       "\n",
       "  electrodes_problems extra_beats pacemaker  strat_fold  \\\n",
       "0                 NaN         NaN       NaN           6   \n",
       "1                 NaN         NaN       NaN           6   \n",
       "2                 NaN         NaN       NaN          10   \n",
       "3                 NaN         NaN       NaN           1   \n",
       "4                 NaN         NaN       NaN           1   \n",
       "\n",
       "                 filename_lr                filename_hr  \n",
       "0  records100/10000/10900_lr  records500/10000/10900_hr  \n",
       "1  records100/10000/10900_lr  records500/10000/10900_hr  \n",
       "2  records100/08000/08209_lr  records500/08000/08209_hr  \n",
       "3  records100/17000/17620_lr  records500/17000/17620_hr  \n",
       "4  records100/12000/12967_lr  records500/12000/12967_hr  \n",
       "\n",
       "[5 rows x 30 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "csv_fn = \"coorteeqsrafva.csv\"\n",
    "df = pd.read_csv(csv_fn, sep=\";\", index_col=0,)\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Format the Data\n",
    "To use this data in a random forrest classifierit will first need to formatted. The script, feature_selection.py, contains the function I use to do that. It keeps the following columns:\n",
    "\n",
    "- diagnosi: Rhythm type\n",
    "- age: Patient age\n",
    "- sex: Patient sex\n",
    "- height: Patient height\n",
    "- weight:Patient weight\n",
    "- heart_axis: The direction of the overall electrical activity of the heart\n",
    "- pacemaker: If the patient has an implanted pacemaker\n",
    "\n",
    "Most of the formatting is done by filling in missing values, and categorizing text numerically. For example, heart_axis can have values of LAD, AXL, MID, RAD, etc., these are converted to 0, 1, 2, 3, 4 ...\n",
    "\n",
    "Some notable formatting happens with age, height, and weight.\n",
    "\n",
    "- Age: missing age values are determined by the mean age of patients that are within +/- 2.5cm of the patient's height, and +/- 2kg of the patient's weight.\n",
    "    - If both height and weight are missing, the age is determined to be the mean age of the entire dataset.\n",
    "- Height: missing height values are determined by the mean height of the patients age demographic, as binned by 0-4yrs, 5-9yrs, 10-19yrs, and >20yrs\n",
    "- Weight: missing height values are determined by the mean height of the patients age demographic, as binned by in 5 year increments from 0 - 10yrs, and ten year increments from 10yrs and onwards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>diagnosi</th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>height</th>\n",
       "      <th>weight</th>\n",
       "      <th>heart_axis</th>\n",
       "      <th>pacemaker</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0</td>\n",
       "      <td>173.459649</td>\n",
       "      <td>84.785714</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>6</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0</td>\n",
       "      <td>173.459649</td>\n",
       "      <td>84.785714</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>173.459649</td>\n",
       "      <td>84.785714</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>1</td>\n",
       "      <td>164.000000</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>0</td>\n",
       "      <td>173.459649</td>\n",
       "      <td>84.785714</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   diagnosi   age  sex      height     weight  heart_axis  pacemaker\n",
       "0         0  54.0    0  173.459649  84.785714           0          0\n",
       "1         6  54.0    0  173.459649  84.785714           0          0\n",
       "2         0  55.0    0  173.459649  84.785714           1          0\n",
       "3         0  29.0    1  164.000000  56.000000           2          0\n",
       "4         0  57.0    0  173.459649  84.785714           0          0"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from feature_selection import format_df\n",
    "\n",
    "df = format_df(df)\n",
    "df.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train, Test, and Evaluate\n",
    "A random forrest classifier from Scikit-Learn is used to classify the data by \"diagnosi\".\n",
    "\n",
    "- The data is split into train and test datasets. This is done by withholding 20% of each rhythm type for testing.\n",
    "- Hyperparameter tuning is performed.\n",
    "- A classifier is trainged with the best hyperparameters on the full training dataset.\n",
    "- The classifier is tested and its results are reported"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5138, 6)\n",
      "(1290, 6)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\danrp\\Projects\\parisai\\ecg\\venv\\Lib\\site-packages\\sklearn\\metrics\\_classification.py:1344: UndefinedMetricWarning:\n",
      "\n",
      "Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "\n",
      "c:\\Users\\danrp\\Projects\\parisai\\ecg\\venv\\Lib\\site-packages\\sklearn\\metrics\\_classification.py:1344: UndefinedMetricWarning:\n",
      "\n",
      "Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "\n",
      "c:\\Users\\danrp\\Projects\\parisai\\ecg\\venv\\Lib\\site-packages\\sklearn\\metrics\\_classification.py:1344: UndefinedMetricWarning:\n",
      "\n",
      "Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from feature_selection import (\n",
    "    train_test_split_by_category,\n",
    "    train,\n",
    "    test\n",
    ")\n",
    "\n",
    "(X_train, y_train), (X_test, y_test) = train_test_split_by_category(df, \"diagnosi\")\n",
    "print(X_train.shape)\n",
    "print(X_test.shape)\n",
    "\n",
    "rf = train(X_train, y_train)\n",
    "report, cm = test(rf, X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\danrp\\Projects\\parisai\\ecg\\venv\\Lib\\site-packages\\sklearn\\metrics\\_classification.py:1344: UndefinedMetricWarning:\n",
      "\n",
      "Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "\n",
      "c:\\Users\\danrp\\Projects\\parisai\\ecg\\venv\\Lib\\site-packages\\sklearn\\metrics\\_classification.py:1344: UndefinedMetricWarning:\n",
      "\n",
      "Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "\n",
      "c:\\Users\\danrp\\Projects\\parisai\\ecg\\venv\\Lib\\site-packages\\sklearn\\metrics\\_classification.py:1344: UndefinedMetricWarning:\n",
      "\n",
      "Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "cells": {
          "values": [
           [
            "Sinus Rhythm",
            "Atrial Fibrillation",
            "Sinus Arrhythmia",
            "Normal Functioning Artificial Pacemaker",
            "Supraventricular Arrhythmia",
            "Bigeminal Pattern (Unknown Origin, SV or Ventricular)",
            "Atrial Flutter",
            "Supraventricular Tachycardia",
            "Paroxysmal Supraventricular Tachycardia",
            "Trigeminal Pattern (Unknown Origin, SV or Ventricular)",
            "macro avg",
            "weighted avg",
            ""
           ],
           [
            61.49,
            43.04,
            30.67,
            96.67,
            0,
            0,
            20,
            0,
            0,
            0,
            25.19,
            51.55,
            ""
           ],
           [
            78.79,
            33.66,
            14.84,
            96.67,
            0,
            0,
            13.33,
            0,
            0,
            0,
            23.73,
            56.67,
            ""
           ],
           [
            69.07,
            37.78,
            20,
            96.67,
            0,
            0,
            16,
            0,
            0,
            0,
            23.95,
            53.06,
            "accuracy"
           ],
           [
            693,
            303,
            155,
            60,
            32,
            17,
            15,
            6,
            5,
            4,
            1290,
            1290,
            "56.67%"
           ]
          ]
         },
         "header": {
          "values": [
           "",
           "precision (%)",
           "recall (%)",
           "f1-score (%)",
           "support"
          ]
         },
         "type": "table"
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "colorscale": [
          [
           0,
           "rgb(247,251,255)"
          ],
          [
           0.125,
           "rgb(222,235,247)"
          ],
          [
           0.25,
           "rgb(198,219,239)"
          ],
          [
           0.375,
           "rgb(158,202,225)"
          ],
          [
           0.5,
           "rgb(107,174,214)"
          ],
          [
           0.625,
           "rgb(66,146,198)"
          ],
          [
           0.75,
           "rgb(33,113,181)"
          ],
          [
           0.875,
           "rgb(8,81,156)"
          ],
          [
           1,
           "rgb(8,48,107)"
          ]
         ],
         "type": "heatmap",
         "x": [
          "NSR",
          "AFIB",
          "SARRH",
          "PACE",
          "SVARR",
          "BIGU",
          "AFLT",
          "SVTAC",
          "PSVT",
          "TRIGU"
         ],
         "y": [
          "TRIGU",
          "PSVT",
          "SVTAC",
          "AFLT",
          "BIGU",
          "SVARR",
          "PACE",
          "SARRH",
          "AFIB",
          "NSR"
         ],
         "z": [
          [
           2,
           2,
           0,
           0,
           0,
           0,
           0,
           0,
           0,
           0
          ],
          [
           4,
           0,
           0,
           0,
           0,
           0,
           0,
           1,
           0,
           0
          ],
          [
           3,
           1,
           0,
           0,
           0,
           0,
           1,
           0,
           1,
           0
          ],
          [
           6,
           4,
           0,
           0,
           0,
           0,
           2,
           3,
           0,
           0
          ],
          [
           9,
           5,
           1,
           0,
           2,
           0,
           0,
           0,
           0,
           0
          ],
          [
           23,
           6,
           2,
           0,
           0,
           0,
           1,
           0,
           0,
           0
          ],
          [
           1,
           1,
           0,
           58,
           0,
           0,
           0,
           0,
           0,
           0
          ],
          [
           112,
           19,
           23,
           0,
           1,
           0,
           0,
           0,
           0,
           0
          ],
          [
           182,
           102,
           9,
           2,
           3,
           1,
           3,
           1,
           0,
           0
          ],
          [
           546,
           97,
           40,
           0,
           5,
           2,
           3,
           0,
           0,
           0
          ]
         ]
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Confusion Matrix"
        },
        "xaxis": {
         "title": {
          "text": "Predicted Label"
         }
        },
        "yaxis": {
         "title": {
          "text": "True Label"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import display\n",
    "\n",
    "from feature_selection import (\n",
    "    select_features,\n",
    "    create_confusion_matrix,\n",
    "    create_report_table,\n",
    "    test\n",
    ")\n",
    "\n",
    "report, cm = test(rf, X_test, y_test)\n",
    "\n",
    "fig1 = create_report_table(report)\n",
    "fig2 = create_confusion_matrix(cm)\n",
    "\n",
    "display(fig1)\n",
    "display(fig2)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Selection\n",
    "\n",
    "The features are scored with impurity-based feature importances. The higher, the more important the feature. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature. It is also known as the Gini importance.\n",
    "\n",
    "### Accuracy\n",
    "We can see from our accuracy measures that patient demographic information, by itself, isn't enough to predict what type of rhythm an ecg strip is presenting (except for pace rhythm, but that info is usually in the patient info). However, accuracy is not the point of this experiment. I just wanted to see what patient info is most indicitive of rhythm types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('age', 0.40534650622274254),\n",
       " ('weight', 0.21370480224947813),\n",
       " ('pacemaker', 0.14582698726070603),\n",
       " ('height', 0.14013217086754282),\n",
       " ('heart_axis', 0.08061118885735595),\n",
       " ('sex', 0.014378344542174477)]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features = select_features(rf, X_train)\n",
    "features"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I'm surprized that sex is not a more important feature. Let's continue this experiment by performing binary classifications for each rhythm type, and see if feature importance changes by rhythm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NSR 0.6263565891472869 [('age', 0.4073111570626327), ('weight', 0.2470546280431409), ('height', 0.15697779479950738), ('heart_axis', 0.08926976906706484), ('pacemaker', 0.08454175446532865), ('sex', 0.014844896562325566)]\n",
      "AFIB 0.7542635658914729 [('age', 0.4737558391343282), ('weight', 0.2109322039154629), ('height', 0.16058781716387296), ('heart_axis', 0.10863000797186372), ('pacemaker', 0.030600830909381253), ('sex', 0.015493300905091014)]\n",
      "SARRH 0.8697674418604651 [('age', 0.5037351378206264), ('weight', 0.27494626660326626), ('height', 0.12893133732810608), ('heart_axis', 0.07056078894170664), ('sex', 0.014995789766459362), ('pacemaker', 0.006830679539835362)]\n",
      "PACE 0.9968992248062015 [('pacemaker', 0.8926332719582352), ('heart_axis', 0.03980415158779869), ('height', 0.02513683943591719), ('age', 0.022580873650903128), ('weight', 0.01782374299730337), ('sex', 0.0020211203698424)]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\danrp\\Projects\\parisai\\ecg\\venv\\Lib\\site-packages\\sklearn\\metrics\\_classification.py:1344: UndefinedMetricWarning:\n",
      "\n",
      "Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "\n",
      "c:\\Users\\danrp\\Projects\\parisai\\ecg\\venv\\Lib\\site-packages\\sklearn\\metrics\\_classification.py:1344: UndefinedMetricWarning:\n",
      "\n",
      "Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "\n",
      "c:\\Users\\danrp\\Projects\\parisai\\ecg\\venv\\Lib\\site-packages\\sklearn\\metrics\\_classification.py:1344: UndefinedMetricWarning:\n",
      "\n",
      "Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVARR 0.9751937984496124 [('age', 0.6696414975998143), ('heart_axis', 0.16315576233293125), ('weight', 0.12275713097948528), ('height', 0.030939891145682637), ('pacemaker', 0.0069277705270664475), ('sex', 0.0065779474150202055)]\n",
      "BIGU 0.986046511627907 [('age', 0.38026545661276506), ('weight', 0.2479253662659412), ('height', 0.22711161330480964), ('heart_axis', 0.11451368316665986), ('sex', 0.024511325061290404), ('pacemaker', 0.005672555588533842)]\n",
      "AFLT 0.9852713178294573 [('age', 0.38880910932772184), ('weight', 0.22906977004932533), ('height', 0.1827470392144765), ('heart_axis', 0.1687134542369751), ('sex', 0.017805539214813267), ('pacemaker', 0.01285508795668805)]\n",
      "SVTAC 0.9922480620155039 [('age', 0.35243784974405024), ('height', 0.28637023028296515), ('weight', 0.2745527867709492), ('heart_axis', 0.0454357006081656), ('sex', 0.03105272487354569), ('pacemaker', 0.01015070772032418)]\n",
      "PSVT 0.9953488372093023 [('age', 0.4124952011655212), ('weight', 0.26454851648086186), ('height', 0.2416128020093787), ('heart_axis', 0.052018662855243356), ('sex', 0.02732178398294316), ('pacemaker', 0.002003033506051681)]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from feature_selection import CM_LABELS\n",
    "\n",
    "orig_trn_lbls = y_train\n",
    "orig_tst_lbls = y_test\n",
    "\n",
    "n_lbls = orig_trn_lbls.max()\n",
    "\n",
    "for i in range(n_lbls):\n",
    "    curr_y_trn = pd.Series(np.where(orig_trn_lbls == i, 1, 0))\n",
    "    curr_y_tst = pd.Series(np.where(orig_tst_lbls == i, 1, 0))\n",
    "\n",
    "    rf = train(X_train, curr_y_trn)\n",
    "    report, cm = test(rf, X_test, curr_y_tst)\n",
    "    features = select_features(rf, X_train)\n",
    "    print(CM_LABELS[i], report[\"accuracy\"], features)\n",
    "    "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results\n",
    "Wow! That works surprizingly well. It's likely that these results are a product of the dataset and would not generalize over a real world population. Never-the-less, what is clear is that age, height, and weight are important for all rhythm types. I'm still surprized that sex is always one of the least important features.\n",
    "\n",
    "In effort to choose features that would easily available for most patients, I will be using age, height, weight, pacemaker, and sex as input the CNN. Even though sex does not seem to be an important feature in this dataset, it is considered by most physicians to be important."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}