test_task_IMS_vk_AppliedAI.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oLRmrYbyJWEN"
      },
      "source": [
        "# TASK:\n",
        "\n",
        "В .csv файле представлена совокупность данных:\n",
        " - заголовок новости,\n",
        " - тема новости,\n",
        " - текст новости.\n",
        "\n",
        "Все слова в тексте лемматизированы, а сами тексты очищены от стоп-слов.\n",
        "\n",
        "Ссылка на .csv файл\n",
        "\n",
        "1. Вам необходимо выбрать и сравнить между собой не менее трех различных алгоритмов **классификации текстов**,\n",
        "\n",
        "2. проанализировать полученные результаты,\n",
        "\n",
        "3. сделать обоснованные выводы о применимости выбранных методов и качестве решения задачи.\n",
        "\n",
        "В качестве ответа необходимо загрузить .ipynb файл с кодом, комментариями и выводами.\n",
        "\n",
        "Успехов!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MP_iVdoIIt1Z"
      },
      "source": [
        "# Libs import"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "-Otio4dIIDP9"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import random\n",
        "\n",
        "random.seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%tensorflow_version 2.x\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense, Embedding, GRU, LSTM, Activation, Flatten, GlobalMaxPool1D, Dropout, Conv1D\n",
        "from tensorflow.keras import utils\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau, EarlyStopping\n",
        "from keras.optimizers import Adam\n",
        "from keras.losses import binary_crossentropy\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7ognKt8c1C1_",
        "outputId": "09b5823d-b77f-44b9-b393-df822f7b3e3e"
      },
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Colab only includes TensorFlow 2.x; %tensorflow_version has no effect.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "id": "263vJdKNbuDn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c0ee1ec5-7381-4cd0-aa69-ee6372acc741"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 201
        },
        "id": "9YdHypYsI3xX",
        "outputId": "fd1b23eb-da9e-4f69-99b8-5ad807f389b8"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                            headline  \\\n",
              "0  Теперь подарки можно выбрать на сайте при помо...   \n",
              "1                     Тренды розничной торговли 2016   \n",
              "2  Порталы сравнения цен вынуждают мелких ритейле...   \n",
              "3              9 беспроигрышных подарков для мужчины   \n",
              "4  Китайские производители сокращают свой бизнес ...   \n",
              "\n",
              "                                                body topic  \n",
              "0  поиск подарок любой праздник гораздо просто бл...    IT  \n",
              "1  платформа соцсеть продолжать служить шопинг-пл...    IT  \n",
              "2  последний время активно расти конкуренция обла...    IT  \n",
              "3  каждый накануне февраль возникать сложный вопр...    IT  \n",
              "4  прекращать поставка новый автомобиль дилер рос...  Авто  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-d1eb1fa4-16ef-4716-88f2-c4fb69e44875\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>headline</th>\n",
              "      <th>body</th>\n",
              "      <th>topic</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Теперь подарки можно выбрать на сайте при помо...</td>\n",
              "      <td>поиск подарок любой праздник гораздо просто бл...</td>\n",
              "      <td>IT</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Тренды розничной торговли 2016</td>\n",
              "      <td>платформа соцсеть продолжать служить шопинг-пл...</td>\n",
              "      <td>IT</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Порталы сравнения цен вынуждают мелких ритейле...</td>\n",
              "      <td>последний время активно расти конкуренция обла...</td>\n",
              "      <td>IT</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>9 беспроигрышных подарков для мужчины</td>\n",
              "      <td>каждый накануне февраль возникать сложный вопр...</td>\n",
              "      <td>IT</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Китайские производители сокращают свой бизнес ...</td>\n",
              "      <td>прекращать поставка новый автомобиль дилер рос...</td>\n",
              "      <td>Авто</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d1eb1fa4-16ef-4716-88f2-c4fb69e44875')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-d1eb1fa4-16ef-4716-88f2-c4fb69e44875 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-d1eb1fa4-16ef-4716-88f2-c4fb69e44875');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-43220d23-5e3c-4e02-8c48-91e4633c7821\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-43220d23-5e3c-4e02-8c48-91e4633c7821')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-43220d23-5e3c-4e02-8c48-91e4633c7821 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "df = pd.read_csv('drive/MyDrive/articles_lemmatized_no_SW.csv', index_col=0)\n",
        "df = df.reset_index().drop('index', axis = 1)\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_xyge-OlIyEg"
      },
      "source": [
        "# Data first view\n",
        "\n",
        "- Количество классов 37 (важно: выборка крайне не сбалансирована, экземпляров одного класса +- 26%, другого 0.0001%) => Некоторые классы детектироваться не будут\n",
        "- Описание данных (мин/средние/макс) кол-во слов/символов в текстах"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lbEBDjHpLY6y",
        "outputId": "a9c597fd-0d5b-4ae0-94af-406f2030c8b6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Теперь подарки можно выбрать на сайте при помощи заданных критериев \n",
            "\n",
            "поиск подарок любой праздник гораздо просто благодаря использование параметр поиск портал иметь функционал давать возможность выбирать подарок использовать детальный параметр поиск любой подарок второй половинка член семья друг родитель выбирать очень просто хороший подарок выбирать заблаговременно учитывать вкус пристрастие интерес человек адресовать выбирать подарок нужно учитывать кто вручать например косметика принимать дарить человек находиться близкий отношение точно постельный белье принимать дарить родственник пара правило этикет принимать считаться взгляд суеверие человек поэтому подарок платок слеза часы разлука суеверный человек дарить стоять стоимость подарок немаловажный фактор многое зависеть отношение человек самый дело важный сколько стоять главное приятный демонстрировать отношение человек поиск критерий помогать находить идеальный подарок начало нужно выбирать раздел подарок далее увидеть раздел следовать начинать выбор критерий праздник бюджет интерес мужчина женщина возраст предполагать необходимо выбирать подарок любимый женщина выбирать категория женщина открываться следующий блок критерий семья отношение дружба детализировать дальнейший поиск подарок выбирать раздел отношение переходить следующий раздел критерий девушка любимый молодожены невеста указывать критерий любимый нажимать кнопка фильтровать выбирать критерий доступный масса раздел состоять идеальный подарок любимый женщина косметика бижутерия украшение сумка мобильный телефон планшет нужно сделать подарок дедушка заходить раздел подарок выбирать категория мужчина видеть следующий блок критерий семья отношение дружба необходимо выбирать параметр семья далеко экран появляться список пункт брат дедушка муж папа сын выбирать раздел дедушка применять фильтр сайт выдавать большой выбор подарок взрослый мужчина мужской парфюмерия галстук визитница зонт лыжа перфоратор многий другой расширенный поиск указывать возраст интерес фильтр позволять упрощать поиск именно цена выбирать необходимый диапазон цена представлять интересовать подарок получать итог образ выбирать любой подарок любой человек любой повод система поиск критерий очень простой справляться человек пользоваться компьютерный техника поиск любой подарок вызывать трудность иметь представление подарить поиск критерий позволять реализовать задача выбирать интересовать критерий сужать перечень сотня тысяча подарок иметься сайт наиболее подходящий подарок доступный покупка кроме подарок сравнивать цена разный магазин \n",
            "\n",
            "IT \n",
            "\n"
          ]
        }
      ],
      "source": [
        "print(df.iloc[0].headline, '\\n')\n",
        "print(df.iloc[0].body, '\\n')\n",
        "print(df.iloc[0].topic, '\\n')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QosILHwsa8vd",
        "outputId": "cfaf3983-ccba-45c2-ff62-29bb39d5eebd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "headline    object\n",
              "body        object\n",
              "topic       object\n",
              "dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "df.dtypes"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6gaILrX0a8vf",
        "outputId": "46ded138-3211-4341-afc0-c8b808b504e5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                  headline  \\\n",
              "count                                56500   \n",
              "unique                               56396   \n",
              "top     Спортивные выходные с «Известиями»   \n",
              "freq                                    32   \n",
              "\n",
              "                                                     body  topic  \n",
              "count                                               56499  56500  \n",
              "unique                                              56498     37  \n",
              "top     футбольный власть расследовать поведение росси...    Мир  \n",
              "freq                                                    2  15131  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-a6ccbd71-45c2-476e-9472-ba99149bbb13\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>headline</th>\n",
              "      <th>body</th>\n",
              "      <th>topic</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>56500</td>\n",
              "      <td>56499</td>\n",
              "      <td>56500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>unique</th>\n",
              "      <td>56396</td>\n",
              "      <td>56498</td>\n",
              "      <td>37</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>top</th>\n",
              "      <td>Спортивные выходные с «Известиями»</td>\n",
              "      <td>футбольный власть расследовать поведение росси...</td>\n",
              "      <td>Мир</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>freq</th>\n",
              "      <td>32</td>\n",
              "      <td>2</td>\n",
              "      <td>15131</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a6ccbd71-45c2-476e-9472-ba99149bbb13')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-a6ccbd71-45c2-476e-9472-ba99149bbb13 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-a6ccbd71-45c2-476e-9472-ba99149bbb13');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-570c19f0-f602-4cf4-b012-2064fcadbcce\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-570c19f0-f602-4cf4-b012-2064fcadbcce')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-570c19f0-f602-4cf4-b012-2064fcadbcce button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "df.describe()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qjRv-8jBI4PZ",
        "outputId": "8275d8ce-872b-4b3d-eaac-f61cc78bc806"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Классов : 37 \n",
            "\n",
            "                                     topic     topic\n",
            "Мир                                  15131  0.267805\n",
            "Общество                             12147  0.214991\n",
            "Политика                              7359  0.130248\n",
            "Страна                                5280  0.093451\n",
            "Экономика                             5032  0.089062\n",
            "Спорт                                 4881  0.086389\n",
            "Культура                              2635  0.046637\n",
            "Армия                                 1295  0.022920\n",
            "Наука                                  824  0.014584\n",
            "Гаджеты & Телеком                      583  0.010319\n",
            "РИО-2016                               449  0.007947\n",
            "Авто                                   131  0.002319\n",
            "Москва                                 118  0.002088\n",
            "День в истории                         108  0.001912\n",
            "Выборы в США                            96  0.001699\n",
            "Выборы-2016                             92  0.001628\n",
            "ПМЭФ                                    63  0.001115\n",
            "День Победы                             62  0.001097\n",
            "ЧЕ 2016                                 56  0.000991\n",
            "Известия: подробности                   38  0.000673\n",
            "Россия - регионы                        26  0.000460\n",
            "Новости компаний                        17  0.000301\n",
            "Финансы                                 10  0.000177\n",
            "Транспорт                                9  0.000159\n",
            "Недвижимость                             7  0.000124\n",
            "Технологическое предпринимательство      6  0.000106\n",
            "Нефть и газ                              6  0.000106\n",
            "Путешествуем по России                   5  0.000088\n",
            "Промышленность                           5  0.000088\n",
            "Образование                              5  0.000088\n",
            "Знай наших                               5  0.000088\n",
            "Шопинг                                   5  0.000088\n",
            "Инновации                                4  0.000071\n",
            "IT                                       4  0.000071\n",
            "КНАУФ                                    2  0.000035\n",
            "Формула лидерства                        2  0.000035\n",
            "Деловая среда                            2  0.000035\n"
          ]
        }
      ],
      "source": [
        "\n",
        "print(\"Классов :\", len(df.topic.unique()), '\\n')\n",
        "counts = df.topic.value_counts()\n",
        "percs = df.topic.value_counts().apply(lambda x: round(x/56500,6))\n",
        "print(pd.concat([counts,percs],axis=1))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7VjB126GOSG-",
        "outputId": "e5c418e8-615e-4c73-992e-4784f9a833f6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "BODY WORDS min:2.0; mean:168.30343899892034; max:5954.0\n",
            " ----------------------------------------\n",
            "BODY SYMB min:9.0; mean:1484.4217596771625; max:48095.0\n",
            " ----------------------------------------\n",
            "HEAD WORDS min:1; mean:7.913911504424779; max:15\n",
            " ----------------------------------------\n",
            "HEAD SYMB min:4; mean:56.1170796460177; max:75\n",
            " ----------------------------------------\n"
          ]
        }
      ],
      "source": [
        "body_words = df.body.str.split().str.len()\n",
        "print(f\"BODY WORDS min:{body_words.min()}; mean:{(body_words.mean())}; max:{body_words.max()}\\n\",'-'*40)\n",
        "\n",
        "body_symb = df.body.str.len()\n",
        "print(f\"BODY SYMB min:{body_symb.min()}; mean:{(body_symb.mean())}; max:{body_symb.max()}\\n\",'-'*40)\n",
        "\n",
        "head_words = df.headline.str.split().str.len()\n",
        "print(f\"HEAD WORDS min:{head_words.min()}; mean:{(head_words.mean())}; max:{head_words.max()}\\n\",'-'*40)\n",
        "\n",
        "head_symb = df.headline.str.len()\n",
        "print(f\"HEAD SYMB min:{head_symb.min()}; mean:{(head_symb.mean())}; max:{head_symb.max()}\\n\",'-'*40)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B-UynRQTa8vm"
      },
      "source": [
        "# Data Preparation\n",
        "\n",
        "В этом разделе я описываю обработку данных:\n",
        "\n",
        "- принято решение обучать модель только на текстах (НЕ включая заголовки), также произведена образка текстов до 100 слов СНАЧАЛА, тк новостные топики пишутся в стиле снижения важности информации\n",
        "\n",
        "- удаление пустых занчений\n",
        "- кодирование классов\n",
        "- разбиение на тренировочную и тестовую выборки\n",
        "- вкторизация текстов + фиксирование их длинны"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(df.shape)\n",
        "df = df.dropna(axis=0)\n",
        "print(df.shape)\n",
        "num_words = 10000\n",
        "max_len = 100\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OHh-5EUnWh4q",
        "outputId": "1114288b-3388-4280-b82b-3a6d33d9856a"
      },
      "execution_count": 74,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(56499, 3)\n",
            "(56499, 3)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 57,
      "metadata": {
        "id": "QshI944Qa8vo",
        "outputId": "c5ebd66c-8bd2-4970-a86d-57c90ae601d9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['IT', 'Авто', 'Армия', 'Выборы в США', 'Выборы-2016',\n",
              "       'Гаджеты & Телеком', 'Деловая среда', 'День Победы',\n",
              "       'День в истории', 'Знай наших', 'Известия: подробности',\n",
              "       'Инновации', 'КНАУФ', 'Культура', 'Мир', 'Москва', 'Наука',\n",
              "       'Недвижимость', 'Нефть и газ', 'Новости компаний', 'Образование',\n",
              "       'Общество', 'ПМЭФ', 'Политика', 'Промышленность',\n",
              "       'Путешествуем по России', 'РИО-2016', 'Россия - регионы', 'Спорт',\n",
              "       'Страна', 'Технологическое предпринимательство', 'Транспорт',\n",
              "       'Финансы', 'Формула лидерства', 'ЧЕ 2016', 'Шопинг', 'Экономика'],\n",
              "      dtype=object)"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ],
      "source": [
        "from sklearn.preprocessing import LabelEncoder\n",
        "le = LabelEncoder()\n",
        "le.fit(df.topic.unique())\n",
        "le.classes_\n",
        "# le.transform([1, 1, 2, 6])\n",
        "# le.inverse_transform([0, 0, 1, 2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 58,
      "metadata": {
        "id": "afbiM4e1a8vr",
        "outputId": "95bcf79d-c459-4cf9-8be3-460e608c8aac",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 201
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                            headline  \\\n",
              "0  Теперь подарки можно выбрать на сайте при помо...   \n",
              "1                     Тренды розничной торговли 2016   \n",
              "2  Порталы сравнения цен вынуждают мелких ритейле...   \n",
              "3              9 беспроигрышных подарков для мужчины   \n",
              "4  Китайские производители сокращают свой бизнес ...   \n",
              "\n",
              "                                                body  class  \n",
              "0  поиск подарок любой праздник гораздо просто бл...      0  \n",
              "1  платформа соцсеть продолжать служить шопинг-пл...      0  \n",
              "2  последний время активно расти конкуренция обла...      0  \n",
              "3  каждый накануне февраль возникать сложный вопр...      0  \n",
              "4  прекращать поставка новый автомобиль дилер рос...      1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-d172ba19-5eb8-437e-9a28-8afd8dab1b9f\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>headline</th>\n",
              "      <th>body</th>\n",
              "      <th>class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Теперь подарки можно выбрать на сайте при помо...</td>\n",
              "      <td>поиск подарок любой праздник гораздо просто бл...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Тренды розничной торговли 2016</td>\n",
              "      <td>платформа соцсеть продолжать служить шопинг-пл...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Порталы сравнения цен вынуждают мелких ритейле...</td>\n",
              "      <td>последний время активно расти конкуренция обла...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>9 беспроигрышных подарков для мужчины</td>\n",
              "      <td>каждый накануне февраль возникать сложный вопр...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Китайские производители сокращают свой бизнес ...</td>\n",
              "      <td>прекращать поставка новый автомобиль дилер рос...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d172ba19-5eb8-437e-9a28-8afd8dab1b9f')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-d172ba19-5eb8-437e-9a28-8afd8dab1b9f button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-d172ba19-5eb8-437e-9a28-8afd8dab1b9f');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-b76f96b7-a5b6-4229-987d-de21a8523af8\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b76f96b7-a5b6-4229-987d-de21a8523af8')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-b76f96b7-a5b6-4229-987d-de21a8523af8 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 58
        }
      ],
      "source": [
        "labels = pd.DataFrame(le.transform(df.topic),columns=['class'], dtype=int)\n",
        "df1 = pd.concat([df,labels], axis=1, join='inner').drop('topic', axis=1)\n",
        "df1.head()\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# разбиение данных на тренировочную и тестовую части\n",
        "df1 = df1.sample(frac=1).reset_index(drop=True)\n",
        "x_train, x_test, y_train, y_test = train_test_split(df1.body, df1['class'], test_size=0.2, random_state=42)"
      ],
      "metadata": {
        "id": "nYNuMS84KcVG"
      },
      "execution_count": 62,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# fit tokeniser\n",
        "tokenizer = Tokenizer(num_words=num_words)\n",
        "tokenizer.fit_on_texts(x_train)\n",
        "\n",
        "#train\n",
        "sequences = tokenizer.texts_to_sequences(x_train)\n",
        "x_train = pad_sequences(sequences, maxlen=max_len)\n",
        "\n",
        "# test\n",
        "# print(x_test[:,0])\n",
        "sequences = tokenizer.texts_to_sequences(x_test)\n",
        "x_test = pad_sequences(sequences, maxlen=max_len)\n",
        "# print(sequences[0])"
      ],
      "metadata": {
        "id": "bu-J5OZa1dhT"
      },
      "execution_count": 63,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# LSTM\n",
        "\n",
        "В этом разделе приведена реализация простой RNN с ячейками LSTM.\n",
        "\n",
        "Результаты предсказаний даже на такой простой сети сравнительно классные (на мой взгляд - самое перспективное мое решение)."
      ],
      "metadata": {
        "id": "ggcSO-ThPbKV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "lstm = Sequential()\n",
        "lstm.add(Embedding(num_words, 64, input_length=max_len))\n",
        "lstm.add(LSTM(128) )\n",
        "lstm.add(Dense(37, activation='softmax'))\n",
        "\n",
        "\n",
        "lstm.compile(optimizer='adam',\n",
        "              loss='sparse_categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "lstm.summary()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "20e6iZRx1CqK",
        "outputId": "b98b56dc-6f46-48d0-c5b5-f50f21e3f0d8"
      },
      "execution_count": 88,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_30\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " embedding_28 (Embedding)    (None, 100, 64)           640000    \n",
            "                                                                 \n",
            " lstm_6 (LSTM)               (None, 128)               98816     \n",
            "                                                                 \n",
            " dense_36 (Dense)            (None, 37)                4773      \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 743589 (2.84 MB)\n",
            "Trainable params: 743589 (2.84 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model_save_path = 'best_model_lstm.h5'\n",
        "checkpoint_callback_lstm = ModelCheckpoint(model_save_path,\n",
        "                                      monitor='val_accuracy',\n",
        "                                      save_best_only=True,\n",
        "                                      verbose=1)\n",
        "\n",
        "history = lstm.fit(x_train,\n",
        "                    y_train,\n",
        "                    epochs=10,\n",
        "                    batch_size=128,\n",
        "                    validation_split=0.1,\n",
        "                    callbacks=[checkpoint_callback_lstm])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "__KFTu131ClR",
        "outputId": "6a4951e8-8a86-4e60-bd75-723bec2ca8c1"
      },
      "execution_count": 89,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "318/318 [==============================] - ETA: 0s - loss: 1.8156 - accuracy: 0.4018\n",
            "Epoch 1: val_accuracy improved from -inf to 0.59049, saving model to best_model_lstm.h5\n",
            "318/318 [==============================] - 82s 248ms/step - loss: 1.8156 - accuracy: 0.4018 - val_loss: 1.3394 - val_accuracy: 0.5905\n",
            "Epoch 2/5\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "318/318 [==============================] - ETA: 0s - loss: 1.1044 - accuracy: 0.6532\n",
            "Epoch 2: val_accuracy improved from 0.59049 to 0.68097, saving model to best_model_lstm.h5\n",
            "318/318 [==============================] - 26s 82ms/step - loss: 1.1044 - accuracy: 0.6532 - val_loss: 1.0421 - val_accuracy: 0.6810\n",
            "Epoch 3/5\n",
            "  1/318 [..............................] - ETA: 4s - loss: 0.9266 - accuracy: 0.7109"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "318/318 [==============================] - ETA: 0s - loss: 0.8729 - accuracy: 0.7294\n",
            "Epoch 3: val_accuracy improved from 0.68097 to 0.71571, saving model to best_model_lstm.h5\n",
            "318/318 [==============================] - 16s 51ms/step - loss: 0.8729 - accuracy: 0.7294 - val_loss: 0.9188 - val_accuracy: 0.7157\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4/5\n",
            "318/318 [==============================] - ETA: 0s - loss: 0.7262 - accuracy: 0.7779\n",
            "Epoch 4: val_accuracy improved from 0.71571 to 0.73761, saving model to best_model_lstm.h5\n",
            "318/318 [==============================] - 18s 56ms/step - loss: 0.7262 - accuracy: 0.7779 - val_loss: 0.8520 - val_accuracy: 0.7376\n",
            "Epoch 5/5\n",
            " 13/318 [>.............................] - ETA: 2s - loss: 0.6377 - accuracy: 0.8191"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "318/318 [==============================] - ETA: 0s - loss: 0.6157 - accuracy: 0.8084\n",
            "Epoch 5: val_accuracy improved from 0.73761 to 0.73783, saving model to best_model_lstm.h5\n",
            "318/318 [==============================] - 11s 35ms/step - loss: 0.6157 - accuracy: 0.8084 - val_loss: 0.8505 - val_accuracy: 0.7378\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "lstm.evaluate(x=x_test,y=y_test)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xcrTdq6F1Chi",
        "outputId": "fd35f239-2bd1-40dd-9c71-3597f987e4a2"
      },
      "execution_count": 90,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "354/354 [==============================] - 3s 8ms/step - loss: 0.8364 - accuracy: 0.7385\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.836440920829773, 0.7384955883026123]"
            ]
          },
          "metadata": {},
          "execution_count": 90
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# CNN\n",
        "\n",
        "В этой главе я пробую обучить сверточную нейросеть, использую таке же параметры компиляции.\n"
      ],
      "metadata": {
        "id": "IgMb4-b96ULo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "cnn = Sequential()\n",
        "cnn.add(Embedding(input_dim=100, output_dim=64, input_length=100))\n",
        "cnn.add(Dropout(rate=0.5))\n",
        "cnn.add(Dense(512))\n",
        "cnn.add(Dropout(rate=0.5))\n",
        "cnn.add(Conv1D(filters=250, kernel_size=3, activation='relu'))\n",
        "cnn.add(GlobalMaxPool1D())\n",
        "cnn.add(Dense(512))\n",
        "cnn.add(Dropout(rate=0.5))\n",
        "cnn.add(Dense(128))\n",
        "cnn.add(Dropout(rate=0.5))\n",
        "cnn.add(Dense(37, activation='softmax'))\n",
        "\n",
        "cnn.compile(optimizer='adam',\n",
        "              loss='sparse_categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "cnn.summary()\n",
        "\n",
        "callbacks = [\n",
        "    ReduceLROnPlateau(),\n",
        "    EarlyStopping(patience=4),\n",
        "    ModelCheckpoint(filepath='model-conv1d.h5', save_best_only=True)\n",
        "]\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3SQfy6fL6WOI",
        "outputId": "133f3007-7174-4e97-e309-7f4b93994e72"
      },
      "execution_count": 102,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_38\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " embedding_32 (Embedding)    (None, 100, 64)           6400      \n",
            "                                                                 \n",
            " dropout_45 (Dropout)        (None, 100, 64)           0         \n",
            "                                                                 \n",
            " dense_57 (Dense)            (None, 100, 512)          33280     \n",
            "                                                                 \n",
            " dropout_46 (Dropout)        (None, 100, 512)          0         \n",
            "                                                                 \n",
            " conv1d_26 (Conv1D)          (None, 98, 250)           384250    \n",
            "                                                                 \n",
            " global_max_pooling1d_23 (G  (None, 250)               0         \n",
            " lobalMaxPooling1D)                                              \n",
            "                                                                 \n",
            " dense_58 (Dense)            (None, 512)               128512    \n",
            "                                                                 \n",
            " dropout_47 (Dropout)        (None, 512)               0         \n",
            "                                                                 \n",
            " dense_59 (Dense)            (None, 128)               65664     \n",
            "                                                                 \n",
            " dropout_48 (Dropout)        (None, 128)               0         \n",
            "                                                                 \n",
            " dense_60 (Dense)            (None, 37)                4773      \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 622879 (2.38 MB)\n",
            "Trainable params: 622879 (2.38 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x_train[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QWu_GbF6MJNF",
        "outputId": "87e605b1-80d3-45de-8d07-6a3a4db07a4a"
      },
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 183,  164, 2253,  137, 5894,   60,  541,  142, 7571,  831, 2915,\n",
              "       1597,   90,  813,  349,  669,  366,  155, 3751,  837,  556,   27,\n",
              "        463,  831,  158,  130, 4746, 5985,  831, 2050,  837,  106,   72,\n",
              "       2696,  229, 9736, 1524,   49,   23,  836,  327,  304, 2604,  905,\n",
              "        255,  702,  105,  255,  837,   56, 2783,  304, 2854, 1082,  106,\n",
              "         99,   58,  278,   15,   25,    7,  541,  142,  326,  312,  124,\n",
              "       1652,  342,   37,  653,  257,  237,  800,  202,  279,  335,   68,\n",
              "          6,   75,  283,  166,  439,  110, 1510,  166,  720, 4497,   56,\n",
              "        908,  202, 7488, 2248,  377,  908, 4237,  728,  166,  564,  183,\n",
              "        113], dtype=int32)"
            ]
          },
          "metadata": {},
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "history = cnn.fit(x_train, y_train,\n",
        "                    epochs=10,\n",
        "                    batch_size=64,\n",
        "                    validation_split=0.1,\n",
        "                    callbacks=callbacks)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HKFGTN7ZGmPG",
        "outputId": "78dce595-e76e-4559-9671-69518accb457"
      },
      "execution_count": 103,
      "outputs": [
        {
          "metadata": {
            "tags": null
          },
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "636/636 [==============================] - 104s 159ms/step - loss: 1.7212 - accuracy: 0.4355 - val_loss: 1.4393 - val_accuracy: 0.5462 - lr: 0.0010\n",
            "Epoch 2/10\n"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "636/636 [==============================] - 37s 58ms/step - loss: 1.4508 - accuracy: 0.5266 - val_loss: 1.3850 - val_accuracy: 0.5500 - lr: 0.0010\n",
            "Epoch 3/10\n",
            "  6/636 [..............................] - ETA: 6s - loss: 1.3329 - accuracy: 0.5573"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "636/636 [==============================] - 28s 44ms/step - loss: 1.4089 - accuracy: 0.5399 - val_loss: 1.3383 - val_accuracy: 0.5591 - lr: 0.0010\n",
            "Epoch 4/10\n",
            "  9/636 [..............................] - ETA: 8s - loss: 1.4102 - accuracy: 0.5625"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "636/636 [==============================] - 23s 36ms/step - loss: 1.3884 - accuracy: 0.5432 - val_loss: 1.3182 - val_accuracy: 0.5684 - lr: 0.0010\n",
            "Epoch 5/10\n",
            "  9/636 [..............................] - ETA: 8s - loss: 1.3775 - accuracy: 0.5399"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "636/636 [==============================] - 20s 31ms/step - loss: 1.3808 - accuracy: 0.5449 - val_loss: 1.3000 - val_accuracy: 0.5810 - lr: 0.0010\n",
            "Epoch 6/10\n",
            "  6/636 [..............................] - ETA: 7s - loss: 1.3333 - accuracy: 0.5807 "
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "636/636 [==============================] - 19s 30ms/step - loss: 1.3711 - accuracy: 0.5476 - val_loss: 1.2985 - val_accuracy: 0.5701 - lr: 0.0010\n",
            "Epoch 7/10\n",
            "  7/636 [..............................] - ETA: 5s - loss: 1.3151 - accuracy: 0.5201"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "636/636 [==============================] - 14s 23ms/step - loss: 1.3659 - accuracy: 0.5504 - val_loss: 1.3117 - val_accuracy: 0.5569 - lr: 0.0010\n",
            "Epoch 8/10\n",
            "636/636 [==============================] - 13s 21ms/step - loss: 1.3540 - accuracy: 0.5546 - val_loss: 1.2959 - val_accuracy: 0.5763 - lr: 0.0010\n",
            "Epoch 9/10\n",
            " 13/636 [..............................] - ETA: 5s - loss: 1.3909 - accuracy: 0.5385"
          ]
        },
        {
          "metadata": {
            "tags": null
          },
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "636/636 [==============================] - 9s 15ms/step - loss: 1.3561 - accuracy: 0.5495 - val_loss: 1.2963 - val_accuracy: 0.5719 - lr: 0.0010\n",
            "Epoch 10/10\n",
            "636/636 [==============================] - 10s 16ms/step - loss: 1.3498 - accuracy: 0.5534 - val_loss: 1.2889 - val_accuracy: 0.5715 - lr: 0.0010\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.\n",
            "  saving_api.save_model(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "cnn.evaluate(x=x_test,y=y_test)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MtVxWupdlSbJ",
        "outputId": "cda41966-bef5-49e3-fa49-f5507d07b07c"
      },
      "execution_count": 104,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "354/354 [==============================] - 2s 6ms/step - loss: 1.2808 - accuracy: 0.5718\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1.2808283567428589, 0.5717698931694031]"
            ]
          },
          "metadata": {},
          "execution_count": 104
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# MLP\n",
        "\n",
        "Построение многослойной сети персептронов.\n",
        "\n",
        "Результаты получились не очень("
      ],
      "metadata": {
        "id": "TVyPZFlOmF3p"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "mlp = Sequential()\n",
        "\n",
        "\n",
        "mlp.add(Dense(512, input_shape=(100,)))\n",
        "mlp.add(Activation('tanh'))\n",
        "mlp.add(Dropout(0.5))\n",
        "mlp.add(Dense(512, input_shape=(100,)))\n",
        "mlp.add(Activation('relu'))\n",
        "mlp.add(Dense(512, input_shape=(100,)))\n",
        "mlp.add(Activation('relu'))\n",
        "mlp.add(Dropout(0.5))\n",
        "mlp.add(Dense(37))\n",
        "mlp.add(Activation('softmax'))\n",
        "\n",
        "\n",
        "mlp.summary()\n",
        "\n",
        "# Add optimization method, loss function, and optimization value\n",
        "mlp.compile(loss='sparse_categorical_crossentropy',\n",
        "              optimizer='adam', metrics=['accuracy'])\n",
        "\n",
        "\n",
        "mlp.fit(x_train, y_train, batch_size=64, epochs=10)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f7xjpdsJmFEt",
        "outputId": "5b9e49d2-bfcf-42a5-fdb2-20beca31d241"
      },
      "execution_count": 98,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential_35\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense_46 (Dense)            (None, 512)               51712     \n",
            "                                                                 \n",
            " activation_15 (Activation)  (None, 512)               0         \n",
            "                                                                 \n",
            " dropout_36 (Dropout)        (None, 512)               0         \n",
            "                                                                 \n",
            " dense_47 (Dense)            (None, 512)               262656    \n",
            "                                                                 \n",
            " activation_16 (Activation)  (None, 512)               0         \n",
            "                                                                 \n",
            " dense_48 (Dense)            (None, 512)               262656    \n",
            "                                                                 \n",
            " activation_17 (Activation)  (None, 512)               0         \n",
            "                                                                 \n",
            " dropout_37 (Dropout)        (None, 512)               0         \n",
            "                                                                 \n",
            " dense_49 (Dense)            (None, 37)                18981     \n",
            "                                                                 \n",
            " activation_18 (Activation)  (None, 37)                0         \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 596005 (2.27 MB)\n",
            "Trainable params: 596005 (2.27 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n",
            "Epoch 1/10\n",
            "707/707 [==============================] - 10s 11ms/step - loss: 2.1360 - accuracy: 0.2473\n",
            "Epoch 2/10\n",
            "707/707 [==============================] - 5s 7ms/step - loss: 2.0779 - accuracy: 0.2589\n",
            "Epoch 3/10\n",
            "707/707 [==============================] - 8s 11ms/step - loss: 2.0713 - accuracy: 0.2617\n",
            "Epoch 4/10\n",
            "707/707 [==============================] - 5s 7ms/step - loss: 2.0639 - accuracy: 0.2632\n",
            "Epoch 5/10\n",
            "707/707 [==============================] - 7s 11ms/step - loss: 2.0623 - accuracy: 0.2634\n",
            "Epoch 6/10\n",
            "707/707 [==============================] - 5s 8ms/step - loss: 2.0592 - accuracy: 0.2643\n",
            "Epoch 7/10\n",
            "707/707 [==============================] - 7s 10ms/step - loss: 2.0559 - accuracy: 0.2653\n",
            "Epoch 8/10\n",
            "707/707 [==============================] - 4s 5ms/step - loss: 2.0565 - accuracy: 0.2653\n",
            "Epoch 9/10\n",
            "707/707 [==============================] - 6s 9ms/step - loss: 2.0543 - accuracy: 0.2658\n",
            "Epoch 10/10\n",
            "707/707 [==============================] - 5s 7ms/step - loss: 2.0532 - accuracy: 0.2657\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x79c21870a890>"
            ]
          },
          "metadata": {},
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "mlp.evaluate(x_test, y_test, batch_size=64)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jHW4lJAUmbhX",
        "outputId": "38b49f1e-4758-4891-d0a8-e17b06fd7cb7"
      },
      "execution_count": 99,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "177/177 [==============================] - 1s 4ms/step - loss: 2.0472 - accuracy: 0.2674\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[2.047179937362671, 0.26743361353874207]"
            ]
          },
          "metadata": {},
          "execution_count": 99
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# SUMMARY\n",
        "\n",
        "Построено 3 разных сети с сопостовимым количеством параметров, на одних и тех же наборах данных, произведена их оценка по метрике accuracy.\n",
        "\n",
        "Accuracy:\n",
        "\n",
        "1. RNN (LSTM): 0.7384955883026123\n",
        "2. CNN       : 0.5718\n",
        "3. MLP       : 0.26743361353874207\n",
        "\n",
        "В соответсвие с оценками можно считать решение основаное на рекурентных нейронных сетях наиболее перспективным, однако не стоит забывать, что набор данных был не сбалансирован и изменение соотношений данных может снизить точность предсказаний.\n",
        "\n",
        "\n",
        "Спасибо за интересную задачу! Буду рад продолжить сотрудничество с вами!)\n",
        "\n"
      ],
      "metadata": {
        "id": "C6MhXZ-qrrPK"
      }
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "TeA4cyjLrw-Z"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "MP_iVdoIIt1Z",
        "_xyge-OlIyEg",
        "B-UynRQTa8vm",
        "ggcSO-ThPbKV",
        "IgMb4-b96ULo",
        "TVyPZFlOmF3p"
      ],
      "gpuType": "T4",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.4"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}