greek_parliament_proceedings_assignment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Greek Parliament Proceedings\n",
    "\n",
    "You will work with a dataset containing Greek Parliament proceedings, available at <https://zenodo.org/record/2587904>. The dataset is described in Zenodo; you can also read <https://lab.imedd.org/en/parliamentary-minutes-from-1989-until-today/>.\n",
    "\n",
    "---\n",
    "\n",
    "> Panos Louridas, Associate Professor <br />\n",
    "> Department of Management Science and Technology <br />\n",
    "> Athens University of Economics and Business <br />\n",
    "> louridas@aueb.gr\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Selection and Preparation\n",
    "\n",
    "As you will find out, the dataset is extensive, as it contains three decades worth of parliamentary proceedings. You can restrict yourself to a subset of the dataset, however that subset must not be trivial. Also keep in mind that a dataset that is too small may be fast to process, but may not be enough to produce good results.\n",
    "\n",
    "You are free to preprocess the data in any way that seems meaningful and that is likely to be conducive to good results. For instance, you may or may not remove punctuation, stopwords, very frequent or very rare words, etc. You can use any library for preprocessing, like [NLTK](https://www.nltk.org/) or [spaCy](https://spacy.io/); spaCy is more modern and faster. When developing the neural network solution you may do the preprocessing beforehand and / or using the TensorFlow facilities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Exploration\n",
    "\n",
    "* Expore the dataset to get an overall idea of what it is about. Think about:\n",
    "\n",
    "  * Speeches per party.\n",
    "  \n",
    "  * Speeches over time.\n",
    "  \n",
    "  * Speeches per party and time.\n",
    "  \n",
    "  * Speakers per party.\n",
    "  \n",
    "  * Etc.\n",
    "  \n",
    "* Your analysis can (and should) include tables and diagrams."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classification without Neural Networks\n",
    "\n",
    "* Train at least two non-neural network algorithms to learn to classify a speech. The target variable should be the political party of the speaker.\n",
    "\n",
    "* If a party does not have enough support (in the Machine Learning sense, i.e., too few speeches), you may excise it from the dataset if you detect that you cannot get results for it.\n",
    "\n",
    "* You should split your data to training and testing datasets, try the different algorithms with cross validation on the training dataset, and find the best hyperparameters for the best algorithm. Note that parameters for bag of words and tf-idf construction can be very important.\n",
    "\n",
    "* Report your scores; once you find the best algorithm and the best hyperparameters, report the score on the test data.\n",
    "\n",
    "* To gauge the efficacy of the algorithm, report also the results of a baseline classifier, using, for instance, scikit-learn's [`DummyClassifier`](https://scikit-learn.org/stable/modules/generated/sklearn.dummy.DummyClassifier.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classification with a Neural Network\n",
    "\n",
    "* Carry out classification, but this time using a neural network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extra (Bonus)\n",
    "\n",
    "* Write a text generator that produces speeches like those in the proceedings."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Honor Code\n",
    "\n",
    "You understand that this is an individual assignment, and as such you must carry it out alone. You may seek help on the Internet, by Googling or searching in StackOverflow for general questions pertaining to the use of Python and pandas libraries and idioms. However, it is not right to ask direct questions that relate to the assignment and where people will actually solve your problem by answering them. You may discuss with your fellow students in order to better understand the questions, if they are not clear enough, but you should not ask them to share their answers with you, or to help you by giving specific advice."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}