Data Manipulation with Pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Manipulations with Pandas\n",
    "\n",
    "Pandas is a very important library in Python for manipulating data in table format. Think of it like Python's version of Excel. Some useful links are pasted below:\n",
    "\n",
    "https://pandas.pydata.org/\n",
    "\n",
    "https://www.youtube.com/watch?v=vmEHCJofslg\n",
    "\n",
    "Merging: https://www.youtube.com/watch?v=h4hOPGo4UVU&t=105s\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing Pandas\n",
    "\n",
    "It is conventional to rename the library as *pd* for short when it is imported."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading data into Pandas\n",
    "#### from csv:\n",
    "    df = pd.read_csv('name_of_file.csv') \n",
    " > save data as csv in same file as notebook\n",
    "#### from excel\n",
    "    df_xlsx = pd.read_excel('name_of_file.xlsx')\n",
    "#### from tab seperated file\n",
    "    df = pd.read_csv('name_of_file.txt', delimiter='\\t')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining some dummy data in a Pandas Dataframe\n",
    "\n",
    "In Pandas, data is loaded into objects called DataFrames. These can be thought of as spreadsheets. Most of the time these read from CSV (comma-separated values) files but in this exampel we will define some dummy data first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_data = {\n",
    "    'city': [\n",
    "        'LONDON',\n",
    "        'MIAMI',\n",
    "        'NEW YORK',\n",
    "        'MANCHESTER',\n",
    "        'LONDON',\n",
    "        'MIAMI',\n",
    "        'NEW YORK',\n",
    "        'MIAMI',\n",
    "        'NEW YORK',\n",
    "    ],\n",
    "    'temperature': [90, 89, 78, 67, 56, 90, 78, 98, 46],\n",
    "    'rain': [0.9, 0.8, 0.7, 0.3, 0.5, 0.9, 0.9, 0.9, 0.3],\n",
    "    'day': ['MON', 'TUE', 'WED', 'FRI', 'THU', 'SAT', 'SUN', 'MON', 'SAT']\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(my_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>89</td>\n",
       "      <td>0.8</td>\n",
       "      <td>TUE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.7</td>\n",
       "      <td>WED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MANCHESTER</td>\n",
       "      <td>67</td>\n",
       "      <td>0.3</td>\n",
       "      <td>FRI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>56</td>\n",
       "      <td>0.5</td>\n",
       "      <td>THU</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  rain  day\n",
       "0      LONDON           90   0.9  MON\n",
       "1       MIAMI           89   0.8  TUE\n",
       "2    NEW YORK           78   0.7  WED\n",
       "3  MANCHESTER           67   0.3  FRI\n",
       "4      LONDON           56   0.5  THU"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Filtering rows in a DataFrame\n",
    "\n",
    "A very common technique in data manipulation with Pandas is to select a subset of rows where some condition is true, i.e., all rows where city is London. To do that we pass in a filter condition as follows:\n",
    "```\n",
    "df[row_filter_condition]\n",
    "```\n",
    "where the *row_filter_condition* itself can be mde up of some condition involving a particular column of the DataFrame itself. This will only return the rows where the condition evaluates to True."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>56</td>\n",
       "      <td>0.5</td>\n",
       "      <td>THU</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     city  temperature  rain  day\n",
       "0  LONDON           90   0.9  MON\n",
       "4  LONDON           56   0.5  THU"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['city'] == 'LONDON']\n",
    "#use & to filter multiple conditions at once\n",
    "#use | to filter conditions seperatley (works as or filter)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>98</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     city  temperature  rain  day\n",
       "0  LONDON           90   0.9  MON\n",
       "7   MIAMI           98   0.9  MON"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['day'] == 'MON']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>89</td>\n",
       "      <td>0.8</td>\n",
       "      <td>TUE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.7</td>\n",
       "      <td>WED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SAT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SUN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>98</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       city  temperature  rain  day\n",
       "0    LONDON           90   0.9  MON\n",
       "1     MIAMI           89   0.8  TUE\n",
       "2  NEW YORK           78   0.7  WED\n",
       "5     MIAMI           90   0.9  SAT\n",
       "6  NEW YORK           78   0.9  SUN\n",
       "7     MIAMI           98   0.9  MON"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['rain'] > 0.5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Filtering columns\n",
    "\n",
    "Another common technique in data manipulation with Pandas is to select a subset of columns. This is done by passing in a list of columns to select as follows\n",
    "```\n",
    "df[[my_col_1, my_col_4, my_col_6]]\n",
    "```\n",
    "where the list of column names should contain the names of the columns you want to select as they are written in the DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rain</th>\n",
       "      <th>temperature</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.9</td>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.8</td>\n",
       "      <td>89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.7</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.3</td>\n",
       "      <td>67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.5</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.9</td>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.9</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.9</td>\n",
       "      <td>98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.3</td>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   rain  temperature\n",
       "0   0.9           90\n",
       "1   0.8           89\n",
       "2   0.7           78\n",
       "3   0.3           67\n",
       "4   0.5           56\n",
       "5   0.9           90\n",
       "6   0.9           78\n",
       "7   0.9           98\n",
       "8   0.3           46"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['rain', 'temperature']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>TUE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>WED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MANCHESTER</td>\n",
       "      <td>FRI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>THU</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>SAT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>SUN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>SAT</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  day\n",
       "0      LONDON  MON\n",
       "1       MIAMI  TUE\n",
       "2    NEW YORK  WED\n",
       "3  MANCHESTER  FRI\n",
       "4      LONDON  THU\n",
       "5       MIAMI  SAT\n",
       "6    NEW YORK  SUN\n",
       "7       MIAMI  MON\n",
       "8    NEW YORK  SAT"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['city', 'day']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Filtering Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Reading data from specified index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "city           MIAMI\n",
      "temperature       89\n",
      "rain             0.8\n",
      "day              TUE\n",
      "Name: 1, dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(df.iloc[1]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Reading data from specified cell (row, column)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "78\n"
     ]
    }
   ],
   "source": [
    "print(df.iloc[2,1]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Filter data based on criteria"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>56</td>\n",
       "      <td>0.5</td>\n",
       "      <td>THU</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     city  temperature  rain  day\n",
       "0  LONDON           90   0.9  MON\n",
       "4  LONDON           56   0.5  THU"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[df['city'].str.contains('LONDON')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Filter out data based on criteria\n",
    "discounts data containing specified element"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>89</td>\n",
       "      <td>0.8</td>\n",
       "      <td>TUE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.7</td>\n",
       "      <td>WED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MANCHESTER</td>\n",
       "      <td>67</td>\n",
       "      <td>0.3</td>\n",
       "      <td>FRI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SAT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SUN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>98</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>46</td>\n",
       "      <td>0.3</td>\n",
       "      <td>SAT</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  rain  day\n",
       "1       MIAMI           89   0.8  TUE\n",
       "2    NEW YORK           78   0.7  WED\n",
       "3  MANCHESTER           67   0.3  FRI\n",
       "5       MIAMI           90   0.9  SAT\n",
       "6    NEW YORK           78   0.9  SUN\n",
       "7       MIAMI           98   0.9  MON\n",
       "8    NEW YORK           46   0.3  SAT"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[~ df['city'].str.contains('LONDON')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sorting Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>56</td>\n",
       "      <td>0.5</td>\n",
       "      <td>THU</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MANCHESTER</td>\n",
       "      <td>67</td>\n",
       "      <td>0.3</td>\n",
       "      <td>FRI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>89</td>\n",
       "      <td>0.8</td>\n",
       "      <td>TUE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SAT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>98</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.7</td>\n",
       "      <td>WED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SUN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>46</td>\n",
       "      <td>0.3</td>\n",
       "      <td>SAT</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  rain  day\n",
       "0      LONDON           90   0.9  MON\n",
       "4      LONDON           56   0.5  THU\n",
       "3  MANCHESTER           67   0.3  FRI\n",
       "1       MIAMI           89   0.8  TUE\n",
       "5       MIAMI           90   0.9  SAT\n",
       "7       MIAMI           98   0.9  MON\n",
       "2    NEW YORK           78   0.7  WED\n",
       "6    NEW YORK           78   0.9  SUN\n",
       "8    NEW YORK           46   0.3  SAT"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#sorts cities alphabetically\n",
    "df.sort_values('city') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.7</td>\n",
       "      <td>WED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SUN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>46</td>\n",
       "      <td>0.3</td>\n",
       "      <td>SAT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>89</td>\n",
       "      <td>0.8</td>\n",
       "      <td>TUE</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SAT</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>98</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MANCHESTER</td>\n",
       "      <td>67</td>\n",
       "      <td>0.3</td>\n",
       "      <td>FRI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>56</td>\n",
       "      <td>0.5</td>\n",
       "      <td>THU</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  rain  day\n",
       "2    NEW YORK           78   0.7  WED\n",
       "6    NEW YORK           78   0.9  SUN\n",
       "8    NEW YORK           46   0.3  SAT\n",
       "1       MIAMI           89   0.8  TUE\n",
       "5       MIAMI           90   0.9  SAT\n",
       "7       MIAMI           98   0.9  MON\n",
       "3  MANCHESTER           67   0.3  FRI\n",
       "0      LONDON           90   0.9  MON\n",
       "4      LONDON           56   0.5  THU"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#sorts cities in descending alphabetical order\n",
    "df.sort_values('city', ascending=False) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Describing data\n",
    "gives statistical information: standard deviation, mean etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>9.000000</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>76.888889</td>\n",
       "      <td>0.688889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>17.417265</td>\n",
       "      <td>0.257121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>46.000000</td>\n",
       "      <td>0.300000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>67.000000</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>78.000000</td>\n",
       "      <td>0.800000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>90.000000</td>\n",
       "      <td>0.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>98.000000</td>\n",
       "      <td>0.900000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       temperature      rain\n",
       "count     9.000000  9.000000\n",
       "mean     76.888889  0.688889\n",
       "std      17.417265  0.257121\n",
       "min      46.000000  0.300000\n",
       "25%      67.000000  0.500000\n",
       "50%      78.000000  0.800000\n",
       "75%      90.000000  0.900000\n",
       "max      98.000000  0.900000"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making Changes to Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "      <th>total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "      <td>90.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>89</td>\n",
       "      <td>0.8</td>\n",
       "      <td>TUE</td>\n",
       "      <td>89.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.7</td>\n",
       "      <td>WED</td>\n",
       "      <td>78.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MANCHESTER</td>\n",
       "      <td>67</td>\n",
       "      <td>0.3</td>\n",
       "      <td>FRI</td>\n",
       "      <td>67.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>56</td>\n",
       "      <td>0.5</td>\n",
       "      <td>THU</td>\n",
       "      <td>56.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SAT</td>\n",
       "      <td>90.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SUN</td>\n",
       "      <td>78.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>98</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "      <td>98.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>46</td>\n",
       "      <td>0.3</td>\n",
       "      <td>SAT</td>\n",
       "      <td>46.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  rain  day  total\n",
       "0      LONDON           90   0.9  MON   90.9\n",
       "1       MIAMI           89   0.8  TUE   89.8\n",
       "2    NEW YORK           78   0.7  WED   78.7\n",
       "3  MANCHESTER           67   0.3  FRI   67.3\n",
       "4      LONDON           56   0.5  THU   56.5\n",
       "5       MIAMI           90   0.9  SAT   90.9\n",
       "6    NEW YORK           78   0.9  SUN   78.9\n",
       "7       MIAMI           98   0.9  MON   98.9\n",
       "8    NEW YORK           46   0.3  SAT   46.3"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#adding sets of data within table\n",
    "\n",
    "df['total'] = df['temperature']+ df['rain']\n",
    "df.head(9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>day</th>\n",
       "      <th>total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "      <td>90.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>89</td>\n",
       "      <td>0.8</td>\n",
       "      <td>TUE</td>\n",
       "      <td>89.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.7</td>\n",
       "      <td>WED</td>\n",
       "      <td>78.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MANCHESTER</td>\n",
       "      <td>67</td>\n",
       "      <td>0.3</td>\n",
       "      <td>FRI</td>\n",
       "      <td>67.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>56</td>\n",
       "      <td>0.5</td>\n",
       "      <td>THU</td>\n",
       "      <td>56.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SAT</td>\n",
       "      <td>90.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.9</td>\n",
       "      <td>SUN</td>\n",
       "      <td>78.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>98</td>\n",
       "      <td>0.9</td>\n",
       "      <td>MON</td>\n",
       "      <td>98.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>46</td>\n",
       "      <td>0.3</td>\n",
       "      <td>SAT</td>\n",
       "      <td>46.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  rain  day  total\n",
       "0      LONDON           90   0.9  MON   90.9\n",
       "1       MIAMI           89   0.8  TUE   89.8\n",
       "2    NEW YORK           78   0.7  WED   78.7\n",
       "3  MANCHESTER           67   0.3  FRI   67.3\n",
       "4      LONDON           56   0.5  THU   56.5\n",
       "5       MIAMI           90   0.9  SAT   90.9\n",
       "6    NEW YORK           78   0.9  SUN   78.9\n",
       "7       MIAMI           98   0.9  MON   98.9\n",
       "8    NEW YORK           46   0.3  SAT   46.3"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#adds columns 1 and 2\n",
    "df['total'] = df.iloc[:,1:3].sum(axis=1)\n",
    "\n",
    "#adds horizontally, axis = 0 adds horizontally\n",
    "\n",
    "df.head(9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>rain</th>\n",
       "      <th>total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>90.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>89</td>\n",
       "      <td>0.8</td>\n",
       "      <td>89.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.7</td>\n",
       "      <td>78.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>MANCHESTER</td>\n",
       "      <td>67</td>\n",
       "      <td>0.3</td>\n",
       "      <td>67.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>LONDON</td>\n",
       "      <td>56</td>\n",
       "      <td>0.5</td>\n",
       "      <td>56.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>90</td>\n",
       "      <td>0.9</td>\n",
       "      <td>90.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>78</td>\n",
       "      <td>0.9</td>\n",
       "      <td>78.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MIAMI</td>\n",
       "      <td>98</td>\n",
       "      <td>0.9</td>\n",
       "      <td>98.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>46</td>\n",
       "      <td>0.3</td>\n",
       "      <td>46.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  rain  total\n",
       "0      LONDON           90   0.9   90.9\n",
       "1       MIAMI           89   0.8   89.8\n",
       "2    NEW YORK           78   0.7   78.7\n",
       "3  MANCHESTER           67   0.3   67.3\n",
       "4      LONDON           56   0.5   56.5\n",
       "5       MIAMI           90   0.9   90.9\n",
       "6    NEW YORK           78   0.9   78.9\n",
       "7       MIAMI           98   0.9   98.9\n",
       "8    NEW YORK           46   0.3   46.3"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# dropping a column from data set\n",
    "\n",
    "df = df.drop(columns=['day'])\n",
    "df.head(9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### moving column\n",
    "reordering columns, specify the order in list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rain</th>\n",
       "      <th>temperature</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.9</td>\n",
       "      <td>90</td>\n",
       "      <td>LONDON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.8</td>\n",
       "      <td>89</td>\n",
       "      <td>MIAMI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.7</td>\n",
       "      <td>78</td>\n",
       "      <td>NEW YORK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.3</td>\n",
       "      <td>67</td>\n",
       "      <td>MANCHESTER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.5</td>\n",
       "      <td>56</td>\n",
       "      <td>LONDON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.9</td>\n",
       "      <td>90</td>\n",
       "      <td>MIAMI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.9</td>\n",
       "      <td>78</td>\n",
       "      <td>NEW YORK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.9</td>\n",
       "      <td>98</td>\n",
       "      <td>MIAMI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.3</td>\n",
       "      <td>46</td>\n",
       "      <td>NEW YORK</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   rain  temperature        city\n",
       "0   0.9           90      LONDON\n",
       "1   0.8           89       MIAMI\n",
       "2   0.7           78    NEW YORK\n",
       "3   0.3           67  MANCHESTER\n",
       "4   0.5           56      LONDON\n",
       "5   0.9           90       MIAMI\n",
       "6   0.9           78    NEW YORK\n",
       "7   0.9           98       MIAMI\n",
       "8   0.3           46    NEW YORK"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df[['rain', 'temperature', 'city']]\n",
    "df.head(9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Merging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1 = pd.DataFrame ({\n",
    "    'city': [ 'London','Miami','New York','Manchester', 'Vienna'],\n",
    "    'temperature': [90, 89, 78, 67, 70],\n",
    " \n",
    "})\n",
    "\n",
    "df2 = pd.DataFrame ({\n",
    "    'city' : ['Manchester', 'New York', 'London', 'Miami', 'Madrid'],\n",
    "    'humidity' : [56, 67, 60, 73, 76],\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>humidity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>London</td>\n",
       "      <td>90</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Miami</td>\n",
       "      <td>89</td>\n",
       "      <td>73</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>New York</td>\n",
       "      <td>78</td>\n",
       "      <td>67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Manchester</td>\n",
       "      <td>67</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  humidity\n",
       "0      London           90        60\n",
       "1       Miami           89        73\n",
       "2    New York           78        67\n",
       "3  Manchester           67        56"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3 = pd.merge(df1,df2, on= 'city') #merges data using city category, extracting common elements\n",
    "df3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>humidity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>London</td>\n",
       "      <td>90.0</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Miami</td>\n",
       "      <td>89.0</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>New York</td>\n",
       "      <td>78.0</td>\n",
       "      <td>67.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Manchester</td>\n",
       "      <td>67.0</td>\n",
       "      <td>56.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Vienna</td>\n",
       "      <td>70.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Madrid</td>\n",
       "      <td>NaN</td>\n",
       "      <td>76.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  humidity\n",
       "0      London         90.0      60.0\n",
       "1       Miami         89.0      73.0\n",
       "2    New York         78.0      67.0\n",
       "3  Manchester         67.0      56.0\n",
       "4      Vienna         70.0       NaN\n",
       "5      Madrid          NaN      76.0"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df4 = pd.merge(df1,df2, on= 'city', how= 'outer') # how includes data not present in both sets\n",
    "df4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>humidity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>London</td>\n",
       "      <td>90</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Miami</td>\n",
       "      <td>89</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>New York</td>\n",
       "      <td>78</td>\n",
       "      <td>67.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Manchester</td>\n",
       "      <td>67</td>\n",
       "      <td>56.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Vienna</td>\n",
       "      <td>70</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  humidity\n",
       "0      London           90      60.0\n",
       "1       Miami           89      73.0\n",
       "2    New York           78      67.0\n",
       "3  Manchester           67      56.0\n",
       "4      Vienna           70       NaN"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df5 = pd.merge(df1,df2, on='city', how='left') #includes all data from first set and common elements from second\n",
    "df5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>humidity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Manchester</td>\n",
       "      <td>67.0</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>New York</td>\n",
       "      <td>78.0</td>\n",
       "      <td>67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>London</td>\n",
       "      <td>90.0</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Miami</td>\n",
       "      <td>89.0</td>\n",
       "      <td>73</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Madrid</td>\n",
       "      <td>NaN</td>\n",
       "      <td>76</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  humidity\n",
       "0  Manchester         67.0        56\n",
       "1    New York         78.0        67\n",
       "2      London         90.0        60\n",
       "3       Miami         89.0        73\n",
       "4      Madrid          NaN        76"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df6 = pd.merge(df1,df2, on='city', how='right') #includes all data from second set and common elements from first\n",
    "df6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>temperature</th>\n",
       "      <th>humidity</th>\n",
       "      <th>_merge</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>London</td>\n",
       "      <td>90.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>both</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Miami</td>\n",
       "      <td>89.0</td>\n",
       "      <td>73.0</td>\n",
       "      <td>both</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>New York</td>\n",
       "      <td>78.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>both</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Manchester</td>\n",
       "      <td>67.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>both</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Vienna</td>\n",
       "      <td>70.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>left_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Madrid</td>\n",
       "      <td>NaN</td>\n",
       "      <td>76.0</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         city  temperature  humidity      _merge\n",
       "0      London         90.0      60.0        both\n",
       "1       Miami         89.0      73.0        both\n",
       "2    New York         78.0      67.0        both\n",
       "3  Manchester         67.0      56.0        both\n",
       "4      Vienna         70.0       NaN   left_only\n",
       "5      Madrid          NaN      76.0  right_only"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df7 = pd.merge(df1,df2, on='city', how='outer', indicator=True) #includes column with details of where data is present\n",
    "df7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conditional Changes\n",
    "replace term with another"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rain</th>\n",
       "      <th>temperature</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.9</td>\n",
       "      <td>90</td>\n",
       "      <td>PARIS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.8</td>\n",
       "      <td>89</td>\n",
       "      <td>MIAMI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.7</td>\n",
       "      <td>78</td>\n",
       "      <td>NEW YORK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.3</td>\n",
       "      <td>67</td>\n",
       "      <td>MANCHESTER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.5</td>\n",
       "      <td>56</td>\n",
       "      <td>PARIS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.9</td>\n",
       "      <td>90</td>\n",
       "      <td>MIAMI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.9</td>\n",
       "      <td>78</td>\n",
       "      <td>NEW YORK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.9</td>\n",
       "      <td>98</td>\n",
       "      <td>MIAMI</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.3</td>\n",
       "      <td>46</td>\n",
       "      <td>NEW YORK</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   rain  temperature        city\n",
       "0   0.9           90       PARIS\n",
       "1   0.8           89       MIAMI\n",
       "2   0.7           78    NEW YORK\n",
       "3   0.3           67  MANCHESTER\n",
       "4   0.5           56       PARIS\n",
       "5   0.9           90       MIAMI\n",
       "6   0.9           78    NEW YORK\n",
       "7   0.9           98       MIAMI\n",
       "8   0.3           46    NEW YORK"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[df['city'] == 'LONDON', 'city'] = 'PARIS'\n",
    "\n",
    "#replaces all LONDONs with PARIS\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Groupby"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "city\n",
       "MANCHESTER    1\n",
       "MIAMI         3\n",
       "NEW YORK      3\n",
       "PARIS         2\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['city']).count()\n",
    "\n",
    "df['count'] = 1\n",
    "df.groupby(['city']).count()['count']\n",
    "# counts number of times term is present"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving Modified Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"city_rainfall.csv\")\n",
    "# saves modified data as new csv file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing CSV file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>rain</th>\n",
       "      <th>temperature</th>\n",
       "      <th>city</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.9</td>\n",
       "      <td>90</td>\n",
       "      <td>PARIS</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.8</td>\n",
       "      <td>89</td>\n",
       "      <td>MIAMI</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.7</td>\n",
       "      <td>78</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0.3</td>\n",
       "      <td>67</td>\n",
       "      <td>MANCHESTER</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0.5</td>\n",
       "      <td>56</td>\n",
       "      <td>PARIS</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>0.9</td>\n",
       "      <td>90</td>\n",
       "      <td>MIAMI</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>0.9</td>\n",
       "      <td>78</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>0.9</td>\n",
       "      <td>98</td>\n",
       "      <td>MIAMI</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>0.3</td>\n",
       "      <td>46</td>\n",
       "      <td>NEW YORK</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  rain  temperature        city  count\n",
       "0           0   0.9           90       PARIS      1\n",
       "1           1   0.8           89       MIAMI      1\n",
       "2           2   0.7           78    NEW YORK      1\n",
       "3           3   0.3           67  MANCHESTER      1\n",
       "4           4   0.5           56       PARIS      1\n",
       "5           5   0.9           90       MIAMI      1\n",
       "6           6   0.9           78    NEW YORK      1\n",
       "7           7   0.9           98       MIAMI      1\n",
       "8           8   0.3           46    NEW YORK      1"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv(\"city_rainfall.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing data from web"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_link = 'https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_United_Kingdom_general_election'\n",
    "all_tables = pd.read_html(my_link)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total tables: 48\n"
     ]
    }
   ],
   "source": [
    "print(f'Total tables: {len(all_tables)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Dates conducted</th>\n",
       "      <th>Pollster</th>\n",
       "      <th>Client</th>\n",
       "      <th>Area</th>\n",
       "      <th>Sample size</th>\n",
       "      <th>Con</th>\n",
       "      <th>Lab</th>\n",
       "      <th>Lib Dems</th>\n",
       "      <th>SNP</th>\n",
       "      <th>Green</th>\n",
       "      <th>Reform</th>\n",
       "      <th>Others</th>\n",
       "      <th>Lead</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Dates conducted</th>\n",
       "      <th>Pollster</th>\n",
       "      <th>Client</th>\n",
       "      <th>Area</th>\n",
       "      <th>Sample size</th>\n",
       "      <th>Unnamed: 5_level_1</th>\n",
       "      <th>Unnamed: 6_level_1</th>\n",
       "      <th>Unnamed: 7_level_1</th>\n",
       "      <th>Unnamed: 8_level_1</th>\n",
       "      <th>Unnamed: 9_level_1</th>\n",
       "      <th>Unnamed: 10_level_1</th>\n",
       "      <th>Others</th>\n",
       "      <th>Lead</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1–4 Mar</td>\n",
       "      <td>Deltapoll</td>\n",
       "      <td>NaN</td>\n",
       "      <td>GB</td>\n",
       "      <td>1500</td>\n",
       "      <td>27%</td>\n",
       "      <td>41%</td>\n",
       "      <td>9%</td>\n",
       "      <td>3%</td>\n",
       "      <td>6%</td>\n",
       "      <td>12%</td>\n",
       "      <td>3%</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3 Mar</td>\n",
       "      <td>Redfield &amp; Wilton</td>\n",
       "      <td>NaN</td>\n",
       "      <td>GB</td>\n",
       "      <td>2000</td>\n",
       "      <td>23%</td>\n",
       "      <td>43%</td>\n",
       "      <td>10%</td>\n",
       "      <td>3%</td>\n",
       "      <td>6%</td>\n",
       "      <td>13%</td>\n",
       "      <td>2%</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1–3 Mar</td>\n",
       "      <td>Savanta</td>\n",
       "      <td>NaN</td>\n",
       "      <td>GB</td>\n",
       "      <td>2245</td>\n",
       "      <td>27%</td>\n",
       "      <td>44%</td>\n",
       "      <td>10%</td>\n",
       "      <td>3%</td>\n",
       "      <td>4%</td>\n",
       "      <td>8%</td>\n",
       "      <td>4%</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>29 Feb – 1 Mar</td>\n",
       "      <td>We Think</td>\n",
       "      <td>NaN</td>\n",
       "      <td>GB</td>\n",
       "      <td>1240</td>\n",
       "      <td>23%</td>\n",
       "      <td>47%</td>\n",
       "      <td>9%</td>\n",
       "      <td>3%</td>\n",
       "      <td>5%</td>\n",
       "      <td>10%</td>\n",
       "      <td>3%</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>29 Feb</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "      <td>Rochdale by-election</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69</th>\n",
       "      <td>7 Jan</td>\n",
       "      <td>Redfield &amp; Wilton</td>\n",
       "      <td>NaN</td>\n",
       "      <td>GB</td>\n",
       "      <td>2000</td>\n",
       "      <td>27%</td>\n",
       "      <td>43%</td>\n",
       "      <td>10%</td>\n",
       "      <td>3%</td>\n",
       "      <td>5%</td>\n",
       "      <td>11%</td>\n",
       "      <td>2%</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>5–7 Jan</td>\n",
       "      <td>Savanta</td>\n",
       "      <td>NaN</td>\n",
       "      <td>UK</td>\n",
       "      <td>2268</td>\n",
       "      <td>26%</td>\n",
       "      <td>45%</td>\n",
       "      <td>10%</td>\n",
       "      <td>3%</td>\n",
       "      <td>5%</td>\n",
       "      <td>8%</td>\n",
       "      <td>4%</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>4–5 Jan</td>\n",
       "      <td>We Think</td>\n",
       "      <td>NaN</td>\n",
       "      <td>GB</td>\n",
       "      <td>1226</td>\n",
       "      <td>25%</td>\n",
       "      <td>47%</td>\n",
       "      <td>9%</td>\n",
       "      <td>2%</td>\n",
       "      <td>5%</td>\n",
       "      <td>10%</td>\n",
       "      <td>2%</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>12 Dec – 4 Jan</td>\n",
       "      <td>YouGov (MRP)[b]</td>\n",
       "      <td>Conservative Britain Alliance[6]</td>\n",
       "      <td>GB</td>\n",
       "      <td>14110</td>\n",
       "      <td>26%</td>\n",
       "      <td>39.5%</td>\n",
       "      <td>12.5%</td>\n",
       "      <td>3%</td>\n",
       "      <td>7.5%</td>\n",
       "      <td>9%</td>\n",
       "      <td>2.5%</td>\n",
       "      <td>13.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>2–3 Jan</td>\n",
       "      <td>YouGov</td>\n",
       "      <td>The Times</td>\n",
       "      <td>GB</td>\n",
       "      <td>2016</td>\n",
       "      <td>22%</td>\n",
       "      <td>46%</td>\n",
       "      <td>10%</td>\n",
       "      <td>3%</td>\n",
       "      <td>7%</td>\n",
       "      <td>9%</td>\n",
       "      <td>2%</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>74 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Dates conducted              Pollster                            Client  \\\n",
       "   Dates conducted              Pollster                            Client   \n",
       "0          1–4 Mar             Deltapoll                               NaN   \n",
       "1            3 Mar     Redfield & Wilton                               NaN   \n",
       "2          1–3 Mar               Savanta                               NaN   \n",
       "3   29 Feb – 1 Mar              We Think                               NaN   \n",
       "4           29 Feb  Rochdale by-election              Rochdale by-election   \n",
       "..             ...                   ...                               ...   \n",
       "69           7 Jan     Redfield & Wilton                               NaN   \n",
       "70         5–7 Jan               Savanta                               NaN   \n",
       "71         4–5 Jan              We Think                               NaN   \n",
       "72  12 Dec – 4 Jan       YouGov (MRP)[b]  Conservative Britain Alliance[6]   \n",
       "73         2–3 Jan                YouGov                         The Times   \n",
       "\n",
       "                    Area           Sample size                   Con  \\\n",
       "                    Area           Sample size    Unnamed: 5_level_1   \n",
       "0                     GB                  1500                   27%   \n",
       "1                     GB                  2000                   23%   \n",
       "2                     GB                  2245                   27%   \n",
       "3                     GB                  1240                   23%   \n",
       "4   Rochdale by-election  Rochdale by-election  Rochdale by-election   \n",
       "..                   ...                   ...                   ...   \n",
       "69                    GB                  2000                   27%   \n",
       "70                    UK                  2268                   26%   \n",
       "71                    GB                  1226                   25%   \n",
       "72                    GB                 14110                   26%   \n",
       "73                    GB                  2016                   22%   \n",
       "\n",
       "                     Lab              Lib Dems                   SNP  \\\n",
       "      Unnamed: 6_level_1    Unnamed: 7_level_1    Unnamed: 8_level_1   \n",
       "0                    41%                    9%                    3%   \n",
       "1                    43%                   10%                    3%   \n",
       "2                    44%                   10%                    3%   \n",
       "3                    47%                    9%                    3%   \n",
       "4   Rochdale by-election  Rochdale by-election  Rochdale by-election   \n",
       "..                   ...                   ...                   ...   \n",
       "69                   43%                   10%                    3%   \n",
       "70                   45%                   10%                    3%   \n",
       "71                   47%                    9%                    2%   \n",
       "72                 39.5%                 12.5%                    3%   \n",
       "73                   46%                   10%                    3%   \n",
       "\n",
       "                   Green                Reform                Others  \\\n",
       "      Unnamed: 9_level_1   Unnamed: 10_level_1                Others   \n",
       "0                     6%                   12%                    3%   \n",
       "1                     6%                   13%                    2%   \n",
       "2                     4%                    8%                    4%   \n",
       "3                     5%                   10%                    3%   \n",
       "4   Rochdale by-election  Rochdale by-election  Rochdale by-election   \n",
       "..                   ...                   ...                   ...   \n",
       "69                    5%                   11%                    2%   \n",
       "70                    5%                    8%                    4%   \n",
       "71                    5%                   10%                    2%   \n",
       "72                  7.5%                    9%                  2.5%   \n",
       "73                    7%                    9%                    2%   \n",
       "\n",
       "                    Lead  \n",
       "                    Lead  \n",
       "0                     14  \n",
       "1                     20  \n",
       "2                     17  \n",
       "3                     24  \n",
       "4   Rochdale by-election  \n",
       "..                   ...  \n",
       "69                    16  \n",
       "70                    19  \n",
       "71                    22  \n",
       "72                  13.5  \n",
       "73                    24  \n",
       "\n",
       "[74 rows x 13 columns]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_tables[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}