PS2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem Set 2\n",
    "## Cecilia Barnes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Legality\n",
    "1. https://en.wikibooks.org/wiki/Cookbook:Recipes is the website I want to scrape\n",
    "\n",
    "2. In wikibooks terms of service, it does mention data collections in their terms of use. It says their \" mission is to empower and engage people around the world to collect and develop content under a free license or in the public domain, and to disseminate it effectively and globally, free of charge.\" It mentions that users are free to read and print articles, share and reuse articles under free licenses, and contribute to and edit their sites.\n",
    "\n",
    "3. Steps to lessen the burden of the provider are to only request the page info once unless you need to entirely rerun the code. I could donate money to wikimedia as well since it is a nonprofit site.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Scrape Recipes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bs4 import BeautifulSoup\n",
    "import requests\n",
    "link = 'https://en.wikibooks.org/wiki/Cookbook:Recipes'\n",
    "result = requests.get(link)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#wiki = BeautifulSoup(result.content)\n",
    "wiki = BeautifulSoup(result.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "li = wiki.select('li')\n",
    "links = [a.find('a') for a in li]\n",
    "hrefs = []\n",
    "for each in links[:50]:\n",
    "    hrefs.append(each.get('href'))\n",
    "    #print(each['href'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "import pandas as pd\n",
    "base_url = \"https://en.wikibooks.org/\"\n",
    "# make dataframe w recipe id, name, text, quantity, unit, ingredient\n",
    "df_dict = {\"id\":[], \"name\":[], \"text\":[], \"quantity\":[], \"unit\":[], \"ingredient\":[]}\n",
    "recipe_id = 0\n",
    "\n",
    "for each in links[:50]:\n",
    "    recipe_pg = requests.get(base_url + each['href'])\n",
    "    soup = BeautifulSoup(recipe_pg.text)\n",
    "    recipe_id = recipe_id\n",
    "    name = soup.h1.text.split(\":\")[1]\n",
    "    # get list of ingredients\n",
    "    #check that item above is h3 with id=ingredients\n",
    "    ul = soup.find_all(\"ul\")\n",
    "    ingredient_ul = []\n",
    "    for each in ul:\n",
    "        h2 = each.find_previous_sibling('h2')\n",
    "        h3 = each.find_previous_sibling('h3')\n",
    "        if h2 is not None:\n",
    "            if \"Ingredient\" in h2.text:\n",
    "                ingredient_ul.append(each)\n",
    "        if h3 is not None:\n",
    "            if \"Ingredient\" in h3.text:\n",
    "                ingredient_ul.append(each)\n",
    "    ingredients = []\n",
    "    for ul in ingredient_ul:\n",
    "        ingredients.append(ul.find_all(\"li\"))\n",
    "    #ingredients = ingredient_ul.find_all(\"li\")\n",
    "    units = [\"cups\", \"cup\", \"tablespoons\", \"tablespoon\", \"Tablespoon\", \"teaspoons\", \"teaspoon\", \"tsp\", \"Tbs\", \"oz\", \"tbsp\"]\n",
    "    # for each ingredient, make list and add to dict\n",
    "    for item in ingredients:\n",
    "        for li in item:\n",
    "            # getting full line of text about ingredient\n",
    "            text = li.text\n",
    "        \n",
    "            # finding digits in text\n",
    "            quantity = re.findall(\"^\\d*½*/*-*–*\\d*\", text)\n",
    "            if len(quantity) > 0:\n",
    "                quantity = quantity[0]\n",
    "            else:\n",
    "                quantity = \"\"\n",
    "\n",
    "            # finding units in text\n",
    "            unit_found = re.findall('|'.join(units), text)\n",
    "            if len(unit_found) > 0:\n",
    "                unit = re.findall('|'.join(units), text)[0]\n",
    "            else:\n",
    "                unit = \"\"\n",
    "\n",
    "            # finding ingredients in text\n",
    "            if quantity == \"\" and unit == \"\":\n",
    "                ingredient = text\n",
    "            elif quantity != \"\" and unit == \"\":\n",
    "                ingredient = text.split(quantity)[1]\n",
    "            elif quantity == \"\" and unit != \"\":\n",
    "                ingredient = text.split(unit)[1]\n",
    "            else:\n",
    "                ingredient = text.split(unit)[1]\n",
    "            if \"of\" in ingredient:\n",
    "                ingredient = ingredient.replace(\"of \", \"\")\n",
    "                \n",
    "        # add all features to dict\n",
    "        df_dict[\"id\"].append(recipe_id)\n",
    "        df_dict[\"name\"].append(name)\n",
    "        df_dict[\"text\"].append(text)\n",
    "        df_dict[\"quantity\"].append(quantity)\n",
    "        df_dict[\"unit\"].append(unit)\n",
    "        df_dict[\"ingredient\"].append(ingredient)\n",
    "    recipe_id += 1\n",
    "#df_dict\n",
    "df = pd.DataFrame(df_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>text</th>\n",
       "      <th>quantity</th>\n",
       "      <th>unit</th>\n",
       "      <th>ingredient</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1-2-3-4 Cake</td>\n",
       "      <td>4 eggs</td>\n",
       "      <td>4</td>\n",
       "      <td></td>\n",
       "      <td>eggs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Green Mango and Cumin Drink (Aam Panna)</td>\n",
       "      <td>A handful of fresh mint leaves</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>A handful fresh mint leaves</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Acorn Squash with Orange Marmalade</td>\n",
       "      <td>1/8 teaspoon salt</td>\n",
       "      <td>1/8</td>\n",
       "      <td>teaspoon</td>\n",
       "      <td>salt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Shrimp Fritters (Accra)</td>\n",
       "      <td>black pepper and salt (really optional here) t...</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>black pepper and salt (really optional here) t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Adobo</td>\n",
       "      <td>½ tsp (2.5 mL) freshly cracked (not ground) bl...</td>\n",
       "      <td>½</td>\n",
       "      <td>tsp</td>\n",
       "      <td>(2.5 mL) freshly cracked (not ground) black p...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>Agedashi Tofu</td>\n",
       "      <td>Oil for deep frying</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Oil for deep frying</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>5</td>\n",
       "      <td>Agedashi Tofu</td>\n",
       "      <td>1½ Tbs (22.5 mL) mirin</td>\n",
       "      <td>1½</td>\n",
       "      <td>Tbs</td>\n",
       "      <td>(22.5 mL) mirin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>5</td>\n",
       "      <td>Agedashi Tofu</td>\n",
       "      <td>Grated daikon (white radish)</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Grated daikon (white radish)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>6</td>\n",
       "      <td>Alfredo Sauce</td>\n",
       "      <td>1 pound (450 g) fettuccine egg noodles</td>\n",
       "      <td>1</td>\n",
       "      <td></td>\n",
       "      <td>pound (450 g) fettuccine egg noodles</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>6</td>\n",
       "      <td>Alfredo Sauce</td>\n",
       "      <td>pinch of fresh-ground nutmeg</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>pinch fresh-ground nutmeg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>7</td>\n",
       "      <td>Potato and Cauliflower Curry (Aloo Gobi)</td>\n",
       "      <td>5 sprig of mint</td>\n",
       "      <td>5</td>\n",
       "      <td></td>\n",
       "      <td>sprig mint</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>8</td>\n",
       "      <td>Potato Samosas (Aloo Pies)</td>\n",
       "      <td>Oil for deep frying</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Oil for deep frying</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>9</td>\n",
       "      <td>Potato Curry (Aloo Masala)</td>\n",
       "      <td>1 tablespoon oil</td>\n",
       "      <td>1</td>\n",
       "      <td>tablespoon</td>\n",
       "      <td>oil</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>10</td>\n",
       "      <td>Angels on Horseback</td>\n",
       "      <td>6 slices lean bacon</td>\n",
       "      <td>6</td>\n",
       "      <td></td>\n",
       "      <td>slices lean bacon</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>11</td>\n",
       "      <td>A Nice Cup of Tea</td>\n",
       "      <td>Milk or lemon to taste</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Milk or lemon to taste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>11</td>\n",
       "      <td>A Nice Cup of Tea</td>\n",
       "      <td>3–5 measures of loose tea, in an infuser if yo...</td>\n",
       "      <td>3–5</td>\n",
       "      <td></td>\n",
       "      <td>measures loose tea, in an infuser if you like</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>11</td>\n",
       "      <td>A Nice Cup of Tea</td>\n",
       "      <td>Sweetener, such as sugar or honey, to taste</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Sweetener, such as sugar or honey, to taste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>12</td>\n",
       "      <td>Ambrosia Fruit Salad</td>\n",
       "      <td>3 cups whipping cream</td>\n",
       "      <td>3</td>\n",
       "      <td>cups</td>\n",
       "      <td>whipping cream</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>13</td>\n",
       "      <td>Red Bean Paste (Anko)</td>\n",
       "      <td>a dash salt</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>a dash salt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>13</td>\n",
       "      <td>Red Bean Paste (Anko)</td>\n",
       "      <td>3 cups all-purpose flour (reserve 1 tablespoon...</td>\n",
       "      <td>3</td>\n",
       "      <td>cups</td>\n",
       "      <td>all-purpose flour (reserve 1 tablespoon if mi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>14</td>\n",
       "      <td>Anzac Biscuits</td>\n",
       "      <td>2 tablespoons of water</td>\n",
       "      <td>2</td>\n",
       "      <td>tablespoons</td>\n",
       "      <td>water</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>15</td>\n",
       "      <td>Appam</td>\n",
       "      <td>optional: ½ teaspoon yeast or kefir, to start ...</td>\n",
       "      <td></td>\n",
       "      <td>teaspoon</td>\n",
       "      <td>yeast or kefir, to start the ferment</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>16</td>\n",
       "      <td>Apple Bundt Cake</td>\n",
       "      <td>½ cup raisins</td>\n",
       "      <td>½</td>\n",
       "      <td>cup</td>\n",
       "      <td>raisins</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>17</td>\n",
       "      <td>Apple Crisp</td>\n",
       "      <td>2/3 to 3/4 cup (120-150 g)  brown sugar</td>\n",
       "      <td>2/3</td>\n",
       "      <td>cup</td>\n",
       "      <td>(120-150 g)  brown sugar</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>19</td>\n",
       "      <td>Apple Pie I</td>\n",
       "      <td>2 tablespoons cold water</td>\n",
       "      <td>2</td>\n",
       "      <td>tablespoons</td>\n",
       "      <td>cold water</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>19</td>\n",
       "      <td>Apple Pie I</td>\n",
       "      <td>nutmeg</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>nutmeg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>19</td>\n",
       "      <td>Apple Pie I</td>\n",
       "      <td>sugar</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>sugar</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>20</td>\n",
       "      <td>Apple Pie II</td>\n",
       "      <td>2 tablespoons cold water</td>\n",
       "      <td>2</td>\n",
       "      <td>tablespoons</td>\n",
       "      <td>cold water</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>20</td>\n",
       "      <td>Apple Pie II</td>\n",
       "      <td>2 Pillsbury (or other store-bought) pie crusts</td>\n",
       "      <td>2</td>\n",
       "      <td></td>\n",
       "      <td>Pillsbury (or other store-bought) pie crusts</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>20</td>\n",
       "      <td>Apple Pie II</td>\n",
       "      <td>½ teaspoon nutmeg</td>\n",
       "      <td>½</td>\n",
       "      <td>teaspoon</td>\n",
       "      <td>nutmeg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>22</td>\n",
       "      <td>Apple Tapioca</td>\n",
       "      <td>1 tablespoon butter</td>\n",
       "      <td>1</td>\n",
       "      <td>tablespoon</td>\n",
       "      <td>butter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>23</td>\n",
       "      <td>Arroz con Pollo</td>\n",
       "      <td>225 g (8 oz) canned peas (thoroughly drained)</td>\n",
       "      <td>225</td>\n",
       "      <td>oz</td>\n",
       "      <td>) canned peas (thoroughly drained)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>24</td>\n",
       "      <td>Arroz Negro</td>\n",
       "      <td>olive oil</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>olive oil</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>24</td>\n",
       "      <td>Arroz Negro</td>\n",
       "      <td>1 gallon of water</td>\n",
       "      <td>1</td>\n",
       "      <td></td>\n",
       "      <td>gallon water</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>25</td>\n",
       "      <td>Rice - Molded (Arroz Tapado)</td>\n",
       "      <td>1 tablespoon parsley, finely minced</td>\n",
       "      <td>1</td>\n",
       "      <td>tablespoon</td>\n",
       "      <td>parsley, finely minced</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>26</td>\n",
       "      <td>Artichoke Hearts Attack</td>\n",
       "      <td>Parmesan or Romano cheese</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Parmesan or Romano cheese</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>27</td>\n",
       "      <td>Asparagus Soup</td>\n",
       "      <td>to taste: salt, freshly ground black pepper, a...</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>to taste: salt, freshly ground black pepper, a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>28</td>\n",
       "      <td>Atkins-Friendly Tira Misu</td>\n",
       "      <td>1 tsp instant espresso powder</td>\n",
       "      <td>1</td>\n",
       "      <td>tsp</td>\n",
       "      <td>instant espresso powder</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>29</td>\n",
       "      <td>Azteca</td>\n",
       "      <td>mustard seed, salt, and freshly ground black p...</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>mustard seed, salt, and freshly ground black p...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>30</td>\n",
       "      <td>Hard Tack of India (Baati)</td>\n",
       "      <td>1 1/2 teaspoons ghee (for serving)</td>\n",
       "      <td>1</td>\n",
       "      <td>teaspoons</td>\n",
       "      <td>ghee (for serving)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>31</td>\n",
       "      <td>Baba Ganoush</td>\n",
       "      <td>Paprika or cayenne pepper, as a garnish</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Paprika or cayenne pepper, as a garnish</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>32</td>\n",
       "      <td>Salt Cod and Potato Casserole (Bacalhau à Gome...</td>\n",
       "      <td>2 Tbs. parsley, finely chopped</td>\n",
       "      <td>2</td>\n",
       "      <td>Tbs</td>\n",
       "      <td>. parsley, finely chopped</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>33</td>\n",
       "      <td>Bacon Cheese Omelet</td>\n",
       "      <td>1 Tablespoon of butter</td>\n",
       "      <td>1</td>\n",
       "      <td>Tablespoon</td>\n",
       "      <td>butter</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>34</td>\n",
       "      <td>Baingan Bartha (Southern)</td>\n",
       "      <td>salt to taste</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>salt to taste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>35</td>\n",
       "      <td>Baked Lamb and Yogurt</td>\n",
       "      <td>salt, pepper</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>salt, pepper</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>35</td>\n",
       "      <td>Baked Lamb and Yogurt</td>\n",
       "      <td>salt, pepper</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>salt, pepper</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>36</td>\n",
       "      <td>Baked Oatmeal</td>\n",
       "      <td>2 eggs</td>\n",
       "      <td>2</td>\n",
       "      <td></td>\n",
       "      <td>eggs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>37</td>\n",
       "      <td>Baked Pork Chops</td>\n",
       "      <td>Garlic salt, black pepper, and Dijon mustard, ...</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Garlic salt, black pepper, and Dijon mustard, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>39</td>\n",
       "      <td>Banana Nut Muffin</td>\n",
       "      <td>Note: Add sugar to taste, if necessary.</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Note: Add sugar to taste, if necessary.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>40</td>\n",
       "      <td>Banana Chocolate Bread Pudding</td>\n",
       "      <td>3 bananas</td>\n",
       "      <td>3</td>\n",
       "      <td></td>\n",
       "      <td>bananas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>41</td>\n",
       "      <td>Banana Cream Pie</td>\n",
       "      <td>Grandma's Pie Crust.</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Grandma's Pie Crust.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>41</td>\n",
       "      <td>Banana Cream Pie</td>\n",
       "      <td>1/4 cup lemon juice</td>\n",
       "      <td>1/4</td>\n",
       "      <td>cup</td>\n",
       "      <td>lemon juice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>41</td>\n",
       "      <td>Banana Cream Pie</td>\n",
       "      <td>1 1/2 teaspoons powdered sugar</td>\n",
       "      <td>1</td>\n",
       "      <td>teaspoons</td>\n",
       "      <td>powdered sugar</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>42</td>\n",
       "      <td>Banana Cream Pie II</td>\n",
       "      <td>meringue made from the leftover egg whites (op...</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>meringue made from the leftover egg whites (op...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>43</td>\n",
       "      <td>Pancake</td>\n",
       "      <td>1 tbsp white sugar (Optional: this will add a ...</td>\n",
       "      <td>1</td>\n",
       "      <td>tbsp</td>\n",
       "      <td>white sugar (Optional: this will add a sweetn...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>45</td>\n",
       "      <td>Barbecued Spare Ribs</td>\n",
       "      <td>(Optional) 2 teaspoons Cumin</td>\n",
       "      <td></td>\n",
       "      <td>teaspoons</td>\n",
       "      <td>Cumin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>46</td>\n",
       "      <td>Barbecue Sauce</td>\n",
       "      <td>1/2 teaspoon \"concentrated\" liquid smoke flavo...</td>\n",
       "      <td>1/2</td>\n",
       "      <td>teaspoon</td>\n",
       "      <td>\"concentrated\" liquid smoke flavoring</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>47</td>\n",
       "      <td>Barbecue Sauce - Kansas City style</td>\n",
       "      <td>1 teaspoon (5 mL) salt</td>\n",
       "      <td>1</td>\n",
       "      <td>teaspoon</td>\n",
       "      <td>(5 mL) salt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>48</td>\n",
       "      <td>Basic Indian Tomato Gravy</td>\n",
       "      <td>salt to taste</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>salt to taste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>49</td>\n",
       "      <td>Bean Jahni soup</td>\n",
       "      <td>chili powder</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>chili powder</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id                                               name  \\\n",
       "0    0                                       1-2-3-4 Cake   \n",
       "1    1            Green Mango and Cumin Drink (Aam Panna)   \n",
       "2    2                 Acorn Squash with Orange Marmalade   \n",
       "3    3                            Shrimp Fritters (Accra)   \n",
       "4    4                                              Adobo   \n",
       "5    5                                      Agedashi Tofu   \n",
       "6    5                                      Agedashi Tofu   \n",
       "7    5                                      Agedashi Tofu   \n",
       "8    6                                      Alfredo Sauce   \n",
       "9    6                                      Alfredo Sauce   \n",
       "10   7           Potato and Cauliflower Curry (Aloo Gobi)   \n",
       "11   8                         Potato Samosas (Aloo Pies)   \n",
       "12   9                         Potato Curry (Aloo Masala)   \n",
       "13  10                                Angels on Horseback   \n",
       "14  11                                  A Nice Cup of Tea   \n",
       "15  11                                  A Nice Cup of Tea   \n",
       "16  11                                  A Nice Cup of Tea   \n",
       "17  12                               Ambrosia Fruit Salad   \n",
       "18  13                              Red Bean Paste (Anko)   \n",
       "19  13                              Red Bean Paste (Anko)   \n",
       "20  14                                     Anzac Biscuits   \n",
       "21  15                                              Appam   \n",
       "22  16                                   Apple Bundt Cake   \n",
       "23  17                                        Apple Crisp   \n",
       "24  19                                        Apple Pie I   \n",
       "25  19                                        Apple Pie I   \n",
       "26  19                                        Apple Pie I   \n",
       "27  20                                       Apple Pie II   \n",
       "28  20                                       Apple Pie II   \n",
       "29  20                                       Apple Pie II   \n",
       "30  22                                      Apple Tapioca   \n",
       "31  23                                    Arroz con Pollo   \n",
       "32  24                                        Arroz Negro   \n",
       "33  24                                        Arroz Negro   \n",
       "34  25                       Rice - Molded (Arroz Tapado)   \n",
       "35  26                            Artichoke Hearts Attack   \n",
       "36  27                                     Asparagus Soup   \n",
       "37  28                          Atkins-Friendly Tira Misu   \n",
       "38  29                                             Azteca   \n",
       "39  30                         Hard Tack of India (Baati)   \n",
       "40  31                                       Baba Ganoush   \n",
       "41  32  Salt Cod and Potato Casserole (Bacalhau à Gome...   \n",
       "42  33                                Bacon Cheese Omelet   \n",
       "43  34                          Baingan Bartha (Southern)   \n",
       "44  35                              Baked Lamb and Yogurt   \n",
       "45  35                              Baked Lamb and Yogurt   \n",
       "46  36                                      Baked Oatmeal   \n",
       "47  37                                   Baked Pork Chops   \n",
       "48  39                                  Banana Nut Muffin   \n",
       "49  40                     Banana Chocolate Bread Pudding   \n",
       "50  41                                   Banana Cream Pie   \n",
       "51  41                                   Banana Cream Pie   \n",
       "52  41                                   Banana Cream Pie   \n",
       "53  42                                Banana Cream Pie II   \n",
       "54  43                                            Pancake   \n",
       "55  45                               Barbecued Spare Ribs   \n",
       "56  46                                     Barbecue Sauce   \n",
       "57  47                 Barbecue Sauce - Kansas City style   \n",
       "58  48                          Basic Indian Tomato Gravy   \n",
       "59  49                                    Bean Jahni soup   \n",
       "\n",
       "                                                 text quantity         unit  \\\n",
       "0                                              4 eggs        4                \n",
       "1                      A handful of fresh mint leaves                         \n",
       "2                                   1/8 teaspoon salt      1/8     teaspoon   \n",
       "3   black pepper and salt (really optional here) t...                         \n",
       "4   ½ tsp (2.5 mL) freshly cracked (not ground) bl...        ½          tsp   \n",
       "5                                 Oil for deep frying                         \n",
       "6                              1½ Tbs (22.5 mL) mirin       1½          Tbs   \n",
       "7                        Grated daikon (white radish)                         \n",
       "8              1 pound (450 g) fettuccine egg noodles        1                \n",
       "9                        pinch of fresh-ground nutmeg                         \n",
       "10                                    5 sprig of mint        5                \n",
       "11                                Oil for deep frying                         \n",
       "12                                   1 tablespoon oil        1   tablespoon   \n",
       "13                                6 slices lean bacon        6                \n",
       "14                             Milk or lemon to taste                         \n",
       "15  3–5 measures of loose tea, in an infuser if yo...      3–5                \n",
       "16        Sweetener, such as sugar or honey, to taste                         \n",
       "17                              3 cups whipping cream        3         cups   \n",
       "18                                        a dash salt                         \n",
       "19  3 cups all-purpose flour (reserve 1 tablespoon...        3         cups   \n",
       "20                             2 tablespoons of water        2  tablespoons   \n",
       "21  optional: ½ teaspoon yeast or kefir, to start ...              teaspoon   \n",
       "22                                      ½ cup raisins        ½          cup   \n",
       "23            2/3 to 3/4 cup (120-150 g)  brown sugar      2/3          cup   \n",
       "24                           2 tablespoons cold water        2  tablespoons   \n",
       "25                                             nutmeg                         \n",
       "26                                              sugar                         \n",
       "27                           2 tablespoons cold water        2  tablespoons   \n",
       "28     2 Pillsbury (or other store-bought) pie crusts        2                \n",
       "29                                  ½ teaspoon nutmeg        ½     teaspoon   \n",
       "30                                1 tablespoon butter        1   tablespoon   \n",
       "31      225 g (8 oz) canned peas (thoroughly drained)      225           oz   \n",
       "32                                          olive oil                         \n",
       "33                                  1 gallon of water        1                \n",
       "34                1 tablespoon parsley, finely minced        1   tablespoon   \n",
       "35                          Parmesan or Romano cheese                         \n",
       "36  to taste: salt, freshly ground black pepper, a...                         \n",
       "37                      1 tsp instant espresso powder        1          tsp   \n",
       "38  mustard seed, salt, and freshly ground black p...                         \n",
       "39                 1 1/2 teaspoons ghee (for serving)        1    teaspoons   \n",
       "40            Paprika or cayenne pepper, as a garnish                         \n",
       "41                     2 Tbs. parsley, finely chopped        2          Tbs   \n",
       "42                             1 Tablespoon of butter        1   Tablespoon   \n",
       "43                                      salt to taste                         \n",
       "44                                       salt, pepper                         \n",
       "45                                       salt, pepper                         \n",
       "46                                             2 eggs        2                \n",
       "47  Garlic salt, black pepper, and Dijon mustard, ...                         \n",
       "48            Note: Add sugar to taste, if necessary.                         \n",
       "49                                          3 bananas        3                \n",
       "50                               Grandma's Pie Crust.                         \n",
       "51                                1/4 cup lemon juice      1/4          cup   \n",
       "52                     1 1/2 teaspoons powdered sugar        1    teaspoons   \n",
       "53  meringue made from the leftover egg whites (op...                         \n",
       "54  1 tbsp white sugar (Optional: this will add a ...        1         tbsp   \n",
       "55                       (Optional) 2 teaspoons Cumin             teaspoons   \n",
       "56  1/2 teaspoon \"concentrated\" liquid smoke flavo...      1/2     teaspoon   \n",
       "57                             1 teaspoon (5 mL) salt        1     teaspoon   \n",
       "58                                      salt to taste                         \n",
       "59                                       chili powder                         \n",
       "\n",
       "                                           ingredient  \n",
       "0                                                eggs  \n",
       "1                         A handful fresh mint leaves  \n",
       "2                                                salt  \n",
       "3   black pepper and salt (really optional here) t...  \n",
       "4    (2.5 mL) freshly cracked (not ground) black p...  \n",
       "5                                 Oil for deep frying  \n",
       "6                                     (22.5 mL) mirin  \n",
       "7                        Grated daikon (white radish)  \n",
       "8                pound (450 g) fettuccine egg noodles  \n",
       "9                           pinch fresh-ground nutmeg  \n",
       "10                                         sprig mint  \n",
       "11                                Oil for deep frying  \n",
       "12                                                oil  \n",
       "13                                  slices lean bacon  \n",
       "14                             Milk or lemon to taste  \n",
       "15      measures loose tea, in an infuser if you like  \n",
       "16        Sweetener, such as sugar or honey, to taste  \n",
       "17                                     whipping cream  \n",
       "18                                        a dash salt  \n",
       "19   all-purpose flour (reserve 1 tablespoon if mi...  \n",
       "20                                              water  \n",
       "21               yeast or kefir, to start the ferment  \n",
       "22                                            raisins  \n",
       "23                           (120-150 g)  brown sugar  \n",
       "24                                         cold water  \n",
       "25                                             nutmeg  \n",
       "26                                              sugar  \n",
       "27                                         cold water  \n",
       "28       Pillsbury (or other store-bought) pie crusts  \n",
       "29                                             nutmeg  \n",
       "30                                             butter  \n",
       "31                 ) canned peas (thoroughly drained)  \n",
       "32                                          olive oil  \n",
       "33                                       gallon water  \n",
       "34                             parsley, finely minced  \n",
       "35                          Parmesan or Romano cheese  \n",
       "36  to taste: salt, freshly ground black pepper, a...  \n",
       "37                            instant espresso powder  \n",
       "38  mustard seed, salt, and freshly ground black p...  \n",
       "39                                 ghee (for serving)  \n",
       "40            Paprika or cayenne pepper, as a garnish  \n",
       "41                          . parsley, finely chopped  \n",
       "42                                             butter  \n",
       "43                                      salt to taste  \n",
       "44                                       salt, pepper  \n",
       "45                                       salt, pepper  \n",
       "46                                               eggs  \n",
       "47  Garlic salt, black pepper, and Dijon mustard, ...  \n",
       "48            Note: Add sugar to taste, if necessary.  \n",
       "49                                            bananas  \n",
       "50                               Grandma's Pie Crust.  \n",
       "51                                        lemon juice  \n",
       "52                                     powdered sugar  \n",
       "53  meringue made from the leftover egg whites (op...  \n",
       "54   white sugar (Optional: this will add a sweetn...  \n",
       "55                                              Cumin  \n",
       "56              \"concentrated\" liquid smoke flavoring  \n",
       "57                                        (5 mL) salt  \n",
       "58                                      salt to taste  \n",
       "59                                       chili powder  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"recipe_info.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the issues I found that I couldn't easily solve was fully isolating the ingredient itself. Many of my ingredients still have extraneous text attached to them (like the measurement in grams). It is nearly impossible to isolate only the ingredient itself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Sweet or Salty? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1. There are more (or equal) sweet recipes than salty ones.\n"
     ]
    }
   ],
   "source": [
    "sweet = df[df[\"ingredient\"] == \"sugar\"]\n",
    "salty = df[df[\"ingredient\"] == \"salt\"]\n",
    "if len(sweet) >= len(salty):\n",
    "    print(\"1. There are more (or equal) sweet recipes than salty ones.\")\n",
    "else:\n",
    "    print(\"1. There are less sweet recipes than salty ones.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. There are problems with this since the units in each recipe are so varied. You would have to choose one unit and convert all recipes to that unit and then compare, which isn't possible given the weird quanities like \"a handful\". Additionally, this doesn't take into account the amount of sugar in each of the ingredients. For example, chocolate might have 25 grams of sugar that cannot be accounted for because we do not have the nutrition facts of each ingredient and therefore absolute values cannot be accurately compared.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Favorite Recipe\n",
    "My favorite recipe is chocolate chip cookies but I am out of groceries right now so I cant make them :("
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}