notebooks/03.10-Working-With-Strings.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "< [数据透视表](03.09-Pivot-Tables.ipynb) | [目录](Index.ipynb) | [在时间序列上操作](03.11-Working-with-Time-Series.ipynb) >\n",
    "\n",
    "<a href=\"https://colab.research.google.com/github/wangyingsm/Python-Data-Science-Handbook/blob/master/notebooks/03.10-Working-With-Strings.ipynb\"><img align=\"left\" src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\" title=\"Open and Execute in Google Colaboratory\"></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Vectorized String Operations\n",
    "\n",
    "# 向量化的字符串操作"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> One strength of Python is its relative ease in handling and manipulating string data.\n",
    "Pandas builds on this and provides a comprehensive set of *vectorized string operations* that become an essential piece of the type of munging required when working with (read: cleaning up) real-world data.\n",
    "In this section, we'll walk through some of the Pandas string operations, and then take a look at using them to partially clean up a very messy dataset of recipes collected from the Internet.\n",
    "\n",
    "Python的一个强大的特点就是它能相对简单的处理和操作字符串数据。Pandas在此基础上提供了一整套*向量化字符串操作*，这成为了当我们处理（清洗）真实世界数据时非常关键的功能。在本节中，我们将对很多Pandas的字符串操作进行介绍，然后看它们在我们对从互联网采集到的非常不规范的数据集进行清洗时发挥的作用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introducing Pandas String Operations\n",
    "\n",
    "## Pandas字符串操作介绍\n",
    "\n",
    "> We saw in previous sections how tools like NumPy and Pandas generalize arithmetic operations so that we can easily and quickly perform the same operation on many array elements. For example:\n",
    "\n",
    "在前面章节中我们看到NumPy和Pandas的工具能向量化算术运算，让我们可以很容易和快速的对数组的元素进行相同的数学计算。例如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 4,  6, 10, 14, 22, 26])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "x = np.array([2, 3, 5, 7, 11, 13])\n",
    "x * 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> This *vectorization* of operations simplifies the syntax of operating on arrays of data: we no longer have to worry about the size or shape of the array, but just about what operation we want done.\n",
    "For arrays of strings, NumPy does not provide such simple access, and thus you're stuck using a more verbose loop syntax:\n",
    "\n",
    "这种*向量化*的操作能简化数组元素的操作语法：我们不再需要担心数组的大小和形状，只需要关注于需要进行的运算本身。对于字符串数组，NumPy没有提供这种简单的操作，因此你需要继续使用循环语法来处理："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Peter', 'Paul', 'Mary', 'Guido']"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = ['peter', 'Paul', 'MARY', 'gUIDO']\n",
    "[s.capitalize() for s in data]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> This is perhaps sufficient to work with some data, but it will break if there are any missing values.\n",
    "For example:\n",
    "\n",
    "这可能对于一些数据集来说足够了，但是对于含有缺失值的数据集来说就出问题了。例如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'NoneType' object has no attribute 'capitalize'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-3-3b0264c38d59>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m'peter'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'Paul'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'MARY'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'gUIDO'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcapitalize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ms\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-3-3b0264c38d59>\u001b[0m in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m'peter'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'Paul'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'MARY'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'gUIDO'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcapitalize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ms\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m: 'NoneType' object has no attribute 'capitalize'"
     ]
    }
   ],
   "source": [
    "data = ['peter', 'Paul', None, 'MARY', 'gUIDO']\n",
    "[s.capitalize() for s in data]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Pandas includes features to address both this need for vectorized string operations and for correctly handling missing data via the ``str`` attribute of Pandas Series and Index objects containing strings.\n",
    "So, for example, suppose we create a Pandas Series with this data:\n",
    "\n",
    "Pandas包含了前面说到的向量化的字符串操作，而且还能正确的处理缺失值，这可以通过Pandas的Series和Index对象的`str`属性来实现。例如，假设我们如下创建一个Pandas Series："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    peter\n",
       "1     Paul\n",
       "2     None\n",
       "3     MARY\n",
       "4    gUIDO\n",
       "dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "names = pd.Series(data)\n",
    "names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We can now call a single method that will capitalize all the entries, while skipping over any missing values:\n",
    "\n",
    "我们现在可以调用一个方法就能将所有元素首字母变大写的功能，并能跳过缺失值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    Peter\n",
       "1     Paul\n",
       "2     None\n",
       "3     Mary\n",
       "4    Guido\n",
       "dtype: object"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "names.str.capitalize()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Using tab completion on this ``str`` attribute will list all the vectorized string methods available to Pandas.\n",
    "\n",
    "在IPython中在`str`属性上使用制表符自动补全功能可以列出Pandas中支持的所有的向量化字符串操作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tables of Pandas String Methods\n",
    "\n",
    "## Pandas字符串方法列表\n",
    "\n",
    "> If you have a good understanding of string manipulation in Python, most of Pandas string syntax is intuitive enough that it's probably sufficient to just list a table of available methods; we will start with that here, before diving deeper into a few of the subtleties.\n",
    "The examples in this section use the following series of names:\n",
    "\n",
    "如果你已经很好理解了Python中的字符串操作，大多数的Pandas字符串操作语法是很直观的，因此简单的列一张表格说明所有可用的方法就足够理解了；我们在深入到一些细节之前可以先浏览这张表。本节的例子将会使用下面的一个姓名的Series对象："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "monte = pd.Series(['Graham Chapman', 'John Cleese', 'Terry Gilliam',\n",
    "                   'Eric Idle', 'Terry Jones', 'Michael Palin'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Methods similar to Python string methods\n",
    "\n",
    "### 类似Python的字符串方法\n",
    "\n",
    "> Nearly all Python's built-in string methods are mirrored by a Pandas vectorized string method. Here is a list of Pandas ``str`` methods that mirror Python string methods:\n",
    "\n",
    "几乎所有Python內建的字符串方法都有Pandas的向量化版本。下面是Pandas的`str`属性中与Python內建字符串方法一致的方法：\n",
    "\n",
    "|             |                  |                  |                  |\n",
    "|-------------|------------------|------------------|------------------|\n",
    "|``len()``    | ``lower()``      | ``translate()``  | ``islower()``    | \n",
    "|``ljust()``  | ``upper()``      | ``startswith()`` | ``isupper()``    | \n",
    "|``rjust()``  | ``find()``       | ``endswith()``   | ``isnumeric()``  | \n",
    "|``center()`` | ``rfind()``      | ``isalnum()``    | ``isdecimal()``  | \n",
    "|``zfill()``  | ``index()``      | ``isalpha()``    | ``split()``      | \n",
    "|``strip()``  | ``rindex()``     | ``isdigit()``    | ``rsplit()``     | \n",
    "|``rstrip()`` | ``capitalize()`` | ``isspace()``    | ``partition()``  | \n",
    "|``lstrip()`` |  ``swapcase()``  |  ``istitle()``   | ``rpartition()`` |\n",
    "\n",
    "> Notice that these have various return values. Some, like ``lower()``, return a series of strings:\n",
    "\n",
    "要提醒的是，这些方法与內建字符串方法可能有着不同的返回值，如`lower()`返回的是一个字符串的Series对象："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    graham chapman\n",
       "1       john cleese\n",
       "2     terry gilliam\n",
       "3         eric idle\n",
       "4       terry jones\n",
       "5     michael palin\n",
       "dtype: object"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monte.str.lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> But some others return numbers:\n",
    "\n",
    "另外一些返回的是数字的Series对象："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    14\n",
       "1    11\n",
       "2    13\n",
       "3     9\n",
       "4    11\n",
       "5    13\n",
       "dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monte.str.len()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Or Boolean values:\n",
    "\n",
    "或布尔值的Series对象："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2     True\n",
       "3    False\n",
       "4     True\n",
       "5    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monte.str.startswith('T')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Still others return lists or other compound values for each element:\n",
    "\n",
    "还有一些会返回诸如列表那样的复合类型的Series对象："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    [Graham, Chapman]\n",
       "1       [John, Cleese]\n",
       "2     [Terry, Gilliam]\n",
       "3         [Eric, Idle]\n",
       "4       [Terry, Jones]\n",
       "5     [Michael, Palin]\n",
       "dtype: object"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monte.str.split()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We'll see further manipulations of this kind of series-of-lists object as we continue our discussion.\n",
    "\n",
    "我们后面会讨论到如何操作这种列表组成的Series对象。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Methods using regular expressions\n",
    "\n",
    "### 使用正则表达式的方法\n",
    "\n",
    "> In addition, there are several methods that accept regular expressions to examine the content of each string element, and follow some of the API conventions of Python's built-in ``re`` module:\n",
    "\n",
    "除此之外，还有一些方法可以接受正则表达式来检查每个元素字符串是否匹配模式，它们遵从Python內建的`re`模块的API规范：\n",
    "\n",
    "| 方法 | 描述 |\n",
    "|--------|-------------|\n",
    "| ``match()`` | 在每个元素上调用``re.match()``方法，返回布尔类型Series |\n",
    "| ``extract()`` | 在每个元素上调用``re.match()``方法，返回匹配到模式的正则分组的Series |\n",
    "| ``findall()`` | 在每个元素上调用``re.findall()``方法 |\n",
    "| ``replace()`` | 将匹配模式的字符串部分替换成其他字符串值 |\n",
    "| ``contains()`` | 在每个元素上调用``re.search()``，返回布尔类型Series |\n",
    "| ``count()`` | 计算匹配到模式的次数 |\n",
    "| ``split()``   | 等同于``str.split()``，但是能接受正则表达式参数 |\n",
    "| ``rsplit()`` | 等同于``str.rsplit()``, 但是能接受正则表达式参数 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> With these, you can do a wide range of interesting operations.\n",
    "For example, we can extract the first name from each by asking for a contiguous group of characters at the beginning of each element:\n",
    "\n",
    "使用上面的方法，你可以执行很多有趣的操作。例如，我们可以通过匹配连续的一组字母的模式从姓名中提取出名字："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     Graham\n",
       "1       John\n",
       "2      Terry\n",
       "3       Eric\n",
       "4      Terry\n",
       "5    Michael\n",
       "dtype: object"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monte.str.extract('([A-Za-z]+)', expand=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Or we can do something more complicated, like finding all names that start and end with a consonant, making use of the start-of-string (``^``) and end-of-string (``$``) regular expression characters:\n",
    "\n",
    "或者我们可以执行更复杂的操作，如找出所有姓名中首字母和尾字母都是辅音字母的人，这里需要使用字符串开始位置（`^`）和字符串结束位置（`$`）正则表达式特殊符号："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    [Graham Chapman]\n",
       "1                  []\n",
       "2     [Terry Gilliam]\n",
       "3                  []\n",
       "4       [Terry Jones]\n",
       "5     [Michael Palin]\n",
       "dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monte.str.findall(r'^[^AEIOU].*[^aeiou]$')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> The ability to concisely apply regular expressions across ``Series`` or ``Dataframe`` entries opens up many possibilities for analysis and cleaning of data.\n",
    "\n",
    "这种在`Series`或`DataFrame`上简洁的应用正则表达式的特性，在清洗和分析数据任务中非常有用。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Miscellaneous methods\n",
    "\n",
    "### 其他方法\n",
    "\n",
    "> Finally, there are some miscellaneous methods that enable other convenient operations:\n",
    "\n",
    "最后，下面是一些无法分类的其他方法但也是很方便的字符串功能：\n",
    "\n",
    "| 方法 | 描述 |\n",
    "|--------|-------------|\n",
    "| ``get()`` | 对每个元素使用索引值获取字符中的字符 |\n",
    "| ``slice()`` | 对每个元素进行字符串切片 |\n",
    "| ``slice_replace()`` | 将每个元素的字符串切片替换成另一个字符串值 |\n",
    "| ``cat()``      | 将所有字符串元素连接成一个字符串 |\n",
    "| ``repeat()`` | 对每个字符串元素进行重复操作 |\n",
    "| ``normalize()`` | 返回字符串的unicode标准化结果 |\n",
    "| ``pad()`` | 字符串对齐 |\n",
    "| ``wrap()`` | 字符串换行 |\n",
    "| ``join()`` | 字符串中字符的连接 |\n",
    "| ``get_dummies()`` | 将字符串按照分隔符分割后形成一个二维的dummy DataFrame |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Vectorized item access and slicing\n",
    "\n",
    "#### 向量化的索引和切片操作\n",
    "\n",
    "> The ``get()`` and ``slice()`` operations, in particular, enable vectorized element access from each array.\n",
    "For example, we can get a slice of the first three characters of each array using ``str.slice(0, 3)``.\n",
    "Note that this behavior is also available through Python's normal indexing syntax–for example, ``df.str.slice(0, 3)`` is equivalent to ``df.str[0:3]``:\n",
    "\n",
    "`get()`和`slice()`操作，可以对每个字符串元素进行索引访问和切片的操作。例如，我们可以通过`str.slice(0, 3)`获取每个字符串元素的前三个字母。还需要说明的是，这个操作也可以通过Python标准的切片语法来完成，也就是说`df.str[:3]`等同于`df.str.slice(0, 3)`："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    Gra\n",
       "1    Joh\n",
       "2    Ter\n",
       "3    Eri\n",
       "4    Ter\n",
       "5    Mic\n",
       "dtype: object"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monte.str[:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Indexing via ``df.str.get(i)`` and ``df.str[i]`` is likewise similar.\n",
    "\n",
    "索引取值操作也是一样，`df.str[i]`等同于`df.str.get(i)`。\n",
    "\n",
    "> These ``get()`` and ``slice()`` methods also let you access elements of arrays returned by ``split()``.\n",
    "For example, to extract the last name of each entry, we can combine ``split()`` and ``get()``:\n",
    "\n",
    "`get()`和`slice()`方法还能支持对`split()`返回的列表进行取值操作。例如我们使用`split()`和`get()`方法可以提取出每个人的姓："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    Chapman\n",
       "1     Cleese\n",
       "2    Gilliam\n",
       "3       Idle\n",
       "4      Jones\n",
       "5      Palin\n",
       "dtype: object"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "monte.str.split().str.get(-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Indicator variables\n",
    "\n",
    "#### 指示器变量\n",
    "\n",
    "> Another method that requires a bit of extra explanation is the ``get_dummies()`` method.\n",
    "This is useful when your data has a column containing some sort of coded indicator.\n",
    "For example, we might have a dataset that contains information in the form of codes, such as A=\"born in America,\" B=\"born in the United Kingdom,\" C=\"likes cheese,\" D=\"likes spam\":\n",
    "\n",
    "还有一个需要进行说明的方法是`get_dummies()`。这个方法在你的数据中含有某种编码的指示器的时候非常有用。例如，我们有一个数据集，里面有一个编码了的列，A代表“出生在美国”，B代表“出生在英国”，C代表“喜欢芝士”，D代表“喜欢肉罐头”："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>info</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Graham Chapman</td>\n",
       "      <td>B|C|D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>John Cleese</td>\n",
       "      <td>B|D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Terry Gilliam</td>\n",
       "      <td>A|C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Eric Idle</td>\n",
       "      <td>B|D</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Terry Jones</td>\n",
       "      <td>B|C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Michael Palin</td>\n",
       "      <td>B|C|D</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             name   info\n",
       "0  Graham Chapman  B|C|D\n",
       "1     John Cleese    B|D\n",
       "2   Terry Gilliam    A|C\n",
       "3       Eric Idle    B|D\n",
       "4     Terry Jones    B|C\n",
       "5   Michael Palin  B|C|D"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "full_monte = pd.DataFrame({'name': monte,\n",
    "                           'info': ['B|C|D', 'B|D', 'A|C',\n",
    "                                    'B|D', 'B|C', 'B|C|D']})\n",
    "full_monte"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> The ``get_dummies()`` routine lets you quickly split-out these indicator variables into a ``DataFrame``:\n",
    "\n",
    "`get_dummies()`方法能让你快速的将这些编码的指示器变量分解出来，并形成一个`DataFrame`："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "      <th>D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   A  B  C  D\n",
       "0  0  1  1  1\n",
       "1  0  1  0  1\n",
       "2  1  0  1  0\n",
       "3  0  1  0  1\n",
       "4  0  1  1  0\n",
       "5  0  1  1  1"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "full_monte['info'].str.get_dummies('|')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> With these operations as building blocks, you can construct an endless range of string processing procedures when cleaning your data.\n",
    "\n",
    "有了上述的这些向量化字符串方法，你可以在清洗数据时构建无穷无尽的字符串处理流程。\n",
    "\n",
    "> We won't dive further into these methods here, but I encourage you to read through [\"Working with Text Data\"](http://pandas.pydata.org/pandas-docs/stable/text.html) in the Pandas online documentation, or to refer to the resources listed in [Further Resources](03.13-Further-Resources.ipynb).\n",
    "\n",
    "在这里我们不在深入介绍每个方法，作者推荐你去阅读Pandas的在线文档[处理文本数据](http://pandas.pydata.org/pandas-docs/stable/text.html)，或者参考[更多资源](03.13-Further-Resources.ipynb)中的其他资料。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example: Recipe Database\n",
    "\n",
    "## 例子：菜谱数据库\n",
    "\n",
    "> These vectorized string operations become most useful in the process of cleaning up messy, real-world data.\n",
    "Here I'll walk through an example of that, using an open recipe database compiled from various sources on the Web.\n",
    "Our goal will be to parse the recipe data into ingredient lists, so we can quickly find a recipe based on some ingredients we have on hand.\n",
    "\n",
    "上述介绍的这些向量化字符串操作是我们对不规范的真实世界数据进行清洗的最有效工具。下面我们将使用网络中收集的一个菜谱数据库作为例子来总体说明。我们的目标是将这些菜谱数据解析成配方的列表，这样我们就能很快速的根据我们手头的材料找到相应配方的菜谱。\n",
    "\n",
    "> The scripts used to compile this can be found at https://github.com/fictivekin/openrecipes, and the link to the current version of the database is found there as well.\n",
    "\n",
    "用来收集菜谱的脚本可以在https://github.com/fictivekin/openrecipes 这里找到，最新版本的菜谱数据库的连接也在这个页面上。\n",
    "\n",
    "> As of Spring 2016, this database is about 30 MB, and can be downloaded and unzipped with these commands:\n",
    "\n",
    "到2016年春天，这个数据库有大约30MB大小，可以通过下面的命令下载以及解压：\n",
    "\n",
    "译者注：open recipes数据库在2017年后已经无法使用原作者的地址进行下载，参见[Issue#179](https://github.com/jakevdp/PythonDataScienceHandbook/issues/179)。使用新的地址可以正确下载数据库内容。下述shell代码也使用新地址进行了修改。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !curl -O https://s3.amazonaws.com/openrecipes/20170107-061401-recipeitems.json.gz\n",
    "# !gunzip 20170107-061401-recipeitems.json.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> The database is in JSON format, so we will try ``pd.read_json`` to read it:\n",
    "\n",
    "数据库是JSON格式，因此我们需要使用`pd.read_json`方法来读取它："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ValueError: Trailing data\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    recipes = pd.read_json('data/20170107-061401-recipeitems.json')\n",
    "except ValueError as e:\n",
    "    print(\"ValueError:\", e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Oops! We get a ``ValueError`` mentioning that there is \"trailing data.\"\n",
    "Searching for the text of this error on the Internet, it seems that it's due to using a file in which *each line* is itself a valid JSON, but the full file is not.\n",
    "Let's check if this interpretation is true:\n",
    "\n",
    "喔噢，这里会产生一个`ValueError`指出有冗余的数据。通过在网上搜索这个错误信息，我们得到原因是这个文件*每一行*都是一个正确的JSON，但是整个文件不是正确的JSON格式。我们来验证一下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 12)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with open('data/20170107-061401-recipeitems.json') as f:\n",
    "    line = f.readline()\n",
    "pd.read_json(line).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Yes, apparently each line is a valid JSON, so we'll need to string them together.\n",
    "One way we can do this is to actually construct a string representation containing all these JSON entries, and then load the whole thing with ``pd.read_json``:\n",
    "\n",
    "通过读取文件一行我们验证了我们的想法，现在我们需要将这些正确的JSON行合并在一起。实现这个目标的一种方式就是我们手动将所有的行合并成一个JSON Array，然后将这个JSON Array的字符串传递到`pd.read_json`来进行解析："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将每一行JSON对象合并成一整个JSON Array\n",
    "with open('data/20170107-061401-recipeitems.json', 'r') as f:\n",
    "    # 提取每一行\n",
    "    data = (line.strip() for line in f)\n",
    "    # 每两个对象之间用,分隔，最后首尾加上中括号表示数组\n",
    "    data_json = \"[{0}]\".format(','.join(data))\n",
    "# 将结果字符串作为JSON格式读取到Pandas中\n",
    "recipes = pd.read_json(data_json)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(173278, 17)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recipes.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We see there are nearly 200,000 recipes, and 17 columns.\n",
    "Let's take a look at one row to see what we have:\n",
    "\n",
    "从形状可知有接近20万个菜谱，每个菜谱有17列数据。看看其中的一行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "_id                                {'$oid': '5160756b96cc62079cc2db15'}\n",
       "cookTime                                                          PT30M\n",
       "creator                                                             NaN\n",
       "dateModified                                                        NaN\n",
       "datePublished                                                2013-03-11\n",
       "description           Late Saturday afternoon, after Marlboro Man ha...\n",
       "image                 http://static.thepioneerwoman.com/cooking/file...\n",
       "ingredients           Biscuits\\n3 cups All-purpose Flour\\n2 Tablespo...\n",
       "name                                    Drop Biscuits and Sausage Gravy\n",
       "prepTime                                                          PT10M\n",
       "recipeCategory                                                      NaN\n",
       "recipeInstructions                                                  NaN\n",
       "recipeYield                                                          12\n",
       "source                                                  thepioneerwoman\n",
       "totalTime                                                           NaN\n",
       "ts                                             {'$date': 1365276011104}\n",
       "url                   http://thepioneerwoman.com/cooking/2013/03/dro...\n",
       "Name: 0, dtype: object"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recipes.iloc[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> There is a lot of information there, but much of it is in a very messy form, as is typical of data scraped from the Web.\n",
    "In particular, the ingredient list is in string format; we're going to have to carefully extract the information we're interested in.\n",
    "Let's start by taking a closer look at the ingredients:\n",
    "\n",
    "结果中有很多的数据，但大多数列的数据都是混乱不堪的，正如所有从网络中爬取的数据一样。特别注意到，配方列表是一个字符串的格式，因此我们需要特别小心的在我们感兴趣的列中进行数据提取操作。让我们大致看一些配方列的基本情况："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    173278.000000\n",
       "mean        244.617926\n",
       "std         146.705285\n",
       "min           0.000000\n",
       "25%         147.000000\n",
       "50%         221.000000\n",
       "75%         314.000000\n",
       "max        9067.000000\n",
       "Name: ingredients, dtype: float64"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recipes.ingredients.str.len().describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> The ingredient lists average 250 characters long, with a minimum of 0 and a maximum of nearly 10,000 characters!\n",
    "\n",
    "配方的列表平均有250个字符长，最短的是0个字符，而最长的能达到接近10000个字符。\n",
    "\n",
    "> Just out of curiousity, let's see which recipe has the longest ingredient list:\n",
    "\n",
    "为了满足一下好奇心，让我们看看那个菜谱有着最长的配方列表：\n",
    "\n",
    "译者注：在Series上使用`argmax()`方法或者`np.argmax(series)`函数会出现下面的警告，原因后续Pandas版本会修改它们的行为，目前版本这两种方法返回的是最大值所在的行，后续会修改为返回行号的最大值。建议使用`Series.idxmax()`方法替代。译者后续增加了使用`idxmax()`方法的版本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/wangy/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py:56: FutureWarning: \n",
      "The current behaviour of 'Series.argmax' is deprecated, use 'idxmax'\n",
      "instead.\n",
      "The behavior of 'argmax' will be corrected to return the positional\n",
      "maximum in the future. For now, use 'series.values.argmax' or\n",
      "'np.argmax(np.array(values))' to get the position of the maximum\n",
      "row.\n",
      "  return getattr(obj, method)(*args, **kwds)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Carrot Pineapple Spice &amp; Brownie Layer Cake with Whipped Cream &amp; Cream Cheese Frosting and Marzipan Carrots'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recipes.name[np.argmax(recipes.ingredients.str.len())]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Carrot Pineapple Spice &amp; Brownie Layer Cake with Whipped Cream &amp; Cream Cheese Frosting and Marzipan Carrots'"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recipes.name[recipes.ingredients.str.len().idxmax()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> That certainly looks like an involved recipe.\n",
    "\n",
    "这个菜谱看起来就很复杂的样子。\n",
    "\n",
    "> We can do other aggregate explorations; for example, let's see how many of the recipes are for breakfast food:\n",
    "\n",
    "我们可以继续研究一下这个数据集，例如，看看有多少种早餐的菜谱："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3524"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recipes.description.str.contains('[Bb]reakfast').sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Or how many of the recipes list cinnamon as an ingredient:\n",
    "\n",
    "或者多少中菜谱中用到了肉桂做原料："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10526"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recipes.ingredients.str.contains('[Cc]innamon').sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We could even look to see whether any recipes misspell the ingredient as \"cinamon\":\n",
    "\n",
    "我们甚至可以找到有多少种菜谱将肉桂拼写成了“肉挂”（cinamon)："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recipes.ingredients.str.contains('[Cc]inamon').sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> This is the type of essential data exploration that is possible with Pandas string tools.\n",
    "It is data munging like this that Python really excels at.\n",
    "\n",
    "这些类型的基础数据分析工作，都可以通过Pandas的字符串工具进行并获得结果。这正是Python在数据科学领域优于其他语言的地方。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A simple recipe recommender\n",
    "\n",
    "### 一个简单的菜单推荐器\n",
    "\n",
    "> Let's go a bit further, and start working on a simple recipe recommendation system: given a list of ingredients, find a recipe that uses all those ingredients.\n",
    "While conceptually straightforward, the task is complicated by the heterogeneity of the data: there is no easy operation, for example, to extract a clean list of ingredients from each row.\n",
    "So we will cheat a bit: we'll start with a list of common ingredients, and simply search to see whether they are in each recipe's ingredient list.\n",
    "For simplicity, let's just stick with herbs and spices for the time being:\n",
    "\n",
    "我们再深入的研究一点，来试试实现一个简单的菜谱推荐系统：给定一系列的原材料组成的配方，找到应用了所有原料的菜谱。虽然看起来很容易，实际上这个任务的难点在于数据的异构性：即无法找到一个简单的操作，能从每一行中提取出干净的原料列表。因此我们来做个弊：我们构建一个很通用的原料列表，然后在每个菜谱的配方中搜索它们是否存在。为简单起见，首先从草药和香料开始："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "spice_list = ['salt', 'pepper', 'oregano', 'sage', 'parsley',\n",
    "              'rosemary', 'tarragon', 'thyme', 'paprika', 'cumin']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We can then build a Boolean ``DataFrame`` consisting of True and False values, indicating whether this ingredient appears in the list:\n",
    "\n",
    "然后就能创建一个布尔`DataFrame`，显示上述列表的原料是否在每个菜谱中存在："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salt</th>\n",
       "      <th>pepper</th>\n",
       "      <th>oregano</th>\n",
       "      <th>sage</th>\n",
       "      <th>parsley</th>\n",
       "      <th>rosemary</th>\n",
       "      <th>tarragon</th>\n",
       "      <th>thyme</th>\n",
       "      <th>paprika</th>\n",
       "      <th>cumin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    salt  pepper  oregano   sage  parsley  rosemary  tarragon  thyme  paprika  \\\n",
       "0  False   False    False   True    False     False     False  False    False   \n",
       "1  False   False    False  False    False     False     False  False    False   \n",
       "2   True    True    False  False    False     False     False  False    False   \n",
       "3  False   False    False  False    False     False     False  False    False   \n",
       "4  False   False    False  False    False     False     False  False    False   \n",
       "\n",
       "   cumin  \n",
       "0  False  \n",
       "1  False  \n",
       "2   True  \n",
       "3  False  \n",
       "4  False  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import re\n",
    "spice_df = pd.DataFrame(dict((spice, recipes.ingredients.str.contains(spice, re.IGNORECASE))\n",
    "                             for spice in spice_list))\n",
    "spice_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Now, as an example, let's say we'd like to find a recipe that uses parsley, paprika, and tarragon.\n",
    "We can compute this very quickly using the ``query()`` method of ``DataFrame``s, discussed in [High-Performance Pandas: ``eval()`` and ``query()``](03.12-Performance-Eval-and-Query.ipynb):\n",
    "\n",
    "有了这个布尔类型的数据集，比方说如果我们想要找到一个配方使用了香菜、辣椒和龙蒿，那么我们就可以很方便的使用`DataFrame`的`query()`方法来找到这些配方（`query`方法详见[高性能Pandas: eval() 和 query()](03.12-Performance-Eval-and-Query.ipynb)）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "selection = spice_df.query('parsley & paprika & tarragon')\n",
    "len(selection)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We find only 10 recipes with this combination; let's use the index returned by this selection to discover the names of the recipes that have this combination:\n",
    "\n",
    "这种原料组合的菜谱只有10种；我们可以用结果的行索引将这些菜谱的名称筛选出来："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2069      All cremat with a Little Gem, dandelion and wa...\n",
       "74964                         Lobster with Thermidor butter\n",
       "93768      Burton's Southern Fried Chicken with White Gravy\n",
       "113926                     Mijo's Slow Cooker Shredded Beef\n",
       "137686                     Asparagus Soup with Poached Eggs\n",
       "140530                                 Fried Oyster Po’boys\n",
       "158475                Lamb shank tagine with herb tabbouleh\n",
       "158486                 Southern fried chicken in buttermilk\n",
       "163175            Fried Chicken Sliders with Pickles + Slaw\n",
       "165243                        Bar Tartine Cauliflower Salad\n",
       "Name: name, dtype: object"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recipes.name[selection.index]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Now that we have narrowed down our recipe selection by a factor of almost 20,000, we are in a position to make a more informed decision about what we'd like to cook for dinner.\n",
    "\n",
    "现在我们就已经将可选择的菜谱范围缩小了将近两万倍，我们可以很容易的对晚餐的菜谱做出最后的选择。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Going further with recipes\n",
    "\n",
    "### 更加深入的分析菜谱数据库\n",
    "\n",
    "> Hopefully this example has given you a bit of a flavor (ba-dum!) for the types of data cleaning operations that are efficiently enabled by Pandas string methods.\n",
    "Of course, building a very robust recipe recommendation system would require a *lot* more work!\n",
    "Extracting full ingredient lists from each recipe would be an important piece of the task; unfortunately, the wide variety of formats used makes this a relatively time-consuming process.\n",
    "This points to the truism that in data science, cleaning and munging of real-world data often comprises the majority of the work, and Pandas provides the tools that can help you do this efficiently.\n",
    "\n",
    "希望上面的推荐器例子能够让你认识到了使用Pandas的字符串方法我们可以对数据进行异常方便的清洗操作。当然如果希望构建一个成熟的菜谱推荐系统的话，需要比上例*复杂的多*的技巧和工程。将每个菜谱中的原料配方提取出来变成一个列表会是其中很重要的一环；不幸的是，因为数据格式的多样性，这项任务会相对很耗时。这阐述了数据科学中的一个事实，那就是清洗和预处理真实世界的数据是这个领域非常主要的工作之一，Pandas提供了一些工具能帮助你很有效率的完成它。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "< [数据透视表](03.09-Pivot-Tables.ipynb) | [目录](Index.ipynb) | [在时间序列上操作](03.11-Working-with-Time-Series.ipynb) >\n",
    "\n",
    "<a href=\"https://colab.research.google.com/github/wangyingsm/Python-Data-Science-Handbook/blob/master/notebooks/03.10-Working-With-Strings.ipynb\"><img align=\"left\" src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\" title=\"Open and Execute in Google Colaboratory\"></a>\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}