01_constructors.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to DataFrames\n",
    "**[Bogumił Kamiński](http://bogumilkaminski.pl/about/), April 27, 2019**\n",
    "\n",
    "Let's get started by loading the `DataFrames` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "using DataFrames, Random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constructors and conversion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Constructors\n",
    "\n",
    "In this section, you'll see many ways to create a `DataFrame` using the `DataFrame()` constructor.\n",
    "\n",
    "First, we could create an empty DataFrame,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th></tr><tr><th></th></tr></thead><tbody><p>0 rows × 0 columns</p></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|}\n",
       "\t& \\\\\n",
       "\t\\hline\n",
       "\t& \\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "0×0 DataFrame\n"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame() # empty DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we could call the constructor using keyword arguments to add columns to the `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Float64</th><th>String</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>0.211587</td><td>LdB</td></tr><tr><th>2</th><td>2</td><td>0.880327</td><td>yXA</td></tr><tr><th>3</th><td>3</td><td>0.127785</td><td>LJY</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Float64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 0.211587 & LdB \\\\\n",
       "\t2 & 2 & 0.880327 & yXA \\\\\n",
       "\t3 & 3 & 0.127785 & LJY \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×3 DataFrame\n",
       "│ Row │ A     │ B        │ C      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼──────────┼────────┤\n",
       "│ 1   │ 1     │ 0.211587 │ LdB    │\n",
       "│ 2   │ 2     │ 0.880327 │ yXA    │\n",
       "│ 3   │ 3     │ 0.127785 │ LJY    │"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(A=1:3, B=rand(3), C=randstring.([3,3,3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create a `DataFrame` from a dictionary, in which case keys from the dictionary will be sorted to create the `DataFrame` columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Bool</th><th>Char</th></tr></thead><tbody><p>2 rows × 3 columns</p><tr><th>1</th><td>1</td><td>true</td><td>'a'</td></tr><tr><th>2</th><td>2</td><td>false</td><td>'b'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Bool & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & true & 'a' \\\\\n",
       "\t2 & 2 & false & 'b' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×3 DataFrame\n",
       "│ Row │ A     │ B     │ C    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mBool\u001b[39m  │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼───────┼──────┤\n",
       "│ 1   │ 1     │ true  │ 'a'  │\n",
       "│ 2   │ 2     │ false │ 'b'  │"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = Dict(\"A\" => [1,2], \"B\" => [true, false], \"C\" => ['a', 'b'])\n",
    "DataFrame(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rather than explicitly creating a dictionary first, as above, we could pass `DataFrame` arguments with the syntax of dictionary key-value pairs. \n",
    "\n",
    "Note that in this case, we use symbols to denote the column names and arguments are not sorted. For example, `:A`, the symbol, produces `A`, the name of the first column here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Bool</th><th>Char</th></tr></thead><tbody><p>2 rows × 3 columns</p><tr><th>1</th><td>1</td><td>true</td><td>'a'</td></tr><tr><th>2</th><td>2</td><td>false</td><td>'b'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Bool & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & true & 'a' \\\\\n",
       "\t2 & 2 & false & 'b' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×3 DataFrame\n",
       "│ Row │ A     │ B     │ C    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mBool\u001b[39m  │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼───────┼──────┤\n",
       "│ 1   │ 1     │ true  │ 'a'  │\n",
       "│ 2   │ 2     │ false │ 'b'  │"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(:A => [1,2], :B => [true, false], :C => ['a', 'b'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a `DataFrame` from a vector of vectors, and each vector becomes a column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th></tr><tr><th></th><th>Float64</th><th>Float64</th><th>Float64</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>0.139237</td><td>0.37297</td><td>0.79358</td></tr><tr><th>2</th><td>0.612363</td><td>0.0338258</td><td>0.918752</td></tr><tr><th>3</th><td>0.574598</td><td>0.693267</td><td>0.301719</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x2 & x3\\\\\n",
       "\t\\hline\n",
       "\t& Float64 & Float64 & Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 0.139237 & 0.37297 & 0.79358 \\\\\n",
       "\t2 & 0.612363 & 0.0338258 & 0.918752 \\\\\n",
       "\t3 & 0.574598 & 0.693267 & 0.301719 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×3 DataFrame\n",
       "│ Row │ x1       │ x2        │ x3       │\n",
       "│     │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m   │ \u001b[90mFloat64\u001b[39m  │\n",
       "├─────┼──────────┼───────────┼──────────┤\n",
       "│ 1   │ 0.139237 │ 0.37297   │ 0.79358  │\n",
       "│ 2   │ 0.612363 │ 0.0338258 │ 0.918752 │\n",
       "│ 3   │ 0.574598 │ 0.693267  │ 0.301719 │"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([rand(3) for i in 1:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is not allowed to pass a vector of scalars to `DataFrame` constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "ArgumentError",
     "evalue": "ArgumentError: 'Array{Float64,1}' iterates 'Float64' values, which don't satisfy the Tables.jl Row-iterator interface",
     "output_type": "error",
     "traceback": [
      "ArgumentError: 'Array{Float64,1}' iterates 'Float64' values, which don't satisfy the Tables.jl Row-iterator interface",
      "",
      "Stacktrace:",
      " [1] invalidtable(::Array{Float64,1}, ::Float64) at C:\\Users\\bogum\\.julia\\packages\\Tables\\P0aUn\\src\\iteratorwrapper.jl:35",
      " [2] iterate at C:\\Users\\bogum\\.julia\\packages\\Tables\\P0aUn\\src\\iteratorwrapper.jl:41 [inlined]",
      " [3] buildcolumns(::Nothing, ::Tables.IteratorWrapper{Array{Float64,1}}) at C:\\Users\\bogum\\.julia\\packages\\Tables\\P0aUn\\src\\fallbacks.jl:123",
      " [4] #DataFrame#364(::Bool, ::Type, ::Array{Float64,1}) at C:\\Users\\bogum\\.julia\\packages\\Tables\\P0aUn\\src\\fallbacks.jl:154",
      " [5] DataFrame(::Array{Float64,1}) at C:\\Users\\bogum\\.julia\\packages\\DataFrames\\Wvy9E\\src\\other\\tables.jl:20",
      " [6] top-level scope at In[7]:1"
     ]
    }
   ],
   "source": [
    "DataFrame(rand(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead use a transposed vector if you have a vector of atoms (in this way you effectively pass a two dimensional array to the constructor which is supported)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>1 rows × 3 columns</p><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x2 & x3\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "1×3 DataFrame\n",
       "│ Row │ x1    │ x2    │ x3    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 2     │ 3     │"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(permutedims([1, 2, 3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or pass a vector of `NamedTuple`s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>2</td></tr><tr><th>2</th><td>3</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 \\\\\n",
       "\t2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ a     │ b     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 2     │\n",
       "│ 2   │ 3     │ 4     │"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "v = [(a=1, b=2), (a=3, b=4)]\n",
    "DataFrame(v)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pass a second argument to give the columns names in case you pass a vector of vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>4</td><td>7</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 \\\\\n",
       "\t2 & 2 & 5 & 8 \\\\\n",
       "\t3 & 3 & 6 & 9 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×3 DataFrame\n",
       "│ Row │ A     │ B     │ C     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 4     │ 7     │\n",
       "│ 2   │ 2     │ 5     │ 8     │\n",
       "│ 3   │ 3     │ 6     │ 9     │"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([1:3, 4:6, 7:9], [:A, :B, :C])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively you can pass a `NamedTuple` of vectors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>1</td><td>11</td></tr><tr><th>2</th><td>2</td><td>12</td></tr><tr><th>3</th><td>3</td><td>13</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 11 \\\\\n",
       "\t2 & 2 & 12 \\\\\n",
       "\t3 & 3 & 13 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×2 DataFrame\n",
       "│ Row │ a     │ b     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 11    │\n",
       "│ 2   │ 2     │ 12    │\n",
       "│ 3   │ 3     │ 13    │"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n = (a=1:3, b=11:13)\n",
    "DataFrame(n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a `DataFrame` from a matrix,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Float64</th><th>Float64</th><th>Float64</th><th>Float64</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0.883282</td><td>0.0982655</td><td>0.438723</td><td>0.86469</td></tr><tr><th>2</th><td>0.82151</td><td>0.309246</td><td>0.921206</td><td>0.742596</td></tr><tr><th>3</th><td>0.107196</td><td>0.995836</td><td>0.917311</td><td>0.986401</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Float64 & Float64 & Float64 & Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 0.883282 & 0.0982655 & 0.438723 & 0.86469 \\\\\n",
       "\t2 & 0.82151 & 0.309246 & 0.921206 & 0.742596 \\\\\n",
       "\t3 & 0.107196 & 0.995836 & 0.917311 & 0.986401 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×4 DataFrame\n",
       "│ Row │ x1       │ x2        │ x3       │ x4       │\n",
       "│     │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m   │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │\n",
       "├─────┼──────────┼───────────┼──────────┼──────────┤\n",
       "│ 1   │ 0.883282 │ 0.0982655 │ 0.438723 │ 0.86469  │\n",
       "│ 2   │ 0.82151  │ 0.309246  │ 0.921206 │ 0.742596 │\n",
       "│ 3   │ 0.107196 │ 0.995836  │ 0.917311 │ 0.986401 │"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rand(3,4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and here we do the same but also pass column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th>Float64</th><th>Float64</th><th>Float64</th><th>Float64</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0.169398</td><td>0.54164</td><td>0.675899</td><td>0.41232</td></tr><tr><th>2</th><td>0.723424</td><td>0.906382</td><td>0.324795</td><td>0.204721</td></tr><tr><th>3</th><td>0.54897</td><td>0.933663</td><td>0.546234</td><td>0.187317</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Float64 & Float64 & Float64 & Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 0.169398 & 0.54164 & 0.675899 & 0.41232 \\\\\n",
       "\t2 & 0.723424 & 0.906382 & 0.324795 & 0.204721 \\\\\n",
       "\t3 & 0.54897 & 0.933663 & 0.546234 & 0.187317 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×4 DataFrame\n",
       "│ Row │ a        │ b        │ c        │ d        │\n",
       "│     │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │\n",
       "├─────┼──────────┼──────────┼──────────┼──────────┤\n",
       "│ 1   │ 0.169398 │ 0.54164  │ 0.675899 │ 0.41232  │\n",
       "│ 2   │ 0.723424 │ 0.906382 │ 0.324795 │ 0.204721 │\n",
       "│ 3   │ 0.54897  │ 0.933663 │ 0.546234 │ 0.187317 │"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rand(3,4), Symbol.('a':'d'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also construct an uninitialized DataFrame.\n",
    "\n",
    "Here we pass column types, names and number of rows; we get `missing` in column :C because `Any >: Missing`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Float64</th><th>Any</th></tr></thead><tbody><p>1 rows × 3 columns</p><tr><th>1</th><td>97002080</td><td>4.79264e-316</td><td>missing</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Float64 & Any\\\\\n",
       "\t\\hline\n",
       "\t1 & 97002080 & 4.79264e-316 &  \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "1×3 DataFrame\n",
       "│ Row │ A        │ B            │ C       │\n",
       "│     │ \u001b[90mInt64\u001b[39m    │ \u001b[90mFloat64\u001b[39m      │ \u001b[90mAny\u001b[39m     │\n",
       "├─────┼──────────┼──────────────┼─────────┤\n",
       "│ 1   │ 97002080 │ 4.79264e-316 │ \u001b[90mmissing\u001b[39m │"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([Int, Float64, Any], [:A, :B, :C], 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a `DataFrame` where `:C` is `#undef`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Float64</th><th>String</th></tr></thead><tbody><p>1 rows × 3 columns</p><tr><th>1</th><td>4294967296</td><td>5.60914e-316</td><td>#undef</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Float64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 4294967296 & 5.60914e-316 & \\#undef \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "1×3 DataFrame\n",
       "│ Row │ A          │ B            │ C      │\n",
       "│     │ \u001b[90mInt64\u001b[39m      │ \u001b[90mFloat64\u001b[39m      │ \u001b[90mString\u001b[39m │\n",
       "├─────┼────────────┼──────────────┼────────┤\n",
       "│ 1   │ 4294967296 │ 5.60914e-316 │ #undef │"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([Int, Float64, String], [:A, :B, :C], 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To initialize a `DataFrame` with column names, but no rows use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Float64</th><th>String</th></tr></thead><tbody><p>0 rows × 3 columns</p></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Float64 & String\\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "0×3 DataFrame\n"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([Int, Float64, String], [:A, :B, :C])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Float64</th><th>String</th></tr></thead><tbody><p>0 rows × 3 columns</p></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Float64 & String\\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "0×3 DataFrame\n"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(A=Int[], B=Float64[], C=String[])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can create a `DataFrame` by copying an existing `DataFrame`.\n",
    "\n",
    "Note that `copy` also copies the vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true, true, false)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(a=1:2, b='a':'b')\n",
    "y = copy(x)\n",
    "(x === y), isequal(x, y), (x.a == y.a), (x.a === y.a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling `DataFrame` on a `DataFrame` object works like `copy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true, true, false)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(a=1:2, b='a':'b')\n",
    "y = DataFrame(x)\n",
    "(x === y), isequal(x, y), (x.a == y.a), (x.a === y.a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can avoid copying of columns of a data frame by passing `copycols=false` keyword argument or using `DataFrame!` constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true, true, true)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(a=1:2, b='a':'b')\n",
    "y = DataFrame(x, copycols=false)\n",
    "(x === y), isequal(x, y), (x.a == y.a), (x.a === y.a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true, true, true)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(a=1:2, b='a':'b')\n",
    "y = DataFrame!(x)\n",
    "(x === y), isequal(x, y), (x.a == y.a), (x.a === y.a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same rule applies to other constructors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = [1, 2, 3]\n",
    "df1 = DataFrame(a=a)\n",
    "df2 = DataFrame(a=a, copycols=false)\n",
    "df1.a === a, df2.a === a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can create a similar uninitialized `DataFrame` based on an original one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>21725999824</td><td>'\\x00\\x00\\x00\\x01'</td></tr><tr><th>2</th><td>129279664</td><td>'\\0'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 21725999824 & '\\textbackslash{}x00\\textbackslash{}x00\\textbackslash{}x00\\textbackslash{}x01' \\\\\n",
       "\t2 & 129279664 & '\\textbackslash{}0' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ a           │ b                  │\n",
       "│     │ \u001b[90mInt64\u001b[39m       │ \u001b[90mChar\u001b[39m               │\n",
       "├─────┼─────────────┼────────────────────┤\n",
       "│ 1   │ 21725999824 │ '\\x00\\x00\\x00\\x01' │\n",
       "│ 2   │ 129279664   │ '\\0'               │"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>0 rows × 2 columns</p></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "0×2 DataFrame\n"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar(x, 0) # number of rows in a new DataFrame passed as a second argument"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also create a new `DataFrame` from `SubDataFrame` or `DataFrameRow` (discussed in detail later in the tutorial)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>'a'</td></tr><tr><th>2</th><td>1</td><td>'a'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 'a' \\\\\n",
       "\t2 & 1 & 'a' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 SubDataFrame\n",
       "│ Row │ a     │ b    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┤\n",
       "│ 1   │ 1     │ 'a'  │\n",
       "│ 2   │ 1     │ 'a'  │"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sdf = view(x, [1,1], :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SubDataFrame{DataFrame,DataFrames.Index,Array{Int64,1}}"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "typeof(sdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>'a'</td></tr><tr><th>2</th><td>1</td><td>'a'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 'a' \\\\\n",
       "\t2 & 1 & 'a' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ a     │ b    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┤\n",
       "│ 1   │ 1     │ 'a'  │\n",
       "│ 2   │ 1     │ 'a'  │"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(sdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p>DataFrameRow</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>1 rows × 2 columns</p><tr><th>1</th><td>1</td><td>'a'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 'a' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "DataFrameRow\n",
       "│ Row │ a     │ b    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┤\n",
       "│ 1   │ 1     │ 'a'  │"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfr = x[1, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>1 rows × 2 columns</p><tr><th>1</th><td>1</td><td>'a'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 'a' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "1×2 DataFrame\n",
       "│ Row │ a     │ b    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┤\n",
       "│ 1   │ 1     │ 'a'  │"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(dfr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conversion to a matrix\n",
    "\n",
    "Let's start by creating a `DataFrame` with two rows and two columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[\"A\", \"B\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create a matrix by passing this `DataFrame` to `Matrix`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Any,2}:\n",
       " 1  \"A\"\n",
       " 2  \"B\""
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This would work even if the `DataFrame` had some `missing`s:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String⍰</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>missing</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String⍰\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 &  \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y       │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString⍰\u001b[39m │\n",
       "├─────┼───────┼─────────┤\n",
       "│ 1   │ 1     │ \u001b[90mmissing\u001b[39m │\n",
       "│ 2   │ 2     │ B       │"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[missing,\"B\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Any,2}:\n",
       " 1  missing\n",
       " 2  \"B\"    "
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the two previous matrix examples, Julia created matrices with elements of type `Any`. We can see more clearly that the type of matrix is inferred when we pass, for example, a `DataFrame` of integers to `Matrix`, creating a 2D `Array` of `Int64`s:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>3</td></tr><tr><th>2</th><td>2</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 3 \\\\\n",
       "\t2 & 2 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 3     │\n",
       "│ 2   │ 2     │ 4     │"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=3:4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Int64,2}:\n",
       " 1  3\n",
       " 2  4"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this next example, Julia correctly identifies that `Union` is needed to express the type of the resulting `Matrix` (which contains `missing`s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>Int64⍰</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>missing</td></tr><tr><th>2</th><td>2</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64⍰\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 &  \\\\\n",
       "\t2 & 2 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y       │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64⍰\u001b[39m  │\n",
       "├─────┼───────┼─────────┤\n",
       "│ 1   │ 1     │ \u001b[90mmissing\u001b[39m │\n",
       "│ 2   │ 2     │ 4       │"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[missing,4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Union{Missing, Int64},2}:\n",
       " 1   missing\n",
       " 2  4       "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we can't force a conversion of `missing` values to `Int`s!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "ename": "ErrorException",
     "evalue": "cannot convert a DataFrame containing missing values to Matrix{Int64} (found for column y)",
     "output_type": "error",
     "traceback": [
      "cannot convert a DataFrame containing missing values to Matrix{Int64} (found for column y)",
      "",
      "Stacktrace:",
      " [1] error(::String) at .\\error.jl:33",
      " [2] convert(::Type{Array{Int64,2}}, ::DataFrame) at C:\\Users\\bogum\\.julia\\packages\\DataFrames\\Wvy9E\\src\\abstractdataframe\\abstractdataframe.jl:828",
      " [3] Array{Int64,2}(::DataFrame) at C:\\Users\\bogum\\.julia\\packages\\DataFrames\\Wvy9E\\src\\abstractdataframe\\abstractdataframe.jl:838",
      " [4] top-level scope at In[38]:1"
     ]
    }
   ],
   "source": [
    "Matrix{Int}(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conversion to `NamedTuple` related tabular structures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[\"A\", \"B\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Tables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we convert a `DataFrame` into a `NamedTuple` of vectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(x = [1, 2], y = [\"A\", \"B\"])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ct = Tables.columntable(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we convert it into a vector of `NamedTuples`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Array{NamedTuple{(:x, :y),Tuple{Int64,String}},1}:\n",
       " (x = 1, y = \"A\")\n",
       " (x = 2, y = \"B\")"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rt = Tables.rowtable(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can perform the conversions back to a `DataFrame` using a standard constructor call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(ct)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Handling of duplicate column names\n",
    "\n",
    "We can pass the `makeunique` keyword argument to allow passing duplicate names (they get deduplicated)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>a_2</th><th>a_1</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>1 rows × 3 columns</p><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& a & a\\_2 & a\\_1\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "1×3 DataFrame\n",
       "│ Row │ a     │ a_2   │ a_1   │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 2     │ 3     │"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(:a=>1, :a=>2, :a_1=>3; makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Otherwise, duplicates are not allowed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "ename": "ArgumentError",
     "evalue": "ArgumentError: Duplicate variable names: :a. Pass makeunique=true to make them unique using a suffix automatically.",
     "output_type": "error",
     "traceback": [
      "ArgumentError: Duplicate variable names: :a. Pass makeunique=true to make them unique using a suffix automatically.",
      "",
      "Stacktrace:",
      " [1] #make_unique!#1(::Bool, ::Function, ::Array{Symbol,1}, ::Array{Symbol,1}) at C:\\Users\\bogum\\.julia\\packages\\DataFrames\\Wvy9E\\src\\other\\utils.jl:22",
      " [2] #make_unique! at .\\none:0 [inlined]",
      " [3] #make_unique#2 at C:\\Users\\bogum\\.julia\\packages\\DataFrames\\Wvy9E\\src\\other\\utils.jl:44 [inlined]",
      " [4] #make_unique at .\\none:0 [inlined]",
      " [5] #Index#3 at C:\\Users\\bogum\\.julia\\packages\\DataFrames\\Wvy9E\\src\\other\\index.jl:14 [inlined]",
      " [6] (::getfield(Core, Symbol(\"#kw#Type\")))(::NamedTuple{(:makeunique,),Tuple{Bool}}, ::Type{DataFrames.Index}, ::Array{Symbol,1}) at .\\none:0",
      " [7] #DataFrame#89(::Bool, ::Bool, ::Type, ::Pair{Symbol,Int64}, ::Vararg{Pair{Symbol,Int64},N} where N) at C:\\Users\\bogum\\.julia\\packages\\DataFrames\\Wvy9E\\src\\dataframe\\dataframe.jl:141",
      " [8] DataFrame(::Pair{Symbol,Int64}, ::Vararg{Pair{Symbol,Int64},N} where N) at C:\\Users\\bogum\\.julia\\packages\\DataFrames\\Wvy9E\\src\\dataframe\\dataframe.jl:139",
      " [9] top-level scope at In[46]:1"
     ]
    }
   ],
   "source": [
    "df = DataFrame(:a=>1, :a=>2, :a_1=>3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finallly observe that `nothing` is not printed when displaying a `DataFrame`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Union…</th><th>Union…</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td></td></tr><tr><th>2</th><td></td><td>a</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Union… & Union…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 &  \\\\\n",
       "\t2 &  & a \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x      │ y      │\n",
       "│     │ \u001b[90mUnion…\u001b[39m │ \u001b[90mUnion…\u001b[39m │\n",
       "├─────┼────────┼────────┤\n",
       "│ 1   │ 1      │        │\n",
       "│ 2   │        │ a      │"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(x=[1, nothing], y=[nothing, \"a\"])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.1.0",
   "language": "julia",
   "name": "julia-1.1"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.1.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}