20. Matrices with NumPy and SciPy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*********************************************************************************************************\n",
    "# A Tour of Python 3  \n",
    "version 1.0.1  \n",
    "Authors: Phil Pfeiffer, Zack Bunch, and Feyisayo Oyeniyi  \n",
    "East Tennessee State University  \n",
    "Last updated June 2021  \n",
    "\n",
    "Chapter 20: author Katie Wilson; ed. Phil Pfeiffer  \n",
    "*********************************************************************************************************"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  20. Matrices  \n",
    " 20.1 [Prerequisites](#Matrices-Prerequisites)  \n",
    " &ensp; 20.1.1 [Background](#Matrices-Background)  \n",
    " &ensp; 20.1.2 [Libraries](#Matrices-Libraries)  \n",
    " &ensp;&ensp; 20.1.2.1 [Installation via Command Prompt](#Matrices-Installation-via-Command-Prompt)  \n",
    " &ensp;&ensp; 20.1.2.2 [Installation in Jupyter](#Matrices-Installation-in-Jupyter)  \n",
    " 20.2 [Matrices as NumPy Arrays](#Matrices-Matrices-as-NumPy-Arrays)  \n",
    " 20.3 [Matrix Operations](#Matrices-Matrix-Operations)  \n",
    " &ensp; 20.3.1 [Transpose](#Matrices-Transpose)  \n",
    " &ensp; 20.3.2 [Addition and Subtraction](#Matrices-Addition-and-Subtraction)  \n",
    " &ensp; 20.3.3 [Multiplication](#Matrices-Multiplication)  \n",
    " &ensp;&ensp; 20.3.3.1 [Scalar Multiplication](#Matrices-Scalar-Multiplication)  \n",
    " &ensp;&ensp; 20.3.3.2 [Matrix Multiplication](#Matrices-Matrix-Multiplication)  \n",
    " &ensp; 20.3.4 [Inverse](#Matrices-Inverse)  \n",
    " 20.4 [Types of Matrices](#Matrices-Types-of-Matrices)  \n",
    " &ensp; 20.4.1 [Square](#Matrices-Square)  \n",
    " &ensp; 20.4.2 [Diagonal](#Matrices-Diagonal)  \n",
    " &ensp; 20.4.3 [Identity](#Matrices-Identity)  \n",
    " &ensp; 20.4.4 [Triangular](#Matrices-Triangular)  \n",
    " &ensp; 20.4.5 [Orthogonal](#Matrices-Orthogonal)  \n",
    " &ensp; 20.4.6 [Unitary](#Matrices-Unitary)  \n",
    " 20.5 [Additional References](#Matrices-Additional-References)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 20.1 Prerequisites <a name='Matrices-Prerequisites'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.1.1 Background <a name='Matrices-Background'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This unit assumes a basic familiarity with *linear algebra*, a branch of mathematics that uses arrays and matrices to model and solve systems of linear equations. ",
    "*Arrays* are collections of values, typically arranged into rows and columns. ",
    "*Matrices* are rectangular, two-dimensional arrays that support three basic operations: addition, scalar multiplication, and matrix multiplication.\n",
    "\n",
    "Matrices are used in various fields, including mathematics, physics, and computer science, to answer questions about phenomena of interest. For example,\n",
    "- A matrix may represent a way of allocating a set of resources to different projects. ",
    "  Using matrices and their operations, a company could determine an optimal strategy for allocating these resources.\n",
    "- A digital image is often represented as a matrix, ",
    "  where the value in each row and column of the matrix represents the color of a pixel in an image's corresponding row and column. ",
    "  Decoding digital video, a series of digital images, uses matrix multiplication.\n",
    "- Graphs can be represented as matrices whose elements characterize that graph's edges: ",
    "  i.e., the connections between its nodes. ",
    "  The simplest such representations use a 1 at element (i,j) to represent an edge from node i to node j and 0 otherwise. ",
    "  More complex matrices use a range of numbers to characterize some aspect of that connection: e.g., its value, its cost, or the connection's strength.\n",
    "\n",
    "This unit focuses on manipulating matrices whose rows and columns are indexed by nonnegative integers and exploring different common types of matrices."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.1.2 Libraries <a name='Matrices-Libraries'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NumPy** and **SciPy** are third-party Python libraries that support various mathematical operations, including standard operations on matrices. ",
    "These libraries are not included in a standard Python installation: they must be installed separately to run this notebook's exercises. ",
    "Both libraries only need to be installed once; if you already have installed these previously, then this section can be ignored.\n",
    "\n",
    "NumPy and SciPy can be installed from a command prompt or from inside Jupyter itself. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 20.1.2.1 Installation via Command Prompt <a name='Matrices-Installation-via-Command-Prompt'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Go to the Windows Start menu and open a cmd.exe window. ",
    "At the command prompt, enter the following commands:\n",
    "- `python -m pip install --upgrade pip` &nbsp; to ensure pip is up to date with the current version.\n",
    "- `python -m pip install numpy` &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; to install NumPy.\n",
    "- `python -m pip install scipy` &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; to install SciPy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 20.1.2.2 Installation in Jupyter <a name='Matrices-Installation-in-Jupyter'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From within Jupyter, execute the following code cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.1.2.2.a Installing numpy and scipy from Jupyter\n",
    "# Credit to Jake at https://jakevdp.github.io/blog/2017/12/05/installing-python-packages-from-jupyter/\n",
    "\n",
    "import sys\n",
    "\n",
    "!{sys.executable} -m pip install --upgrade pip\n",
    "!{sys.executable} -m pip install numpy\n",
    "!{sys.executable} -m pip install scipy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notes:\n",
    "- The \"!\" symbol before the command passes the command to the shell to execute.\n",
    "- `sys.executable` returns the absolute path of the Python interpreter's executable binary. \n",
    "  This ensures you are installing the libraries for the same version of Python you are using to run the notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 20.2 Matrices as NumPy Arrays <a name='Matrices-Matrices-as-NumPy-Arrays'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy is often used to define two-dimensional arrays to use as matrices. ",
    "NumPy array objects are smaller than comparable Python list-based representations of arrays. ",
    "NumPy also provides array operations that are faster than comparable Python operations on lists, ",
    "along with optimized functions for mathematics that require NumPy arrays as their inputs.\n",
    "\n",
    "For more information about NumPy arrays, see the [official NumPy documentation](https://numpy.org/doc/stable/reference/generated/numpy.array.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy's `array` method creates arrays like the two-dimensional one shown below. These arrays will be used throughout this unit to represent matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.2.a Using numpy's array() method to define a two dimensional matrix\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "A = np.array([[1,2,3],  # Define a matrix, \n",
    "              [4,5,6],  # where each row in the matrix is defined using a Python list\n",
    "              [7,8,9]])\n",
    "\n",
    "A # Display the resulting matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The array shown above is a *square* array: one that has the same number of rows and columns. `array` can also create non-square arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.2.b Showing that arrays can be non-square\n",
    "import numpy as np\n",
    "\n",
    "B = np.array([[1,2,3,4,5],  # Define a matrix with 2 rows and 5 columns\n",
    "              [6,7,8,9,0]])\n",
    "\n",
    "B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style='color:blue'>",
    "&#128073;&ensp;&ensp;**Exercise 20.2.1:**\n\n",
    "</span>",
    "<span style='color:navy' >",
    "In the following code cell, ",
    "explore what happens when you attempt to use NumPy to create a *ragged* array: i.e., one with rows of different lengths.",
    "</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style='color:blue'>",
    "&#128073;&ensp;&ensp;**Exercise 20.2.2:**\n\n",
    "</span>",
    "<span style='color:navy' >",
    "In the following markdown cell, ",
    "explain whether a *ragged* array is appropriate for use as a matrix. Why or why not?",
    "</span>\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 20.3 Matrix Operations <a name='Matrices-Matrix-Operations'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.3.1 Transpose <a name='Matrices-Transpose'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A matrix's *transpose* is a matrix whose rows are the columns of the original. ",
    "For a matrix $A$ –"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$",
    "A = \\begin{bmatrix} a_{11} & a_{12} & ... & a_{1n} \\\\\n",
    "                       a_{21} & a_{22} & ... & a_{2n} \\\\ \n",
    "                       \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "                       a_{m1} & a_{m2} & ... & a_{mn} \\end{bmatrix} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "– $A$'s transpose, denoted $A^T$, is"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$",
    "A^T = \\begin{bmatrix} a_{11} & a_{21} & ... & a_{m1} \\\\\n",
    "                      a_{12} & a_{22} & ... & a_{m2} \\\\ \n",
    "                      \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "                      a_{1n} & a_{2n} & ... & a_{mn} \\end{bmatrix} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A matrix's `T` property returns its transposition: i.e., `A.T` for matrix A. ",
    "Taking the transposition using `T` does not alter the original matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.1.a  Computing a matrix's transpose\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define two matrices\n",
    "A = np.array([[1,2,3],\n",
    "              [4,5,6],\n",
    "              [7,8,9]])\n",
    "\n",
    "print(\"A, before transposition:\", A, sep=\"\\n\")             # The original matrix\n",
    "print(\"A.T:\", A.T, sep=\"\\n\")                               # The transpose\n",
    "print(\"A, after transposition (unchanged):\", A, sep=\"\\n\")  # The original matrix, unchanged"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.3.2 Addition and Subtraction <a name='Matrices-Addition-and-Subtraction'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two matrices are *added* by summing their corresponding elements.",
    "That is, for matrices $A$ and $B$ –"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \n",
    "   A = \\begin{bmatrix} a_{11} & a_{12} & ... & a_{1n} \\\\\n",
    "                       a_{21} & a_{22} & ... & a_{2n} \\\\ \n",
    "                       \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "                       a_{m1} & a_{m2} & ... & a_{mn} \\end{bmatrix} \n",
    "   B = \\begin{bmatrix} b_{11} & b_{12} & ... & b_{1n} \\\\\n",
    "                       b_{21} & b_{22} & ... & b_{2n} \\\\ \n",
    "                       \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "                       b_{m1} & b_{m2} & ... & b_{mn} \\end{bmatrix} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "– the sum of $A$ and $B$, denoted $A+B$, is"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "    A+B = \\begin{bmatrix} a_{11}+b_{11} & a_{12}+b_{12} & ... & a_{1n}+b_{1n} \\\\\n",
    "                          a_{21}+b_{21} & a_{22}+b_{22} & ... & a_{2n}+b_{2n} \\\\ \n",
    "                          \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "                          a_{m1}+b_{m1} & a_{m2}+b_{m2} & ... & a_{mn}+b_{mn} \\end{bmatrix} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Matrices are added using an infix `+` symbol: i.e., `A+B` for matrices A and B."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.2.a Adding two matrices\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define two matrices\n",
    "A = np.array([[1,2,3],\n",
    "              [4,5,6],\n",
    "              [7,8,9]])\n",
    "\n",
    "B = np.array([[9,8,7],\n",
    "              [6,5,4],\n",
    "              [3,2,1]])\n",
    "\n",
    "A+B   # Do the addition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To add two matrices, both must be the same shape, that is, have the same number of rows and columns.\n",
    "\n",
    "A matrix's `shape` property returns a pair *(number of rows, number of columns)* that denotes the matrix's shape."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.2.b Determining a matrix's shape\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define two matrices\n",
    "A = np.array([[1,2,3,4,5],\n",
    "              [6,7,8,9,0]])\n",
    "\n",
    "B = np.array([[9,8,7],\n",
    "              [6,5,4],\n",
    "              [3,2,1]])\n",
    "\n",
    "# Examine their shapes\n",
    "print(\"A.shape =\", A.shape)\n",
    "print(\"A has\", A.shape[0], \"rows and\", A.shape[1], \"columns.\")\n",
    "\n",
    "print(\"B.shape =\", B.shape)\n",
    "print(\"B has\", B.shape[0], \"rows and\", B.shape[1], \"columns.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More generally, the `shape` of an n-dimensional array is given by an n-tuple whose successive elements give the size of that array's successive dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.2.c Showing that matrix addition between matrices of a different size fails\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define two matrices\n",
    "A = np.array([[1,2,3,4,5],\n",
    "              [6,7,8,9,0]])\n",
    "\n",
    "B = np.array([[9,8,7],\n",
    "              [6,5,4],\n",
    "              [3,2,1]])\n",
    "\n",
    "A+B    # Fails!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly to `+`, the `-` operator subtracts a second matrix from a first. The two matrices must have the same shape."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.2.d Subtracting two matrices\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define two matrices\n",
    "A = np.array([[1,2,3],\n",
    "              [4,5,6],\n",
    "              [7,8,9]])\n",
    "\n",
    "B = np.array([[9,8,7],\n",
    "              [6,5,4],\n",
    "              [3,2,1]])\n",
    "\n",
    "A-B    # Do the subtraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.2.e Showing that matrix subtraction between matrices of a different size fails\n",
    "import numpy as np\n",
    "\n",
    "# Define two matrices\n",
    "A = np.array([[1,2,3,4,5],\n",
    "              [6,7,8,9,0]])\n",
    "\n",
    "B = np.array([[9,8,7],\n",
    "              [6,5,4],\n",
    "              [3,2,1]])\n",
    "\n",
    "A-B # Fails!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style='color:blue'>",
    "&#128073;&ensp;&ensp;**Exercise 20.3.2.1:**\n\n",
    "</span>",
    "<span style='color:navy' >",
    "In the following markdown cell, ",
    "discuss whether matrix addition is commutative, that is, A+B=B+A. What about matrix subtraction?",
    "</span>\n",
    "***\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.3.3 Multiplication <a name='Matrices-Multiplication'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 20.3.3.1 Scalar Multiplication <a name='Matrices-Scalar-Multiplication'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Scalar multiplication* multiplies each element of a matrix by a constant.",
    "For a matrix $A$ and a constant $c$ –"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$",
    "A = \\begin{bmatrix} a_{11} & a_{12} & ... & a_{1n} \\\\\n",
    "                       a_{21} & a_{22} & ... & a_{2n} \\\\ \n",
    "                       \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "                       a_{m1} & a_{m2} & ... & a_{mn} \\end{bmatrix} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "– the scalar multiplication of **$A$** by **$c$**, denoted **$c*A$**, is"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ c*A = \\begin{bmatrix} c*a_{11} & c*a_{12} & ... & c*a_{1n} \\\\\n",
    "                         c*a_{21} & c*a_{22} & ... & c*a_{2n} \\\\ \n",
    "                         \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "                         c*a_{m1} & c*a_{m2} & ... & c*a_{mn} \\end{bmatrix} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To multiply a matrix by a scalar, use NumPy's `*` operator: i.e., `c*A` for matrix A and constant c.\n",
    "\n",
    "Scalar multiplication is commutative, that is, `c*A=A*c`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.3.1.a Multiplication of a matrix with a constant\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define a matrix\n",
    "A = np.array([[1,2,3],\n",
    "              [4,5,6],\n",
    "              [7,8,9]])\n",
    "\n",
    "# Define a constant\n",
    "c = 10\n",
    "\n",
    "# Perform scalar multiplication...\n",
    "print(\"c*A =\")\n",
    "print(c*A)\n",
    "print()\n",
    "\n",
    "# ...and show that it's commutative.\n",
    "print(\"A*c =\")\n",
    "print(A*c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 20.3.3.2 Matrix Multiplication <a name='Matrices-Matrix-Multiplication'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two matrices are *multiplied* by computing the dot product of each row i of the first matrix and the corresponding column j of the second matrix. ",
    "This value then becomes the result's (i,j)th value.\n",
    "\n",
    "The *dot product* of two *vectors* (i.e., one-dimensional arrays) $a$ and $b$ is computed by multiplying the vectors' corresponding elements and summing the results.",
    "For one-dimensional arrays $a$ and $b$ –"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "a = \\begin{bmatrix} a_{11} & a_{12} & ... & a_{1n} \\end{bmatrix} \n",
    "b = \\begin{bmatrix} b_{11} & b_{12} & ... & b_{1n} \\end{bmatrix} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "– the dot product of $a$ and $b$, denoted $a \\cdot b$, is"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "a\\cdot b = \\begin{bmatrix} a_{11}b_{11}+a_{12}b_{12}+...+a_{1n}b_{1n} \\end{bmatrix} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Returning to the definition of matrix multiplication, for matrices $A$ and $B$ –"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "A = \\left[ a_{ij} \\right]_{i,j = 1,1}^{m,n}\n",
    "B = \\left[ b_{jk} \\right]_{j,k = 1,1}^{n,r}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "– the product of $A$ and $B$, denoted $AB$ is"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "AB = \\left[ \\sum_{j=1}^n a_{ij}b_{jk} \\right]_{i,k=1,1}^{m,r}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To multiply two matrices, use NumPy's `@` operator: i.e., `A@B` for matrices A and B."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.3.2.a Multiplying two matrices of the same shape\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define two matrices\n",
    "A = np.array([[1,2,3],\n",
    "              [4,5,6],\n",
    "              [7,8,9]])\n",
    "\n",
    "B = np.array([[9,8,7],\n",
    "              [6,5,4],\n",
    "              [3,2,1]])\n",
    "\n",
    "A@B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To multiply two matrices, the number of columns in the first must match the number of rows in the second. ",
    "If the first matrix's shape is (a,b) and the second's is (b,c), the resulting matrix A@B will be of shape (a,c)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.3.2.b Multiplying two matrices of different, but corresponding, shape\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define two matrices\n",
    "A = np.array([[1,2,3],  # 2x3 matrix\n",
    "              [4,5,6]])\n",
    "\n",
    "B = np.array([[9,8],    # 3x2 matrix\n",
    "              [6,5],\n",
    "              [3,2]])\n",
    "\n",
    "A@B  # Do the matrix multiplication - this yields a 2x2 result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.3.2.c Multiplying two matrices of different and not corresponding shape\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define two matrices\n",
    "A = np.array([[1,2],  # 2x2 matrix\n",
    "              [4,5]])\n",
    "\n",
    "B = np.array([[9,8],  # 3x2 matrix\n",
    "              [6,5],\n",
    "              [3,2]])\n",
    "\n",
    "A@B    # Fails! 2x2 <-Not equal!-> 3x2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style='color:blue'>",
    "&#128073;&ensp;&ensp;**Exercise 20.3.3.1:**\n\n",
    "</span>",
    "<span style='color:navy' >",
    "In the following markdown cell, discuss whether matrix multiplication is commutative, that is, A@B=B@A.",
    "</span>\n",
    "***\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy supports three additional methods for multiplying vectors. ",
    "`dot` computes the dot product of two vectors. ",
    "NumPy's `dot` method has been generalized to support matrices, as shown in the following example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.3.2.d Multiplying matrices using dot products with dot()\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "A = np.array([[1,2,3],  # 2x3 matrix\n",
    "              [4,5,6]])\n",
    "\n",
    "B = np.array([[9,8],    # 3x2 matrix\n",
    "              [6,5],\n",
    "              [3,2]])\n",
    "\n",
    "print(A@B)        # Multiplying with @ notation\n",
    "print(A.dot(B))   # Multiplying using dot products; result is the same"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy's `outer` method computes the *outer product* of two vectors $a$ and $b$, defined as $ab^T$. ",
    "Outer products can be used to implement matrix multiplication, as shown by the next example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.3.2.e Using outer() to perform matrix multiplication\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "A = np.array([[1,2,3],  # 2x3 matrix\n",
    "              [4,5,6]])\n",
    "\n",
    "B = np.array([[9,8],    # 3x2 matrix\n",
    "              [6,5],\n",
    "              [3,2]])\n",
    "\n",
    "# Multiply outer products\n",
    "outer = np.zeros((A.shape[0], B.shape[1]))  # Start with an array of all zeroes in the shape of the result\n",
    "for i in range(A.shape[1]):                 # Repeat for the number of columns in A\n",
    "    outer += np.outer(A[:,i], B[i,:])       # Add in the outer product of the ith column of A with the ith row of B\n",
    "\n",
    "print(A@B)   # Multiplying with @ notation\n",
    "print(outer) # Result is the same with outer products, though outer is composed of floats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy's `inner` method computes the *inner product* of two vectors $a$ and $b$, defined as $b^Ta$. ",
    "Inner products can be used to implement matrix multiplication, as shown by the next example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.3.2.f Using inner() to perform matrix multiplication\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "A = np.array([[1,2,3],  # 2x3 matrix\n",
    "              [4,5,6]])\n",
    "\n",
    "B = np.array([[9,8],    # 3x2 matrix\n",
    "              [6,5],\n",
    "              [3,2]])\n",
    "\n",
    "# Multiply inner products\n",
    "inner = np.zeros((A.shape[0], B.shape[1]))   # Start with an array of all zeros in the shape of the result\n",
    "for i in range(A.shape[0]):                  # Repeat for each of the rows in A...\n",
    "    for j in range(B.shape[1]):              # ...and the columns in B\n",
    "        inner[i,j]=np.inner(A[i,:], B[:,j])  # Set the (i,j) coordinate of the result to the inner product of \n",
    "                                             # the ith row of A with the jth column of B\n",
    "\n",
    "print(A@B)    # Multiplying with @ notation\n",
    "print(inner)  # Result is the same with inner products, though inner is composed of floats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.3.4 Inverse <a name='Matrices-Inverse'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Matrices don't support a division operator per se. ",
    " However, multiplying by the *inverse* of a matrix-- when one exists-- achieves a similar effect.\n",
    "\n",
    "The *inverse* of a matrix $A$, $A^{-1}$, is the matrix such that $AA^{-1}=I$, ",
    "where $I$ is the *identity* matrix:  a matrix with 1's along the diagonal, and 0's everywhere else."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SciPy's `linalg` module with the `inv` command can be used to find a matrix's inverse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.4.a Finding the inverse for a matrix, and noticing that A@Ainv != I...?\n",
    "\n",
    "import numpy as np\n",
    "from scipy import linalg\n",
    "\n",
    "# Define a matrix\n",
    "A = np.array([[1,2],\n",
    "              [3,4]])\n",
    "\n",
    "Ainv = linalg.inv(A) # Calculate A's inverse\n",
    "print(\"A's inverse:\")\n",
    "print(Ainv, end=\"\\n\\n\") \n",
    "\n",
    "print(\"A@Ainv:\")\n",
    "print(A@Ainv, end=\"\\n\\n\")    # If the inverse was calculated correctly, this should be the identity matrix\n",
    "\n",
    "print(\"Identity:\")\n",
    "print(np.eye(2), end=\"\\n\\n\") \n",
    "\n",
    "# Compare the calculated inverse to the identity matrix\n",
    "result = \"same\" if np.all(np.eye(2) == A@Ainv) else \"not same\"\n",
    "print(\"Ainv and Identity are \", result) # Not the same...?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Due to floating point rounding errors, this multiplication yields a matrix that is not quite an identity matrix, but close to exact; ",
    "the result matrix's diagonal elements are very close to 1, while the off-diagonal elements are very close to zero.\n",
    "\n",
    "To gauge whether a computation is sufficiently close to ideal, use NumPy's `around` method, as shown in the next example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.4.b Finding the inverse for a matrix, avoiding the roundoff error issue\n",
    "\n",
    "import numpy as np\n",
    "from scipy import linalg\n",
    "\n",
    "# Define a matrix\n",
    "A = np.array([[1,2],\n",
    "              [3,4]])\n",
    "\n",
    "Ainv = linalg.inv(A) # Calculate A's inverse\n",
    "print(\"A's inverse:\")\n",
    "print(Ainv, end=\"\\n\\n\") \n",
    "\n",
    "print(\"A@Ainv:\")\n",
    "ATimesAinv = np.around(A@Ainv) # Avoid roundoff error with around to round to the nearest integer\n",
    "print(ATimesAinv, end=\"\\n\\n\") # If the inverse was calculated correctly, this should be the identity matrix\n",
    "\n",
    "print(\"Identity:\")\n",
    "print(np.eye(2), end=\"\\n\\n\") \n",
    "\n",
    "# Compare the calculated inverse to the identity matrix\n",
    "result = \"same\" if np.all(np.eye(2) == ATimesAinv) else \"not same\"\n",
    "print(\"Ainv and Identity are\", result) # The same!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For matrices that lack inverses, `inv` raises a LinAlgError."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.3.4.c Showing that some matrices do not have an inverse\n",
    "\n",
    "import numpy as np\n",
    "from scipy import linalg\n",
    "\n",
    "A = np.array([[3,2],  # A matrix, which does not have an inverse\n",
    "              [6,4]])\n",
    "\n",
    "linalg.inv(A) # Fails!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 20.4 Types of Matrices <a name='Matrices-Types-of-Matrices'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.4.1 Square <a name='Matrices-Square'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A *square* matrix is one that has the same number of rows and columns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style='color:blue'>",
    "&#128073;&ensp;&ensp;**Exercise 20.4.1.1:**\n\n",
    "</span>",
    "<span style='color:navy' >",
    "In the following code cell, ",
    "write a function to determine whether a matrix is square using a matrix's `shape` property. ",
    "Verify the function works as expected for square and non-square matrix inputs. ",
    "</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.4.2 Diagonal <a name='Matrices-Diagonal'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A *diagonal* matrix is one that has zeroes in all positions except for along its diagonal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is, the matrix $D$ –\n",
    "\n",
    "$$\n",
    "D = \\left[ d_{ij} \\right]\n",
    "$$\n",
    "\n",
    "– is diagonal if $d_{ij} = 0$ for all $i \\ne j$. This includes the identity matrix, which has 1's along its diagonal and 0's everywhere else."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy's `diag` method can be used to make a diagonal matrix when given the coefficients to put along the diagonal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.4.2.a Creating a diagonal matrix using diag()\n",
    "import numpy as np\n",
    "\n",
    "d = np.array([1,2,3]) # Values to put along the diagonal\n",
    "\n",
    "D = np.diag(d) # Create a diagonal matrix using those values\n",
    "\n",
    "D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 20.4.2.b Creating a diagonal matrix using diag(), and showing that values on the diagonal can be zero\n",
    "import numpy as np\n",
    "\n",
    "d = np.array([1,0,2,0,3,0,0,4]) # Values to put along the diagonal\n",
    "\n",
    "D = np.diag(d) # Create a diagonal matrix using those values\n",
    "\n",
    "D"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, when passed a matrix, `diag` returns the values along the diagonal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.4.2.c Getting the diagonal from a diagonal matrix\n",
    "import numpy as np\n",
    "\n",
    "D = np.array([[1,0,0],  # Create a matrix with [1,2,3] along the diagonal\n",
    "              [0,2,0],\n",
    "              [0,0,3]])\n",
    "\n",
    "d = np.diag(D) # Take the diagonal from the matrix\n",
    "\n",
    "d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`diag` can be used to extract a matrix's diagonal elements regardless of whether that matrix is diagonal or not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.4.2.d Getting the diagonal from a non-diagonal matrix\n",
    "import numpy as np\n",
    "\n",
    "A = np.array([[1,2,3],  # Create a matrix with [1,5,9] along the diagonal, but is NOT a diagonal matrix\n",
    "              [4,5,6],\n",
    "              [7,8,9]])\n",
    "\n",
    "d = np.diag(A) # Take the diagonal from the matrix\n",
    "\n",
    "d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.4.3 Identity <a name='Matrices-Identity'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An *identity* matrix, often referred to as $I$, is a diagonal matrix with 1's along its diagonal. ",
    "NumPy's `eye` method creates identity matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 20.4.3.a Using the eye method to create identity matrices\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# The eye method, when passed a single parameter m, makes an identity matrix of size m x m\n",
    "print(np.eye(1), end=\"\\n\\n\")\n",
    "print(np.eye(2), end=\"\\n\\n\")\n",
    "print(np.eye(3), end=\"\\n\\n\")\n",
    "\n",
    "# The eye method can also make non-square identity matrices by passing the dimensions m,n for a matrix of size m x n\n",
    "print(np.eye(2,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multiplying any matrix by the identity matrix leaves the original matrix unchanged. For a matrix $A$ of size $m$ by $n$,\n",
    "\n",
    "$$I_m A = A I_n = A$$\n",
    "\n",
    "where $I_m$ is an identity matrix of size $m$ by $m$ and $I_n$ is an identity matrix of size $n$ by $n$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.4.3.b Showing that a matrix multiplied by the identity matrix is the given matrix\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# Define a square matrix A\n",
    "A = np.array([[1,2,3],  # 3x3 square matrix\n",
    "              [4,5,6],\n",
    "              [7,8,9]])\n",
    "\n",
    "identityM = np.eye(A.shape[0]) # 3x3 identity\n",
    "\n",
    "print(A@identityM) # Same as A\n",
    "\n",
    "print(identityM@A) # Same as A\n",
    "\n",
    "# Define a non-square matrix B\n",
    "B = np.array([[1,2,3],  # 2x3 matrix\n",
    "              [4,5,6]])\n",
    "\n",
    "identityM = np.eye(B.shape[0]) # 2x2 identity\n",
    "identityN = np.eye(B.shape[1]) # 3x3 identity\n",
    "\n",
    "print(B@identityN) # Same as B\n",
    "\n",
    "print(identityM@B) # Same as B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.4.4 Triangular <a name='Matrices-Triangular'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A *triangular* matrix is a matrix whose diagonal splits that matrix into two regions, at least one of which has all zero elements. ",
    "Strategies for converting a matrix into an equivalent, triangular matrix are a key element of strategies for solving matrix equations: ",
    "problems involving the need to identify a matrix that, when multiplied with a second, known matrix, yield a desired result.\n",
    "\n",
    "Types of triangular matrices include *upper* and *lower* triangular matrices.\n",
    "\n",
    "A matrix **$U$** is *upper triangular* if all entries of the matrix below the diagonal are zero: e.g., \n",
    "\n",
    "$$\n",
    "U = \\begin{bmatrix} u_{11} & u_{12} & u_{13} & ...    & u_{1n}   \\\\\n",
    "                           & u_{22} & u_{23} & ...    & u_{2n}   \\\\ \n",
    "                           &        & \\ddots & \\ddots & \\vdots   \\\\\n",
    "                           &        &        & \\ddots & u_{m-1n} \\\\\n",
    "                    0      &        &        &        & u_{mn} \\end{bmatrix} \n",
    "$$\n",
    "\n",
    "is upper triangular.\n",
    "\n",
    "A zero in the bottom left-hand corner is commonly used to indicate that all empty spaces are filled with zeroes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.4.4.a Creating an example of an upper triangular matrix\n",
    "import numpy as np\n",
    "\n",
    "U = np.array( [[max(0,j-i+1) for j in range(5)] \n",
    "               for i in range(5)])\n",
    "U"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Triangular matrices are not required to be square."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.4.4.b Creating a non-square upper triangular matrix\n",
    "import numpy as np\n",
    "\n",
    "U = np.array( [[max(0,j-i+1) for j in range(8)] \n",
    "               for i in range(3)])\n",
    "U"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, a matrix is *lower triangular* if all entries of the matrix above the diagonal are zero: e.g., \n",
    "\n",
    "$$\n",
    "L = \\begin{bmatrix} l_{11} &        &        &          &     0    \\\\\n",
    "                    l_{21} & l_{22} &        &          &          \\\\ \n",
    "                    l_{31} & l_{32} & \\ddots &          &          \\\\\n",
    "                    \\vdots & \\vdots & \\ddots & \\ddots   &          \\\\\n",
    "                    l_{m1} & l_{m2} & ...    & l_{m-1n} & l_{mn} \\end{bmatrix} \n",
    "$$\n",
    "\n",
    "is lower triangular."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 20.4.4.c Creating an example of a lower triangular matrix\n",
    "import numpy as np\n",
    "\n",
    "L = np.array( [[max(0,j-i+1) for i in range(5)] \n",
    "               for j in range(5)])\n",
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.4.4.d Creating a lower triangular matrix that is non-square\n",
    "import numpy as np\n",
    "\n",
    "L = np.array( [[max(0,j-i+1) for i in range(3)] \n",
    "               for j in range(8)])\n",
    "L"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.4.5 Orthogonal <a name='Matrices-Orthogonal'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A matrix **$Q$** is *orthogonal* if\n",
    "\n",
    "$$Q^TQ=QQ^T=I_n$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 20.4.5.a Creating an orthogonal matrix and showing that it's orthogonal\n",
    "import numpy as np\n",
    "\n",
    "Q = np.array([[-1,0], # An orthogonal matrix\n",
    "              [0,1]])\n",
    "\n",
    "# All are the 2x2 identity matrix:\n",
    "print(Q@Q.T)\n",
    "print(Q.T@Q)\n",
    "print(np.eye(Q.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 20.4.6 Unitary <a name='Matrices-Unitary'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A matrix **$U$** is *unitary* if\n",
    "\n",
    "$$U^HU=UU^H=I_n$$\n",
    "\n",
    "where $U^H$ is the Hermitian transpose of $U$, that is, the conjugate transpose.\n",
    "\n",
    "For real numbers, unitary and orthogonal are the same. The conjugate transpose differs only for matrices with complex values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A matrix's *conjugate* can be calculated by using NumPy's `conjugate` method.\n",
    "\n",
    "A matrix's *Hermitian transpose* can be calculated by using NumPy's `conjugate` method and passing in the matrix's transpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 20.4.6.a Using conjugate() along with a matrix's transpose to calculate the Hermitian transpose using real numbers\n",
    "import numpy as np\n",
    "\n",
    "A = np.array([[1,2,3],  # 3x3 square matrix\n",
    "              [4,5,6],\n",
    "              [7,8,9]])\n",
    "\n",
    "print(np.conjugate(A.T)) # Perform the Hermitian transpose\n",
    "\n",
    "print(A.T) # Notice that the Hermitian transpose = transpose for real numbers\n",
    "\n",
    "sameOrNot = \"\" if np.all(np.conjugate(A.T) == A.T) else \"not\"\n",
    "print(\"Hermitian transpose and transpose are\", sameOrNot, \"the same.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To denote $\\sqrt{-1}$, Python uses `j`, as is standard in engineering, rather than `i`, as is standard in mathematics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.4.6.b Using conjugate() along with a matrix's transpose to calculate the Hermitian transpose using complex numbers\n",
    "import numpy as np\n",
    "\n",
    "A = np.array([[1   ,2j  ,3   ],  # A matrix with complex numbers\n",
    "              [-2j ,3   ,3-4j],\n",
    "              [3   ,3+4j,5   ]])\n",
    "\n",
    "print(np.conjugate(A.T)) # Perform the Hermitian transpose\n",
    "print(A.T) # Notice that it is DIFFERENT from the regular transpose for complex numbers\n",
    "\n",
    "sameOrNot = \"\" if np.all(np.conjugate(A.T) == A.T) else \"not\"\n",
    "print(\"Hermitian transpose and transpose are:\", sameOrNot, \"the same.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 20.4.6.c Creating a unitary matrix and showing that it's unitary\n",
    "import numpy as np\n",
    "\n",
    "U = np.array([[ 1/3, -2/3,  -2/3], # A unitary matrix\n",
    "              [2j/3, 2j/3, -1j/3],\n",
    "              [ 2/3, -1/3,   2/3]])\n",
    "\n",
    "# Define a function for the Hermitian transpose\n",
    "def H(matrix):\n",
    "    return np.conjugate(matrix.T)\n",
    "\n",
    "# All are the 3x3 identity matrix:\n",
    "print(np.around(U@H(U))) # Avoid roundoff error with around to round to the nearest integer\n",
    "print(np.around(H(U)@U))\n",
    "print(np.eye(U.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 20.5 Additional References <a name='Matrices-Additional-References'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other topics in linear algebra, such as computing norms, finding pseudo-inverses for matrices without actual inverses, ",
    "finding eigenvalues/eigenvectors, and matrix decompositions are also supported by NumPy and SciPy.\n",
    "\n",
    "For more information about NumPy, see the [official NumPy documentation](https://numpy.org/doc/stable/reference/index.html).\n",
    "\n",
    "For more information about SciPy, see the [official SciPy documentation](https://docs.scipy.org/doc/scipy/reference/). \n",
    "For information directly related to the `linalg` package, see SciPy's [`linalg` tutorial](https://docs.scipy.org/doc/scipy/reference/tutorial/linalg.html) \n",
    "and [`linalg` reference](https://docs.scipy.org/doc/scipy/reference/linalg.html)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}