Untitled.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9f2bdea4",
   "metadata": {},
   "source": [
    "# A partial derivative is a derivative of a function with respect to one of its variables, holding the other variables constant. It is denoted using the symbol $\\frac{\\partial}{\\partial x}$, where $x$ is the variable with respect to which the derivative is taken.\n",
    "\n",
    "For example, consider the function $f(x,y)=x^2y+y^3$. The partial derivative of this function with respect to $x$ is $\\frac{\\partial}{\\partial x}f(x,y)=2xy$. This represents the rate of change of the function $f$ with respect to the variable $x$, holding the variable $y$ constant. Similarly, the partial derivative with respect to $y$ is $\\frac{\\partial}{\\partial y}f(x,y)=x^2+3y^2$.\n",
    "\n",
    "To compute partial derivatives in a program, you can use automatic differentiation, which is a technique that uses the chain rule of calculus to compute derivatives of a function defined by a computer program. This allows you to compute derivatives of arbitrary order and complexity with high accuracy and efficiency, without having to manually derive and implement the derivative formulas.\n",
    "\n",
    "In most deep learning libraries, automatic differentiation is implemented using a technique called reverse-mode differentiation, which is an efficient algorithm for computing the gradient of a scalar-valued function with respect to a vector of inputs. To use automatic differentiation in a program, you typically define a function using a sequence of operations (such as addition, multiplication, etc.), and then call a backward method on the output to compute the gradient with respect to the inputs. The gradient can then be used to update the model parameters using gradient descent, as described above.\n",
    "\n",
    "\n",
    "\\begin{split}\\begin{aligned} \\left[\\mathbf{H}\\right]_{i, j} &= [\\mathbf{U}]_{i, j} + \\sum_k \\sum_l[\\mathsf{W}]_{i, j, k, l}  [\\mathbf{X}]_{k, l}\\\\ &=  [\\mathbf{U}]_{i, j} +\n",
    "\\sum_a \\sum_b [\\mathsf{V}]_{i, j, a, b}  [\\mathbf{X}]_{i+a, j+b}.\\end{aligned}\\end{split}\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ca4833c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}