1.1_movie_recommendation_ubcf.ipynb

{
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5-final"
  },
  "orig_nbformat": 2,
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3",
   "language": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2,
 "cells": [
  {
   "source": [
    "# Movie recommendation system\n",
    "\n",
    "It uses **User Based Collaborative Filtering (CBF)**\n",
    "\n",
    "It's a memory-based type of collaborative filtering, based on similarity between users.\n",
    "The main idea here - to recommend same movies for similar users. For example if user A watched video V1,V2,V3 \n",
    "and user B is watched video V2,V3 then users are similar and we could recommend video V1 to user B.\n",
    "\n",
    "\n",
    "At the beginning we do an initial setup and import.\n",
    "\n"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "import pandas as pd\n",
    "import numpy as np \n",
    "from sklearn.metrics import pairwise_distances\n",
    "\n",
    "#initial output setup\n",
    "pd.set_option('display.max_rows', 70)\n",
    "pd.set_option('display.max_columns', 70)\n",
    "pd.set_option('display.width', 500)"
   ]
  },
  {
   "source": [
    "Now we are loading movies and ratings movielens' datasets.\n",
    "We can't use IMDB here, because it doesn't provide with user ratings. And we need it for UBCF."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "   userId  movieId  rating   timestamp\n",
       "0       1      296     5.0  1147880044\n",
       "1       1      306     3.5  1147868817\n",
       "2       1      307     5.0  1147868828\n",
       "3       1      665     5.0  1147878820\n",
       "4       1      899     3.5  1147868510"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>userId</th>\n      <th>movieId</th>\n      <th>rating</th>\n      <th>timestamp</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>296</td>\n      <td>5.0</td>\n      <td>1147880044</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1</td>\n      <td>306</td>\n      <td>3.5</td>\n      <td>1147868817</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1</td>\n      <td>307</td>\n      <td>5.0</td>\n      <td>1147868828</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1</td>\n      <td>665</td>\n      <td>5.0</td>\n      <td>1147878820</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1</td>\n      <td>899</td>\n      <td>3.5</td>\n      <td>1147868510</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 7
    }
   ],
   "source": [
    "\n",
    "movies=pd.read_csv(\"data/movies.csv\")\n",
    "# here we set max number of rows for ratings dataset, \n",
    "# algorithm is memory based and and leads or memory overflow on the large number of ratings\n",
    "ratings=pd.read_csv(\"data/ratings.csv\",nrows=99999)\n",
    "\n",
    "ratings.head()"
   ]
  },
  {
   "source": [
    "I'm looking for Star Wars -like movies recommendation. So we need to search for Star Wars movies, and get their ids."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "       movieId                                              title                                genres\n257        260          Star Wars: Episode IV - A New Hope (1977)               Action|Adventure|Sci-Fi\n1166      1196  Star Wars: Episode V - The Empire Strikes Back...               Action|Adventure|Sci-Fi\n1179      1210  Star Wars: Episode VI - Return of the Jedi (1983)               Action|Adventure|Sci-Fi\n2537      2628   Star Wars: Episode I - The Phantom Menace (1999)               Action|Adventure|Sci-Fi\n5270      5378  Star Wars: Episode II - Attack of the Clones (...          Action|Adventure|Sci-Fi|IMAX\n9952     33493  Star Wars: Episode III - Revenge of the Sith (...               Action|Adventure|Sci-Fi\n12593    61160                   Star Wars: The Clone Wars (2008)     Action|Adventure|Animation|Sci-Fi\n14912    79006  Empire of Dreams: The Story of the 'Star Wars'...                           Documentary\n21250   109713               Star Wars: Threads of Destiny (2014)               Action|Adventure|Sci-Fi\n25055   122886  Star Wars: Episode VII - The Force Awakens (2015)  Action|Adventure|Fantasy|Sci-Fi|IMAX\n29997   135216               The Star Wars Holiday Special (1978)      Adventure|Children|Comedy|Sci-Fi\n30524   136485                    Robot Chicken: Star Wars (2007)               Animation|Comedy|Sci-Fi\n36606   150688  Plastic Galaxy: The Story of Star Wars Toys (2...                           Documentary\n43646   166528                Rogue One: A Star Wars Story (2016)       Action|Adventure|Fantasy|Sci-Fi\n49883   179819                    Star Wars: The Last Jedi (2017)       Action|Adventure|Fantasy|Sci-Fi\n50585   181355         Robot Chicken: Star Wars Episode II (2008)                         Comedy|Sci-Fi\n50586   181357        Robot Chicken: Star Wars Episode III (2010)                         Comedy|Sci-Fi\n51308   182857              Star Wars: The Legacy Revealed (2007)                           Documentary\n52111   184593                         Star Wars Downunder (2013)                                Sci-Fi\n53524   187595                     Solo: A Star Wars Story (2018)      Action|Adventure|Children|Sci-Fi\n56918   195217                                  Star Wars: Dresca                                Sci-Fi\n"
     ]
    }
   ],
   "source": [
    "print(movies[movies.title.str.contains(\"Star Wars\")])"
   ]
  },
  {
   "source": [
    "Now we creating a new user - myself, and provide ratings for these 2 movies, taken from the previous step\n",
    "\n",
    "Star Wars: Episode IV - A New Hope (1977)   \n",
    "Star Wars: Episode VI - Return of the Jedi (1983)  \n",
    "Star Wars: Episode VI - Return of the Jedi (1983)  \n",
    "\n",
    "we create a new user with id is bigger than the maximum number of users in the movielens dataset."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "           userId  movieId  rating     timestamp\n",
       "99997         757     2118     4.0  1.184014e+09\n",
       "99998         757     2124     3.5  1.184074e+09\n",
       "99999   999999999      260     5.0           NaN\n",
       "100000  999999999     1196     5.0           NaN\n",
       "100001  999999999     1210     5.0           NaN"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>userId</th>\n      <th>movieId</th>\n      <th>rating</th>\n      <th>timestamp</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>99997</th>\n      <td>757</td>\n      <td>2118</td>\n      <td>4.0</td>\n      <td>1.184014e+09</td>\n    </tr>\n    <tr>\n      <th>99998</th>\n      <td>757</td>\n      <td>2124</td>\n      <td>3.5</td>\n      <td>1.184074e+09</td>\n    </tr>\n    <tr>\n      <th>99999</th>\n      <td>999999999</td>\n      <td>260</td>\n      <td>5.0</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>100000</th>\n      <td>999999999</td>\n      <td>1196</td>\n      <td>5.0</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>100001</th>\n      <td>999999999</td>\n      <td>1210</td>\n      <td>5.0</td>\n      <td>NaN</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 8
    }
   ],
   "source": [
    "#add new user - myself to  check the predictions\n",
    "#user  - get max user value here, for now just 999999999\n",
    "my_user_id = 999999999\n",
    "my_rating = pd.DataFrame([[my_user_id,260,5],[my_user_id,1196,5],[my_user_id,1210,5]],columns=['userId','movieId',\"rating\"])\n",
    "ratings = ratings.append(my_rating,ignore_index=True)\n",
    "\n",
    "#check last 5 rows to ensure user properly added\n",
    "ratings.tail()"
   ]
  },
  {
   "source": [
    "Now we need to create a single dataset which will contain all movies and user ratings."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "       movieId                                              title                   genres     userId  rating  timestamp\n",
       "7674       260          Star Wars: Episode IV - A New Hope (1977)  Action|Adventure|Sci-Fi  999999999     5.0        NaN\n",
       "26905     1196  Star Wars: Episode V - The Empire Strikes Back...  Action|Adventure|Sci-Fi  999999999     5.0        NaN\n",
       "28237     1210  Star Wars: Episode VI - Return of the Jedi (1983)  Action|Adventure|Sci-Fi  999999999     5.0        NaN"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>movieId</th>\n      <th>title</th>\n      <th>genres</th>\n      <th>userId</th>\n      <th>rating</th>\n      <th>timestamp</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>7674</th>\n      <td>260</td>\n      <td>Star Wars: Episode IV - A New Hope (1977)</td>\n      <td>Action|Adventure|Sci-Fi</td>\n      <td>999999999</td>\n      <td>5.0</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>26905</th>\n      <td>1196</td>\n      <td>Star Wars: Episode V - The Empire Strikes Back...</td>\n      <td>Action|Adventure|Sci-Fi</td>\n      <td>999999999</td>\n      <td>5.0</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>28237</th>\n      <td>1210</td>\n      <td>Star Wars: Episode VI - Return of the Jedi (1983)</td>\n      <td>Action|Adventure|Sci-Fi</td>\n      <td>999999999</td>\n      <td>5.0</td>\n      <td>NaN</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 10
    }
   ],
   "source": [
    "#merge both datasets (inner join operation) by movieId\n",
    "movies_and_ratings = pd.merge(movies,ratings,on=\"movieId\")\n",
    "#show me my ratings\n",
    "movies_and_ratings[movies_and_ratings['userId']==my_user_id].head()\n"
   ]
  },
  {
   "source": [
    "Now we need to create a temporary table which will contain users/movies data. Every column is a movie related user's rating.\n",
    "\n",
    "In simple words we have a table which contains all movies ratings per user."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "movieId  1       2       3       4       5       6       7       8       9       10      11      12      13      14      15      16      17      18      19      20      21      22      23      24      25      26      27      28      29      30      31      32      33      34      35      ...  200838  201200  201340  201588  201646  201749  201773  201811  202263  202265  202429  202439  202759  202934  203208  203218  203222  203244  203322  203375  203513  203519  203649  204352  204542  \\\n",
       "0           0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0  ...     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0   \n",
       "1           3.5     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0  ...     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0   \n",
       "2           4.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     4.5     0.0     0.0     4.5     0.0     0.0     0.0  ...     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0   \n",
       "3           3.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0  ...     3.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     4.5     0.0     2.5     0.0     0.0     0.0   \n",
       "4           4.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     4.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     5.0     0.0     0.0     0.0  ...     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0   \n",
       "\n",
       "movieId  204692  204698  204704  205054  205072  205106  205413  205499  205557  206272  \n",
       "0           0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0  \n",
       "1           0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0  \n",
       "2           0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0  \n",
       "3           0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0  \n",
       "4           0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0     0.0  \n",
       "\n",
       "[5 rows x 9786 columns]"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th>movieId</th>\n      <th>1</th>\n      <th>2</th>\n      <th>3</th>\n      <th>4</th>\n      <th>5</th>\n      <th>6</th>\n      <th>7</th>\n      <th>8</th>\n      <th>9</th>\n      <th>10</th>\n      <th>11</th>\n      <th>12</th>\n      <th>13</th>\n      <th>14</th>\n      <th>15</th>\n      <th>16</th>\n      <th>17</th>\n      <th>18</th>\n      <th>19</th>\n      <th>20</th>\n      <th>21</th>\n      <th>22</th>\n      <th>23</th>\n      <th>24</th>\n      <th>25</th>\n      <th>26</th>\n      <th>27</th>\n      <th>28</th>\n      <th>29</th>\n      <th>30</th>\n      <th>31</th>\n      <th>32</th>\n      <th>33</th>\n      <th>34</th>\n      <th>35</th>\n      <th>...</th>\n      <th>200838</th>\n      <th>201200</th>\n      <th>201340</th>\n      <th>201588</th>\n      <th>201646</th>\n      <th>201749</th>\n      <th>201773</th>\n      <th>201811</th>\n      <th>202263</th>\n      <th>202265</th>\n      <th>202429</th>\n      <th>202439</th>\n      <th>202759</th>\n      <th>202934</th>\n      <th>203208</th>\n      <th>203218</th>\n      <th>203222</th>\n      <th>203244</th>\n      <th>203322</th>\n      <th>203375</th>\n      <th>203513</th>\n      <th>203519</th>\n      <th>203649</th>\n      <th>204352</th>\n      <th>204542</th>\n      <th>204692</th>\n      <th>204698</th>\n      <th>204704</th>\n      <th>205054</th>\n      <th>205072</th>\n      <th>205106</th>\n      <th>205413</th>\n      <th>205499</th>\n      <th>205557</th>\n      <th>206272</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>3.5</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>4.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>4.5</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>4.5</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>3.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>3.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>4.5</td>\n      <td>0.0</td>\n      <td>2.5</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>4.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>4.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>5.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 9786 columns</p>\n</div>"
     },
     "metadata": {},
     "execution_count": 12
    }
   ],
   "source": [
    "#create pivot table, which is anologue of excel spreadsheet, with userId as rows, movieId as columns and ratings as values\n",
    "#we drops the oroginal indexes, and use our indexes\n",
    "ratings_matrix_users = movies_and_ratings.pivot_table(index=['userId'],columns=['movieId'],values='rating').reset_index(drop=True)\n",
    "#now we fill NaN values with 0 (NaN values - it's movies without user's rating)\n",
    "ratings_matrix_users.fillna( 0, inplace = True )\n",
    "\n",
    "ratings_matrix_users.head()"
   ]
  },
  {
   "source": [
    "Now we are calculating cosine similarity between users. We calculate this similarity per user and based on the movie's recommendations"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "        0         1         2         3         4         5         6         7         8         9         10        11        12        13        14        15        16        17        18        19        20        21        22        23        24        25        26        27        28        29        30        31        32        33        34   ...       723       724       725       726       727       728       729       730       731       732       733       734       735       736  \\\n",
       "0  0.000000  0.762322  0.795569  0.740310  0.585380  0.713644  0.483813  0.613872  0.587894  0.650287  0.757955  0.791427  0.812564  0.839441  0.732855  0.787865  0.460789  0.734377  0.824454  0.746610  0.748297  0.382547  0.686465  0.721393  0.502705  0.697569  0.656762  0.630851  0.542527  0.672086  0.785477  0.518198  0.786187  0.757059  0.732197  ...  0.504968  0.832759  0.302022  0.846140  0.308225  0.732292  0.778153  0.601921  0.729441  0.565407  0.734289  0.798535  0.842420  0.766931   \n",
       "1  0.762322  0.000000  0.851134  0.845573  0.778461  0.853804  0.595940  0.823053  0.826056  0.839609  0.808417  0.919295  0.937771  0.760281  0.905279  0.789124  0.542125  0.941470  0.891440  0.948573  0.787132  0.517673  0.865426  0.787593  0.652304  0.864380  0.884432  0.851070  0.559998  0.862938  0.931246  0.662353  0.822460  0.791767  0.832617  ...  0.591909  0.826404  0.473745  0.913559  0.472995  0.788991  0.821289  0.750972  0.882246  0.792028  0.897152  0.929801  0.856294  0.910188   \n",
       "2  0.795569  0.851134  0.000000  0.949809  0.589485  0.756184  0.416750  0.636739  0.611415  0.684263  0.759393  0.807306  0.940765  0.850471  0.761767  0.806257  0.740240  0.783419  0.955911  0.831225  0.934976  0.334861  0.690867  0.779418  0.466520  0.750047  0.695535  0.728849  0.555639  0.761565  0.821270  0.486860  0.806443  0.764098  0.827921  ...  0.442365  0.848849  0.338528  0.928882  0.342932  0.899557  0.888350  0.580027  0.738277  0.596108  0.755349  0.899591  0.869580  0.882710   \n",
       "3  0.740310  0.845573  0.949809  0.000000  0.553947  0.735843  0.364200  0.602503  0.596383  0.653252  0.726999  0.768222  0.916451  0.799266  0.742920  0.772563  0.763080  0.766609  0.942304  0.840308  0.945414  0.287860  0.658415  0.729692  0.411259  0.722708  0.686375  0.743533  0.494002  0.780663  0.775722  0.436994  0.761455  0.725148  0.788552  ...  0.380036  0.796465  0.323227  0.914223  0.320482  0.908957  0.840103  0.538248  0.702827  0.574708  0.728259  0.899477  0.816300  0.901190   \n",
       "4  0.585380  0.778461  0.589485  0.553947  0.000000  0.749003  0.852828  0.960714  0.911527  0.931860  0.664093  0.845964  0.747585  0.635868  0.831131  0.553453  0.335918  0.891353  0.651776  0.767177  0.515223  0.797138  0.868240  0.646965  0.910249  0.902670  0.806332  0.746899  0.575804  0.668492  0.875108  0.885178  0.734351  0.708642  0.750078  ...  0.861111  0.702467  0.696448  0.701281  0.710149  0.539070  0.584750  0.945139  0.846486  0.841572  0.822564  0.770698  0.774381  0.698273   \n",
       "\n",
       "        737       738       739       740       741       742       743       744       745       746       747       748       749       750       751       752       753       754       755       756       757  \n",
       "0  0.820321  0.803252  0.798151  0.797270  0.728536  0.768867  0.681737  0.723453  0.663986  0.811305  0.482705  0.820605  0.731827  0.740974  0.658093  0.573658  0.507467  0.799494  0.832629  0.706292  0.435819  \n",
       "1  0.923578  0.872068  0.907889  0.921949  0.878565  0.840458  0.793979  0.775799  0.816721  0.817747  0.644709  0.907388  0.820333  0.908850  0.868289  0.705866  0.633376  0.802113  0.942991  0.860851  0.684739  \n",
       "2  0.904518  0.926165  0.959958  0.819772  0.827453  0.901127  0.893223  0.716109  0.645743  0.887117  0.445917  0.894193  0.898252  0.790688  0.691411  0.544246  0.458736  0.950494  0.891995  0.713025  0.584674  \n",
       "3  0.854153  0.893316  0.928362  0.784796  0.829117  0.915091  0.903814  0.659800  0.594408  0.856195  0.398828  0.878092  0.888267  0.757690  0.679258  0.487419  0.410016  0.935265  0.851681  0.684626  0.641613  \n",
       "4  0.785566  0.595770  0.707167  0.813837  0.603989  0.538680  0.558256  0.604386  0.914295  0.630178  0.895607  0.778780  0.588616  0.786225  0.916750  0.907414  0.892208  0.566304  0.792438  0.858039  0.558378  \n",
       "\n",
       "[5 rows x 758 columns]"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>0</th>\n      <th>1</th>\n      <th>2</th>\n      <th>3</th>\n      <th>4</th>\n      <th>5</th>\n      <th>6</th>\n      <th>7</th>\n      <th>8</th>\n      <th>9</th>\n      <th>10</th>\n      <th>11</th>\n      <th>12</th>\n      <th>13</th>\n      <th>14</th>\n      <th>15</th>\n      <th>16</th>\n      <th>17</th>\n      <th>18</th>\n      <th>19</th>\n      <th>20</th>\n      <th>21</th>\n      <th>22</th>\n      <th>23</th>\n      <th>24</th>\n      <th>25</th>\n      <th>26</th>\n      <th>27</th>\n      <th>28</th>\n      <th>29</th>\n      <th>30</th>\n      <th>31</th>\n      <th>32</th>\n      <th>33</th>\n      <th>34</th>\n      <th>...</th>\n      <th>723</th>\n      <th>724</th>\n      <th>725</th>\n      <th>726</th>\n      <th>727</th>\n      <th>728</th>\n      <th>729</th>\n      <th>730</th>\n      <th>731</th>\n      <th>732</th>\n      <th>733</th>\n      <th>734</th>\n      <th>735</th>\n      <th>736</th>\n      <th>737</th>\n      <th>738</th>\n      <th>739</th>\n      <th>740</th>\n      <th>741</th>\n      <th>742</th>\n      <th>743</th>\n      <th>744</th>\n      <th>745</th>\n      <th>746</th>\n      <th>747</th>\n      <th>748</th>\n      <th>749</th>\n      <th>750</th>\n      <th>751</th>\n      <th>752</th>\n      <th>753</th>\n      <th>754</th>\n      <th>755</th>\n      <th>756</th>\n      <th>757</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0.000000</td>\n      <td>0.762322</td>\n      <td>0.795569</td>\n      <td>0.740310</td>\n      <td>0.585380</td>\n      <td>0.713644</td>\n      <td>0.483813</td>\n      <td>0.613872</td>\n      <td>0.587894</td>\n      <td>0.650287</td>\n      <td>0.757955</td>\n      <td>0.791427</td>\n      <td>0.812564</td>\n      <td>0.839441</td>\n      <td>0.732855</td>\n      <td>0.787865</td>\n      <td>0.460789</td>\n      <td>0.734377</td>\n      <td>0.824454</td>\n      <td>0.746610</td>\n      <td>0.748297</td>\n      <td>0.382547</td>\n      <td>0.686465</td>\n      <td>0.721393</td>\n      <td>0.502705</td>\n      <td>0.697569</td>\n      <td>0.656762</td>\n      <td>0.630851</td>\n      <td>0.542527</td>\n      <td>0.672086</td>\n      <td>0.785477</td>\n      <td>0.518198</td>\n      <td>0.786187</td>\n      <td>0.757059</td>\n      <td>0.732197</td>\n      <td>...</td>\n      <td>0.504968</td>\n      <td>0.832759</td>\n      <td>0.302022</td>\n      <td>0.846140</td>\n      <td>0.308225</td>\n      <td>0.732292</td>\n      <td>0.778153</td>\n      <td>0.601921</td>\n      <td>0.729441</td>\n      <td>0.565407</td>\n      <td>0.734289</td>\n      <td>0.798535</td>\n      <td>0.842420</td>\n      <td>0.766931</td>\n      <td>0.820321</td>\n      <td>0.803252</td>\n      <td>0.798151</td>\n      <td>0.797270</td>\n      <td>0.728536</td>\n      <td>0.768867</td>\n      <td>0.681737</td>\n      <td>0.723453</td>\n      <td>0.663986</td>\n      <td>0.811305</td>\n      <td>0.482705</td>\n      <td>0.820605</td>\n      <td>0.731827</td>\n      <td>0.740974</td>\n      <td>0.658093</td>\n      <td>0.573658</td>\n      <td>0.507467</td>\n      <td>0.799494</td>\n      <td>0.832629</td>\n      <td>0.706292</td>\n      <td>0.435819</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0.762322</td>\n      <td>0.000000</td>\n      <td>0.851134</td>\n      <td>0.845573</td>\n      <td>0.778461</td>\n      <td>0.853804</td>\n      <td>0.595940</td>\n      <td>0.823053</td>\n      <td>0.826056</td>\n      <td>0.839609</td>\n      <td>0.808417</td>\n      <td>0.919295</td>\n      <td>0.937771</td>\n      <td>0.760281</td>\n      <td>0.905279</td>\n      <td>0.789124</td>\n      <td>0.542125</td>\n      <td>0.941470</td>\n      <td>0.891440</td>\n      <td>0.948573</td>\n      <td>0.787132</td>\n      <td>0.517673</td>\n      <td>0.865426</td>\n      <td>0.787593</td>\n      <td>0.652304</td>\n      <td>0.864380</td>\n      <td>0.884432</td>\n      <td>0.851070</td>\n      <td>0.559998</td>\n      <td>0.862938</td>\n      <td>0.931246</td>\n      <td>0.662353</td>\n      <td>0.822460</td>\n      <td>0.791767</td>\n      <td>0.832617</td>\n      <td>...</td>\n      <td>0.591909</td>\n      <td>0.826404</td>\n      <td>0.473745</td>\n      <td>0.913559</td>\n      <td>0.472995</td>\n      <td>0.788991</td>\n      <td>0.821289</td>\n      <td>0.750972</td>\n      <td>0.882246</td>\n      <td>0.792028</td>\n      <td>0.897152</td>\n      <td>0.929801</td>\n      <td>0.856294</td>\n      <td>0.910188</td>\n      <td>0.923578</td>\n      <td>0.872068</td>\n      <td>0.907889</td>\n      <td>0.921949</td>\n      <td>0.878565</td>\n      <td>0.840458</td>\n      <td>0.793979</td>\n      <td>0.775799</td>\n      <td>0.816721</td>\n      <td>0.817747</td>\n      <td>0.644709</td>\n      <td>0.907388</td>\n      <td>0.820333</td>\n      <td>0.908850</td>\n      <td>0.868289</td>\n      <td>0.705866</td>\n      <td>0.633376</td>\n      <td>0.802113</td>\n      <td>0.942991</td>\n      <td>0.860851</td>\n      <td>0.684739</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0.795569</td>\n      <td>0.851134</td>\n      <td>0.000000</td>\n      <td>0.949809</td>\n      <td>0.589485</td>\n      <td>0.756184</td>\n      <td>0.416750</td>\n      <td>0.636739</td>\n      <td>0.611415</td>\n      <td>0.684263</td>\n      <td>0.759393</td>\n      <td>0.807306</td>\n      <td>0.940765</td>\n      <td>0.850471</td>\n      <td>0.761767</td>\n      <td>0.806257</td>\n      <td>0.740240</td>\n      <td>0.783419</td>\n      <td>0.955911</td>\n      <td>0.831225</td>\n      <td>0.934976</td>\n      <td>0.334861</td>\n      <td>0.690867</td>\n      <td>0.779418</td>\n      <td>0.466520</td>\n      <td>0.750047</td>\n      <td>0.695535</td>\n      <td>0.728849</td>\n      <td>0.555639</td>\n      <td>0.761565</td>\n      <td>0.821270</td>\n      <td>0.486860</td>\n      <td>0.806443</td>\n      <td>0.764098</td>\n      <td>0.827921</td>\n      <td>...</td>\n      <td>0.442365</td>\n      <td>0.848849</td>\n      <td>0.338528</td>\n      <td>0.928882</td>\n      <td>0.342932</td>\n      <td>0.899557</td>\n      <td>0.888350</td>\n      <td>0.580027</td>\n      <td>0.738277</td>\n      <td>0.596108</td>\n      <td>0.755349</td>\n      <td>0.899591</td>\n      <td>0.869580</td>\n      <td>0.882710</td>\n      <td>0.904518</td>\n      <td>0.926165</td>\n      <td>0.959958</td>\n      <td>0.819772</td>\n      <td>0.827453</td>\n      <td>0.901127</td>\n      <td>0.893223</td>\n      <td>0.716109</td>\n      <td>0.645743</td>\n      <td>0.887117</td>\n      <td>0.445917</td>\n      <td>0.894193</td>\n      <td>0.898252</td>\n      <td>0.790688</td>\n      <td>0.691411</td>\n      <td>0.544246</td>\n      <td>0.458736</td>\n      <td>0.950494</td>\n      <td>0.891995</td>\n      <td>0.713025</td>\n      <td>0.584674</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0.740310</td>\n      <td>0.845573</td>\n      <td>0.949809</td>\n      <td>0.000000</td>\n      <td>0.553947</td>\n      <td>0.735843</td>\n      <td>0.364200</td>\n      <td>0.602503</td>\n      <td>0.596383</td>\n      <td>0.653252</td>\n      <td>0.726999</td>\n      <td>0.768222</td>\n      <td>0.916451</td>\n      <td>0.799266</td>\n      <td>0.742920</td>\n      <td>0.772563</td>\n      <td>0.763080</td>\n      <td>0.766609</td>\n      <td>0.942304</td>\n      <td>0.840308</td>\n      <td>0.945414</td>\n      <td>0.287860</td>\n      <td>0.658415</td>\n      <td>0.729692</td>\n      <td>0.411259</td>\n      <td>0.722708</td>\n      <td>0.686375</td>\n      <td>0.743533</td>\n      <td>0.494002</td>\n      <td>0.780663</td>\n      <td>0.775722</td>\n      <td>0.436994</td>\n      <td>0.761455</td>\n      <td>0.725148</td>\n      <td>0.788552</td>\n      <td>...</td>\n      <td>0.380036</td>\n      <td>0.796465</td>\n      <td>0.323227</td>\n      <td>0.914223</td>\n      <td>0.320482</td>\n      <td>0.908957</td>\n      <td>0.840103</td>\n      <td>0.538248</td>\n      <td>0.702827</td>\n      <td>0.574708</td>\n      <td>0.728259</td>\n      <td>0.899477</td>\n      <td>0.816300</td>\n      <td>0.901190</td>\n      <td>0.854153</td>\n      <td>0.893316</td>\n      <td>0.928362</td>\n      <td>0.784796</td>\n      <td>0.829117</td>\n      <td>0.915091</td>\n      <td>0.903814</td>\n      <td>0.659800</td>\n      <td>0.594408</td>\n      <td>0.856195</td>\n      <td>0.398828</td>\n      <td>0.878092</td>\n      <td>0.888267</td>\n      <td>0.757690</td>\n      <td>0.679258</td>\n      <td>0.487419</td>\n      <td>0.410016</td>\n      <td>0.935265</td>\n      <td>0.851681</td>\n      <td>0.684626</td>\n      <td>0.641613</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0.585380</td>\n      <td>0.778461</td>\n      <td>0.589485</td>\n      <td>0.553947</td>\n      <td>0.000000</td>\n      <td>0.749003</td>\n      <td>0.852828</td>\n      <td>0.960714</td>\n      <td>0.911527</td>\n      <td>0.931860</td>\n      <td>0.664093</td>\n      <td>0.845964</td>\n      <td>0.747585</td>\n      <td>0.635868</td>\n      <td>0.831131</td>\n      <td>0.553453</td>\n      <td>0.335918</td>\n      <td>0.891353</td>\n      <td>0.651776</td>\n      <td>0.767177</td>\n      <td>0.515223</td>\n      <td>0.797138</td>\n      <td>0.868240</td>\n      <td>0.646965</td>\n      <td>0.910249</td>\n      <td>0.902670</td>\n      <td>0.806332</td>\n      <td>0.746899</td>\n      <td>0.575804</td>\n      <td>0.668492</td>\n      <td>0.875108</td>\n      <td>0.885178</td>\n      <td>0.734351</td>\n      <td>0.708642</td>\n      <td>0.750078</td>\n      <td>...</td>\n      <td>0.861111</td>\n      <td>0.702467</td>\n      <td>0.696448</td>\n      <td>0.701281</td>\n      <td>0.710149</td>\n      <td>0.539070</td>\n      <td>0.584750</td>\n      <td>0.945139</td>\n      <td>0.846486</td>\n      <td>0.841572</td>\n      <td>0.822564</td>\n      <td>0.770698</td>\n      <td>0.774381</td>\n      <td>0.698273</td>\n      <td>0.785566</td>\n      <td>0.595770</td>\n      <td>0.707167</td>\n      <td>0.813837</td>\n      <td>0.603989</td>\n      <td>0.538680</td>\n      <td>0.558256</td>\n      <td>0.604386</td>\n      <td>0.914295</td>\n      <td>0.630178</td>\n      <td>0.895607</td>\n      <td>0.778780</td>\n      <td>0.588616</td>\n      <td>0.786225</td>\n      <td>0.916750</td>\n      <td>0.907414</td>\n      <td>0.892208</td>\n      <td>0.566304</td>\n      <td>0.792438</td>\n      <td>0.858039</td>\n      <td>0.558378</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 758 columns</p>\n</div>"
     },
     "metadata": {},
     "execution_count": 14
    }
   ],
   "source": [
    "movie_similarity = 1 - pairwise_distances( ratings_matrix_users.values, metric=\"cosine\" )\n",
    "#Filling diagonals with 0s for future use when sorting is done (will be explained later), \n",
    "np.fill_diagonal( movie_similarity, 0 ) \n",
    "#create a new matrix with user/user similarities\n",
    "ratings_matrix_users = pd.DataFrame( movie_similarity )\n",
    "ratings_matrix_users.head()\n",
    "#we see here a distance between users (user,user)!!! this similarity we will use for items recommendations"
   ]
  },
  {
   "source": [
    "Now we find the maximum values for every user,based on the maximum similarity value.\n",
    "The most similar user is himself, and we fill these values with 0 (diagonal values in the table above)"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "     similarUser\n",
       "0            106\n",
       "1            604\n",
       "2            242\n",
       "3            646\n",
       "4            668\n",
       "..           ...\n",
       "753          351\n",
       "754          207\n",
       "755          216\n",
       "756          369\n",
       "757          181\n",
       "\n",
       "[758 rows x 1 columns]"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>similarUser</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>106</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>604</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>242</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>646</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>668</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>753</th>\n      <td>351</td>\n    </tr>\n    <tr>\n      <th>754</th>\n      <td>207</td>\n    </tr>\n    <tr>\n      <th>755</th>\n      <td>216</td>\n    </tr>\n    <tr>\n      <th>756</th>\n      <td>369</td>\n    </tr>\n    <tr>\n      <th>757</th>\n      <td>181</td>\n    </tr>\n  </tbody>\n</table>\n<p>758 rows × 1 columns</p>\n</div>"
     },
     "metadata": {},
     "execution_count": 15
    }
   ],
   "source": [
    "# here is why we need to use 0,0 in diagonal, to not stick with these 1.0 at diagonals!\n",
    "similar_users = ratings_matrix_users.idxmax(axis=1).to_frame() #converting pivot to   datagrame \n",
    "similar_users.columns=[\"similarUser\"]\n",
    "similar_users\n",
    "#1st column is user\n",
    "#2nd columnt is the most similar user"
   ]
  },
  {
   "source": [
    "Now we create 2 functions:\n",
    "get list of recommended movies, based on the user similarity.\n",
    "and a helper function to get the movie information based on movieId"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "movieId_recommended=list()\n",
    "def getRecommendedMoviesAsperUserSimilarity(userId):\n",
    "    \"\"\"\n",
    "     Recommending movies which user hasn't watched as per User Similarity\n",
    "    :param user_id: user_id to whom movie needs to be recommended\n",
    "    :return: movieIds to user \n",
    "    \"\"\"\n",
    "    user2Movies= ratings[ratings['userId']== userId]['movieId']\n",
    "    sim_user=similar_users.iloc[0,0]\n",
    "    df_recommended=pd.DataFrame(columns=['movieId','title','genres','userId','rating','timestamp'])\n",
    "    for movieId in ratings[ratings['userId']== sim_user]['movieId']:\n",
    "        if movieId not in user2Movies:\n",
    "            df_new= movies_and_ratings[(movies_and_ratings.userId==sim_user) & (movies_and_ratings.movieId==movieId)]\n",
    "            df_recommended=pd.concat([df_recommended,df_new])\n",
    "        best10=df_recommended.sort_values(['rating'], ascending = False )[1:10]  \n",
    "    return best10['movieId']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def movieIdToTitle(listMovieIDs):\n",
    "    \"\"\"\n",
    "     Converting movieId to titles\n",
    "    :param user_id: List of movies\n",
    "    :return: movie titles\n",
    "    \"\"\"\n",
    "    movie_titles= list()\n",
    "    for id in listMovieIDs:\n",
    "        movie_titles.append(movies[movies['movieId']==id]['title'])\n",
    "    return movie_titles"
   ]
  },
  {
   "source": [
    "And finally we ask to provide recommendations"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Movies you should watch are:\n\n[149    Rob Roy (1995)\nName: title, dtype: object, 375    True Lies (1994)\nName: title, dtype: object, 314    Shawshank Redemption, The (1994)\nName: title, dtype: object, 310    Secret of Roan Inish, The (1994)\nName: title, dtype: object, 292    Pulp Fiction (1994)\nName: title, dtype: object, 287    Once Were Warriors (1994)\nName: title, dtype: object, 243    Hoop Dreams (1994)\nName: title, dtype: object, 549    True Romance (1993)\nName: title, dtype: object, 20    Get Shorty (1995)\nName: title, dtype: object]\n"
     ]
    }
   ],
   "source": [
    "recommend_movies= movieIdToTitle(getRecommendedMoviesAsperUserSimilarity(my_user_id))\n",
    "print(\"Movies you should watch are:\\n\")\n",
    "print(recommend_movies)"
   ]
  }
 ]
}