Tweepy - Tweet Extraction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"text-align: center;\"><div  style=\"color:#03bafc; font-size:30px; font-weight:bold; line-height:40px;\">Tweepy - Tweet Extraction</div></div>\n",
    "<div style=\"text-align: center; color:#006666\"><strong>Created By: </strong>Parikshit Koya</div>\n",
    "<br>\n",
    "<div style=\"text-align: center; color:#006666\"><strong>GitHub: </strong><a href='https://github.com/parikshitkoya'>https://github.com/parikshitkoya</a></div>\n",
    "<div style=\"text-align: center; color:#006666\"><strong>Kaggle: </strong><a href='https://www.kaggle.com/parikshitkoya'>https://www.kaggle.com/parikshitkoya</a></div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### What are we doing:\n",
    "\n",
    "- Downloading 3000 Tweets for where a twitter handle has been mentioned.\n",
    "- Extracting specific columns by parsing the JSON (JavaScript Object Notation) output from Tweepy\n",
    "- Extracting full tweets using the tweet ID as tweets are Truncated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "from secrets import apikey, apikeysecret, accesstoken, accesstokensecret"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tweepy\n",
    "from tweepy import Cursor\n",
    "import pandas as pd\n",
    "import json\n",
    "import openpyxl\n",
    "from datetime import datetime, timedelta\n",
    "import re\n",
    "import string\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import numpy as np\n",
    "import nltk\n",
    "import sklearn as sk\n",
    "from sklearn.metrics import precision_recall_fscore_support as score\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "auth = tweepy.OAuthHandler(apikey, apikeysecret)\n",
    "auth.set_access_token(accesstoken, accesstokensecret)\n",
    "api = tweepy.API(auth, wait_on_rate_limit=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Importing Tweets from @AmazonIN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = '@amazonin -filter:retweets'\n",
    "max_tweets = 4000\n",
    "searched_tweets_amazon = [status for status in tweepy.Cursor(api.search, lang = 'en', q=query).items(max_tweets)]\n",
    "tweet_id = []\n",
    "text = []\n",
    "source = []\n",
    "user = []\n",
    "created = []\n",
    "user_verified = []\n",
    "for i in range(len(searched_tweets_amazon)):\n",
    "    json_str = json.dumps(searched_tweets_amazon[i]._json)\n",
    "    json.loads(json_str)['id']\n",
    "    tweet_id.append(str(json.loads(json_str)['id']))\n",
    "    text.append(json.loads(json_str)['text'])\n",
    "    source.append(json.loads(json_str)['source'])\n",
    "    user.append(json.loads(json_str)['user']['screen_name'])\n",
    "    created.append(json.loads(json_str)['created_at'])\n",
    "    user_verified.append(json.loads(json_str)['user']['verified'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tweet_id</th>\n",
       "      <th>text</th>\n",
       "      <th>user</th>\n",
       "      <th>created</th>\n",
       "      <th>user_verified</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1351150534800117760</td>\n",
       "      <td>@amazonIN @AmazonHelp what's wrong??? After 6 ...</td>\n",
       "      <td>YashDaga</td>\n",
       "      <td>Mon Jan 18 12:52:45 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1351150515703455752</td>\n",
       "      <td>@amazonIN A .3\\nB. 2\\nC. 1\\nD. 4</td>\n",
       "      <td>Chandan19408317</td>\n",
       "      <td>Mon Jan 18 12:52:40 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1351150469343809543</td>\n",
       "      <td>@amazonIN A) REDMI 9A - 3) Rs. 6,999\\nB) Samsu...</td>\n",
       "      <td>MollaNew</td>\n",
       "      <td>Mon Jan 18 12:52:29 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1351150342893948931</td>\n",
       "      <td>@amazonIN A. Redmi 9A - 3. Rs 6,999🔥\\nB. Samsu...</td>\n",
       "      <td>MadhuChand07</td>\n",
       "      <td>Mon Jan 18 12:51:59 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1351150232634064901</td>\n",
       "      <td>Aura by Samyak Naturals is now Amazon's Choice...</td>\n",
       "      <td>SamyakNaturals</td>\n",
       "      <td>Mon Jan 18 12:51:33 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3995</th>\n",
       "      <td>1350771816193810433</td>\n",
       "      <td>@amazonIN 19 th January #AmazonPrimeMembership...</td>\n",
       "      <td>Royal__Jadeja</td>\n",
       "      <td>Sun Jan 17 11:47:51 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3996</th>\n",
       "      <td>1350771771130085378</td>\n",
       "      <td>@amazonIN 19th January , 2021\\n\\n#AmazonPrimeM...</td>\n",
       "      <td>Param_Sidhu77</td>\n",
       "      <td>Sun Jan 17 11:47:41 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3997</th>\n",
       "      <td>1350771761063759873</td>\n",
       "      <td>@amazonIN 26th january</td>\n",
       "      <td>Pradhanadityaj1</td>\n",
       "      <td>Sun Jan 17 11:47:38 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3998</th>\n",
       "      <td>1350771684282810373</td>\n",
       "      <td>@RNagothu @PrakashJavdekar @KapilMishra_IND @R...</td>\n",
       "      <td>giamvt</td>\n",
       "      <td>Sun Jan 17 11:47:20 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3999</th>\n",
       "      <td>1350771670924095488</td>\n",
       "      <td>@amazonIN Answer - 19th January 2021\\n#AmazonP...</td>\n",
       "      <td>GreatPunekar</td>\n",
       "      <td>Sun Jan 17 11:47:17 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4000 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 tweet_id                                               text  \\\n",
       "0     1351150534800117760  @amazonIN @AmazonHelp what's wrong??? After 6 ...   \n",
       "1     1351150515703455752                   @amazonIN A .3\\nB. 2\\nC. 1\\nD. 4   \n",
       "2     1351150469343809543  @amazonIN A) REDMI 9A - 3) Rs. 6,999\\nB) Samsu...   \n",
       "3     1351150342893948931  @amazonIN A. Redmi 9A - 3. Rs 6,999🔥\\nB. Samsu...   \n",
       "4     1351150232634064901  Aura by Samyak Naturals is now Amazon's Choice...   \n",
       "...                   ...                                                ...   \n",
       "3995  1350771816193810433  @amazonIN 19 th January #AmazonPrimeMembership...   \n",
       "3996  1350771771130085378  @amazonIN 19th January , 2021\\n\\n#AmazonPrimeM...   \n",
       "3997  1350771761063759873                             @amazonIN 26th january   \n",
       "3998  1350771684282810373  @RNagothu @PrakashJavdekar @KapilMishra_IND @R...   \n",
       "3999  1350771670924095488  @amazonIN Answer - 19th January 2021\\n#AmazonP...   \n",
       "\n",
       "                 user                         created  user_verified  \n",
       "0            YashDaga  Mon Jan 18 12:52:45 +0000 2021          False  \n",
       "1     Chandan19408317  Mon Jan 18 12:52:40 +0000 2021          False  \n",
       "2            MollaNew  Mon Jan 18 12:52:29 +0000 2021          False  \n",
       "3        MadhuChand07  Mon Jan 18 12:51:59 +0000 2021          False  \n",
       "4      SamyakNaturals  Mon Jan 18 12:51:33 +0000 2021          False  \n",
       "...               ...                             ...            ...  \n",
       "3995    Royal__Jadeja  Sun Jan 17 11:47:51 +0000 2021          False  \n",
       "3996    Param_Sidhu77  Sun Jan 17 11:47:41 +0000 2021          False  \n",
       "3997  Pradhanadityaj1  Sun Jan 17 11:47:38 +0000 2021          False  \n",
       "3998           giamvt  Sun Jan 17 11:47:20 +0000 2021          False  \n",
       "3999     GreatPunekar  Sun Jan 17 11:47:17 +0000 2021          False  \n",
       "\n",
       "[4000 rows x 5 columns]"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "amzn_dict = {'tweet_id': tweet_id, 'text': text, 'user': user, 'created': created, 'user_verified': user_verified}\n",
    "amzn_tweets = pd.DataFrame(amzn_dict)\n",
    "amzn_tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tweet_id</th>\n",
       "      <th>text</th>\n",
       "      <th>user</th>\n",
       "      <th>created</th>\n",
       "      <th>user_verified</th>\n",
       "      <th>full_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1351150534800117760</td>\n",
       "      <td>@amazonIN @AmazonHelp what's wrong??? After 6 ...</td>\n",
       "      <td>YashDaga</td>\n",
       "      <td>Mon Jan 18 12:52:45 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@amazonIN @AmazonHelp what's wrong??? After 6 ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1351150515703455752</td>\n",
       "      <td>@amazonIN A .3\\nB. 2\\nC. 1\\nD. 4</td>\n",
       "      <td>Chandan19408317</td>\n",
       "      <td>Mon Jan 18 12:52:40 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@amazonIN A .3\\nB. 2\\nC. 1\\nD. 4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1351150469343809543</td>\n",
       "      <td>@amazonIN A) REDMI 9A - 3) Rs. 6,999\\nB) Samsu...</td>\n",
       "      <td>MollaNew</td>\n",
       "      <td>Mon Jan 18 12:52:29 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@amazonIN A) REDMI 9A - 3) Rs. 6,999\\nB) Samsu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1351150342893948931</td>\n",
       "      <td>@amazonIN A. Redmi 9A - 3. Rs 6,999🔥\\nB. Samsu...</td>\n",
       "      <td>MadhuChand07</td>\n",
       "      <td>Mon Jan 18 12:51:59 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@amazonIN A. Redmi 9A - 3. Rs 6,999🔥\\nB. Samsu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1351150232634064901</td>\n",
       "      <td>Aura by Samyak Naturals is now Amazon's Choice...</td>\n",
       "      <td>SamyakNaturals</td>\n",
       "      <td>Mon Jan 18 12:51:33 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>Aura by Samyak Naturals is now Amazon's Choice...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3995</th>\n",
       "      <td>1350771816193810433</td>\n",
       "      <td>@amazonIN 19 th January #AmazonPrimeMembership...</td>\n",
       "      <td>Royal__Jadeja</td>\n",
       "      <td>Sun Jan 17 11:47:51 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@amazonIN 19 th January #AmazonPrimeMembership...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3996</th>\n",
       "      <td>1350771771130085378</td>\n",
       "      <td>@amazonIN 19th January , 2021\\n\\n#AmazonPrimeM...</td>\n",
       "      <td>Param_Sidhu77</td>\n",
       "      <td>Sun Jan 17 11:47:41 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@amazonIN 19th January , 2021\\n\\n#AmazonPrimeM...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3997</th>\n",
       "      <td>1350771761063759873</td>\n",
       "      <td>@amazonIN 26th january</td>\n",
       "      <td>Pradhanadityaj1</td>\n",
       "      <td>Sun Jan 17 11:47:38 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@amazonIN 26th january</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3998</th>\n",
       "      <td>1350771684282810373</td>\n",
       "      <td>@RNagothu @PrakashJavdekar @KapilMishra_IND @R...</td>\n",
       "      <td>giamvt</td>\n",
       "      <td>Sun Jan 17 11:47:20 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@RNagothu @PrakashJavdekar @KapilMishra_IND @R...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3999</th>\n",
       "      <td>1350771670924095488</td>\n",
       "      <td>@amazonIN Answer - 19th January 2021\\n#AmazonP...</td>\n",
       "      <td>GreatPunekar</td>\n",
       "      <td>Sun Jan 17 11:47:17 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@amazonIN Answer - 19th January 2021\\n#AmazonP...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4000 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 tweet_id                                               text  \\\n",
       "0     1351150534800117760  @amazonIN @AmazonHelp what's wrong??? After 6 ...   \n",
       "1     1351150515703455752                   @amazonIN A .3\\nB. 2\\nC. 1\\nD. 4   \n",
       "2     1351150469343809543  @amazonIN A) REDMI 9A - 3) Rs. 6,999\\nB) Samsu...   \n",
       "3     1351150342893948931  @amazonIN A. Redmi 9A - 3. Rs 6,999🔥\\nB. Samsu...   \n",
       "4     1351150232634064901  Aura by Samyak Naturals is now Amazon's Choice...   \n",
       "...                   ...                                                ...   \n",
       "3995  1350771816193810433  @amazonIN 19 th January #AmazonPrimeMembership...   \n",
       "3996  1350771771130085378  @amazonIN 19th January , 2021\\n\\n#AmazonPrimeM...   \n",
       "3997  1350771761063759873                             @amazonIN 26th january   \n",
       "3998  1350771684282810373  @RNagothu @PrakashJavdekar @KapilMishra_IND @R...   \n",
       "3999  1350771670924095488  @amazonIN Answer - 19th January 2021\\n#AmazonP...   \n",
       "\n",
       "                 user                         created  user_verified  \\\n",
       "0            YashDaga  Mon Jan 18 12:52:45 +0000 2021          False   \n",
       "1     Chandan19408317  Mon Jan 18 12:52:40 +0000 2021          False   \n",
       "2            MollaNew  Mon Jan 18 12:52:29 +0000 2021          False   \n",
       "3        MadhuChand07  Mon Jan 18 12:51:59 +0000 2021          False   \n",
       "4      SamyakNaturals  Mon Jan 18 12:51:33 +0000 2021          False   \n",
       "...               ...                             ...            ...   \n",
       "3995    Royal__Jadeja  Sun Jan 17 11:47:51 +0000 2021          False   \n",
       "3996    Param_Sidhu77  Sun Jan 17 11:47:41 +0000 2021          False   \n",
       "3997  Pradhanadityaj1  Sun Jan 17 11:47:38 +0000 2021          False   \n",
       "3998           giamvt  Sun Jan 17 11:47:20 +0000 2021          False   \n",
       "3999     GreatPunekar  Sun Jan 17 11:47:17 +0000 2021          False   \n",
       "\n",
       "                                              full_text  \n",
       "0     @amazonIN @AmazonHelp what's wrong??? After 6 ...  \n",
       "1                      @amazonIN A .3\\nB. 2\\nC. 1\\nD. 4  \n",
       "2     @amazonIN A) REDMI 9A - 3) Rs. 6,999\\nB) Samsu...  \n",
       "3     @amazonIN A. Redmi 9A - 3. Rs 6,999🔥\\nB. Samsu...  \n",
       "4     Aura by Samyak Naturals is now Amazon's Choice...  \n",
       "...                                                 ...  \n",
       "3995  @amazonIN 19 th January #AmazonPrimeMembership...  \n",
       "3996  @amazonIN 19th January , 2021\\n\\n#AmazonPrimeM...  \n",
       "3997                             @amazonIN 26th january  \n",
       "3998  @RNagothu @PrakashJavdekar @KapilMishra_IND @R...  \n",
       "3999  @amazonIN Answer - 19th January 2021\\n#AmazonP...  \n",
       "\n",
       "[4000 rows x 6 columns]"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "full_text = []\n",
    "for i in range(len(amzn_tweets)):\n",
    "    try:\n",
    "        full_text.append(api.get_status(amzn_tweets.iloc[i][0], tweet_mode='extended')._json['full_text'])\n",
    "    except:\n",
    "        full_text.append(amzn_tweets.iloc[i][1])\n",
    "amzn_tweets['full_text'] = full_text\n",
    "amzn_tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "amzn_tweets.to_excel('amazon_tweets.xlsx')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "auth = tweepy.OAuthHandler(apikey, apikeysecret)\n",
    "auth.set_access_token(accesstoken, accesstokensecret)\n",
    "api = tweepy.API(auth, wait_on_rate_limit=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Importing Tweets from @Flipkart"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = '@flipkart -filter:retweets'\n",
    "max_tweets = 4000\n",
    "searched_tweets_flipkart = [status for status in tweepy.Cursor(api.search, lang = 'en', q=query).items(max_tweets)]\n",
    "tweet_id = []\n",
    "text = []\n",
    "user = []\n",
    "created = []\n",
    "user_verified = []\n",
    "for i in range(len(searched_tweets_flipkart)):\n",
    "    json_str = json.dumps(searched_tweets_flipkart[i]._json)\n",
    "    json.loads(json_str)['id']\n",
    "    tweet_id.append(str(json.loads(json_str)['id']))\n",
    "    text.append(json.loads(json_str)['text'])\n",
    "    user.append(json.loads(json_str)['user']['screen_name'])\n",
    "    created.append(json.loads(json_str)['created_at'])\n",
    "    user_verified.append(json.loads(json_str)['user']['verified'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tweet_id</th>\n",
       "      <th>text</th>\n",
       "      <th>user</th>\n",
       "      <th>created</th>\n",
       "      <th>user_verified</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1351081210743820289</td>\n",
       "      <td>@Euro_Fashions @amazonIN @Flipkart Fake contes...</td>\n",
       "      <td>Ramprasad43</td>\n",
       "      <td>Mon Jan 18 08:17:17 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1351080341906333698</td>\n",
       "      <td>The #OPPOReno5Pro 5G priced at Rs. 35,990 and ...</td>\n",
       "      <td>waquarhaider10</td>\n",
       "      <td>Mon Jan 18 08:13:50 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1351080322461536258</td>\n",
       "      <td>@flipkartsupport Even if you won't accept, eve...</td>\n",
       "      <td>JayashreeKawad2</td>\n",
       "      <td>Mon Jan 18 08:13:45 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1351079901173051392</td>\n",
       "      <td>@Flipkart Mera ek product 11th jan ko pickup h...</td>\n",
       "      <td>ruturaj_art</td>\n",
       "      <td>Mon Jan 18 08:12:04 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1351079541733797890</td>\n",
       "      <td>@oppomobileindia EMI Months Rs. 1000\\n@Flipkart</td>\n",
       "      <td>IslamNabiAnsari</td>\n",
       "      <td>Mon Jan 18 08:10:39 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3304</th>\n",
       "      <td>1350703146494877699</td>\n",
       "      <td>@IndiaPOCO @NatwarlalMe @Flipkart what about #...</td>\n",
       "      <td>Laxmipr44761376</td>\n",
       "      <td>Sun Jan 17 07:14:59 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3305</th>\n",
       "      <td>1350702989753806849</td>\n",
       "      <td>@Flipkart What the hell is going on ??\\nWhy ar...</td>\n",
       "      <td>yash_kushwah_</td>\n",
       "      <td>Sun Jan 17 07:14:22 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3306</th>\n",
       "      <td>1350702947987058688</td>\n",
       "      <td>@flipkartsupport Fake @Flipkart</td>\n",
       "      <td>GoharTasha</td>\n",
       "      <td>Sun Jan 17 07:14:12 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3307</th>\n",
       "      <td>1350702895759384576</td>\n",
       "      <td>@Mayankujjaini @Flipkart heard about display i...</td>\n",
       "      <td>majestic_minal</td>\n",
       "      <td>Sun Jan 17 07:13:59 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3308</th>\n",
       "      <td>1350702631950323712</td>\n",
       "      <td>@WBCGamerYT @motorolaindia @Flipkart Thanks 👍 ...</td>\n",
       "      <td>Venkat72313461</td>\n",
       "      <td>Sun Jan 17 07:12:56 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3309 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 tweet_id                                               text  \\\n",
       "0     1351081210743820289  @Euro_Fashions @amazonIN @Flipkart Fake contes...   \n",
       "1     1351080341906333698  The #OPPOReno5Pro 5G priced at Rs. 35,990 and ...   \n",
       "2     1351080322461536258  @flipkartsupport Even if you won't accept, eve...   \n",
       "3     1351079901173051392  @Flipkart Mera ek product 11th jan ko pickup h...   \n",
       "4     1351079541733797890    @oppomobileindia EMI Months Rs. 1000\\n@Flipkart   \n",
       "...                   ...                                                ...   \n",
       "3304  1350703146494877699  @IndiaPOCO @NatwarlalMe @Flipkart what about #...   \n",
       "3305  1350702989753806849  @Flipkart What the hell is going on ??\\nWhy ar...   \n",
       "3306  1350702947987058688                    @flipkartsupport Fake @Flipkart   \n",
       "3307  1350702895759384576  @Mayankujjaini @Flipkart heard about display i...   \n",
       "3308  1350702631950323712  @WBCGamerYT @motorolaindia @Flipkart Thanks 👍 ...   \n",
       "\n",
       "                 user                         created  user_verified  \n",
       "0         Ramprasad43  Mon Jan 18 08:17:17 +0000 2021          False  \n",
       "1      waquarhaider10  Mon Jan 18 08:13:50 +0000 2021          False  \n",
       "2     JayashreeKawad2  Mon Jan 18 08:13:45 +0000 2021          False  \n",
       "3         ruturaj_art  Mon Jan 18 08:12:04 +0000 2021          False  \n",
       "4     IslamNabiAnsari  Mon Jan 18 08:10:39 +0000 2021          False  \n",
       "...               ...                             ...            ...  \n",
       "3304  Laxmipr44761376  Sun Jan 17 07:14:59 +0000 2021          False  \n",
       "3305    yash_kushwah_  Sun Jan 17 07:14:22 +0000 2021          False  \n",
       "3306       GoharTasha  Sun Jan 17 07:14:12 +0000 2021          False  \n",
       "3307   majestic_minal  Sun Jan 17 07:13:59 +0000 2021          False  \n",
       "3308   Venkat72313461  Sun Jan 17 07:12:56 +0000 2021          False  \n",
       "\n",
       "[3309 rows x 5 columns]"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "flipkart_dict = {'tweet_id': tweet_id, 'text': text, 'user': user, 'created': created, 'user_verified': user_verified}\n",
    "flipkart_tweets = pd.DataFrame(flipkart_dict)\n",
    "flipkart_tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tweet_id</th>\n",
       "      <th>text</th>\n",
       "      <th>user</th>\n",
       "      <th>created</th>\n",
       "      <th>user_verified</th>\n",
       "      <th>full_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1351081210743820289</td>\n",
       "      <td>@Euro_Fashions @amazonIN @Flipkart Fake contes...</td>\n",
       "      <td>Ramprasad43</td>\n",
       "      <td>Mon Jan 18 08:17:17 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@Euro_Fashions @amazonIN @Flipkart Fake contes...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1351080341906333698</td>\n",
       "      <td>The #OPPOReno5Pro 5G priced at Rs. 35,990 and ...</td>\n",
       "      <td>waquarhaider10</td>\n",
       "      <td>Mon Jan 18 08:13:50 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>The #OPPOReno5Pro 5G priced at Rs. 35,990 and ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1351080322461536258</td>\n",
       "      <td>@flipkartsupport Even if you won't accept, eve...</td>\n",
       "      <td>JayashreeKawad2</td>\n",
       "      <td>Mon Jan 18 08:13:45 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@flipkartsupport Even if you won't accept, eve...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1351079901173051392</td>\n",
       "      <td>@Flipkart Mera ek product 11th jan ko pickup h...</td>\n",
       "      <td>ruturaj_art</td>\n",
       "      <td>Mon Jan 18 08:12:04 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@Flipkart Mera ek product 11th jan ko pickup h...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1351079541733797890</td>\n",
       "      <td>@oppomobileindia EMI Months Rs. 1000\\n@Flipkart</td>\n",
       "      <td>IslamNabiAnsari</td>\n",
       "      <td>Mon Jan 18 08:10:39 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@oppomobileindia EMI Months Rs. 1000\\n@Flipkart</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3304</th>\n",
       "      <td>1350703146494877699</td>\n",
       "      <td>@IndiaPOCO @NatwarlalMe @Flipkart what about #...</td>\n",
       "      <td>Laxmipr44761376</td>\n",
       "      <td>Sun Jan 17 07:14:59 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@IndiaPOCO @NatwarlalMe @Flipkart what about #...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3305</th>\n",
       "      <td>1350702989753806849</td>\n",
       "      <td>@Flipkart What the hell is going on ??\\nWhy ar...</td>\n",
       "      <td>yash_kushwah_</td>\n",
       "      <td>Sun Jan 17 07:14:22 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@Flipkart What the hell is going on ??\\nWhy ar...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3306</th>\n",
       "      <td>1350702947987058688</td>\n",
       "      <td>@flipkartsupport Fake @Flipkart</td>\n",
       "      <td>GoharTasha</td>\n",
       "      <td>Sun Jan 17 07:14:12 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@flipkartsupport Fake @Flipkart</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3307</th>\n",
       "      <td>1350702895759384576</td>\n",
       "      <td>@Mayankujjaini @Flipkart heard about display i...</td>\n",
       "      <td>majestic_minal</td>\n",
       "      <td>Sun Jan 17 07:13:59 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@Mayankujjaini @Flipkart heard about display i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3308</th>\n",
       "      <td>1350702631950323712</td>\n",
       "      <td>@WBCGamerYT @motorolaindia @Flipkart Thanks 👍 ...</td>\n",
       "      <td>Venkat72313461</td>\n",
       "      <td>Sun Jan 17 07:12:56 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@WBCGamerYT @motorolaindia @Flipkart Thanks 👍 ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3309 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 tweet_id                                               text  \\\n",
       "0     1351081210743820289  @Euro_Fashions @amazonIN @Flipkart Fake contes...   \n",
       "1     1351080341906333698  The #OPPOReno5Pro 5G priced at Rs. 35,990 and ...   \n",
       "2     1351080322461536258  @flipkartsupport Even if you won't accept, eve...   \n",
       "3     1351079901173051392  @Flipkart Mera ek product 11th jan ko pickup h...   \n",
       "4     1351079541733797890    @oppomobileindia EMI Months Rs. 1000\\n@Flipkart   \n",
       "...                   ...                                                ...   \n",
       "3304  1350703146494877699  @IndiaPOCO @NatwarlalMe @Flipkart what about #...   \n",
       "3305  1350702989753806849  @Flipkart What the hell is going on ??\\nWhy ar...   \n",
       "3306  1350702947987058688                    @flipkartsupport Fake @Flipkart   \n",
       "3307  1350702895759384576  @Mayankujjaini @Flipkart heard about display i...   \n",
       "3308  1350702631950323712  @WBCGamerYT @motorolaindia @Flipkart Thanks 👍 ...   \n",
       "\n",
       "                 user                         created  user_verified  \\\n",
       "0         Ramprasad43  Mon Jan 18 08:17:17 +0000 2021          False   \n",
       "1      waquarhaider10  Mon Jan 18 08:13:50 +0000 2021          False   \n",
       "2     JayashreeKawad2  Mon Jan 18 08:13:45 +0000 2021          False   \n",
       "3         ruturaj_art  Mon Jan 18 08:12:04 +0000 2021          False   \n",
       "4     IslamNabiAnsari  Mon Jan 18 08:10:39 +0000 2021          False   \n",
       "...               ...                             ...            ...   \n",
       "3304  Laxmipr44761376  Sun Jan 17 07:14:59 +0000 2021          False   \n",
       "3305    yash_kushwah_  Sun Jan 17 07:14:22 +0000 2021          False   \n",
       "3306       GoharTasha  Sun Jan 17 07:14:12 +0000 2021          False   \n",
       "3307   majestic_minal  Sun Jan 17 07:13:59 +0000 2021          False   \n",
       "3308   Venkat72313461  Sun Jan 17 07:12:56 +0000 2021          False   \n",
       "\n",
       "                                              full_text  \n",
       "0     @Euro_Fashions @amazonIN @Flipkart Fake contes...  \n",
       "1     The #OPPOReno5Pro 5G priced at Rs. 35,990 and ...  \n",
       "2     @flipkartsupport Even if you won't accept, eve...  \n",
       "3     @Flipkart Mera ek product 11th jan ko pickup h...  \n",
       "4       @oppomobileindia EMI Months Rs. 1000\\n@Flipkart  \n",
       "...                                                 ...  \n",
       "3304  @IndiaPOCO @NatwarlalMe @Flipkart what about #...  \n",
       "3305  @Flipkart What the hell is going on ??\\nWhy ar...  \n",
       "3306                    @flipkartsupport Fake @Flipkart  \n",
       "3307  @Mayankujjaini @Flipkart heard about display i...  \n",
       "3308  @WBCGamerYT @motorolaindia @Flipkart Thanks 👍 ...  \n",
       "\n",
       "[3309 rows x 6 columns]"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "full_text = []\n",
    "for i in range(len(flipkart_tweets)):\n",
    "    try:\n",
    "        full_text.append(api.get_status(flipkart_tweets.iloc[i][0], tweet_mode='extended')._json['full_text'])\n",
    "    except:\n",
    "        full_text.append(flipkart_tweets.iloc[i][1])\n",
    "flipkart_tweets['full_text'] = full_text\n",
    "flipkart_tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [],
   "source": [
    "flipkart_tweets.to_excel('flipkart_tweets.xlsx')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Importing Tweets from @Snapdeal\n",
    "\n",
    "Attempting to download 3000 Tweets with a different set of Credentials:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convtodatetime(text):\n",
    "    text = datetime.strptime(text, '%a %b %d %H:%M:%S %z %Y')\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "202101180704\n"
     ]
    }
   ],
   "source": [
    "starttime = datetime.utcnow()\n",
    "starttime = starttime - timedelta(hours=0, minutes=0, seconds = 15)\n",
    "startdate = starttime.strftime('%Y%m%d%H%M')\n",
    "print(startdate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = '@snapdeal'\n",
    "tweet_id = []\n",
    "text = []\n",
    "user = []\n",
    "created = []\n",
    "user_verified = []\n",
    "\n",
    "while len(tweet_id)<4000:\n",
    "    searched_tweets_snapdeal = [status for status in api.search_full_archive(environment_name='TestEnv', query=query, toDate = startdate, maxResults = 100)]\n",
    "\n",
    "    for i in range(len(searched_tweets_snapdeal)):\n",
    "        json_str = json.dumps(searched_tweets_snapdeal[i]._json)\n",
    "        tweet_id.append(str(json.loads(json_str)['id']))\n",
    "        text.append(json.loads(json_str)['text'])\n",
    "        user.append(json.loads(json_str)['user']['screen_name'])\n",
    "        created.append(json.loads(json_str)['created_at'])\n",
    "        user_verified.append(json.loads(json_str)['user']['verified'])\n",
    "\n",
    "    temp_timestamp = pd.DataFrame(created ,columns=['temp_time'])\n",
    "    temp_timestamp['temp_time'] = temp_timestamp['temp_time'].apply(convtodatetime)\n",
    "    startdate = min(temp_timestamp['temp_time'])\n",
    "    startdate = startdate.strftime('%Y%m%d%H%M')\n",
    "    startdate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wanted to download 4000 tweets but was able to download only 3000 due to API restrictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tweet_id</th>\n",
       "      <th>text</th>\n",
       "      <th>user</th>\n",
       "      <th>created</th>\n",
       "      <th>user_verified</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1351027592799932424</td>\n",
       "      <td>It's a fraud or I really win price from snapde...</td>\n",
       "      <td>Deshbha32859266</td>\n",
       "      <td>Mon Jan 18 04:44:13 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1351027067803070467</td>\n",
       "      <td>#फ़सवनुक\\n@Flipkart हे ग्राहकांची फ़सवनुक करत ...</td>\n",
       "      <td>SAGARANDHARE4</td>\n",
       "      <td>Mon Jan 18 04:42:08 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1351004828315561987</td>\n",
       "      <td>Dear @Flipkart  ur courier partner is a lier. ...</td>\n",
       "      <td>Adhithy33736019</td>\n",
       "      <td>Mon Jan 18 03:13:46 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1350953560008175619</td>\n",
       "      <td>@snapdeal 91287989</td>\n",
       "      <td>Rakeshk92153278</td>\n",
       "      <td>Sun Jan 17 23:50:02 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1350865862547316736</td>\n",
       "      <td>RT @forbes_india: #2021LivingInHope As we stay...</td>\n",
       "      <td>PrachiSharma707</td>\n",
       "      <td>Sun Jan 17 18:01:34 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4089</th>\n",
       "      <td>1322228171173892096</td>\n",
       "      <td>Big billion stories 🤷‍♂️\\n\\nHey @Flipkart Why ...</td>\n",
       "      <td>trendtrader93</td>\n",
       "      <td>Fri Oct 30 17:25:36 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4090</th>\n",
       "      <td>1322221761300099072</td>\n",
       "      <td>@UniverseUNLTD @devgandhi1988 @Dilljaley @Megh...</td>\n",
       "      <td>Lucille_Is_Mine</td>\n",
       "      <td>Fri Oct 30 17:00:08 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4091</th>\n",
       "      <td>1322214169353990150</td>\n",
       "      <td>@Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...</td>\n",
       "      <td>devgandhi1988</td>\n",
       "      <td>Fri Oct 30 16:29:58 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4092</th>\n",
       "      <td>1322213873672355841</td>\n",
       "      <td>@devgandhi1988 @Dilljaley @MeghUpdates @amazon...</td>\n",
       "      <td>Lucille_Is_Mine</td>\n",
       "      <td>Fri Oct 30 16:28:47 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4093</th>\n",
       "      <td>1322211624044818432</td>\n",
       "      <td>@Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...</td>\n",
       "      <td>devgandhi1988</td>\n",
       "      <td>Fri Oct 30 16:19:51 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4094 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 tweet_id                                               text  \\\n",
       "0     1351027592799932424  It's a fraud or I really win price from snapde...   \n",
       "1     1351027067803070467  #फ़सवनुक\\n@Flipkart हे ग्राहकांची फ़सवनुक करत ...   \n",
       "2     1351004828315561987  Dear @Flipkart  ur courier partner is a lier. ...   \n",
       "3     1350953560008175619                                 @snapdeal 91287989   \n",
       "4     1350865862547316736  RT @forbes_india: #2021LivingInHope As we stay...   \n",
       "...                   ...                                                ...   \n",
       "4089  1322228171173892096  Big billion stories 🤷‍♂️\\n\\nHey @Flipkart Why ...   \n",
       "4090  1322221761300099072  @UniverseUNLTD @devgandhi1988 @Dilljaley @Megh...   \n",
       "4091  1322214169353990150  @Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...   \n",
       "4092  1322213873672355841  @devgandhi1988 @Dilljaley @MeghUpdates @amazon...   \n",
       "4093  1322211624044818432  @Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...   \n",
       "\n",
       "                 user                         created  user_verified  \n",
       "0     Deshbha32859266  Mon Jan 18 04:44:13 +0000 2021          False  \n",
       "1       SAGARANDHARE4  Mon Jan 18 04:42:08 +0000 2021          False  \n",
       "2     Adhithy33736019  Mon Jan 18 03:13:46 +0000 2021          False  \n",
       "3     Rakeshk92153278  Sun Jan 17 23:50:02 +0000 2021          False  \n",
       "4     PrachiSharma707  Sun Jan 17 18:01:34 +0000 2021          False  \n",
       "...               ...                             ...            ...  \n",
       "4089    trendtrader93  Fri Oct 30 17:25:36 +0000 2020          False  \n",
       "4090  Lucille_Is_Mine  Fri Oct 30 17:00:08 +0000 2020          False  \n",
       "4091    devgandhi1988  Fri Oct 30 16:29:58 +0000 2020          False  \n",
       "4092  Lucille_Is_Mine  Fri Oct 30 16:28:47 +0000 2020          False  \n",
       "4093    devgandhi1988  Fri Oct 30 16:19:51 +0000 2020          False  \n",
       "\n",
       "[4094 rows x 5 columns]"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "snapdeal_dict = {'tweet_id': tweet_id, 'text': text, 'user': user, 'created': created, 'user_verified': user_verified}\n",
    "snapdeal_tweets = pd.DataFrame(snapdeal_dict)\n",
    "snapdeal_tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tweet_id</th>\n",
       "      <th>text</th>\n",
       "      <th>user</th>\n",
       "      <th>created</th>\n",
       "      <th>user_verified</th>\n",
       "      <th>full_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1351027592799932424</td>\n",
       "      <td>It's a fraud or I really win price from snapde...</td>\n",
       "      <td>Deshbha32859266</td>\n",
       "      <td>Mon Jan 18 04:44:13 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>It's a fraud or I really win price from snapde...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1351027067803070467</td>\n",
       "      <td>#फ़सवनुक\\n@Flipkart हे ग्राहकांची फ़सवनुक करत ...</td>\n",
       "      <td>SAGARANDHARE4</td>\n",
       "      <td>Mon Jan 18 04:42:08 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>#फ़सवनुक\\n@Flipkart हे ग्राहकांची फ़सवनुक करत ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1351004828315561987</td>\n",
       "      <td>Dear @Flipkart  ur courier partner is a lier. ...</td>\n",
       "      <td>Adhithy33736019</td>\n",
       "      <td>Mon Jan 18 03:13:46 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>Dear @Flipkart  ur courier partner is a lier. ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1350953560008175619</td>\n",
       "      <td>@snapdeal 91287989</td>\n",
       "      <td>Rakeshk92153278</td>\n",
       "      <td>Sun Jan 17 23:50:02 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>@snapdeal 91287989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1350865862547316736</td>\n",
       "      <td>RT @forbes_india: #2021LivingInHope As we stay...</td>\n",
       "      <td>PrachiSharma707</td>\n",
       "      <td>Sun Jan 17 18:01:34 +0000 2021</td>\n",
       "      <td>False</td>\n",
       "      <td>RT @forbes_india: #2021LivingInHope As we stay...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4089</th>\n",
       "      <td>1322228171173892096</td>\n",
       "      <td>Big billion stories 🤷‍♂️\\n\\nHey @Flipkart Why ...</td>\n",
       "      <td>trendtrader93</td>\n",
       "      <td>Fri Oct 30 17:25:36 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "      <td>Big billion stories 🤷‍♂️\\n\\nHey @Flipkart Why ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4090</th>\n",
       "      <td>1322221761300099072</td>\n",
       "      <td>@UniverseUNLTD @devgandhi1988 @Dilljaley @Megh...</td>\n",
       "      <td>Lucille_Is_Mine</td>\n",
       "      <td>Fri Oct 30 17:00:08 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "      <td>@UniverseUNLTD @devgandhi1988 @Dilljaley @Megh...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4091</th>\n",
       "      <td>1322214169353990150</td>\n",
       "      <td>@Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...</td>\n",
       "      <td>devgandhi1988</td>\n",
       "      <td>Fri Oct 30 16:29:58 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "      <td>@Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4092</th>\n",
       "      <td>1322213873672355841</td>\n",
       "      <td>@devgandhi1988 @Dilljaley @MeghUpdates @amazon...</td>\n",
       "      <td>Lucille_Is_Mine</td>\n",
       "      <td>Fri Oct 30 16:28:47 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "      <td>@devgandhi1988 @Dilljaley @MeghUpdates @amazon...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4093</th>\n",
       "      <td>1322211624044818432</td>\n",
       "      <td>@Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...</td>\n",
       "      <td>devgandhi1988</td>\n",
       "      <td>Fri Oct 30 16:19:51 +0000 2020</td>\n",
       "      <td>False</td>\n",
       "      <td>@Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4094 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 tweet_id                                               text  \\\n",
       "0     1351027592799932424  It's a fraud or I really win price from snapde...   \n",
       "1     1351027067803070467  #फ़सवनुक\\n@Flipkart हे ग्राहकांची फ़सवनुक करत ...   \n",
       "2     1351004828315561987  Dear @Flipkart  ur courier partner is a lier. ...   \n",
       "3     1350953560008175619                                 @snapdeal 91287989   \n",
       "4     1350865862547316736  RT @forbes_india: #2021LivingInHope As we stay...   \n",
       "...                   ...                                                ...   \n",
       "4089  1322228171173892096  Big billion stories 🤷‍♂️\\n\\nHey @Flipkart Why ...   \n",
       "4090  1322221761300099072  @UniverseUNLTD @devgandhi1988 @Dilljaley @Megh...   \n",
       "4091  1322214169353990150  @Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...   \n",
       "4092  1322213873672355841  @devgandhi1988 @Dilljaley @MeghUpdates @amazon...   \n",
       "4093  1322211624044818432  @Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...   \n",
       "\n",
       "                 user                         created  user_verified  \\\n",
       "0     Deshbha32859266  Mon Jan 18 04:44:13 +0000 2021          False   \n",
       "1       SAGARANDHARE4  Mon Jan 18 04:42:08 +0000 2021          False   \n",
       "2     Adhithy33736019  Mon Jan 18 03:13:46 +0000 2021          False   \n",
       "3     Rakeshk92153278  Sun Jan 17 23:50:02 +0000 2021          False   \n",
       "4     PrachiSharma707  Sun Jan 17 18:01:34 +0000 2021          False   \n",
       "...               ...                             ...            ...   \n",
       "4089    trendtrader93  Fri Oct 30 17:25:36 +0000 2020          False   \n",
       "4090  Lucille_Is_Mine  Fri Oct 30 17:00:08 +0000 2020          False   \n",
       "4091    devgandhi1988  Fri Oct 30 16:29:58 +0000 2020          False   \n",
       "4092  Lucille_Is_Mine  Fri Oct 30 16:28:47 +0000 2020          False   \n",
       "4093    devgandhi1988  Fri Oct 30 16:19:51 +0000 2020          False   \n",
       "\n",
       "                                              full_text  \n",
       "0     It's a fraud or I really win price from snapde...  \n",
       "1     #फ़सवनुक\\n@Flipkart हे ग्राहकांची फ़सवनुक करत ...  \n",
       "2     Dear @Flipkart  ur courier partner is a lier. ...  \n",
       "3                                    @snapdeal 91287989  \n",
       "4     RT @forbes_india: #2021LivingInHope As we stay...  \n",
       "...                                                 ...  \n",
       "4089  Big billion stories 🤷‍♂️\\n\\nHey @Flipkart Why ...  \n",
       "4090  @UniverseUNLTD @devgandhi1988 @Dilljaley @Megh...  \n",
       "4091  @Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...  \n",
       "4092  @devgandhi1988 @Dilljaley @MeghUpdates @amazon...  \n",
       "4093  @Lucille_Is_Mine @Dilljaley @MeghUpdates @amaz...  \n",
       "\n",
       "[4094 rows x 6 columns]"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "full_text = []\n",
    "for i in range(len(snapdeal_tweets)):\n",
    "    try:\n",
    "        full_text.append(api.get_status(snapdeal_tweets.iloc[i][0], tweet_mode='extended')._json['full_text'])\n",
    "    except:\n",
    "        full_text.append(snapdeal_tweets2.iloc[i][0])\n",
    "snapdeal_tweets['full_text'] = full_text\n",
    "snapdeal_tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "snapdeal_tweets.to_excel('snapdeal_tweets.xlsx')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}