Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I fixed scrapers.py #23

Open
ghost opened this issue Jul 8, 2021 · 3 comments
Open

I fixed scrapers.py #23

ghost opened this issue Jul 8, 2021 · 3 comments

Comments

@ghost
Copy link

ghost commented Jul 8, 2021

I fixed the inactive websites in scrapers.py but it is still broke when you search

https://github.com/UdemyAdmin/UdemyCourseGrabber

@chamomile93
Copy link

Another website got down : myfreecourses.com
See trace :
root@desktop:/UdemyCourseGrabber# python3 main.py Paste in your Udemy course URL, or the course name: <course (name | url)> The value entered is not a link, taking it as the course name. 91%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉ | 10/11 [00:04<00:00, 2.21it/s]Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/aiohttp/connector.py", line 969, in _wrap_create_connection return await self._loop.create_connection(*args, **kwargs) # type: ignore # noqa File "/usr/lib/python3.6/asyncio/base_events.py", line 794, in create_connection raise exceptions[0] File "/usr/lib/python3.6/asyncio/base_events.py", line 781, in create_connection yield from self.sock_connect(sock, address) File "/usr/lib/python3.6/asyncio/selector_events.py", line 439, in sock_connect return (yield from fut) File "/usr/lib/python3.6/asyncio/selector_events.py", line 469, in _sock_connect_cb raise OSError(err, 'Connect call failed %s' % (address,)) ConnectionRefusedError: [Errno 111] Connect call failed ('72.52.178.23', 443) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "main.py", line 94, in <module> interactive() File "main.py", line 65, in interactive course_info = scraper.search_course() File "/UdemyCourseGrabber/scrapers.py", line 73, in search_course results = self.loop.run_until_complete(self.__search_course()) File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete return future.result() File "/UdemyCourseGrabber/scrapers.py", line 84, in __search_course result = await func File "/usr/lib/python3.6/asyncio/tasks.py", line 458, in _wait_for_one return f.result() # May raise f.exception(). File "/UdemyCourseGrabber/scrapers.py", line 226, in __myfreecourses async with session.get(f'https://myfreecourses.com/?s={self.__mod_name}') as response: File "/usr/local/lib/python3.6/dist-packages/aiohttp/client.py", line 1117, in __aenter__ self._resp = await self._coro File "/usr/local/lib/python3.6/dist-packages/aiohttp/client.py", line 521, in _request req, traces=traces, timeout=real_timeout File "/usr/local/lib/python3.6/dist-packages/aiohttp/connector.py", line 535, in connect proto = await self._create_connection(req, traces, timeout) File "/usr/local/lib/python3.6/dist-packages/aiohttp/connector.py", line 892, in _create_connection _, proto = await self._create_direct_connection(req, traces, timeout) File "/usr/local/lib/python3.6/dist-packages/aiohttp/connector.py", line 1051, in _create_direct_connection raise last_exc File "/usr/local/lib/python3.6/dist-packages/aiohttp/connector.py", line 1032, in _create_direct_connection client_error=client_error, File "/usr/local/lib/python3.6/dist-packages/aiohttp/connector.py", line 975, in _wrap_create_connection raise client_error(req.connection_key, exc) from exc aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host myfreecourses.com:443 ssl:default [Connect call failed ('72.52.178.23', 443)]

@bend-n
Copy link

bend-n commented Dec 23, 2021

i fixed my scrapers.py too, however
90%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ | 9/10 [00:10<00:01, 1.15s/it] Traceback (most recent call last): File "/home/bendn/UdemyCourseGrabber/main.py", line 94, in <module> interactive() File "/home/bendn/UdemyCourseGrabber/main.py", line 65, in interactive course_info = scraper.search_course() File "/home/bendn/UdemyCourseGrabber/scrapers.py", line 73, in search_course results = self.loop.run_until_complete(self.__search_course()) File "/usr/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete return future.result() File "/home/bendn/UdemyCourseGrabber/scrapers.py", line 84, in __search_course result = await func File "/usr/lib/python3.10/asyncio/tasks.py", line 571, in _wait_for_one return f.result() # May raise f.exception(). File "/home/bendn/UdemyCourseGrabber/scrapers.py", line 204, in __tutorialsplanet async with session.get(f'https://tutorialsplanet.net/?s={self.__mod_name}') as response: File "/home/bendn/.local/lib/python3.10/site-packages/aiohttp/client.py", line 1138, in __aenter__ self._resp = await self._coro File "/home/bendn/.local/lib/python3.10/site-packages/aiohttp/client.py", line 559, in _request await resp.start(conn) File "/home/bendn/.local/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 893, in start with self._timer: File "/home/bendn/.local/lib/python3.10/site-packages/aiohttp/helpers.py", line 721, in __exit__ raise asyncio.TimeoutError from None asyncio.exceptions.TimeoutError

@keethesh
Copy link
Owner

I'm back! In the following days, I'll see if the script needs a rewrite, else I'll just fix all the errors!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants