Skip to content

Latest commit

 

History

History
27 lines (21 loc) · 1.69 KB

README.md

File metadata and controls

27 lines (21 loc) · 1.69 KB

DataPixie: Unleash Knowledge, Uncover Opportunities.

It brings a desktop based GUI, backed with Web scrapping algorithm, that scrapes the courses from youtube and udemy (as of now) and sorts them, based on user preference and stores in SQL DB and later showcases them on a GUI.

How to use it:

  1. Pre-requisite is that the java files are already present in local repo, and config (pom.xml dependencies etc) are set up.
  2. Set up and load the SQL server via JDBC connector. (Here MySQL 8.0 is used)
  3. You would also need to register to API client 2.0 Affiliate API Program by Udemy and a reliable Youtube API. Registration will get you the public key to access the courses and their details.
  4. Some files are therfore hidden from other developers, you will have to create the requests and add appropriate headers to the requests and use them.
  5. We are good to try the App.

SOP:

  1. First query has to be given in the search bar.
  2. Press the search button (it is a custom Jbutton class, doesn't give look and feel of a button, I know!)
  3. Then Apply the sorting feature. (Optional)
  4. Results will start displaying using the "Next" and "Previous" options in the content panel.

Upcoming features:

  1. Websites other than Youtube and Udemy to be incorporated in next release.
  2. Java Multithreading to be added for processing the requests parallelly and also visit all pages of a website.
  3. Front end GUI to be made more aesthetic and good looking. JavaFX with Scenebuilder to be used.

Adding one snapshot for the viewers for high level understanding of the project! image

Feedbacks will always be apreciated!