IPB Repository Scrapper & Downloader

Simple repository scrapper & downloader written in Go
View Demo · Report Bug · Request Feature

Note (Important!)

As of this date (07/04/2022), this application is no longer allowed to run. However, you could be able to run it locally. The source code is still open-source, until someone asked me to remove it 😂 I'm done with this. Thank you.

(back to top)

About

In the internet, there are many website that provide research journals and/or theses, such as IPB Repository, LIPI, e-Journal Perpusnas, and Google Scholar. IPB repository is one of the most popular university repositories on the internet. Unfortunately, only people with access can download locked files (students, lecturers, or staff). Therefore, I create this app to help anyone who wants to get free access to it. (my credentials are stored in the server-side).

Built With

• Go v1.16
• Bootstrap v5
• goquery v1.7.1
• godotenv v1.4.0

Prerequisites

To run this app, you need Go in your environment. Please go to the following link: https://golang.org/

(back to top)

Installation

1. Clone the repository.

   git clone https://github.com/naufalist/repository.ipb.ac.id.git

2. Copy .env.example to .env.

   cp .env.example .env

3. In .env, you should change the credentials.

   LDAP_USERNAME=CHANGE_WITH_YOUR_LDAP_USERNAME
   LDAP_PASSWORD=CHANGE_WITH_YOUR_LDAP_PASSWORD

4. Run the app

   go run main.go

5. Now, this app can be accessed in http://localhost:9000.

6. (Optional). If you want to use a different port, you can change this value in .env file.

   PORT=9000

(back to top)

Usage

1. Access this site: http://localhost:9000.
2. Copy-paste the repository url into the form. Example repository url: https://repository.ipb.ac.id/handle/123456789/39884.
3. Click the generate button.
4. If success, information from the repository will be displayed below the form.
5. To download repository file, you can click each download button.

(back to top)

Contributing

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".

(back to top)

Contact

@naufalist - [email protected]

(back to top)