Are you a heavy user of Zotero and Medium? Do you want to apply your Zotero color coding to Medium articles and export the notes to Obsidian? This tool makes it possible!
This Python script allows you to scrape articles from multiple lists on Medium.com and save them into a library.csv. The library serves as a local database, and the script checks for differences between the library and scraped articles for every scrape attempt, saving new articles into the library.
Additionally, this web scraper downloads each article as a PDF using scribe.rip, an alternative frontend for Medium articles, for more beautifully formatted PDFs. This process uses Percollate, a command-line tool that turns web pages into elegant PDFs.
The downloaded PDFs are saved into separate folders (one for each list) within a downloads folder.
Finally, you can add these downloaded PDFs to Zotero to begin your note-taking and color-coding process.
- Scraping Medium Lists: The script scrapes articles from your specified Medium lists and saves them into a local library.csv file.
- PDF Generation: The scraper downloads each article as a PDF using scribe.rip and Percollate for a visually appealing format.
- File Organization: The downloaded PDFs are organized into separate folders for each list within a downloads folder.
- Zotero Integration: You can import the downloaded PDFs into Zotero for note-taking and color-coding.
- Python 3.x
- Brave Browser
- Percollate
- Clone or download the repository.
- Set up a virtual environment by running
python3 -m venv .venvand activate it. - Install all dependencies with
pip install -r requirements.txt. - Install Percollate by following the instructions here.
- Edit the medium_lists.csv file to add your own public lists.
Run the scraper with the following command: python3 -m medium_scraper.py.
- This scraper is built for the Brave Browser. You must install this browser to use the scraper, or edit the code to use Chrome (for more information, read this).
- The scraper will only work with public lists.