Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/main'
Browse files Browse the repository at this point in the history
  • Loading branch information
@Evil0ctal
Evil0ctal committed Jun 14, 2024
2 parents bbaf20c + 2a0184b commit 0348205
Showing 1 changed file with 59 additions and 32 deletions.
91 changes: 59 additions & 32 deletions README.en.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

> 🚨If you need to use a private server to run this project, please refer to:[Deployment preparations](./README.md#%EF%B8%8F%E9%83%A8%E7%BD%B2%E5%89%8D%E7%9A%84%E5%87%86%E5%A4%87%E5%B7%A5%E4%BD%9C%E8%AF%B7%E4%BB%94%E7%BB%86%E9%98%85%E8%AF%BB),[Docker deployment](./README.md#%E9%83%A8%E7%BD%B2%E6%96%B9%E5%BC%8F%E4%BA%8C-docker),[One-click deployment](./README.md#%E9%83%A8%E7%BD%B2%E6%96%B9%E5%BC%8F%E4%B8%80-linux)
This project is based on[PyWebIO](https://github.com/pywebio/PyWebIO)，[FastAPI](https://fastapi.tiangolo.com/)，[HTTPX](https://www.python-httpx.org/), fast and asynchronous[Tik Tok](https://www.douyin.com/)/[TikTok](https://www.tiktok.com/)Data crawling tool, and realizes online batch parsing and downloading of videos or photo albums without watermarks, data crawling API, and iOS shortcut command without watermark downloads through the Web. You can deploy or modify this project yourself to achieve more functions, or you can call it directly in your project[scraper.py](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/Stable/scraper.py)or install an existing[pip package](https://pypi.org/project/douyin-tiktok-scraper/)As a parsing library, it is easy to crawl data, etc.....
This project is based on[PyWebIO](https://github.com/pywebio/PyWebIO)，[FastAPI](https://fastapi.tiangolo.com/)，[HTTPX](https://www.python-httpx.org/), fast and asynchronous[Tik Tok](https://www.douyin.com/)/[TikTok](https://www.tiktok.com/)Data crawling tool, and realizes online batch parsing and downloading of videos or photo albums without watermarks, data crawling API, and iOS shortcut commands without watermark downloads through the Web. You can deploy or modify this project yourself to achieve more functions, or you can call it directly in your project[scraper.py](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/Stable/scraper.py)or install an existing[pip package](https://pypi.org/project/douyin-tiktok-scraper/)As a parsing library, it is easy to crawl data, etc.....

_Some simple application scenarios:_

Expand All @@ -26,13 +26,38 @@ _Download prohibited videos, perform data analysis, download without watermark o
## 🔊 V4 version notes

- If you are interested in writing this project together, please add us on WeChat`Evil0ctal`Note: Github project reconstruction, everyone can communicate and learn from each other in the group. Advertising and illegal things are not allowed. It is purely for making friends and technical exchanges.
- used in this project`X-Bogus`The algorithm can still call Douyin and TikTok APIs normally.`A-Bogus`The algorithm will not be open source for the time being.
- This project uses`X-Bogus`Algorithms and`A_Bogus`The algorithm requests the Web API of Douyin and TikTok.
- Due to Douyin's risk control, after deploying this project, please**Obtain the cookie of Douyin website in the browser and replace it in config.yaml.**
- Please read the document below before raising an issue. Solutions to most problems will be included in the document.
- This project is completely free, but when using it, please comply with:[Apache-2.0 license](https://github.com/Evil0ctal/Douyin_TikTok_Download_API?tab=Apache-2.0-1-ov-file#readme)
- This project has a closed source branch version, which contains more interfaces and services. Please see the information below for details.

## 🔖TikHub.io API

[TikHub.io](https://beta-web.tikhub.io/en-us/users/signin)It is an API platform that provides various public data interfaces including Douyin and TikTok. If you want to support[Douyin_TikTok_Download_API](https://github.com/Evil0ctal/Douyin_TikTok_Download_API)For project development, we strongly recommend that you choose[TikHub.io](https://beta-web.tikhub.io/en-us/users/signin)。

#### Features:

> 📦 Ready to use right out of the box
Eliminate cumbersome usage processes and use the encapsulated SDK for rapid development, making calls easier. All API interfaces are written in accordance with OpenAPI specifications and come with sample parameters.

> 💰 Cost advantage
There are no preset package restrictions and no monthly usage thresholds. All consumption is billed immediately based on actual usage, and tiered billing is performed based on the user's daily requests. At the same time, free quota can be obtained through daily sign-in in the user backend. , and these free credits will not expire.

> ⚡️ Fast support
We have a large Discord community server, where administrators and other users will quickly reply to you and help you quickly solve current problems.

> 🎉Embrace open source
Part of TikHub's source code will be open sourced on Github, and it will sponsor authors of some open source projects.

#### Link:

- Discord:[Tikhub discord](https://discord.com/invite/aMEAS8Xsvz)
- Free Douyin/TikTok API:[Tikhub Beta Opi](https://beta.tikhub.io/)
- Register:[TikHub signup](https://beta-web.tikhub.io/en-us/users/signup)
- API Docs:[Cheers to my father, Dex](https://api.tikhub.io/)

## 🖥Demo site: I am very vulnerable...please do not stress test (·•᷄ࡇ•᷅ )

Expand Down Expand Up @@ -68,46 +93,46 @@ _Download prohibited videos, perform data analysis, download without watermark o

> **_/app/web_**
- use`PyWebIO`A simple web program created to process the values ​​entered on the web page and then use them`Crawlers`The related class processing interface outputs related data on the web page.
- use`PyWebIO`A simple web program created to process the values ​​entered on the web page and use them`Crawlers`The related class processing interface outputs related data on the web page.

**_Most of the parameters of the above files can be found in the corresponding`config.yaml`Modify in_**

## 💡Project file structure

./Douyin_TikTok_Download_API
├─app
│ ├─api
│ │ ├─endpoints
│ │ └─models
│ ├─download
│ └─web
│ └─views
└─crawlers
├─douyin
│ └─web
├─hybrid
├─tiktok
│ ├─app
│ └─web
└─utils
├─app
│ ├─api
│ │ ├─endpoints
│ │ └─models
│ ├─download
│ └─web
│ └─views
└─crawlers
├─douyin
│ └─web
├─hybrid
├─tiktok
│ ├─app
│ └─web
└─utils

## ✨Supported functions:

- Batch parsing on the web page (supports Douyin/TikTok mixed parsing)
- Batch analysis on the web page (supports Douyin/TikTok mixed analysis)
- Download videos or photo albums online.
- make[pip package](https://pypi.org/project/douyin-tiktok-scraper/)Conveniently and quickly import your projects
- [iOS shortcut commands to quickly call API](https://apps.apple.com/cn/app/%E5%BF%AB%E6%8D%B7%E6%8C%87%E4%BB%A4/id915249334)Achieve in-app download of watermark-free videos/photo albums
- Complete API documentation ([Demo/Demonstration](https://api.douyin.wtf/docs))
- Rich API interface:

- Douyin web version API

- [x] Video data analysis
- [x] Get user homepage work data
- [x] Obtain the data of works liked by the user's homepage
- [x] Obtain the data of collected works on the user's homepage
- [x] Get user homepage information
- [x] Get user collection work data
- [x] Get user live streaming data
- [x] Get user live stream data
- [x] Get the live streaming data of a specified user
- [x] Get the ranking of users who give gifts in the live broadcast room
- [x] Get single video comment data
Expand All @@ -116,19 +141,20 @@ _Download prohibited videos, perform data analysis, download without watermark o
- [x] Generate verify_fp
- [x] Generate s_v_web_id
- [x] Generate X-Bogus parameters using interface URL
- [x] Generate A_Bogus parameters using interface URL
- [x] Extract a single user id
- [x] Extract list user id
- [x] Extract a single work id
- [x] Extract list work id
- [x] Extract live broadcast room number from list
- [x] Extract live broadcast room number from list

- TikTok web version API

- [x] Video data analysis
- [x] Get user homepage work data
- [x] Obtain the data of works liked by the user's homepage
- [x] Get user homepage information
- [x] Get user homepage fan data
- [x] Get user home page fan data
- [x] Get user homepage follow data
- [x] Get user homepage collection work data
- [x] Get user homepage collection data
Expand All @@ -149,7 +175,7 @@ _Download prohibited videos, perform data analysis, download without watermark o

## 📦Call the parsing library (obsolete and needs to be updated):

> 💡PyPi:<https://pypi.org/project/douyin-tiktok-scraper/>
> 💡PyPi：<https://pypi.org/project/douyin-tiktok-scraper/>
Install the parsing library:`pip install douyin-tiktok-scraper`

Expand All @@ -170,7 +196,7 @@ asyncio.run(hybrid_parsing(url=input("Paste Douyin/TikTok/Bilibili share URL her

## 🗺️Supported submission formats:

> 💡Tip: Including but not limited to the following examples. If you encounter link parsing failure, please open a new one.[issue](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/issues)
> 💡Tip: Including but not limited to the following examples, if you encounter link parsing failure, please open a new one[issue](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/issues)
- Douyin sharing password (copy in APP)

Expand Down Expand Up @@ -243,6 +269,7 @@ Online:<https://api.douyin.wtf/docs>
- <https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/30e56e5a7f97f87d60b1045befb1f6db147f8590/crawlers/tiktok/web/config.yaml#L6>
- I turned off the online download function of the demo site. The video someone downloaded was so huge that it crashed the server. You can right-click on the web page parsing results page to save the video...
- The cookies of the demo site are my own and are not guaranteed to be valid for a long time. They only serve as a demonstration. If you deploy it yourself, please obtain the cookies yourself.
- If you need to directly access the video link returned by TikTok Web API, an HTTP 403 error will occur. Please use the API in this project.`/api/download`The interface downloads TikTok videos. This interface has been manually closed in the demo site, and you need to deploy this project by yourself.
- here is one**Video tutorial**You can refer to:**_<https://www.bilibili.com/video/BV1vE421j7NR/>_**

## 💻Deployment (Method 1 Linux)
Expand Down Expand Up @@ -274,7 +301,7 @@ My invitation link:

> Turn on/off automatic operation at startup
- Use the following commands to set the service to run automatically at boot or cancel automatic run at boot:
- Use the following command to set the service to run automatically at boot or cancel automatic run at boot:
- `sudo systemctl enable Douyin_TikTok_Download_API.service`
- `sudo systemctl disable Douyin_TikTok_Download_API.service`

Expand All @@ -290,7 +317,7 @@ My invitation link:
### Preparation

开始之前，请确保您的系统已安装Docker。如果还未安装Docker，可以从[Docker official website](https://www.docker.com/products/docker-desktop/)Download and install.
Before you begin, make sure Docker is installed on your system. If you haven't installed Docker yet, you can install it from[Docker official website](https://www.docker.com/products/docker-desktop/)Download and install.

### Step 1: Pull the Docker image

Expand Down Expand Up @@ -347,7 +374,7 @@ docker run -d --name douyin_tiktok_api -p 80:80 \

### Configuration file modification

Most of the configuration of the project can be found in the following directories:`config.yaml`File modification:
Most of the project configuration can be found in the following directories:`config.yaml`File modification:

- `/crawlers/douyin/web/config.yaml`
- `/crawlers/tiktok/web/config.yaml`
Expand All @@ -362,7 +389,7 @@ When you need to stop and remove a container, use the following commands:
docker stop douyin_tiktok_api

# Remove
docker rm douyin_tiktok_api
docker rm douyin_tiktok_api
```

## 📸Screenshot
Expand All @@ -375,7 +402,7 @@ Douyin official API:![](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/

API of this project:![](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/Screenshots/benchmarks/Douyin_API_Douyin_wtf.png?raw=true)

TikTok official API:![](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/Screenshots/benchmarks/TikTok_API.png?raw=true)
TikTok官方API:![](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/Screenshots/benchmarks/TikTok_API.png?raw=true)

API of this project:![](https://github.com/Evil0ctal/Douyin_TikTok_Download_API/blob/main/Screenshots/benchmarks/TikTok_API_Douyin_wtf.png?raw=true)

Expand Down

0 comments on commit 0348205

Please sign in to comment.