Project Name

"News Crawler" Web App

Description

This project is a Razor web app that utilizes Docker, Elasticsearch, Kibana, and Python Scrapy library to crawl a news website and store the crawled data in Elasticsearch. The app then uses Razor pages to display the data and provides various functionalities such as a "What's New" section, a paginated list of all news, a search function, and a details page for each news article.

Features

• "What's New" section displaying random 6 news articles from Elasticsearch
• Paginated list of all news articles
• Search function to find articles based on keywords
• Details page for each news article

Installation

To run this project, follow these steps:

1. Install Docker on your machine.
2. Run Elasticsearch and Kibana containers using Docker.
3. Configure Elasticsearch to ensure proper functionality.
4. Install Python and the Scrapy library.
5. Crawl the news website using Scrapy and save the data to Elasticsearch.
6. Open the Razor web app project in your preferred development environment.
7. Build and run the project.

Usage

Once the project is running, you can access the following functionalities:

• "What's New" section: Displays random 6 news articles from Elasticsearch.
• All News: Lists all news articles with pagination.
• Search: Allows you to search for news articles based on keywords.
• Details: Shows the details of a selected news article.

Contributing

If you would like to contribute to this project, please follow these guidelines:

• Fork the repository.
• Create a new branch for your feature or bug fix.
• Make your changes and commit them.
• Push your changes to your forked repository.
• Submit a pull request.

License

This project is licensed under the Mozilla Public License Version 2.0. Please refer to the LICENSE file for more information.

Contact

If you have any questions or suggestions, feel free to contact the project maintainer at [email protected].