Skip to content

aktheroy/Multilingual_Translation

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

61 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐ŸŒŸ Demo

Demo.mp4

Multilingual Translation Model

A scalable and efficient multilingual translation system powered by fine-tuned Large Language Models, featuring real-time translation capabilities across multiple language pairs.

Table of Contents

Hereโ€™s the updated section with the live link separated below the image and set to open in a new tab:


๐Ÿ› ๏ธ Flowchart

Live Link

Untitled design

Live Deployment on Google Cloud Platform

Note: The live deployment is hosted on Google Cloud Platform. Please allow up to 2 minutes for cold start if the service is inactive.


This ensures the image and link are separate, and the link opens in a new tab when clicked. Let me know if you need further adjustments! ๐Ÿš€


๐Ÿ› ๏ธ Technical Implementation

Model Architecture

  • Fine-tuned Large Language Model using LoRA (Low-Rank Adaptation)
  • Quantization techniques for model optimization
  • Automated validation and preprocessing pipelines

Backend

  • Flask-based REST API server
  • Scikit-learn for data preprocessing and validation
  • Docker containerization for consistent deployment
  • CI/CD pipeline integration for automated deployment

Frontend

  • Responsive web interface
  • Real-time translation updates
  • Concurrent translation support

๐Ÿ“Š Performance Metrics

  • Translation Accuracy: 32% BLEU score
  • Training Efficiency: 20% reduction in training cycle time
  • Inference Latency: Sub-400ms response time
  • Scalability: Handles concurrent translation requests

๐Ÿ’ป Tech Stack

๐Ÿค– ML/Deep Learning

Python PyTorch HuggingFace PEFT Pandas NumPy Matplotlib

๐ŸŽจ Frontend & ๐Ÿ”ง Backend

Flask HTML5 CSS3 JavaScript Jinja

โ˜๏ธ Cloud/DevOps

Google Cloud Docker

๐Ÿš€ Getting Started

Prerequisites

# Clone the repository
git clone https://github.com/yourusername/multilingual-translation.git

# Install dependencies
pip install -r requirements.txt

Running Locally

# Start the Flask server
python app.py

# Access the web interface
open http://localhost:5000

Docker Deployment

# Build Docker image
docker build -t translation-model .

# Run container (map port 5000 to 8080)
docker run -p 8080:5000 translation-model

๐Ÿ“ Project Structure

.
โ”œโ”€โ”€ Backend/
โ”‚   โ”œโ”€โ”€ ml/
โ”‚   โ”‚   โ”œโ”€โ”€ models/                                         # Saved model files for local/production use
โ”‚   โ”‚   โ””โ”€โ”€ notebooks/                                      # Jupyter notebooks for experiments
โ”‚   โ”‚       โ”œโ”€โ”€ EDA.ipynb                                   # Exploratory Data Analysis
โ”‚   โ”‚       โ”œโ”€โ”€ NLP pipeline.ipynb                          # Data preprocessing pipeline
โ”‚   โ”‚       โ”œโ”€โ”€ M2M100.ipynb                                # Base M2M100 implementation
โ”‚   โ”‚       โ”œโ”€โ”€ M2M100(Lora).ipynb                          # LoRA fine-tuning
โ”‚   โ”‚       โ”œโ”€โ”€ 4bit FB(M2M100)+lora.ipynb                  # 4-bit quantization
โ”‚   โ”‚       โ”œโ”€โ”€ 4bit FB(M2m100)+lora+Greek+Hindi.ipynb      # Multi-language support
โ”‚   โ”‚       โ””โ”€โ”€ Bleu_score_BFT.ipynb                        # Performance evaluation
โ”‚   โ”œโ”€โ”€ api/                                                # REST API endpoints
โ”‚   โ”œโ”€โ”€ dataset/                                            # Training and evaluation datasets
โ”‚   โ””โ”€โ”€ preprocessing/                                      # Production preprocessing scripts
โ”œโ”€โ”€ Frontend/
โ”‚   โ”œโ”€โ”€ Templates/
โ”‚   โ”‚   โ””โ”€โ”€ index.html                                      # Main application template
โ”‚   โ””โ”€โ”€ static/
โ”‚       โ”œโ”€โ”€ css/
โ”‚       โ”‚   โ””โ”€โ”€ styles.css                                 # Custom styling
โ”‚       โ””โ”€โ”€ js/
โ”‚           โ””โ”€โ”€ script.js                                  # Frontend logic
โ”œโ”€โ”€ tests/                                                 # Unit and integration tests
โ”œโ”€โ”€ Dockerfile                                             # Container configuration
โ””โ”€โ”€ requirements.txt                                       # Python dependencies

๐Ÿ“š API Documentation

Translation Endpoint

POST /translate
Content-Type: application/json

Request Body

{
    "source_text": "Hi friends, how are you",
    "source_lang": "en",
    "target_lang": "hi"
}

Response

{
    "translated_text": "เคจเคฎเคธเฅเค•เคพเคฐ เคฆเฅ‹เคธเฅเคคเฅ‹เค‚, เค†เคช เค•เฅˆเคธเฅ‡ เคนเฅˆเค‚?"
}

Example: Greek Translation

{
    "source_text": "Hello, how are you?",
    "source_lang": "en",
    "target_lang": "el"
}

Response

{
    "translated_text": "ฮ“ฮตฮนฮฌ ฯƒฮฟฯ…, ฯ€ฯŽฯ‚ ฮตฮฏฯƒฮฑฮน;"
}

Supported Languages

  • English (en)
  • Hindi (hi)
  • Greek (el)

Error Responses

{
    "error": "Model or tokenizer not loaded."
}

Status: 500

{
    "error": "Translation failed."
}

Status: 500

๐Ÿค Contributing

  1. Fork the repository
  2. Create your feature branch: git checkout -b feature/NewFeature
  3. Commit your changes: git commit -m 'Add NewFeature'
  4. Push to the branch: git push origin feature/NewFeature
  5. Submit a pull request

๐Ÿ“ License

This project is licensed under the MIT License - see the LICENSE file for details.


About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages